An airport intelligent operation and maintenance method and system based on VR and the Internet of Things

Abstract

This invention belongs to the field of intelligent operation and maintenance management technology and discloses a method and system for intelligent airport operation and maintenance based on VR and the Internet of Things. The method includes: establishing a BIM model of the airport pavement area, generating a lightweight 3D model of the airport pavement area through modeling analysis and image rendering, and extracting a pavement image of the predetermined airport section from the model; introducing deep learning-based image processing technology to perform multi-level feature extraction and joint perception on the pavement image of the predetermined airport section to intelligently identify whether the predetermined airport section requires maintenance, and based on the analysis results, corresponding marking and prompting are made in the 3D model of the airport pavement area. The present invention can update pavement status information in real time, helping operation and maintenance personnel better understand the actual conditions of the airport pavement, thereby effectively improving the efficiency and accuracy of airport operation and maintenance.

Description

Technical Field

[0001] The present invention relates to the field of intelligent operation and maintenance management technology, and more specifically, to an airport intelligent operation and maintenance method and system based on VR and the Internet of Things. Background Art

[0002] With the rapid development of the global aviation industry, airports, as crucial transportation hubs connecting the world, are increasingly becoming a focus of attention within and beyond the industry regarding their operational efficiency and safety. Airport pavement, as critical infrastructure for aircraft takeoff and landing, taxiing, and the movement of various vehicles, is crucial for overall operational efficiency and flight safety.

[0003] Traditional airport pavement maintenance methods mainly rely on manual inspections and regular testing. Regular inspection cameras directly capture two-dimensional images. This method is not only time-consuming and labor-intensive, but also difficult to achieve real-time monitoring and comprehensive coverage of airport pavement conditions. Especially when faced with large-scale, complex and changeable airport pavement environments, its limitations and shortcomings become increasingly obvious.

[0004] In recent years, the rapid development of virtual reality (VR) technology and the Internet of Things (IoT) has provided new solutions for intelligent airport operations. VR technology simulates a real-world airport environment, providing operators with an immersive and intuitive visual platform for operations and maintenance, significantly improving the efficiency and accuracy of operational decisions. Simultaneously, IoT technology, through various sensors and devices, enables real-time monitoring and data collection of airport facility status, providing a rich data source for intelligent operations and maintenance. Therefore, the rich data sources provided by IoT technology can be combined with VR technology for intelligent airport operations and maintenance. Consequently, a method and system for intelligent airport operations and maintenance based on VR and IoT are highly anticipated. Summary of the Invention

[0005] This invention application provides an airport intelligent operation and maintenance method and system based on VR and the Internet of Things, which aims to partially or completely solve the technical problems in the existing technology. It can help operation and maintenance personnel better understand the actual situation of airport pavement based on existing sensor data, BIM model data, etc., thereby effectively improving the efficiency and accuracy of airport operation and maintenance. This invention application adopts the following technical solutions:

[0006] First, a method for intelligent airport operation and maintenance based on VR and the Internet of Things, including:

[0007] Establish a BIM model of the airport pavement area;

[0008] Modeling and analyzing the BIM model of the airport pavement area to achieve information conversion and extraction to obtain a JSON file;

[0009] Sending the JSON file to a VR server, which loads the JSON file into a WebGL model and performs image rendering to obtain a lightweight 3D model of the airport pavement area;

[0010] Extracting a road surface image of a predetermined airport road section from the lightweight airport road surface area 3D model;

[0011] determining, based on the road surface image of the predetermined airport road section, whether the predetermined airport road section requires maintenance to obtain a determination result;

[0012] In response to the determination that the predetermined airport road section requires maintenance, marking the predetermined airport road section in the 3D model of the lightweight airport pavement area, the marking being "maintenance required";

[0013] The 3D model of the lightweight airfield pavement area is displayed with markings.

[0014] Optionally, judging whether the predetermined airport road section requires maintenance based on the road surface image of the predetermined airport road section to obtain a judgment result includes:

[0015] Performing multi-level image context feature extraction on the road surface image of the predetermined airport section to obtain a shallow context coding feature map of the airport section road surface and a deep context coding feature map of the airport section road surface;

[0016] Performing a priori-guided feature saliency joint perception on the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface to obtain a shallow-deep joint perception feature map of the airport section road surface;

[0017] The judgment result is generated based on the shallow-deep joint perception feature map of the airport road section.

[0018] Optionally, performing multi-level image context feature extraction on the road surface image of the predetermined airport section to obtain a shallow context coding feature map of the airport section road surface and a deep context coding feature map of the airport section road surface includes:

[0019] The road surface image of the predetermined airport road section is input into an airport road surface feature extractor based on the FPT model to obtain a shallow context coding feature map of the airport road section road surface and a deep context coding feature map of the airport road section road surface.

[0020] Optionally, performing a priori-guided feature saliency joint perception on the shallow context encoding feature map of the airport section road surface and the deep context encoding feature map of the airport section road surface to obtain a shallow-deep joint perception feature map of the airport section road surface includes:

[0021] Reshaping the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface to obtain a shallow context coding feature matrix of the airport section road surface and a deep context coding feature matrix of the airport section road surface;

[0022] Inputting the airport section road surface shallow context coding feature matrix and the airport section road surface deep context coding feature matrix into the prior guided modulation module to obtain the prior modulated airport section road surface shallow context coding feature matrix and the prior modulated airport section road surface deep context coding feature matrix;

[0023] The shallow context encoding feature matrix of the prior modulated airport section road surface and the deep context encoding feature matrix of the prior modulated airport section road surface are subjected to attention interaction fusion to obtain the shallow-deep joint perception feature map of the airport section road surface.

[0024] Optionally, performing attention interaction fusion on the a priori modulated airport section road surface shallow context encoding feature matrix and the a priori modulated airport section road surface deep context encoding feature matrix to obtain the airport section road surface shallow-deep joint perception feature map, including:

[0025] Inputting the prior modulated airport section road surface shallow context coding feature matrix and the prior modulated airport section road surface deep context coding feature matrix into the attention interaction coding module based on the transformer-like structure to obtain the airport section road surface shallow feature enhancement interaction coding matrix and the airport section road surface deep feature enhancement interaction coding matrix;

[0026] Reshaping the shallow-layer feature enhancement interaction coding matrix and the deep-layer feature enhancement interaction coding matrix of the airport section road surface to obtain a shallow-layer feature enhancement interaction coding feature map and a deep-layer feature enhancement interaction coding feature map of the airport section road surface;

[0027] The weighted sum of the shallow-layer feature enhancement interactive coding feature map of the airport section road surface and the deep-layer feature enhancement interactive coding feature map of the airport section road surface is calculated to obtain the shallow-deep joint perception feature map of the airport section road surface.

[0028] Optionally, the a priori modulated airport section road surface shallow context coding feature matrix and the a priori modulated airport section road surface deep context coding feature matrix are input into an attention interaction coding module based on a transformer-like structure to obtain an airport section road surface shallow feature enhancement interaction coding matrix and an airport section road surface deep feature enhancement interaction coding matrix, including:

[0029] Using the a priori modulated airport section road surface shallow context encoding feature matrix as the query feature matrix and the value feature matrix, and using the a priori modulated airport section road surface deep context encoding feature matrix as the key feature matrix, the query feature matrix, the value feature matrix, and the key feature matrix are subjected to attention interaction based on the converter structure to obtain the airport section road surface shallow feature enhanced interaction encoding matrix;

[0030] The prior modulated airport section road surface deep context encoding feature matrix is ​​used as the query feature matrix and the value feature matrix, and the prior modulated airport section road surface shallow context encoding feature matrix is ​​used as the key feature matrix. The query feature matrix, the value feature matrix and the key feature matrix are subjected to attention interaction based on the converter structure to obtain the airport section road surface deep feature enhanced interaction encoding matrix.

[0031] Optionally, generating the judgment result based on the shallow-deep joint perception feature map of the airport road section includes:

[0032] The shallow-deep joint perception feature map of the airport road section is input into the classifier-based airport operation and maintenance judgment module to obtain the judgment result.

[0033] Optionally, inputting the shallow-deep joint perception feature map of the airport road section into a classifier-based airport operation and maintenance judgment module to obtain the judgment result includes:

[0034] Expanding the airport section road surface shallow-deep joint perception feature map into an airport section road surface shallow-deep joint perception feature vector;

[0035] Using the fully connected layer of the airport operation and maintenance judgment module to perform fully connected encoding on the shallow-deep joint perception feature vector of the airport section road surface to obtain a shallow-deep joint perception fully connected encoding vector of the airport section road surface;

[0036] Inputting the airport road section pavement shallow-deep joint perception fully connected coding vector into the Softmax classification function of the airport operation and maintenance judgment module to obtain the probability value of the airport road section pavement shallow-deep joint perception feature map belonging to each classification label, wherein the classification label includes maintenance required and maintenance not required;

[0037] The classification label corresponding to the largest probability value among the probability values ​​is determined as the judgment result.

[0038] In a second aspect, an airport intelligent operation and maintenance system based on VR and the Internet of Things adopts any one of the airport intelligent operation and maintenance methods based on VR and the Internet of Things described in the first aspect, comprising:

[0039] BIM model building module, used to build the BIM model of the airport pavement area;

[0040] An information conversion and extraction module is used to perform modeling and analysis on the BIM model of the airport pavement area to achieve information conversion and extraction to obtain a JSON file;

[0041] A model loading and image rendering module is used to send the JSON file to the VR server, and the VR server loads the JSON file into the model and renders the image in WebGL to obtain a lightweight 3D model of the airport pavement area;

[0042] a road surface image extraction module, configured to extract a road surface image of a predetermined airport road section from the lightweight airport road surface area 3D model;

[0043] a road condition assessment module, configured to determine, based on the road surface image of the predetermined airport road section, whether the predetermined airport road section requires maintenance to obtain a determination result;

[0044] a maintenance requirement marking module, configured to mark the predetermined airport road section in the 3D model of the lightweight airport pavement area as "maintenance required" in response to the determination result that the predetermined airport road section requires maintenance;

[0045] The 3D model display module is used to display the 3D model of the lightweight airport pavement area with markings.

[0046] Compared with the prior art, the beneficial technical effects of the present invention are as follows: BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above and other objects, features, and advantages of the present invention will become more apparent through a more detailed description of the present invention in conjunction with the accompanying drawings. The accompanying drawings are provided to provide a further understanding of the present invention and constitute a part of the specification. Together with the present invention, they are used to explain the present invention and are not intended to limit the present invention. In the drawings, the same reference numerals generally represent the same components or steps.

[0048] Figure 1 This is a flow chart of an airport intelligent operation and maintenance method based on VR and the Internet of Things applied for by the present invention.

[0049] Figure 2This is a data flow diagram of the airport intelligent operation and maintenance method based on VR and the Internet of Things applied for by the present invention.

[0050] Figure 3 This is a flow chart of step S5 of the present invention.

[0051] Figure 4 This is a flow chart of step S52 of the present invention.

[0052] Figure 5 This is a flow chart of step S523 of the present invention.

[0053] Figure 6 This is a block diagram of an airport intelligent operation and maintenance system based on VR and the Internet of Things applied for by the present invention. DETAILED DESCRIPTION

[0054] As used in the present application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0055] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are illustrative only, and different aspects of the system and method can use different modules.

[0056] Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the various steps may be processed in reverse order or simultaneously, as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0057] Below, the exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0058] It is worth noting that in the present application, all actions of acquiring data are carried out in compliance with the relevant data protection laws and policies of the country where the data is located and with the authorization given by the owner of the corresponding device.

[0059] like Figure 1 and Figure 2 As shown, in the first aspect, a method for intelligent airport operation and maintenance based on VR and the Internet of Things includes:

[0060] Step S1, establishing a BIM model of the airport pavement area;

[0061] Step S2, modeling and analyzing the BIM model of the airport pavement area to achieve information conversion and extraction to obtain a JSON file;

[0062] Step S3: sending the JSON file to a VR server, and the VR server performs model loading and image rendering on the JSON file in WebGL to obtain a lightweight 3D model of the airport pavement area;

[0063] Step S4, extracting a road surface image of a predetermined airport road section from the lightweight airport road surface area 3D model;

[0064] Step S5, based on the road surface image of the predetermined airport road section, determining whether the predetermined airport road section requires maintenance to obtain a determination result;

[0065] Step S6, in response to the determination that the predetermined airport road section requires maintenance, marking the predetermined airport road section in the 3D model of the lightweight airport pavement area, wherein the mark is "maintenance required";

[0066] Step S7: displaying the 3D model of the lightweight airport pavement area with the markings.

[0067] In the aforementioned VR and IoT-based intelligent airport operation and maintenance method, step S1 involves establishing a BIM model of the airport pavement area. It should be understood that Building Information Modeling (BIM) technology is a comprehensive building management technology. It is not just a three-dimensional graphical display but also includes a wealth of non-geometric information, such as material information (e.g., material, color, supplier, etc.), cost information (e.g., cost, budget, etc.), and progress information (e.g., construction plan, duration, etc.). Currently, BIM models have been widely used in the construction industry, as well as in various fields such as urban planning, civil engineering, and industrial design, and play a vital role in the construction and maintenance of airport infrastructure. By establishing a BIM model of the airport pavement area, digital management of the airport pavement area can be achieved, providing a comprehensive and precise management platform for airport operations and maintenance.

[0068] Specifically, when establishing the BIM model of the airport pavement area, IoT technology can be used to collect basic data about the airport pavement area. The basic data of the airport pavement area includes but is not limited to: topographic survey data, which uses high-precision laser scanners or drones to conduct large-area topographic surveys to obtain accurate ground elevation data; existing facility information, including the size, layout, material of runways, taxiways, aprons, etc., and the relative position relationship between facilities; underground facility information, such as the location and direction of underground pipelines and cable trenches, which is crucial for future excavation work; historical maintenance records, understanding past maintenance conditions can help predict problems that may be encountered in the future.

[0069] To ensure smooth sharing and exchange of BIM models and related information between different parties, unified standards and specifications must be adhered to. For example, Level of Development (LOD) defines the depth of information required at different levels, from conceptual design to completion; Industry Foundation Classes (IFC), an internationally recognized data format that supports cross-platform data interoperability; and the ISO 19650 series of international standards for information management, particularly for construction projects using BIM technology, provide a framework for information integration.

[0070] Selecting appropriate BIM software depends on the project's scale, complexity, and the skill level of the individuals or team. Numerous professional BIM software options are available, such as Autodesk Revit, Bentley MicroStation, and Graphisoft ARCHICAD. These BIM software offer powerful modeling capabilities and can handle complex structural and system integration issues. Furthermore, given the unique characteristics of airport pavement, it's also important to evaluate whether the BIM software supports features such as large-scale scene rendering optimization and large-scale point cloud data processing.

[0071] Based on the collected basic data, the basic framework of the airport pavement area is first constructed, including the outlines of major structures such as runways, taxiways, and aprons. Further details are then added, such as the specific locations and parameters of auxiliary facilities such as marking lines, signage, lighting, and drainage systems. Furthermore, non-geometric information such as material properties and cost estimates is required. Tools such as collision detection are then used to check for conflicts and irregularities within the model, and adjustments are made accordingly.

[0072] Given the high security and sensitivity of airports, strict measures must be taken regarding data management and information security within BIM models. On one hand, a comprehensive data backup mechanism must be established to prevent the loss of important information due to unexpected events; on the other hand, access control must be implemented to ensure that only authorized personnel can view or edit specific data.

[0073] In the aforementioned VR and IoT-based intelligent airport operation and maintenance method, step S2 involves modeling and analyzing the BIM model of the airport pavement area to convert and extract information to produce a JSON file. It should be understood that relying solely on BIM models for facility management still makes it difficult to quickly identify and accurately locate minor damage to airport pavements, which limits the in-depth application of BIM technology in airport operations and maintenance. Because BIM models contain a large amount of different types of data, in order to efficiently process and utilize this data, it is necessary to further convert the information in the BIM model into a format that is easier to process and analyze. Based on this, the present invention considers JSON (JavaScript Object Notation) files as a lightweight data exchange format that can effectively store and transmit structured data and is highly readable and easy to parse. Therefore, the present invention further models and analyzes the BIM model of the airport pavement area, extracts key information, and converts it into a JSON file format to facilitate data sharing and exchange between different systems, simplifying subsequent data processing.

[0074] Specifically, the BIM model is first comprehensively analyzed, including evaluating the geometry, size and relative position of key areas such as runways, taxiways, and aprons to ensure that all physical features are correctly identified; detailed inspection of the material types used in each part and their properties, such as wear resistance and compressive strength, and recording of specific construction methods and processes; understanding the functional role of each component, such as lighting systems, drainage systems, marking lines, etc., as well as the interaction between components; for dynamically changing parts, such as maintenance history or planned renovation projects, record important event points on the timeline; based on design requirements and actual construction conditions, calculate the human and material resources required for each stage and the corresponding cost budget.

[0075] After completing the above analysis, the information is divided into different categories for better understanding and use. Entity objects represent physical objects, such as runways, taxiways, buildings, etc. Each entity has its own unique identifier and set of attributes. Relationship links describe the associations between entities, such as a certain taxiway is connected to a specific runway endpoint, or which areas a maintenance facility serves. Operational instructions specify how to operate entities or perform specific tasks, such as regularly checking the status of a piece of equipment or taking emergency measures in the event of a failure. Spatiotemporal coordinates provide information about time and spatial location, which helps to track the historical changes of objects or predict future trends.

[0076] With a clear data structure as a foundation, information transformation can begin. Specifically, parametric modeling engines, leveraging software development interfaces such as the Autodesk Revit API and Dynamo, can directly access the data layer within the BIM model, automatically extracting the required information and performing preliminary organization. Semantic Web technologies, using standard protocols such as RDF (Resource Description Framework) and OWL (Web Ontology Language), can assign clear semantic meaning to various types of information, enabling computers to understand its content and context. Rule engines and reasoning mechanisms, based on pre-set business logic rules, can automatically derive new conclusions or recommendations, such as optimizing route planning based on current road conditions. Big data processing frameworks, leveraging distributed computing platforms such as Hadoop and Spark, can rapidly process massive data sets, ensuring the speed and quality of information transformation. Geographic Information Systems (GIS), integrated with GIS technology, not only enhance understanding of geographic location but also incorporate external factors such as weather conditions and traffic flow, further enriching the information dimension.

[0077] The last step is to encapsulate the analyzed, classified and transformed information into a JSON file. This involves defining a suitable JSON schema to guide the specific representation of information. The basic framework of the JSON document is built according to the main categories such as entity objects, relationship links, and operation instructions to form a multi-level data tree structure. For example, a key-value pair is used to describe a specific information item, where the key is a unique string identifier and the value is the corresponding actual content, which may be a simple text, number, or a more complex nested object. When there are multiple elements of the same type, they are organized in an array form; for objects that appear repeatedly but do not want to be copied multiple times, they can be simplified by using ID references. Although JSON itself does not support formal comment syntax, in some cases, additional explanatory text can be added to the value field to help users understand the information intent more quickly.

[0078] In the aforementioned VR and IoT-based intelligent airport operation and maintenance method, step S3 sends the JSON file to a VR server, which then loads and renders the JSON file in WebGL to generate a lightweight 3D model of the airport pavement area. Specifically, to optimize the original BIM model, removing unnecessary details while retaining the core elements for display and analysis, thereby creating a lightweight 3D model suitable for real-time interaction, the present invention further sends the JSON file to the VR server, which then uses WebGL technology to load and render it into a visual, lightweight 3D model of the airport pavement area. It should be understood that WebGL is an open-source framework that allows developers to draw complex graphical objects directly in the browser without installing additional plug-ins. Specifically, the WebGL engine quickly reconstructs a lightweight 3D scene based on the description in the JSON file, running smoothly even on mobile devices. Furthermore, the 3D model rendered using WebGL technology not only provides intuitive visual effects but also supports interactive operations. Operations and maintenance personnel can control perspective changes, zooming, and panning using gestures or a mouse, further enhancing their perception and understanding of the airport pavement conditions.

[0079] First, during the data transmission phase, it's necessary to ensure that JSON files are securely and quickly delivered from the information extraction system to the VR server. This typically involves using HTTP or HTTPS protocols, combined with a RESTful API interface, to exchange data between the client and server. To ensure efficient transmission, you may also need to consider using compression algorithms such as Gzip or Brotli to reduce file size and speed up downloads. Furthermore, given the potential for transmission interruptions caused by network fluctuations, a reasonable retry mechanism and error handling logic should be designed to enhance system robustness.

[0080] Once the JSON file successfully arrives at the VR server, the next task is to parse the file contents. Because the JSON format itself is easy to parse, most modern programming languages ​​have built-in parser libraries that can directly read and interpret the key-value pair structure in the file. For large or complex JSON files, it is recommended to use a streaming parsing method, reading in data block by block rather than loading the entire file into memory at once, to reduce resource usage. During the parsing process, special attention should be paid to mapping abstract data items back to specific geometric entities, such as converting numerical values ​​representing the length and width of a runway into line segments or planes in three-dimensional space; at the same time, visual characteristics such as material properties and texture mapping should be correctly handled to prepare for subsequent rendering.

[0081] After completing the data parsing, the crucial model loading link is entered. At this stage, WebGL technology is mainly relied on to achieve efficient rendering of three-dimensional graphics in a browser environment. WebGL is a low-level JavaScript API that allows direct access to GPU hardware acceleration functions, so that high-quality 3D images can be presented on the web page. In order to fully utilize the capabilities of WebGL, an advanced framework such as Three.js is generally selected as a development tool, which simplifies many underlying operations, provides convenient methods for creating basic elements such as scenes, cameras, light sources, and supports the import of 3D model files in various formats. For the application of the present invention, Three.js can be used to easily load BIM model components described by JSON to build a complete airport pavement area view.

[0082] Once all necessary 3D objects have been correctly loaded into the WebGL scene, image rendering can begin. The rendering process involves more than simply displaying objects; it also applies a range of advanced graphics techniques, including lighting effects, shadow calculations, and reflection and refraction simulations, to make the final image more realistic and vivid. To optimize rendering performance, several key measures were taken: first, appropriate LOD (Level of Detail) settings were implemented, automatically adjusting the model's level of detail based on the viewer's distance to avoid unnecessary high-precision rendering at long distances; second, batching was enabled to combine similar draw calls, reducing communication overhead between the CPU and GPU; third, instanced rendering was utilized. For recurring objects (such as signs and lamps), multiple instances can be generated in batches with a single draw command, significantly improving efficiency; and fourth, deferred shading was implemented, centralizing the calculation of lighting effects in the post-processing stage to reduce the rendering burden per frame.

[0083] Finally, after the above series of processing steps, a lightweight yet detailed 3D model of the airport pavement area is obtained. It not only retains the core information of the original BIM data, but also has real-time interactive features. Users can use VR equipment to fully explore the airport environment and gain an intuitive experience.

[0084] In the aforementioned VR and IoT-based intelligent airport operation and maintenance method, step S4 extracts a pavement image of a predetermined airport road section from the lightweight 3D model of the airport pavement area. It will be appreciated that, given that airports typically cover a wide area and often include multiple functional areas, in order to accurately monitor and analyze airport pavement conditions, the present invention further extracts pavement images of specific airport road sections from the lightweight 3D model. This allows for focusing attention on critical areas requiring inspection or maintenance, thereby improving operation and maintenance efficiency and avoiding unnecessary global scans.

[0085] To accurately identify and select the intended airport road section, a virtual camera view must first be created to capture images of that road section. During this process, the camera's angle, height, and field of view must be carefully set to ensure the best possible view of the road surface. For example, to inspect for cracks or wear on the road surface, a lower-angle oblique view might be selected; to observe the overall layout, a higher-angle bird's-eye view might be used. Furthermore, the camera's height and field of view must be optimized based on the specific observation purpose. They should not be too narrow to miss important details, nor too wide to distort the image.

[0086] Next comes the rendering stage, which fully utilizes WebGL and its various shader technologies to optimize and adjust properties such as color, brightness, and contrast for selected road sections. Advanced effects such as environment mapping and normal mapping are applied to enhance the realism of the image. Furthermore, to address image quality issues for large areas or complex structures, techniques such as anti-aliasing and anisotropic filtering are used to reduce edge artifacts and improve texture clarity. Poor lighting conditions can be improved by adding additional light sources or adjusting lighting parameters.

[0087] Once all preparations are complete, the final step is to extract the desired 2D image from the rendered scene. This is typically achieved through a framebuffer object (FBO). This involves creating an off-screen render target, rendering the camera's captured content into this off-screen buffer, and then copying its contents into a texture or exporting it directly to an image file format such as PNG or JPEG. This completes the conversion from a 3D model to a 2D image. To ensure the extracted image is suitable for its intended use, it undergoes necessary post-processing, including resizing, color correction, annotation, compression, and optimization. This ensures that the image is more realistic, facilitating storage and transmission while also helping users better understand the image content. This effectively transforms complex 3D data into intuitive and easily understandable 2D images, providing powerful support for subsequent image analysis. Compared to traditional cameras capturing 2D images directly, this method enables real-time monitoring and comprehensive coverage of airport pavement conditions. It offers greater adaptability, coverage, and information richness for large, complex, and ever-changing airport pavement environments.

[0088] In the above-mentioned airport intelligent operation and maintenance method based on VR and the Internet of Things, the step S5 is to determine whether the predetermined airport road section needs maintenance based on the road surface image of the predetermined airport road section to obtain a judgment result. To this end, the present invention further introduces a deep learning algorithm to perform intelligent analysis on the extracted road surface image. Figure 3 As shown, the step S5 includes: step S51, performing multi-level image context feature extraction on the road surface image of the predetermined airport section to obtain a shallow context coding feature map of the airport section road surface and a deep context coding feature map of the airport section road surface; step S52, performing feature saliency joint perception based on prior guidance on the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface to obtain a shallow-deep joint perception feature map of the airport section road surface; step S53, generating the judgment result based on the shallow-deep joint perception feature map of the airport section road surface.

[0089] Specifically, in a specific example of the present invention, step S51 includes: inputting the pavement image of the predetermined airport section into an airport pavement feature extractor based on the FPT model to obtain a shallow context encoding feature map of the airport section pavement and a deep context encoding feature map of the airport section pavement. It should be understood that the present invention takes into account that the pavement image of the predetermined airport section may contain a variety of different types and degrees of damage, such as local cracks and potholes, as well as overall wear and structural deformation. Therefore, in order to achieve a comprehensive analysis of the airport pavement, the present invention adopts the FPT (Feature Pyramid Transformer) model as the airport pavement feature extractor to perform multi-level feature extraction on the pavement image of the predetermined airport section. By applying the Transformer mechanism to the Feature Pyramid Network (FPN) structure, the FPT model can achieve fully active feature interaction across space and scales. Specifically, the FPT model uses three Transformer encoding layers, corresponding to non-local interactions within the same-level feature maps, top-down non-local interactions, and bottom-up non-local interactions, respectively. It can effectively capture the shallow detail features and deep semantic information in the road surface image to generate shallow context encoding feature maps and deep context encoding feature maps of the airport section road surface, thereby providing a rich information basis for subsequent road maintenance analysis.

[0090] Specifically, in a specific example of the present invention, step S52 performs a priori-guided joint perception of feature saliency on the shallow context-encoded feature map of the airport section road surface and the deep context-encoded feature map of the airport section road surface to obtain a shallow-deep joint perception feature map of the airport section road surface. It should be understood that the present invention takes into account that the shallow features of the road surface image mainly include low-level visual features such as edges and textures, while the deep features capture more complex structural patterns and semantic information. In order to achieve a more comprehensive and detailed feature description of the road surface condition, the present invention further performs a joint perception of the shallow context-encoded feature map of the airport section road surface and the deep context-encoded feature map of the airport section road surface. In particular, considering that simple fusion methods such as feature splicing may not be able to fully express the correlation between shallow and deep features, the present invention application proposes a feature saliency joint perception method based on prior guidance, which enhances the understanding and expression ability of road surface features by introducing prior information in historical data, and on this basis, performs two-way attention interaction on the shallow and deep features of the road surface, thereby effectively capturing the key information of road damage and improving the accuracy of road maintenance analysis.

[0091] like Figure 4 As shown, the step S52 includes:

[0092] Step S521, reshaping the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface to obtain a shallow context coding feature matrix of the airport section road surface and a deep context coding feature matrix of the airport section road surface;

[0093] Step S522: inputting the shallow context coding feature matrix of the airport section road surface and the deep context coding feature matrix of the airport section road surface into a priori guided modulation module to obtain a priori modulated shallow context coding feature matrix of the airport section road surface and a priori modulated deep context coding feature matrix of the airport section road surface;

[0094] Step S523, performing attention interaction fusion on the prior modulated airport section road surface shallow context coding feature matrix and the prior modulated airport section road surface deep context coding feature matrix to obtain the airport section road surface shallow-deep joint perception feature map.

[0095] More specifically, step S521 includes the following formula:

[0096]

[0097]

[0098] in, represents the shallow context coding feature map of the airport road section, represents the deep context encoding feature map of the airport road section, represents the feature shape reshaping, represents the shallow context encoding feature matrix of the airport road section, Represents the deep context encoding feature matrix of the airport road section.

[0099] That is, the feature shapes of the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface are reshaped into matrix form to adapt them to the format requirements of subsequent data interaction processing. In a specific example, the local feature matrices along the channel dimension of the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface are expanded into one-dimensional vectors to obtain the shallow context coding feature matrix of the airport section road surface and the deep context coding feature matrix of the airport section road surface.

[0100] More specifically, step S522 includes the following formula:

[0101]

[0102]

[0103] in, represents matrix multiplication, represents the transpose of the matrix, 、 Both represent the learnable memory parameter matrix of the prior guided modulation module, represents the normalization function, represents the prior modulated airport road section shallow context encoding feature matrix, Represents the deep context encoding feature matrix of the prior modulated airport road section pavement.

[0104] Specifically, the prior-guided modulation module modulates the shallow and deep pavement features after the characteristic shape has been reshaped. This module leverages prior knowledge from historical data to train its internal learnable memory parameter matrix by learning pavement image features of different damage types and degrees. This module then determines the importance weights of different features, dynamically modulating the current shallow and deep pavement features to enhance the recognition and representation of key pavement damage characteristics.

[0105] like Figure 5 As shown, the step S523 includes:

[0106] Step S5231: Input the a priori modulated airport section road surface shallow context coding feature matrix and the a priori modulated airport section road surface deep context coding feature matrix into an attention interaction coding module based on a transformer-like structure to obtain an airport section road surface shallow feature enhancement interaction coding matrix and an airport section road surface deep feature enhancement interaction coding matrix;

[0107] Step S5232, reshaping the shallow-layer feature enhancement interaction coding matrix and the deep-layer feature enhancement interaction coding matrix of the airport section road surface to obtain a shallow-layer feature enhancement interaction coding feature map and a deep-layer feature enhancement interaction coding feature map of the airport section road surface;

[0108] Step S5233, calculating the weighted sum of the shallow feature enhancement interactive coding feature map of the airport section road surface and the deep feature enhancement interactive coding feature map of the airport section road surface to obtain the shallow-deep joint perception feature map of the airport section road surface.

[0109] In a specific example of the present invention, step S5231 includes: first, using the prior modulated airport section road surface shallow context coding feature matrix as the query feature matrix and the value feature matrix, and using the prior modulated airport section road surface deep context coding feature matrix as the key feature matrix, performing a converter-based attention interaction on the query feature matrix, the value feature matrix, and the key feature matrix to obtain the airport section road surface shallow feature enhancement interaction coding matrix, including the following formula:

[0110]

[0111] in, is the feature scale scaling factor of the attention interaction encoding module, represents the soft maximization function, Represents the enhanced interactive coding matrix of shallow features of the pavement on the airport section.

[0112] Then, the prior modulated airport section road surface deep context encoding feature matrix is ​​used as the query feature matrix and the value feature matrix, and the prior modulated airport section road surface shallow context encoding feature matrix is ​​used as the key feature matrix. The query feature matrix, the value feature matrix and the key feature matrix are subjected to attention interaction based on the converter structure to obtain the airport section road surface deep feature enhanced interaction encoding matrix, including the following formula:

[0113]

[0114] in, Represents the enhanced interactive coding matrix of deep features of the airport road section.

[0115] That is, the shallow context encoding feature matrix of the airport section pavement and the deep context encoding feature matrix of the airport section pavement after prior guidance modulation are further subjected to bidirectional attention interaction of the converter architecture, so as to utilize the detailed information in the shallow features to enhance the semantic understanding of the deep features, and at the same time utilize the semantic information of the deep features to guide the attention focus of the shallow features. Through this complementary learning process, the expressive power of the shallow features and the deep features of the pavement are enhanced respectively, and the shallow feature enhanced interaction encoding matrix and the deep feature enhanced interaction encoding matrix of the airport section pavement are generated.

[0116] In a specific example of the present invention, step S5232 includes the following formula:

[0117]

[0118]

[0119] in, Represents the enhanced interactive coding feature map of the shallow features of the airport road section. Represents the interactive coding feature map of the airport road section with enhanced deep features.

[0120] That is, the shallow feature enhancement interaction coding matrix of the airport section pavement after attention interaction and the deep feature enhancement interaction coding matrix of the airport section pavement are further reshaped to restore the original feature shape.

[0121] In a specific example of the present invention, step S5233 includes the following formula:

[0122]

[0123] in, and For different weight parameters, Represents the joint shallow-deep perception feature map of the airport road section.

[0124] Specifically, through a pixel-by-pixel weighted aggregation operation, shallow and deep features are fused to form a joint shallow-deep perception feature map of the airport road section. This approach effectively leverages the complementary strengths of shallow and deep features, preserving detailed information in road imagery while enhancing semantic understanding of road damage patterns, providing strong support for subsequent road maintenance decisions.

[0125] Specifically, in a specific example of the present application, step S53 includes: inputting the shallow-deep joint perception feature map of the airport road section pavement into the airport operation and maintenance judgment module based on the classifier to obtain the judgment result. Specifically, the classifier is based on a neural network architecture, and through feature learning of the shallow-deep joint perception feature map of the airport road section pavement, it analyzes the pavement damage status, determines the pavement maintenance needs, and outputs the final judgment result, i.e., whether pavement maintenance is required, through the Softmax function of the classification layer. In this way, airport operation and maintenance personnel can quickly identify the pavement areas that need maintenance based on the obtained judgment results, so as to carry out repair work in a timely manner to ensure the safety and efficiency of airport operations.

[0126] More specifically, the shallow-deep joint perception feature map of the airport section pavement is input into the classifier-based airport operation and maintenance judgment module to obtain the judgment result, including: expanding the shallow-deep joint perception feature map of the airport section pavement into a shallow-deep joint perception feature vector of the airport section pavement; using the fully connected layer of the airport operation and maintenance judgment module to fully connect the shallow-deep joint perception feature vector of the airport section pavement to obtain a shallow-deep joint perception fully connected encoding vector of the airport section pavement; inputting the shallow-deep joint perception fully connected encoding vector of the airport section pavement into the Softmax classification function of the airport operation and maintenance judgment module to obtain the probability value of the shallow-deep joint perception feature map of the airport section pavement belonging to each classification label, wherein the classification labels include maintenance required and maintenance not required; and determining the classification label corresponding to the largest of the probability values ​​as the judgment result.

[0127] Here, considering that the shallow context coding feature map of the airport section pavement and the deep context coding map of the airport section pavement respectively represent the shallow image semantic context coding features and deep image semantic context coding features of the pavement image of the predetermined airport section, when performing feature significant joint perception based on prior guidance, due to the difference in the guidance ability of prior guidance for the shallow and deep layers of the image, the shallow-deep joint perception feature map of the airport section pavement will have insufficient long-distance joint perception representation based on enhanced interactivity of semantic features of images of different depths, thereby reducing the expression effect of the shallow-deep joint perception feature map of the airport section pavement, thereby affecting the accuracy of the judgment result obtained by the input classifier-based airport operation and maintenance judgment module.

[0128] Preferably, when the shallow-deep joint perception feature map of the airport road section pavement is input into the classifier-based airport operation and maintenance judgment module to obtain a judgment result, the shallow-deep joint perception feature map of the airport road section pavement is optimized, including:

[0129] Calculate the absolute value sum of all eigenvalues ​​of the shallow-deep joint perception feature map of the airport road section , and the square root of the sum of the squares of all eigenvalues ​​of the shallow-deep joint perception feature map of the airport road section , and Divide by To obtain the joint perception probability value of shallow and deep layers of the airport road section ,Right now:

[0130]

[0131]

[0132]

[0133] in, The first one represents the shallow-deep joint perception feature map of the airport road section eigenvalues;

[0134] The shallow-deep joint perception feature map of the airport road section Each eigenvalue in is normalized to its maximum value to obtain the shallow-deep joint perception probability feature map of the airport road section :

[0135]

[0136] in, represents the maximum value in the shallow-deep joint perception feature map of the airport road section, The first one represents the joint perception probability feature map of the shallow and deep layers of the road surface of the airport section. eigenvalues;

[0137] The shallow-deep joint perception probability feature map of the airport section road surface is subtracted from the shallow-deep joint perception probability value of the airport section road surface, and then multiplied by the reciprocal of the shallow-deep joint perception probability value of the airport section road surface to obtain the shallow-deep joint perception domain gradient feature map of the airport section road surface. ,in, Indicates point reduction, represents dot product;

[0138] Subtract the shallow-deep joint perception probability feature map of the airport road section from the unit feature map point Then, the shallow-deep joint perception domain gradient feature map of the airport section road surface is divided by point to obtain the shallow-deep joint perception differential feature map of the airport section road surface, that is:

[0139]

[0140] in, represents the unit feature map, represents the bit-by-bit inverse of the shallow-deep joint perception domain gradient feature map of the airport section road surface, that is, the inverse of each eigenvalue of the shallow-deep joint perception domain gradient feature map of the airport section road surface is calculated. Represents the differential feature map of the shallow-deep joint perception of the airport road section;

[0141] Calculate the gradient feature map of the shallow-deep joint perception domain of the airport section road surface, using the power function with the shallow-deep joint perception probability value of the airport section road surface as the exponent , and the exponential function with natural constant as the base of the shallow-deep joint perception differential feature map of the airport road section The weighted sum is performed to obtain the optimized shallow-deep joint perception feature map of the airport road section, namely:

[0142]

[0143] in, and Both represent weighted hyperparameters, Indicates point addition, Represents the optimized shallow-deep joint perception feature map of the airport road section.

[0144] Then, the optimized airport section road surface shallow-deep joint perception feature map (which can be used as a new airport section road surface shallow-deep joint perception feature map) is input into the classifier-based airport operation and maintenance judgment module to obtain a judgment result.

[0145] Based on this, the mutually exclusive generalized representation behavior units of the shallow-deep joint perception feature map of the airport section road surface are constructed through the discrete differential of the probability density domain gradient corresponding to the shallow-deep joint perception feature map of the airport section road surface. The different behavior unit organization spaces under the non-uniform topological architecture of the shallow-deep joint perception feature map of the airport section road surface are used to enhance the static response characteristics of the short-range correlation micro-information configuration of the shallow-deep joint perception feature map of the airport section road surface to the generative probabilistic generalized representation behavior, thereby ensuring the steady-state convergence of the optimization process between the generated target and the extracted features in the feature space-generation probability mapping, and improving the accuracy of the judgment results obtained by the airport operation and maintenance judgment module based on the classifier when the shallow-deep joint perception feature map of the airport section road surface is input.

[0146] In the above-mentioned airport intelligent operation and maintenance method based on VR and the Internet of Things, in step S6, in response to the judgment result that the predetermined airport road section requires maintenance, the predetermined airport road section is marked in the 3D model of the lightweight airport pavement area, and the mark is "maintenance required".

[0147] Specifically, when the system determines that a designated airport road section requires maintenance, it visually marks the section as "Maintenance Required" within the 3D model of the lightweight airport pavement area. Furthermore, the system generates a report and notifies relevant personnel via email or text message, ensuring that maintenance activities are efficient and do not impact flight operations, thereby improving the safety and reliability of airport operations.

[0148] In the aforementioned VR and IoT-based intelligent airport operation and maintenance method, step S7 displays the 3D model of the lightweight airport pavement area with markers. It should be understood that the markers are highlighted using visual elements such as bright colors, special icons, or flashing effects to ensure clarity from different viewing angles and without obscuring important structural features. Users can interactively click on the markers to view detailed maintenance information and recommended measures.

[0149] In summary, the VR and IoT-based intelligent airport operation and maintenance method proposed in this invention is illustrated. First, a BIM model of the airport pavement area is established. Through modeling analysis and image rendering, a lightweight 3D model of the airport pavement area is generated, from which the pavement image of the predetermined airport section is extracted. Next, deep learning-based image processing technology is introduced to perform multi-level feature extraction and joint perception on the pavement image of the predetermined airport section, intelligently identifying whether the predetermined airport section requires maintenance. Based on the analysis results, corresponding markings and prompts are made in the 3D model of the airport pavement area. In this way, pavement status information can be updated in real time, helping operation and maintenance personnel better understand the actual conditions of the airport pavement, thereby effectively improving the efficiency and accuracy of airport operation and maintenance.

[0150] like Figure 6 As shown, in a second aspect, an airport intelligent operation and maintenance system 100 based on VR and the Internet of Things adopts any one of the airport intelligent operation and maintenance methods based on VR and the Internet of Things described in the first aspect, including:

[0151] A BIM model building module 110 is used to build a BIM model of the airport pavement area;

[0152] An information conversion and extraction module 120 is used to perform modeling analysis on the BIM model of the airport pavement area to achieve information conversion and extraction to obtain a JSON file;

[0153] A model loading and image rendering module 130 is used to send the JSON file to a VR server, and the VR server loads the JSON file into a model and renders the image in WebGL to obtain a lightweight 3D model of the airport pavement area;

[0154] a road surface image extraction module 140 for extracting a road surface image of a predetermined airport road section from the lightweight airport road surface area 3D model;

[0155] a road condition assessment module 150 for determining whether the predetermined airport road section requires maintenance based on the road surface image of the predetermined airport road section to obtain a determination result;

[0156] a maintenance requirement marking module 160 for marking the predetermined airport road section in the lightweight airport pavement area 3D model as "maintenance required" in response to the determination result that the predetermined airport road section requires maintenance;

[0157] The 3D model display module 170 is configured to display the 3D model of the lightweight airport pavement area with markings.

[0158] Here, those skilled in the art will understand that the specific operations of each module in the above-mentioned airport intelligent operation and maintenance system based on VR and the Internet of Things have been referred to above. Figures 1 to 5 The above description has been introduced in detail in the description of an airport intelligent operation and maintenance method based on VR and the Internet of Things, so its repeated description is omitted.

[0159] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in the present invention are merely illustrative and non-limiting, and should not be construed as necessarily possessed by each embodiment of the present invention. Furthermore, the specific details of the above embodiments are provided for illustrative purposes and to facilitate understanding, and are not intended to be limiting. These details do not necessarily limit the present invention to being implemented using these specific details.

[0160] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0161] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be encompassed therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0162] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0163] Finally, it should be noted that the above description has been provided for purposes of illustration and description. Furthermore, the above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to be limiting. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art will appreciate that the technical solutions of the present invention may be modified or replaced with equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An airport intelligent operation and maintenance method based on VR and the Internet of Things, characterized by: include: Establish a BIM model of the airport pavement area; Modeling and analyzing the BIM model of the airport pavement area to achieve information conversion and extraction to obtain a JSON file; Sending the JSON file to a VR server, which loads the JSON file into a WebGL model and performs image rendering to obtain a lightweight 3D model of the airport pavement area; Extracting a road surface image of a predetermined airport road section from the lightweight airport road surface area 3D model; Determining whether the predetermined airport road section requires maintenance based on the road surface image of the predetermined airport road section to obtain a determination result includes: Performing multi-level image context feature extraction on the road surface image of the predetermined airport section to obtain a shallow context coding feature map of the airport section road surface and a deep context coding feature map of the airport section road surface; The shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface are subjected to a priori-guided feature saliency joint perception to obtain a shallow-deep joint perception feature map of the airport section road surface, which includes: Reshaping the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface to obtain a shallow context coding feature matrix of the airport section road surface and a deep context coding feature matrix of the airport section road surface; Inputting the airport section road surface shallow context coding feature matrix and the airport section road surface deep context coding feature matrix into the prior guided modulation module to obtain the prior modulated airport section road surface shallow context coding feature matrix and the prior modulated airport section road surface deep context coding feature matrix; The prior modulated airport section road surface shallow context coding feature matrix M1′ and the prior modulated airport section road surface deep context coding feature matrix M2′ are input into the attention interaction coding module based on the transformer-like structure to obtain the airport section road surface shallow feature enhancement interaction coding matrix M 1i ' and the airport road section deep feature enhanced interactive coding matrix M 2i '; S is the feature scale scaling factor of the attention interaction encoding module, and softmax represents the soft maximization function. The detailed information in the shallow features is used to enhance the semantic understanding of the deep features, and the semantic information of the deep features is used to guide the attention focus of the shallow features. Reshaping the shallow-layer feature enhancement interaction coding matrix and the deep-layer feature enhancement interaction coding matrix of the airport section road surface to obtain a shallow-layer feature enhancement interaction coding feature map and a deep-layer feature enhancement interaction coding feature map of the airport section road surface; Calculating a weighted sum of the shallow-layer feature enhancement interactive coding feature map of the airport section road surface and the deep-layer feature enhancement interactive coding feature map of the airport section road surface to obtain a shallow-deep joint perception feature map of the airport section road surface; Optimizing the shallow-deep joint perception feature map of the airport road section includes: Calculate the sum w1 of the absolute values ​​of all eigenvalues ​​of the shallow-deep joint perception feature map of the airport section road surface and the square root w2 of the sum of the squares of all eigenvalues ​​of the shallow-deep joint perception feature map of the airport section road surface, and divide w2 by w1 to obtain the shallow-deep joint perception probability value p of the airport section road surface; Each eigenvalue in the shallow-deep joint perception feature map of the airport section road surface is normalized to its maximum value to obtain the shallow-deep joint perception probability feature map F′ of the airport section road surface: Subtracting the airport section road surface shallow-deep joint perception probability feature map from the airport section road surface shallow-deep joint perception probability value, and further multiplying it by the reciprocal of the airport section road surface shallow-deep joint perception probability value to obtain the airport section road surface shallow-deep joint perception domain gradient feature map F3; Subtracting the airport section road surface shallow-deep joint perception probability feature map F′ from the unit feature map, and then dividing the result by the airport section road surface shallow-deep joint perception domain gradient feature map to obtain the airport section road surface shallow-deep joint perception differential feature map F4; Calculate the power function F3 of the shallow-deep joint perception domain gradient feature map of the airport road section with the shallow-deep joint perception probability value of the airport road section as the exponent ⊙p , and perform weighted summation with the exponential function exp(F4) of the shallow-deep joint perception differential feature map of the airport road section with a natural constant as the base to obtain the optimized airport road section shallow-deep joint perception feature map F o ,Right now: Among them, r and s are weighted hyperparameters, Indicates point addition; The optimized shallow-deep joint perception feature map of the airport road section is input into the classifier-based airport operation and maintenance judgment module to obtain the judgment result; In response to the determination that the predetermined airport road section requires maintenance, marking the predetermined airport road section in the 3D model of the lightweight airport pavement area, the marking being "maintenance required"; The 3D model of the lightweight airfield pavement area is displayed with markings.

2. The airport intelligent operation and maintenance method based on VR and the Internet of Things according to claim 1 is characterized in that: Performing multi-level image context feature extraction on the road surface image of the predetermined airport section to obtain a shallow context coding feature map of the airport section road surface and a deep context coding feature map of the airport section road surface, including: The road surface image of the predetermined airport road section is input into an airport road surface feature extractor based on the FPT model to obtain a shallow context coding feature map of the airport road section road surface and a deep context coding feature map of the airport road section road surface.

3. The airport intelligent operation and maintenance method based on VR and the Internet of Things according to claim 2 is characterized in that: The a priori modulated airport section road surface shallow context coding feature matrix and the a priori modulated airport section road surface deep context coding feature matrix are input into an attention interaction coding module based on a transformer-like structure to obtain an airport section road surface shallow feature enhancement interaction coding matrix and an airport section road surface deep feature enhancement interaction coding matrix, including: Using the a priori modulated airport section road surface shallow context encoding feature matrix as the query feature matrix and the value feature matrix, and using the a priori modulated airport section road surface deep context encoding feature matrix as the key feature matrix, the query feature matrix, the value feature matrix, and the key feature matrix are subjected to attention interaction based on the converter structure to obtain the airport section road surface shallow feature enhanced interaction encoding matrix; The prior modulated airport section road surface deep context encoding feature matrix is ​​used as the query feature matrix and the value feature matrix, and the prior modulated airport section road surface shallow context encoding feature matrix is ​​used as the key feature matrix. The query feature matrix, the value feature matrix and the key feature matrix are subjected to attention interaction based on the converter structure to obtain the airport section road surface deep feature enhanced interaction encoding matrix.

4. The method for intelligent airport operation and maintenance based on VR and the Internet of Things according to claim 3 is characterized in that: The optimized shallow-deep joint perception feature map of the airport road section is input into the classifier-based airport operation and maintenance judgment module to obtain the judgment result, including: Expanding the optimized airport section road surface shallow-deep joint perception feature map into an airport section road surface shallow-deep joint perception feature vector; Using the fully connected layer of the airport operation and maintenance judgment module to perform fully connected encoding on the shallow-deep joint perception feature vector of the airport section road surface to obtain a shallow-deep joint perception fully connected encoding vector of the airport section road surface; Inputting the airport road section pavement shallow-deep joint perception fully connected coding vector into the Softmax classification function of the airport operation and maintenance judgment module to obtain the probability value of the optimized airport road section pavement shallow-deep joint perception feature map belonging to each classification label, wherein the classification label includes maintenance required and maintenance not required; The classification label corresponding to the largest probability value among the probability values ​​is determined as the judgment result.

5. An airport intelligent operation and maintenance system based on VR and the Internet of Things, adopting the airport intelligent operation and maintenance method based on VR and the Internet of Things according to any one of claims 1 to 4, characterized in that: include: BIM model building module, used to build the BIM model of the airport pavement area; An information conversion and extraction module is used to perform modeling and analysis on the BIM model of the airport pavement area to achieve information conversion and extraction to obtain a JSON file; A model loading and image rendering module is used to send the JSON file to the VR server, and the VR server loads the JSON file into the WebGL model and performs image rendering to obtain a lightweight 3D model of the airport pavement area; a road surface image extraction module, configured to extract a road surface image of a predetermined airport road section from the lightweight airport road surface area 3D model; a road condition assessment module, configured to determine, based on the road surface image of the predetermined airport road section, whether the predetermined airport road section requires maintenance to obtain a determination result; a maintenance requirement marking module, configured to mark the predetermined airport road section in the 3D model of the lightweight airport pavement area as "maintenance required" in response to the determination result that the predetermined airport road section requires maintenance; The 3D model display module is used to display the 3D model of the lightweight airport pavement area with markings.

Images