Airport intelligent operation and maintenance method and system based on VR and Internet of Things

Abstract

The invention belongs to the technical field of intelligent operation and maintenance management, and discloses an airport intelligent operation and maintenance method and system based on VR and Internet of Things, and the method comprises the steps: building a BIM model of an airport road surface region, generating a lightweight 3D model of the airport road surface region through modeling analysis and image rendering, and carrying out the building of the BIM model of the airport road surface region; extracting a road surface image of a predetermined airport road section; and introducing an image processing technology based on deep learning to carry out multi-level feature extraction and joint perception on a pavement image of a predetermined airport road section so as to intelligently identify whether the predetermined airport road section needs to be maintained, and carrying out corresponding marking and prompting in the airport pavement area 3D model based on an analysis result. The pavement state information can be updated in real time, and operation and maintenance personnel can be helped to better understand the actual condition of the airport pavement, so that the airport operation and maintenance efficiency and accuracy are effectively improved.

Description

Technical Field

[0001] The present invention application relates to the field of intelligent operation and maintenance management technology, and more specifically, to an airport intelligent operation and maintenance method and system based on VR and the Internet of Things. Background Art

[0002] With the rapid development of the global aviation industry, airports, as important transportation hubs connecting the world, are becoming the focus of attention within and outside the industry for their operational efficiency and safety. The pavement area of ​​an airport is a key infrastructure for aircraft takeoff and landing, taxiing, and the driving of various vehicles. Its maintenance status is directly related to the overall operational efficiency and flight safety of the airport.

[0003] Traditional airport pavement maintenance methods mainly rely on manual inspections and regular inspections. Regular inspection cameras directly capture two-dimensional images. This method is not only time-consuming and labor-intensive, but also difficult to achieve real-time monitoring and comprehensive coverage of airport pavement conditions. Especially when faced with large-scale, complex and changeable airport pavement environments, its limitations and shortcomings become increasingly obvious.

[0004] In recent years, the rapid development of virtual reality (VR) technology and the Internet of Things (IoT) has provided new solutions for airport intelligent operation and maintenance. VR technology provides an immersive and intuitive visual operation and maintenance platform for operation and maintenance personnel by simulating the real airport environment, greatly improving the efficiency and accuracy of operation and maintenance decisions. At the same time, IoT technology uses various sensors and devices to achieve real-time monitoring and data collection of airport facility status, providing a rich data source for intelligent operation and maintenance. Therefore, the rich data sources provided by IoT technology can be used and combined with virtual reality (VR) technology for airport intelligent operation and maintenance. Accordingly, a method and system for airport intelligent operation and maintenance based on VR and IoT are expected. Summary of the invention

[0005] The present invention application provides an airport intelligent operation and maintenance method and system based on VR and the Internet of Things, which aims to partially or completely solve the technical problems in the prior art, and can help operation and maintenance personnel better understand the actual situation of the airport pavement based on existing sensor data, BIM model data, etc., thereby effectively improving the efficiency and accuracy of airport operation and maintenance. The present invention application adopts the following technical solutions: First, an airport intelligent operation and maintenance method based on VR and the Internet of Things, including: Build a BIM model of the airport pavement area; Modeling and analyzing the BIM model of the airport pavement area to achieve information conversion and extraction to obtain a JSON file; Sending the JSON file to a VR server, the VR server performs model loading and image rendering on the JSON file in WebGL to obtain a lightweight airport pavement area 3D model; Extracting a road surface image of a predetermined airport road section from the lightweight airport road surface area 3D model; Based on the road surface image of the predetermined airport road section, determining whether the predetermined airport road section needs maintenance to obtain a determination result; In response to the determination result that the predetermined airport road section needs maintenance, marking the predetermined airport road section in the lightweight airport pavement area 3D model, the mark being "maintenance required"; The 3D model of the lightweight airfield pavement area is displayed with markings.

[0006] Optionally, judging whether the predetermined airport road section needs maintenance based on the road surface image of the predetermined airport road section to obtain a judgment result includes: Performing multi-level image context feature extraction on the road surface image of the predetermined airport section to obtain a shallow context coding feature map of the airport section road surface and a deep context coding feature map of the airport section road surface; Performing a priori-guided feature saliency joint perception on the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface to obtain a shallow-deep joint perception feature map of the airport section road surface; The judgment result is generated based on the shallow-deep joint perception feature map of the airport road section.

[0007] Optionally, performing multi-level image context feature extraction on the road surface image of the predetermined airport section to obtain a shallow context coding feature map of the road surface of the airport section and a deep context coding feature map of the road surface of the airport section includes: The road surface image of the predetermined airport road section is input into an airport road surface feature extractor based on the FPT model to obtain a shallow context coding feature map of the airport road section road surface and a deep context coding feature map of the airport road section road surface.

[0008] Optionally, performing a priori-guided feature saliency joint perception on the airport section road surface shallow context encoding feature map and the airport section road surface deep context encoding feature map to obtain the airport section road surface shallow-deep joint perception feature map, including: Reshaping the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface to obtain a shallow context coding feature matrix of the airport section road surface and a deep context coding feature matrix of the airport section road surface; Inputting the airport section road surface shallow context coding feature matrix and the airport section road surface deep context coding feature matrix into the prior guided modulation module to obtain the prior modulated airport section road surface shallow context coding feature matrix and the prior modulated airport section road surface deep context coding feature matrix; The shallow context encoding feature matrix of the prior modulated airport road section pavement and the deep context encoding feature matrix of the prior modulated airport road section pavement are interactively fused with attention to obtain the shallow-deep joint perception feature map of the airport road section pavement.

[0009] Optionally, the a priori modulated airport section road surface shallow context encoding feature matrix and the a priori modulated airport section road surface deep context encoding feature matrix are subjected to attention interaction fusion to obtain the airport section road surface shallow-deep joint perception feature map, including: Input the a priori modulated airport section road surface shallow context coding feature matrix and the a priori modulated airport section road surface deep context coding feature matrix into the attention interaction coding module based on the transformer-like structure to obtain the airport section road surface shallow feature reinforcement interaction coding matrix and the airport section road surface deep feature reinforcement interaction coding matrix; Reshaping the shallow-layer feature enhancement interactive coding matrix of the airport section road surface and the deep-layer feature enhancement interactive coding matrix of the airport section road surface to obtain a shallow-layer feature enhancement interactive coding feature map of the airport section road surface and a deep-layer feature enhancement interactive coding feature map of the airport section road surface; The weighted sum of the shallow-layer feature enhancement interactive coding feature map of the airport road section pavement and the deep-layer feature enhancement interactive coding feature map of the airport road section pavement is calculated to obtain the shallow-deep joint perception feature map of the airport road section pavement.

[0010] Optionally, the a priori modulated airport section road surface shallow context coding feature matrix and the a priori modulated airport section road surface deep context coding feature matrix are input into an attention interaction coding module based on a transformer-like structure to obtain an airport section road surface shallow feature enhancement interaction coding matrix and an airport section road surface deep feature enhancement interaction coding matrix, including: The a priori modulated airport section road surface shallow context coding feature matrix is ​​used as the query feature matrix and the value feature matrix, and the a priori modulated airport section road surface deep context coding feature matrix is ​​used as the key feature matrix, and the query feature matrix, the value feature matrix and the key feature matrix are subjected to attention interaction based on the converter structure to obtain the airport section road surface shallow feature reinforcement interaction coding matrix; The prior modulated airport road section pavement deep context encoding feature matrix is ​​used as the query feature matrix and the value feature matrix, and the prior modulated airport road section pavement shallow context encoding feature matrix is ​​used as the key feature matrix. The query feature matrix, the value feature matrix and the key feature matrix are subjected to attention interaction based on the converter structure to obtain the airport road section pavement deep feature enhanced interaction encoding matrix.

[0011] Optionally, generating the judgment result based on the shallow-deep joint perception feature map of the road surface of the airport section includes: The shallow-deep joint perception feature map of the airport road section is input into the classifier-based airport operation and maintenance judgment module to obtain the judgment result.

[0012] Optionally, the shallow-deep joint perception feature map of the airport road section pavement is input into a classifier-based airport operation and maintenance judgment module to obtain the judgment result, including: Expanding the airport section road surface shallow layer-deep layer joint perception feature map into an airport section road surface shallow layer-deep layer joint perception feature vector; Use the fully connected layer of the airport operation and maintenance judgment module to fully connect the shallow-deep joint perception feature vector of the airport road section pavement to obtain the shallow-deep joint perception fully connected encoding vector of the airport road section pavement; Input the airport road section pavement shallow-deep joint perception fully connected encoding vector into the Softmax classification function of the airport operation and maintenance judgment module to obtain the probability value of the airport road section pavement shallow-deep joint perception feature map belonging to each classification label, wherein the classification label includes maintenance required and maintenance not required; The classification label corresponding to the largest probability value among the probability values ​​is determined as the judgment result.

[0013] In a second aspect, an airport intelligent operation and maintenance system based on VR and the Internet of Things adopts any one of the airport intelligent operation and maintenance methods based on VR and the Internet of Things described in the first aspect, including: BIM model building module, used to build the BIM model of the airport pavement area; An information conversion and extraction module is used to perform modeling and analysis on the BIM model of the airport pavement area to achieve information conversion and extraction to obtain a JSON file; A model loading and image rendering module, used for sending the JSON file to a VR server, and the VR server performs model loading and image rendering on the JSON file in WebGL to obtain a lightweight airport pavement area 3D model; A road surface image extraction module, used to extract a road surface image of a predetermined airport road section from the lightweight airport road surface area 3D model; a road condition assessment module, used to determine whether the predetermined airport road section needs maintenance based on the road surface image of the predetermined airport road section to obtain a determination result; A maintenance requirement marking module, for marking the predetermined airport road section in the lightweight airport pavement area 3D model in response to the judgment result that the predetermined airport road section needs maintenance, wherein the mark is "maintenance required"; The 3D model display module is used to display the 3D model of the lightweight airport pavement area with markings.

[0014] Compared with the prior art, the beneficial technical effects of the present invention are as follows: BRIEF DESCRIPTION OF THE DRAWINGS The above and other purposes, features and advantages of the present invention will become more apparent by describing the present invention in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0015] Figure 1 This is a flow chart of an airport intelligent operation and maintenance method based on VR and the Internet of Things applied for by the present invention.

[0016] Figure 2 This is a data flow diagram of an airport intelligent operation and maintenance method based on VR and the Internet of Things applied for by the present invention.

[0017] Figure 3 This is a flow chart of step S5 of the present invention.

[0018] Figure 4 This is a flow chart of step S52 of the present invention.

[0019] Figure 5 This is a flow chart of step S523 of the present invention.

[0020] Figure 6 This is a block diagram of an airport intelligent operation and maintenance system based on VR and the Internet of Things applied for by the present invention. DETAILED DESCRIPTION

[0021] As shown in the present application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.

[0022] Although the present invention application has made various references to certain modules in the system according to the embodiment of the present invention application, however, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0023] The present invention application uses a flow chart to illustrate the operations performed by the system of the embodiment of the present invention application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0024] Below, the exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described here.

[0025] It is worth noting that in the present invention application, all actions of acquiring data are carried out in compliance with the relevant data protection laws and policies of the country where the data is located, and with the authorization given by the owner of the corresponding device.

[0026] like Figure 1 and Figure 2 As shown, in the first aspect, an airport intelligent operation and maintenance method based on VR and the Internet of Things includes: Step S1, establishing a BIM model of the airport pavement area; Step S2, modeling and analyzing the BIM model of the airport pavement area to achieve information conversion and extraction to obtain a JSON file; Step S3, sending the JSON file to a VR server, and the VR server performs model loading and image rendering on the JSON file in WebGL to obtain a lightweight airport pavement area 3D model; Step S4, extracting a road surface image of a predetermined airport road section from the lightweight airport road surface area 3D model; Step S5, based on the road surface image of the predetermined airport road section, determining whether the predetermined airport road section needs maintenance to obtain a determination result; Step S6, in response to the judgment result that the predetermined airport road section needs maintenance, marking the predetermined airport road section in the lightweight airport pavement area 3D model, the mark being "maintenance required"; Step S7, displaying the 3D model of the lightweight airport pavement area with markings.

[0027] In the above-mentioned airport intelligent operation and maintenance method based on VR and the Internet of Things, the step S1 is to establish a BIM model of the airport pavement area. It should be understood that Building Information Modeling (BIM) technology is a comprehensive building management technology, which is not only a three-dimensional graphic display, but also contains a large amount of non-geometric information, such as material information (such as material, color, supplier, etc.), cost information (such as cost, budget, etc.), progress information (such as construction plan, construction period, etc.), etc. At present, BIM models have been widely used in the construction industry as well as urban planning, civil engineering, industrial design and other fields, and play an important role in the construction and maintenance of airport infrastructure. By establishing a BIM model of the airport pavement area, the digital management of the airport pavement area can be realized, providing a comprehensive and accurate management platform for airport operation and maintenance.

[0028] Specifically, when establishing the BIM model of the airport pavement area, the Internet of Things technology can be used to collect basic data about the airport pavement area. The basic data of the airport pavement area includes but is not limited to: topographic survey data, which uses high-precision laser scanners or drones to conduct large-area topographic surveys to obtain accurate ground elevation data; existing facility information, including the size, layout, material of runways, taxiways, aprons, etc., and the relative position relationship between facilities; underground facility information, such as the location and direction of underground pipelines and cable trenches, which is crucial for future excavation work; historical maintenance records, understanding past maintenance conditions will help predict possible problems in the future.

[0029] In order to ensure that BIM models and related information can be shared and exchanged smoothly between different parties, unified standards and specifications must be followed. For example, LOD (Level of Development) defines the information depth requirements at different levels from conceptual design to completion; IFC (Industry Foundation Classes), an internationally accepted data format that supports cross-platform data interoperability; ISO19650 series of international standards for information management, especially for construction projects using BIM technology, these standards provide a framework for information integration.

[0030] Choose the appropriate BIM software based on the project size, complexity, and skill level of the individual or team. There are many professional BIM software available on the market, such as Autodesk Revit, Bentley MicroStation, Graphisoft ARCHICAD, etc. These BIM software provide powerful modeling functions and can handle complex structural and system integration issues. At the same time, considering the particularity of airport pavement, it is also necessary to evaluate whether the BIM software supports large scene rendering optimization, large-scale point cloud data processing and other functions.

[0031] Based on the collected basic data, the basic framework of the airport pavement area is first constructed, including the outlines of the main structures such as runways, taxiways, and aprons. Then more details are added, such as the specific location and parameters of auxiliary facilities such as marking lines, signboards, lamps, and drainage systems. In addition, non-geometric information such as material properties and cost estimates need to be input. Then, tools such as collision detection are used to check whether there are conflicts or unreasonable aspects within the model, and adjustments are made according to the actual situation.

[0032] Given the high security and sensitivity of airports, strict measures must be taken in the data management and information security of BIM models. On the one hand, a complete data backup mechanism should be established to prevent the loss of important data due to accidents; on the other hand, access control should be set up to ensure that only authorized personnel can view or edit specific parts of the data.

[0033] In the above-mentioned airport intelligent operation and maintenance method based on VR and the Internet of Things, the step S2 is to model and analyze the BIM model of the airport pavement area to realize information conversion and extraction to obtain a JSON file. It should be understood that relying solely on the BIM model for facility management is still difficult to quickly identify and accurately locate minor damage to the airport pavement, which limits the in-depth application of BIM technology in the field of airport operation and maintenance. Since the BIM model contains a large number of different types of data, in order to achieve efficient processing and utilization of data, it is necessary to further convert the information in the BIM model into a format that is easier to process and analyze. Based on this, the present invention application considers that JSON (JavaScript Object Notation) files, as a lightweight data exchange format, can effectively store and transmit structured data, and have good readability and easy parsing characteristics. Therefore, the present invention application further models and analyzes the BIM model of the airport pavement area, extracts key information, and converts it into a JSON file format to facilitate sharing and exchanging data between different systems and simplify the subsequent data processing process.

[0034] Specifically, the BIM model is first comprehensively analyzed, including evaluating the geometry, size and relative position of key areas such as runways, taxiways and aprons to ensure that all physical features are correctly identified; checking in detail the type of materials used in each part and their properties, such as wear resistance, compressive strength, etc., and recording specific construction methods and processes; understanding the functional role of each component, such as lighting systems, drainage systems, marking lines, etc., and the interaction between components; for dynamically changing parts, such as maintenance history or planned renovation projects, recording important event points on the timeline; calculating the human and material resources required for each stage and the corresponding cost budget based on design requirements and actual construction conditions.

[0035] After completing the above analysis, the information is divided into different categories for better understanding and use. Entity objects represent physical objects, such as runways, taxiways, buildings, etc. Each entity has its own unique identifier and set of attributes. Relationship links describe the associations between entities, such as a certain taxiway is connected to a specific runway endpoint, or which areas a maintenance facility serves. Operational instructions specify how to operate entities or perform specific tasks, such as regularly checking the status of a piece of equipment or taking emergency measures in the event of a failure. Spatiotemporal coordinates provide information about time and space locations, which helps track the historical changes of objects or predict future trends.

[0036] With a clear data structure as the basis, the information conversion work can begin. Specifically, the parametric modeling engine uses software development interfaces such as Autodesk Revit API and Dynamo to directly access the data layer inside the BIM model, automatically extract the required information and perform preliminary sorting. Semantic Web technology can give clear semantic meanings to various types of information through standard protocols such as RDF (Resource Description Framework) and OWL (Web Ontology Language), so that computers can understand the content and context of the information. The rule engine and reasoning mechanism can automatically derive new conclusions or suggestions based on preset business logic rules, such as proposing an optimized path planning plan based on the current road conditions. The big data processing framework uses distributed computing platforms such as Hadoop and Spark to quickly process massive data sets and ensure the speed and quality of information conversion. The Geographic Information System (GIS) integrates GIS technology to not only enhance the understanding of geographic location, but also introduce more external factors, such as weather conditions and traffic flow, to further enrich the information dimension.

[0037] The last step is to encapsulate the analyzed, classified and transformed information into a JSON file. This involves defining a suitable JSON schema to guide the specific representation of information. The basic framework of the JSON document is built according to the main categories such as entity objects, relationship links, and operation instructions to form a multi-level data tree structure. For example, a key-value pair is used to describe a specific information item, where the key is a unique string identifier and the value is the corresponding actual content, which may be a simple text, number, or a more complex nested object. When there are multiple elements of the same type, they are organized in array form; for those objects that appear repeatedly but do not want to be copied multiple times, the expression can be simplified by ID reference. Although JSON itself does not support formal comment syntax, in some cases, additional explanatory text can be added to the value field to help users understand the information intent more quickly.

[0038] In the above-mentioned intelligent airport operation and maintenance method based on VR and the Internet of Things, in step S3, the JSON file is sent to the VR server, and the VR server loads the JSON file in WebGL for model loading and image rendering to obtain a lightweight airport pavement area 3D model. Specifically, in order to optimize the original BIM model, remove unnecessary details, retain the core elements for display and analysis, and create a lightweight 3D model suitable for real-time interaction, the present invention further sends the JSON file to the VR server, and the VR server further uses WebGL technology to load and render it into a visual lightweight airport pavement area 3D model. It should be understood that WebGL is an open source framework that allows developers to draw complex graphic objects directly in the browser without installing additional plug-ins. Specifically, the WebGL engine will quickly reconstruct a lightweight 3D scene based on the description in the JSON file, and it can run smoothly even on mobile devices. In addition, the 3D model rendered by WebGL technology can not only provide intuitive visual effects, but also support interactive operations. The operation and maintenance personnel can control the perspective transformation, zooming and panning through gestures or mice, further enhancing the perception and understanding of the airport pavement conditions.

[0039] First, during the data transmission phase, it is necessary to ensure that the JSON file is securely and quickly delivered from the information extraction system to the VR server. This usually involves using HTTP or HTTPS protocols in combination with RESTful API interfaces to implement data exchange between the client and the server. In order to ensure transmission efficiency, it may also be necessary to consider the application of compression algorithms, such as Gzip or Brotli, to reduce file size and speed up downloads. In addition, considering the transmission interruption problem that may be caused by network fluctuations, a reasonable retry mechanism and error handling logic should be designed to enhance the robustness of the system.

[0040] Once the JSON file successfully arrives at the VR server, the next task is to parse the file content. Since the JSON format itself is easy to parse, most modern programming languages ​​have built-in parser libraries that can directly read and interpret the key-value pair structure in the file. For large or complex JSON files, it is recommended to use a streaming parsing method to read data block by block instead of loading the entire file into memory at once to reduce resource usage. During the parsing process, special attention should be paid to mapping abstract data items back to specific geometric entities, such as converting numerical values ​​representing the length and width of the runway into line segments or planes in three-dimensional space; at the same time, visual characteristics such as material properties and texture mapping should be correctly handled to prepare for subsequent rendering.

[0041] After completing the data parsing, the crucial model loading link is entered. At this stage, WebGL technology is mainly relied on to achieve efficient rendering of three-dimensional graphics in the browser environment. WebGL is a low-level JavaScript API that allows direct access to GPU hardware acceleration functions, so that high-quality 3D images can be presented on the web page. In order to make full use of the capabilities of WebGL, an advanced framework such as Three.js is generally selected as a development tool, which simplifies many underlying operations, provides convenient methods for creating basic elements such as scenes, cameras, light sources, and supports the import of 3D model files in various formats. For the present invention application, Three.js can be used to easily load the BIM model components described by JSON to build a complete airport pavement area view.

[0042] When all necessary 3D objects are correctly loaded into the WebGL scene, image rendering can begin. The rendering process is not just a simple display of objects, but also includes the application of a series of advanced graphics technologies such as lighting effects, shadow calculations, reflection and refraction simulation, so that the final image is more realistic and vivid. In order to optimize rendering performance, several key measures have been taken: First, the LOD (Level of Detail) is set reasonably, that is, the model detail level is automatically adjusted according to the observer's distance to avoid unnecessary high-precision rendering at a long distance; second, batching technology is enabled to merge similar types of drawing calls to reduce the communication overhead between the CPU and GPU; third, instanced rendering is used. For a large number of recurring objects (such as signs and lamps), only one drawing command is required to batch generate multiple instances, which greatly improves efficiency; fourth, the deferred shading strategy is implemented to uniformly calculate the lighting impact in the post-processing stage to reduce the burden of rendering each frame.

[0043] Finally, after the above series of steps, a lightweight but refined 3D model of the airport pavement area is obtained, which not only retains the core information of the original BIM data, but also has real-time interactive features. Users can explore the airport environment in all directions through VR devices and gain intuitive experience.

[0044] In the above-mentioned airport intelligent operation and maintenance method based on VR and the Internet of Things, the step S4 extracts the road surface image of the predetermined airport section from the lightweight airport road surface area 3D model. It should be understood that, considering that airports usually cover a wide area and usually include multiple areas with different functions, in order to achieve accurate monitoring and analysis of airport road conditions, the present invention further extracts the road surface image of a specific airport section from the above-mentioned lightweight 3D model to focus on the key areas that need to be inspected or maintained, thereby improving the operation and maintenance efficiency and avoiding unnecessary global scanning.

[0045] In order to accurately identify and select the predetermined airport road section, first, a virtual camera view needs to be created to capture the image of the road section. In this process, the angle, height and field of view of the camera need to be carefully set to ensure that the road conditions can be obtained from the best perspective. For example, in order to check the cracks or wear of the road surface, a lower-angle oblique view can be selected; and in order to observe the overall layout, a higher-angle top view may be used. At the same time, the height and field of view of the camera also need to be optimized according to the specific observation purpose. It should not be too narrow to miss important details, nor too wide to cause distortion of the picture.

[0046] Next is the rendering stage, where WebGL and the various shader technologies it supports are fully utilized to optimize the color, brightness, contrast and other attributes of the selected road section, and advanced effects such as environment mapping and normal mapping are applied to enhance the realism of the image. In addition, in order to solve the image quality problem of large areas or complex structures, anti-aliasing, anisotropic filtering and other technologies are used to reduce edge artifacts and improve texture clarity, and poor lighting conditions are improved by adding additional light sources or adjusting lighting parameters.

[0047] When all preparations are completed, the required two-dimensional image needs to be extracted from the rendered scene. This is usually achieved through a framebuffer object (FBO), that is, creating an off-screen rendering target, rendering the content captured by the camera into this off-screen buffer, and then copying its content as a texture or directly exporting it to an image file format (such as PNG, JPEG). This completes the conversion from 3D model to 2D image. In order to ensure that the extracted image meets the intended purpose, it is also necessary to perform necessary post-processing operations, including resizing, color correction, annotation addition, compression and optimization, so that the image is closer to the actual situation, convenient for storage and transmission, and also helps users better understand the image content. In this way, complex three-dimensional data can be effectively converted into intuitive and easy-to-understand two-dimensional images, providing strong support for subsequent image analysis. Compared with traditional cameras that directly shoot two-dimensional images, it can realize real-time monitoring and comprehensive coverage of airport road conditions. When facing large-scale, complex and changeable airport road environments, it has better adaptability, coverage and information richness.

[0048] In the above-mentioned airport intelligent operation and maintenance method based on VR and the Internet of Things, the step S5 is to determine whether the predetermined airport section needs maintenance based on the road surface image of the predetermined airport section to obtain a determination result. To this end, the present invention further introduces a deep learning algorithm to perform intelligent analysis on the extracted road surface image. Figure 3 As shown, the step S5 includes: step S51, performing multi-level image context feature extraction on the road surface image of the predetermined airport section to obtain a shallow context coding feature map of the airport section road surface and a deep context coding feature map of the airport section road surface; step S52, performing a priori-guided feature saliency joint perception on the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface to obtain a shallow-deep joint perception feature map of the airport section road surface; step S53, generating the judgment result based on the shallow-deep joint perception feature map of the airport section road surface.

[0049] Specifically, in a specific example of the present invention application, the step S51 includes: inputting the road surface image of the predetermined airport section into the airport road surface feature extractor based on the FPT model to obtain the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface. It should be understood that the present invention application takes into account that the road surface image of the predetermined airport section may contain a variety of different types and degrees of damage, such as local cracks, potholes, and overall wear and structural deformation. Therefore, in order to achieve a comprehensive analysis of the airport road surface, the present invention application adopts the FPT (Feature Pyramid Transformer) model as the airport road surface feature extractor to perform multi-level feature extraction on the road surface image of the predetermined airport section. The FPT model can achieve full-active feature interaction across space and scale by applying the Transformer mechanism to the feature pyramid FPN (Feature Pyramid Network) structure. Specifically, the FPT model uses three Transformer encoding layers, corresponding to the non-local interactions within the same-level feature maps, top-down non-local interactions, and bottom-up non-local interactions, respectively. It can effectively capture the shallow detail features and deep semantic information in the road surface image to generate the shallow context encoding feature map of the airport section road surface and the deep context encoding feature map of the airport section road surface, thereby providing a rich information basis for subsequent road maintenance analysis.

[0050] Specifically, in a specific example of the present invention application, the step S52 performs a priori-guided joint perception of the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface to obtain a shallow-deep joint perception feature map of the airport section road surface. It should be understood that the present invention application takes into account that the shallow features of the road surface image mainly include low-level visual features such as edges and textures, while the deep features capture more complex structural patterns and semantic information. In order to achieve a more comprehensive and detailed feature description of the road surface conditions, the present invention application further performs a joint perception of the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface. In particular, considering that simple fusion methods such as feature splicing may not be able to fully express the correlation between shallow and deep features, the present invention application proposes a feature-significant joint perception method based on prior guidance, which enhances the understanding and expression of pavement features by introducing prior information in historical data, and on this basis, performs two-way attention interaction on the shallow and deep features of the pavement, thereby effectively capturing the key information of pavement damage and improving the accuracy of pavement maintenance analysis.

[0051] like Figure 4 As shown, the step S52 includes: Step S521, reshaping the airport road section pavement shallow context coding feature map and the airport road section pavement deep context coding feature map to obtain an airport road section pavement shallow context coding feature matrix and an airport road section pavement deep context coding feature matrix; Step S522, inputting the airport section road surface shallow context coding feature matrix and the airport section road surface deep context coding feature matrix into a priori guided modulation module to obtain a priori modulated airport section road surface shallow context coding feature matrix and a priori modulated airport section road surface deep context coding feature matrix; Step S523, performing attention interaction fusion on the a priori modulated airport road section pavement shallow context coding feature matrix and the a priori modulated airport road section pavement deep context coding feature matrix to obtain the airport road section pavement shallow-deep joint perception feature map.

[0052] More specifically, the step S521 includes the following formula:

[0053]

[0054] in, represents the shallow context encoding feature map of the road surface of the airport section, represents the deep context encoding feature map of the road surface of the airport section, represents the feature shape reshaping, represents the shallow context encoding feature matrix of the airport road section, Represents the deep context encoding feature matrix of the airport road section.

[0055] That is, the characteristic shapes of the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface are reshaped into a matrix form to adapt them to the form requirements of subsequent data interaction processing. In a specific example, each local feature matrix along the channel dimension of the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface is expanded into a one-dimensional vector to obtain the shallow context coding feature matrix of the airport section road surface and the deep context coding feature matrix of the airport section road surface.

[0056] More specifically, the step S522 includes the following formula:

[0057]

[0058] in, represents matrix multiplication, represents the transpose of a matrix, , All represent the learnable memory parameter matrix of the a priori guided modulation module, represents the normalization function, represents the prior modulated airport road section shallow context encoding feature matrix, Represents the deep context encoding feature matrix of the prior modulated airport road section pavement.

[0059] That is, the prior-guided modulation module is used to perform feature modulation on the shallow and deep features of the road surface after the feature shape is reshaped. The prior-guided modulation module uses the prior knowledge in the historical data to train its internal learnable memory parameter matrix by learning the road surface image features of different damage types and degrees, and determines the importance weights of different features, thereby realizing dynamic modulation of the current shallow and deep features of the road surface to enhance the recognition and expression of key road damage features.

[0060] like Figure 5 As shown, the step S523 includes: Step S5231, inputting the a priori modulated airport section road surface shallow context coding feature matrix and the a priori modulated airport section road surface deep context coding feature matrix into the attention interaction coding module based on the class converter structure to obtain the airport section road surface shallow feature enhancement interaction coding matrix and the airport section road surface deep feature enhancement interaction coding matrix; Step S5232, reshaping the airport section road surface shallow layer feature enhancement interactive coding matrix and the airport section road surface deep layer feature enhancement interactive coding matrix to obtain an airport section road surface shallow layer feature enhancement interactive coding feature map and an airport section road surface deep layer feature enhancement interactive coding feature map; Step S5233, calculating the weighted sum of the airport road section pavement shallow feature enhancement interactive coding feature map and the airport road section pavement deep feature enhancement interactive coding feature map to obtain the airport road section pavement shallow-deep joint perception feature map.

[0061] In a specific example of the present invention, the step S5231 includes: first, using the a priori modulated airport section road surface shallow context coding feature matrix as the query feature matrix and the value feature matrix, using the a priori modulated airport section road surface deep context coding feature matrix as the key feature matrix, performing attention interaction based on the converter structure on the query feature matrix, the value feature matrix and the key feature matrix to obtain the airport section road surface shallow feature enhancement interaction coding matrix, including the following formula:

[0062] in, is the feature scale scaling factor of the attention interaction encoding module, represents the soft maximization function, Represents the enhanced interactive coding matrix of shallow features of the pavement on the airport section.

[0063] Then, the prior modulated airport section road surface deep context encoding feature matrix is ​​used as the query feature matrix and the value feature matrix, and the prior modulated airport section road surface shallow context encoding feature matrix is ​​used as the key feature matrix. The query feature matrix, the value feature matrix and the key feature matrix are subjected to attention interaction based on the converter structure to obtain the airport section road surface deep feature enhanced interaction encoding matrix, including the following formula:

[0064] in, Represents the enhanced interactive coding matrix of deep features of the pavement of the airport section.

[0065] That is, the shallow context encoding feature matrix of the airport section pavement and the deep context encoding feature matrix of the airport section pavement after prior guidance modulation are further subjected to bidirectional attention interaction of the converter architecture, so as to utilize the detail information in the shallow features to enhance the semantic understanding of the deep features, and at the same time utilize the semantic information of the deep features to guide the attention focus of the shallow features. Through this complementary learning process, the expressive power of the shallow features and the deep features of the pavement are enhanced respectively, and the airport section pavement shallow feature enhanced interaction coding matrix and the airport section pavement deep feature enhanced interaction coding matrix are generated.

[0066] In a specific example of the present invention, step S5232 includes the following formula:

[0067]

[0068] in, Represents the airport road section shallow feature enhanced interactive coding feature map, Represents the interactively encoded feature map of the airport road section with enhanced deep features.

[0069] That is, the shallow feature enhancement interaction coding matrix of the airport section pavement and the deep feature enhancement interaction coding matrix of the airport section pavement after attention interaction are further reshaped to restore the original feature shapes.

[0070] In a specific example of the present invention, step S5233 includes the following formula:

[0071] in, and For different weight parameters, Represents the shallow-deep joint perception feature map of the airport road section.

[0072] That is, through the pixel-by-pixel weighted aggregation operation, the shallow and deep features are fused to form the shallow-deep joint perception feature map of the airport road section. In this way, the complementary advantages of shallow and deep features can be effectively utilized, which not only retains the detailed information of the road image, but also enhances the semantic understanding of the road damage pattern, providing strong support for subsequent road maintenance decisions.

[0073] Specifically, in a specific example of the present application, the step S53 includes: inputting the shallow-deep joint perception feature map of the airport road section pavement into the airport operation and maintenance judgment module based on the classifier to obtain the judgment result. Specifically, the classifier is based on a neural network architecture, and analyzes the pavement damage state and judges the pavement maintenance requirements by performing feature learning on the shallow-deep joint perception feature map of the airport road section pavement, and outputs the final judgment result, i.e., whether pavement maintenance is required, through the Softmax function of the classification layer. In this way, airport operation and maintenance personnel can quickly identify the pavement areas that need maintenance based on the obtained judgment results, so as to carry out repair work in a timely manner to ensure the safety and efficiency of airport operations.

[0074] More specifically, the shallow-deep joint perception feature map of the airport road section pavement is input into an airport operation and maintenance judgment module based on a classifier to obtain the judgment result, including: expanding the shallow-deep joint perception feature map of the airport road section pavement into a shallow-deep joint perception feature vector of the airport road section pavement; using the fully connected layer of the airport operation and maintenance judgment module to fully connect the airport road section pavement shallow-deep joint perception feature vector to obtain an airport road section pavement shallow-deep joint perception fully connected encoding vector; inputting the airport road section pavement shallow-deep joint perception fully connected encoding vector into the Softmax classification function of the airport operation and maintenance judgment module to obtain the probability value of the airport road section pavement shallow-deep joint perception feature map belonging to each classification label, wherein the classification labels include maintenance required and maintenance not required; and determining the classification label corresponding to the largest of the probability values ​​as the judgment result.

[0075] Here, considering that the shallow context coding feature map of the airport section pavement and the deep context coding map of the airport section pavement respectively represent the shallow image semantic context coding features and deep image semantic context coding features of the pavement image of the predetermined airport section, when performing feature significant joint perception based on prior guidance, due to the difference in the guiding ability of prior guidance for the shallow and deep layers of the image, the shallow-deep joint perception feature map of the airport section pavement will have insufficient long-distance joint perception representation based on enhanced interactivity of semantic features of images of different depths, thereby reducing the expression effect of the shallow-deep joint perception feature map of the airport section pavement, thereby affecting the accuracy of the judgment result obtained by the airport operation and maintenance judgment module based on the classifier.

[0076] Preferably, when the airport section road surface shallow layer-deep layer joint perception feature map is input into the classifier-based airport operation and maintenance judgment module to obtain a judgment result, the airport section road surface shallow layer-deep layer joint perception feature map is optimized, including: Calculate the absolute value sum of all eigenvalues ​​of the shallow-deep joint perception feature map of the airport road section , and the square root of the sum of squares of all eigenvalues ​​of the shallow-deep joint perception feature map of the airport road section , and Divide by To obtain the joint perception probability value of the shallow and deep layers of the airport road section ,Right now:

[0077]

[0078]

[0079] in, The first one represents the shallow-deep joint perception feature map of the road surface of the airport section eigenvalues; The shallow-deep joint perception feature map of the airport road section Each eigenvalue in is normalized to the maximum value to obtain the airport road section shallow-deep joint perception probability feature map :

[0080] in, represents the maximum value in the shallow-deep joint perception feature map of the airport road section, The first one represents the joint perception probability feature map of the shallow and deep layers of the road surface of the airport section. eigenvalues; The shallow-deep joint perception probability feature map of the airport section road surface is subtracted from the shallow-deep joint perception probability value of the airport section road surface, and then multiplied by the reciprocal of the shallow-deep joint perception probability value of the airport section road surface to obtain the shallow-deep joint perception domain gradient feature map of the airport section road surface ,in, Indicates point reduction, represents dot product; Subtract the shallow-deep joint perception probability feature map of the airport road section from the unit feature map point After that, the shallow-deep joint perception domain gradient feature map of the airport road section is divided by points to obtain the shallow-deep joint perception differential feature map of the airport road section, that is:

[0081] in, represents the unit feature map, represents the bit-by-bit inverse of the shallow-deep joint perception domain gradient feature map of the road surface of the airport section, that is, the inverse of each eigenvalue of the shallow-deep joint perception domain gradient feature map of the road surface of the airport section is calculated, Represents the differential feature map of the shallow-deep joint perception of the road surface of the airport section; Calculate the shallow-deep joint perception domain gradient feature map of the airport road section road surface with the shallow-deep joint perception probability value of the airport road section road surface as the power function of the exponent , and an exponential function with a natural constant as the base of the shallow-deep joint perception differential feature map of the airport road section The weighted sum is performed to obtain the optimized shallow-deep joint perception feature map of the airport section road surface, namely:

[0082] in, and Both represent weighted hyperparameters, Indicates point addition, Represents the optimized joint shallow-deep perception feature map of the pavement on the airport section.

[0083] Then, the optimized airport road section pavement shallow-deep joint perception feature map (which can be used as a new airport road section pavement shallow-deep joint perception feature map) is input into the classifier-based airport operation and maintenance judgment module to obtain a judgment result.

[0084] Based on this, the mutually exclusive generalized representation behavior units of the shallow-deep joint perception feature map of the airport section road surface are constructed through the discrete differential of the probability density domain gradient corresponding to the shallow-deep joint perception feature map of the airport section road surface, so as to enhance the static response characteristics of the short-range correlation mesoscopic information configuration of the shallow-deep joint perception feature map of the airport section road surface to the generative probabilistic generalized representation behavior based on the different behavior unit organization space under the non-uniform topological architecture of the shallow-deep joint perception feature map of the airport section road surface, thereby ensuring the steady-state convergence of the optimization process between the generated target and the extracted features in the feature space-generated probability mapping, and improving the accuracy of the judgment result obtained by the airport operation and maintenance judgment module based on the classifier when the shallow-deep joint perception feature map of the airport section road surface is input.

[0085] In the above-mentioned airport intelligent operation and maintenance method based on VR and the Internet of Things, in step S6, in response to the judgment result that the predetermined airport road section requires maintenance, the predetermined airport road section is marked in the 3D model of the lightweight airport pavement area, and the mark is "maintenance required".

[0086] Specifically, when the system determines that a scheduled airport road section needs maintenance, it will visually mark the section in the 3D model of the lightweight airport pavement area as "requires maintenance." In addition, the system will generate a report and notify relevant personnel via email or SMS to ensure that maintenance activities are efficient and do not affect flight operations, thereby improving the safety and reliability of airport operations.

[0087] In the above-mentioned airport intelligent operation and maintenance method based on VR and the Internet of Things, the step S7 displays the 3D model of the lightweight airport pavement area with the mark. It should be understood that the mark is highlighted by visual elements such as highlight color, special icons or flashing effects to ensure that it is clearly visible from different viewing angles and does not block important structural features. The user can click on the mark through the interactive function to view detailed maintenance information and recommended measures.

[0088] In summary, the intelligent airport operation and maintenance method based on VR and the Internet of Things based on the present invention is explained. First, a BIM model of the airport pavement area is established, and a lightweight 3D model of the airport pavement area is generated through modeling analysis and image rendering, and the pavement image of the predetermined airport section is extracted from it; then, the image processing technology based on deep learning is introduced to perform multi-level feature extraction and joint perception on the pavement image of the predetermined airport section, so as to intelligently identify whether the predetermined airport section needs maintenance, and make corresponding marks and prompts in the 3D model of the airport pavement area based on the analysis results. In this way, the pavement status information can be updated in real time, helping the operation and maintenance personnel to better understand the actual situation of the airport pavement, thereby effectively improving the efficiency and accuracy of airport operation and maintenance.

[0089] like Figure 6 As shown, in the second aspect, an airport intelligent operation and maintenance system 100 based on VR and the Internet of Things adopts any one of the airport intelligent operation and maintenance methods based on VR and the Internet of Things described in the first aspect, including: A BIM model building module 110, for building a BIM model of the airport pavement area; An information conversion and extraction module 120 is used to perform modeling analysis on the BIM model of the airport pavement area to achieve information conversion and extraction to obtain a JSON file; A model loading and image rendering module 130 is used to send the JSON file to a VR server, and the VR server loads the JSON file into a model and renders the image in WebGL to obtain a lightweight airport pavement area 3D model; A road surface image extraction module 140 is used to extract a road surface image of a predetermined airport road section from the lightweight airport road surface area 3D model; A road condition assessment module 150, configured to determine whether the predetermined airport road section needs maintenance based on the road surface image of the predetermined airport road section to obtain a determination result; A maintenance requirement marking module 160 is used to mark the predetermined airport road section in the lightweight airport pavement area 3D model in response to the judgment result that the predetermined airport road section needs maintenance, and the mark is "maintenance required"; The 3D model display module 170 is used to display the 3D model of the lightweight airport pavement area with markings.

[0090] Here, those skilled in the art can understand that the specific operations of each module in the above-mentioned airport intelligent operation and maintenance system based on VR and the Internet of Things have been referred to above. Figures 1 to 5 It has been introduced in detail in the description of an airport intelligent operation and maintenance method based on VR and the Internet of Things, so its repeated description is omitted.

[0091] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0092] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0093] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0094] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0095] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. An airport intelligent operation and maintenance method based on VR and the Internet of Things, characterized in that: include: Build a BIM model of the airport pavement area; Modeling and analyzing the BIM model of the airport pavement area to achieve information conversion and extraction to obtain a JSON file; Sending the JSON file to a VR server, the VR server performs model loading and image rendering on the JSON file in WebGL to obtain a lightweight airport pavement area 3D model; Extracting a road surface image of a predetermined airport road section from the lightweight airport road surface area 3D model; Based on the road surface image of the predetermined airport road section, determining whether the predetermined airport road section needs maintenance to obtain a determination result; In response to the determination result that the predetermined airport road section needs maintenance, marking the predetermined airport road section in the lightweight airport pavement area 3D model, the marking being "maintenance required"; The 3D model of the lightweight airfield pavement area is displayed with markings.

2. According to claim 1, the intelligent operation and maintenance method of an airport based on VR and the Internet of Things is characterized in that: Based on the road surface image of the predetermined airport road section, determining whether the predetermined airport road section needs maintenance to obtain a determination result includes: Performing multi-level image context feature extraction on the road surface image of the predetermined airport section to obtain a shallow context coding feature map of the airport section road surface and a deep context coding feature map of the airport section road surface; Performing a priori-guided feature saliency joint perception on the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface to obtain a shallow-deep joint perception feature map of the airport section road surface; The judgment result is generated based on the shallow-deep joint perception feature map of the road surface of the airport section.

3. According to the method of airport intelligent operation and maintenance based on VR and Internet of Things according to claim 1 or 2, it is characterized in that: Performing multi-level image context feature extraction on the road surface image of the predetermined airport section to obtain a shallow context coding feature map of the road surface of the airport section and a deep context coding feature map of the road surface of the airport section, including: The road surface image of the predetermined airport road section is input into an airport road surface feature extractor based on the FPT model to obtain a shallow context coding feature map of the airport road section road surface and a deep context coding feature map of the airport road section road surface.

4. The airport intelligent operation and maintenance method based on VR and the Internet of Things according to claim 3 is characterized in that: The shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface are subjected to a priori-guided feature saliency joint perception to obtain a shallow-deep joint perception feature map of the airport section road surface, including: Reshaping the shallow context coding feature map of the airport section road surface and the deep context coding feature map of the airport section road surface to obtain a shallow context coding feature matrix of the airport section road surface and a deep context coding feature matrix of the airport section road surface; Inputting the airport section road surface shallow context coding feature matrix and the airport section road surface deep context coding feature matrix into the prior guided modulation module to obtain the prior modulated airport section road surface shallow context coding feature matrix and the prior modulated airport section road surface deep context coding feature matrix; The shallow context encoding feature matrix of the prior modulated airport road section pavement and the deep context encoding feature matrix of the prior modulated airport road section pavement are interactively fused with attention to obtain the shallow-deep joint perception feature map of the airport road section pavement.

5. The method for intelligent airport operation and maintenance based on VR and the Internet of Things according to claim 4 is characterized in that: The prior modulated airport section road surface shallow context encoding feature matrix and the prior modulated airport section road surface deep context encoding feature matrix are subjected to attention interaction fusion to obtain the airport section road surface shallow-deep joint perception feature map, including: Input the a priori modulated airport section road surface shallow context coding feature matrix and the a priori modulated airport section road surface deep context coding feature matrix into the attention interaction coding module based on the transformer-like structure to obtain the airport section road surface shallow feature reinforcement interaction coding matrix and the airport section road surface deep feature reinforcement interaction coding matrix; Reshaping the shallow-layer feature enhancement interactive coding matrix of the airport section road surface and the deep-layer feature enhancement interactive coding matrix of the airport section road surface to obtain a shallow-layer feature enhancement interactive coding feature map of the airport section road surface and a deep-layer feature enhancement interactive coding feature map of the airport section road surface; The weighted sum of the shallow-layer feature enhancement interactive coding feature map of the airport road section pavement and the deep-layer feature enhancement interactive coding feature map of the airport road section pavement is calculated to obtain the shallow-deep joint perception feature map of the airport road section pavement.

6. The method for intelligent airport operation and maintenance based on VR and the Internet of Things according to claim 5 is characterized in that: The a priori modulated airport section road surface shallow context coding feature matrix and the a priori modulated airport section road surface deep context coding feature matrix are input into the attention interaction coding module based on the transformer-like structure to obtain the airport section road surface shallow feature reinforcement interaction coding matrix and the airport section road surface deep feature reinforcement interaction coding matrix, including: The a priori modulated airport section road surface shallow context coding feature matrix is ​​used as the query feature matrix and the value feature matrix, and the a priori modulated airport section road surface deep context coding feature matrix is ​​used as the key feature matrix, and the query feature matrix, the value feature matrix and the key feature matrix are subjected to attention interaction based on the converter structure to obtain the airport section road surface shallow feature reinforcement interaction coding matrix; The prior modulated airport road section pavement deep context encoding feature matrix is ​​used as the query feature matrix and the value feature matrix, and the prior modulated airport road section pavement shallow context encoding feature matrix is ​​used as the key feature matrix. The query feature matrix, the value feature matrix and the key feature matrix are subjected to attention interaction based on the converter structure to obtain the airport road section pavement deep feature enhanced interaction encoding matrix.

7. The method for intelligent airport operation and maintenance based on VR and the Internet of Things according to claim 6 is characterized in that: The judgment result is generated based on the shallow-deep joint perception feature map of the airport road section, including: The shallow-deep joint perception feature map of the airport road section is input into the classifier-based airport operation and maintenance judgment module to obtain the judgment result.

8. The method for intelligent airport operation and maintenance based on VR and the Internet of Things according to claim 7 is characterized in that: Inputting the shallow-deep joint perception feature map of the airport road section into the classifier-based airport operation and maintenance judgment module to obtain the judgment result, including: Expanding the airport section road surface shallow layer-deep layer joint perception feature map into an airport section road surface shallow layer-deep layer joint perception feature vector; Use the fully connected layer of the airport operation and maintenance judgment module to fully connect the shallow-deep joint perception feature vector of the airport road section pavement to obtain the shallow-deep joint perception fully connected encoding vector of the airport road section pavement; Input the airport road section pavement shallow-deep joint perception fully connected encoding vector into the Softmax classification function of the airport operation and maintenance judgment module to obtain the probability value of the airport road section pavement shallow-deep joint perception feature map belonging to each classification label, wherein the classification label includes maintenance required and maintenance not required; The classification label corresponding to the largest probability value among the probability values ​​is determined as the judgment result.

9. An airport intelligent operation and maintenance system based on VR and the Internet of Things, adopting an airport intelligent operation and maintenance method based on VR and the Internet of Things as described in any one of claims 1 to 8, characterized in that: include: BIM model building module, used to build the BIM model of the airport pavement area; An information conversion and extraction module is used to perform modeling and analysis on the BIM model of the airport pavement area to achieve information conversion and extraction to obtain a JSON file; A model loading and image rendering module, used for sending the JSON file to a VR server, and the VR server performs model loading and image rendering on the JSON file in WebGL to obtain a lightweight airport pavement area 3D model; A road surface image extraction module, used to extract a road surface image of a predetermined airport road section from the lightweight airport road surface area 3D model; A road condition assessment module, used to determine whether the predetermined airport road section needs maintenance based on the road surface image of the predetermined airport road section to obtain a determination result; A maintenance requirement marking module, for marking the predetermined airport road section in the lightweight airport pavement area 3D model in response to the judgment result that the predetermined airport road section needs maintenance, wherein the mark is "maintenance required"; The 3D model display module is used to display the 3D model of the lightweight airport pavement area with markings.

Images