Intelligent inspection method and system for overline steel structure canopy in no-fly zone environment

Abstract

The invention discloses an intelligent inspection method and system for an overline steel structure canopy in a no-fly zone environment. The method comprises the following steps: S1, forming a digital twinborn model; s2, establishing a disease three-level classification system; s3, double positioning of disease marks; s4, generating a maintenance work order and matching a maintenance scheme; s5, eliminating disease marks; and S6, regularly outputting an inspection report. According to the invention, a three-dimensional model, a component code, a mobile terminal, disease identification, early warning and a knowledge graph are combined to form an intelligent inspection scheme of the railway cross-line steel structure canopy applied to a no-fly zone environment, a closed-loop management operation process of perception-diagnosis-decision-verification is realized, and the overall operation and maintenance efficiency is improved.

Description

Technical Field

[0001] This invention belongs to the technical field of computer systems based on specific calculation models, and in particular, it is a method and system for intelligent inspection of cross-line steel structure canopies used in no-fly zone environments. Background Technology

[0002] Railway canopy inspection refers to the regular inspection, maintenance and management of canopies above railway stations, platforms or other railway facilities. The purpose is to ensure the safety and normal function of the facilities, and to guarantee the safety of railway operations and the convenience of passenger travel.

[0003] The steel-structured canopy area involves railway operation safety and has high electromagnetic radiation levels, generally falling within a no-fly zone. Conventional inspection equipment such as drones cannot be used in this area. Furthermore, the open structure of the steel-structured canopy exposes the area to harsh environmental conditions caused by wind and sun, making maintenance difficult. Therefore, track inspection robots are also unsuitable. In most cases, inspections are conducted manually by personnel, which places high demands on their skills. From the discovery of defects to work order assignment, the process relies heavily on the inspectors' experience and judgment, lacking knowledge base support, resulting in inefficient maintenance decision-making and delayed maintenance response. The paper records and defect images kept by inspectors are also stored in a scattered manner, lacking structured management and resulting in low utilization of historical data, leading to data silos.

[0004] Based on the above analysis, the problems and shortcomings of the existing technology are as follows: Maintenance plans rely on human experience, making it difficult to detect safety hazards in a timely manner. Minor defects may develop into major structural problems due to delayed treatment.

[0005] The process from disease discovery to work order dispatch is lengthy and lacks intelligent decision support.

[0006] It is difficult to track the evolution trend of diseases and cannot provide data support for preventive maintenance. Summary of the Invention

[0007] In response to the aforementioned problems, this invention proposes an intelligent inspection method and system for railway overpass steel structure canopies in no-fly zones. By combining 3D models, component coding, mobile terminals, defect identification, early warning, and knowledge graphs, an intelligent inspection solution for railway overpass steel structure canopies applied in no-fly zones is formed.

[0008] The specific technical solution is as follows: A smart inspection method for cross-line steel structure canopies used in no-fly zone environments includes the following steps: S1: Forming a digital twin model A model of the cross-line steel structure canopy was established, and a component coding system was created. The latitude and longitude of the canopy columns measured on site were used as positioning reference points to obtain the latitude and longitude of each component and correspond one-to-one with the component code. The cross-line steel structure canopy model was then lightweighted and loaded into the GIS platform to form a digital twin model. S2: Establish a three-level classification system for diseases Establish a three-level classification system for diseases, set thresholds for early warning modules, and create a disease database, a disease maintenance knowledge base, and a disease early warning database. The disease database collects original images of damaged components of cross-line steel structures from various railway bureaus, performs image enhancement on the damaged component images, uses the UNet segmentation model to pre-annotate the damaged areas of the original damaged component images after image enhancement, generates a JSON format annotation file, and adds three-level classification labels for the diseases: the first level is structural diseases, the second level is technical sub-categories, and the third level is damage features. S3: Dual-location disease marker The system initializes and sets the inspection route and sends it to the mobile terminal. It compares the real-time location of the mobile terminal with the inspection route of the inspection personnel holding the mobile terminal. It receives images of damaged components uploaded by the inspection personnel through the mobile terminal, and performs dual location marking of the defects on the digital twin model using the component code and the latitude and longitude coordinates of the canopy column until the inspection personnel complete the inspection. S4: Generate a repair work order and match a repair plan The system generates maintenance work orders based on the safety level of the defects and calls the Neo4j knowledge graph to match maintenance solutions; S5: Eliminate disease markers After the maintenance personnel verify the repair, they remove the corresponding component defect markers from the digital twin model. S6: Regularly generate inspection reports The system summarizes inspection, maintenance, and disease early warning records based on the disease database, disease maintenance knowledge base, and disease early warning database, and regularly outputs inspection reports.

[0009] In one embodiment, in step S1, the component coding hierarchy is coded according to "structural classification - structural type - component type - component number", and each component code is unique.

[0010] In one embodiment, in step S2, the threshold values ​​for the three-level early warning module are set according to the three-level disease classification system, and the early warning levels are as follows: Yellow alert: Defects have occurred in non-critical components; Orange alert: Major components have suffered damage, but structural safety has not yet been affected; Red alert: Major components have suffered damage, affecting structural safety.

[0011] In one embodiment, the early warning module includes setting an early warning push mechanism and message templates, setting early warning messages according to the early warning level, and selecting the corresponding message template for push.

[0012] In one embodiment, in step S2, after creating the disease database, a distributed file system is used to store the original disease component images, and a relational database stores the structured disease metadata information.

[0013] In one embodiment, in step S3, the mobile terminal positioning method uses GNSS, UWB or Bluetooth positioning technology, and selects an auxiliary gyroscope sensor and an accelerometer sensor to sense the dynamics of the mobile terminal.

[0014] In one embodiment, in step S4, the Neo4j knowledge graph is a "disease-cause-treatment" relationship network constructed using Neo4j in the disease repair database. The network contains various disease repair processes and supports similarity matching retrieval.

[0015] Another objective of this invention is to provide an intelligent inspection system for cross-line steel structure canopies in no-fly zone environments, and to implement an intelligent inspection method for cross-line steel structure canopies in no-fly zone environments, comprising the following modules: Modeling Generation and Conversion Module: Used to create a model of the cross-line steel structure canopy, establish a component coding system, use the latitude and longitude of the canopy columns measured on site as the positioning reference point, obtain the latitude and longitude of each component, correspond one-to-one with the component code, and load the lightweight cross-line steel structure canopy model into the GIS platform to form a digital twin model. Disease Data Management Module: Used to establish a three-level disease classification system, set thresholds for the early warning module, and create a disease database, a disease maintenance knowledge base, and a disease early warning database; Mobile inspection interaction module: used for system initialization, setting inspection routes and sending them to mobile terminals. It compares the real-time positioning of the mobile terminal with the inspection route of the inspector holding the mobile terminal. It receives images of damaged components uploaded by the inspector through the mobile terminal and performs dual positioning and marking of defects on the digital twin model using component codes and the latitude and longitude coordinates of the canopy columns until the inspector completes the inspection. Disease analysis and processing module: used by the system to generate maintenance work orders based on disease safety levels and call Neo4j knowledge graph to match maintenance solutions; Defect Marker Removal Module: Used by maintenance personnel to remove defect markers from the corresponding components on the digital twin model after verification of maintenance. Analysis and output report module: The system summarizes inspection, maintenance and disease early warning records based on the disease database, disease maintenance knowledge base and disease early warning database, and outputs inspection reports regularly.

[0016] Another object of the present invention is to provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the intelligent inspection method for cross-line steel structure canopies for no-fly zone environments.

[0017] The beneficial effects of this invention are: 1. By using both component coding and latitude / longitude coordinates for dual anchoring, the problem of ambiguous positioning of defective components in cross-line steel structures is solved; 2. Maintenance has been upgraded from manual dispatch to system push. The fault maintenance knowledge base provides suggestions for maintenance solutions. Based on the fault classification, the corresponding maintenance solution is matched and the corresponding maintenance work order is issued, which is accurate and targeted, improving maintenance response time and maintenance efficiency. 3. Implement a closed-loop management process of "perception-diagnosis-decision-verification" to improve overall operation and maintenance efficiency. Attached Figure Description

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. However, it should be understood that these drawings are designed for illustrative purposes only and are not intended to limit the scope of the present invention. Furthermore, unless specifically indicated, these drawings are intended only to conceptually illustrate the structural construction described herein and are not necessarily drawn to scale.

[0019] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the system modules of the present invention; Figure 3 This is a flowchart of the inspection and maintenance process of this invention; In the picture: 1-Modeling generation and conversion module; 2-Disease data management module; 3-Mobile inspection interaction module; 4-Disease analysis and processing module; 5-Disease mark removal module; 6-Analysis output report module. Detailed Implementation

[0020] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0021] I. Explanation of the Implementation Example: The following is in conjunction with the appendix Figure 1 - Appendix Figure 3 This invention will be described in detail.

[0022] Example 1, such as Figure 1 As shown in the figure, an intelligent inspection method for a cross-line steel structure canopy in a no-fly zone environment provided by an embodiment of the present invention includes the following steps:

[0023] S1: Forming a digital twin model A model of the cross-line steel structure canopy was established, and a component coding system was created. The latitude and longitude of the canopy columns measured on site were used as positioning reference points to obtain the latitude and longitude of each component and correspond one-to-one with the component code. The cross-line steel structure canopy model was then lightweighted and loaded into the GIS platform to form a digital twin model. S2: Establish a three-level classification system for diseases Establish a three-level classification system for diseases, set thresholds for early warning modules, and create a disease database, a disease maintenance knowledge base, and a disease early warning database. The disease database collects original images of damaged components of cross-line steel structures from various railway bureaus, performs image enhancement on the damaged component images, uses the UNet segmentation model to pre-annotate the damaged areas of the original damaged component images after image enhancement, generates a JSON format annotation file, and adds three-level classification labels for the diseases: the first level is structural diseases, the second level is technical sub-categories, and the third level is damage features. S3: Dual-location disease marker The system initializes and sets the inspection route and sends it to the mobile terminal. It compares the real-time location of the mobile terminal with the inspection route of the inspection personnel holding the mobile terminal. It receives images of damaged components uploaded by the inspection personnel through the mobile terminal, and marks the defects on the digital twin model using the component code and the latitude and longitude coordinates of the canopy column, until the inspection personnel complete the inspection. S4: Generate a repair work order and match a repair plan The system generates maintenance work orders based on the safety level of the defects and calls the Neo4j knowledge graph to match maintenance solutions; S5: Eliminate disease markers After the maintenance personnel verify the repair, they remove the corresponding component defect markers from the digital twin model. S6: Regularly generate inspection reports The system summarizes inspection, maintenance, and disease early warning records based on the disease database, disease maintenance knowledge base, and disease early warning database, and regularly outputs inspection reports.

[0024] In this embodiment, in step S1, since some stations were built a long time ago, some as-built drawings may be missing or inconsistent with the current situation. Therefore, the digital twin model is double-checked using as-built drawings and three-dimensional laser scanning point cloud data to ensure the accuracy of the cross-line steel structure canopy model.

[0025] First, the as-built drawings are received and a model of the cross-line steel structure canopy is established. The model is then verified using point cloud data obtained from on-site 3D laser scanning to ensure consistency between the model and the actual on-site structure. After confirmation of consistency, each component of the cross-line steel structure canopy model is individually coded using the established component coding system. The lightweight cross-line steel structure canopy model is then loaded into the GIS platform to form a digital twin model.

[0026] The resulting digital twin model uses the actual latitude and longitude of the canopy columns measured on-site as the positioning reference point. The actual latitude and longitude of the canopy columns are measured using RTK measurement tools to obtain the latitude and longitude of each component, which corresponds one-to-one with the component code. Therefore, it has high precision characteristics and can realize spatial mapping of steel structure components (such as web members and gutters). Its positioning error is <0.5m, which provides accurate accuracy for marking the location of defects on the digital twin model later.

[0027] In step S2, the images of the damaged components are taken and uploaded by the inspection personnel of each railway bureau via mobile terminals.

[0028] Image enhancement of diseased component images is performed based on the OpenCV image enhancement algorithm, and then the UNet segmentation model is used to pre-annotate the diseased images. After manual verification, the annotation time can be greatly reduced.

[0029] The data in the disease database needs to be trained using a deep learning model to extract and identify features from diseased component images. The trained disease feature model is then used for subsequent label classification of diseased component images.

[0030] First, representative disease images are extracted from the disease database as training set data and validation set data. The validation set data includes 20% adversarial examples. Then, a disease feature model is constructed, using the three-level disease classification system as the extracted features, such as corrosion rate and corrosion depth. The model is constructed in stages and levels, with one CNN model for each level. The first, second, and third levels are respectively used to classify diseased component images, outputting the classification results from the first to the third level, and adding three-level disease classification labels. The primary classification is structural defects, specifically corrosion and damage; the secondary classification is technical subcategories, specifically rust and coating deterioration; and the tertiary classification is damage characteristics, specifically rust rate and rust depth. Finally, after training is completed, the trained model is tested using validation set data until the recognition accuracy meets the requirements.

[0031] In step S3, the inspection personnel use a mobile terminal to conduct inspections according to the inspection route set during system initialization. The mobile terminal assists the inspection personnel in conducting inspections by using real-time positioning and comparing it with the set inspection route. After discovering defects, the inspection personnel select the corresponding component to take a picture and upload it. During the photo upload process, text descriptions or videos can be attached.

[0032] Based on the received images of damaged components, the system accurately locates the damaged components in the digital twin model using the previous component codes and the latitude and longitude coordinates of the canopy columns, achieving a 100% accuracy rate in locating the damaged components, and adds a three-level classification label for the damaged components.

[0033] In step S4, the system first determines the safety level of the damaged component, and then calls the damaged feature model trained in step S2 to automatically identify the image of the damaged component and classify the three levels of damage.

[0034] If the damage occurs in non-critical components, the corresponding warning level is yellow. If major components are damaged, and the number of damaged major components accounts for less than 20% of the total number of major components, then an orange alert is issued. If major components are damaged, and the number of damaged major components accounts for more than 20% of the total number of major components, a red alert will be issued. Based on the warning level, the system will provide corresponding alerts: When a yellow alert is issued, the system will immediately send a reminder message to administrators and maintenance personnel; When an orange or red alert is triggered, the system will send a strong reminder, such as ringing, to the mobile terminals of management and maintenance personnel, requiring immediate action. The alert response time can be reduced from 2 hours to less than 15 minutes.

[0035] After the identification and classification of defective components are completed, the review process is initiated, such as... Figure 3 As shown: If the disease classification is correct, the images of the diseased components and their corresponding classification tags are stored in the virus database. The virus repair knowledge base is called up, and the repair plan is retrieved based on the specific disease type using the "disease-cause-treatment" relationship network built by Neo4j knowledge graph. The location of the diseased components, the disease classification, and the repair plan are summarized into a repair work order and issued to the repair personnel. If the disease classification is incorrect during the review, the classification label corresponding to the disease component image is corrected, and the disease feature model is incrementally trained and updated.

[0036] In this step, the mobile terminals used by maintenance personnel and the mobile terminals used by inspection personnel use the same application.

[0037] In step S5, after the maintenance personnel verify the repair, the system receives the repair completion image uploaded by the maintenance personnel through the mobile terminal, transfers it to the review process, and after approval, removes the corresponding component defect mark on the digital twin model.

[0038] The entire inspection and maintenance process, through steps S4 and S5, enables the system to form a closed-loop management process of "perception-diagnosis-decision-verification", keeping the response time of maintenance work orders to within 2 hours.

[0039] The closed-loop management process of "perception-diagnosis-decision-verification" is as follows: Sensing: The system receives images of damaged components after inspectors inspect and upload them. Diagnosis: The system identifies diseased component images using a disease feature model to classify the disease. Decision: Based on the disease classification and the set warning threshold, the system determines whether to issue a warning and issues a maintenance work order according to the corresponding maintenance plan for the disease; Verification: Maintenance personnel repair the defects according to the maintenance work order, and upload the maintenance results so that the system can remove the defect markers.

[0040] In step S6, the inspection report includes inspection, maintenance, and disease early warning records. These records are stored separately in different databases and can be accessed by the system individually.

[0041] Example 2: Based on the intelligent inspection method for cross-line steel structure canopies in no-fly zones provided in Embodiment 1 of the present invention, preferably, in step S1, the component coding hierarchy is coded according to "structure classification - structure type - component type - component number", and each component code is unique.

[0042] In this embodiment, the component coding is a four-level coding system, namely "structural classification - structural type - component type - component number".

[0043] Example 3: Based on the intelligent inspection method for cross-line steel structure canopies in no-fly zones provided in Embodiment 1 of the present invention, preferably, in step S2, the threshold values ​​of the three-level early warning module are set according to the three-level classification system of defects, and the early warning levels are as follows: Yellow alert: Defects have occurred in non-critical components; Orange alert: Major components have suffered damage, but structural safety has not yet been affected; Red alert: Major components have suffered damage, affecting structural safety.

[0044] In this embodiment, the threshold of the early warning module can be further optimized. An early warning is set after a certain percentage is reached to reduce the false alarm rate, as detailed below: After completing a full inspection: If the damage occurs in non-critical components, the corresponding warning level is yellow. If major components are damaged, and the number of damaged major components accounts for less than 20% of the total number of major components, then an orange alert is issued. If major components are damaged, and the number of damaged major components accounts for more than 20% of the total number of major components, a red alert will be issued.

[0045] Example 4: Based on the intelligent inspection method for cross-line steel structure canopies in no-fly zones provided in Embodiment 3 of the present invention, preferably, the early warning module includes setting an early warning push mechanism and message templates, setting early warning messages according to the early warning level, and selecting the corresponding message template for push.

[0046] Based on the warning level, the system's warning push mechanism is as follows: When a yellow alert is issued, the system will immediately send a reminder message to administrators and maintenance personnel; In the event of an orange or red alert, the system will issue a strong reminder, such as ringing, to the mobile terminals of administrators and maintenance personnel, requiring immediate action.

[0047] Example 5: Based on the intelligent inspection method for cross-line steel structure canopies in no-fly zones provided in Embodiment 1 of the present invention, preferably, in step S2, after creating the defect database, a distributed file system is used to store the original defect component images, and a relational database is used to store structured defect metadata information.

[0048] In this embodiment, the original images are stored using the Hadoop HDFS distributed file system, and the structured disease metadata information is stored using a MySQL relational database.

[0049] Example 6: Based on the intelligent inspection method for cross-line steel structure canopies in no-fly zones provided in Embodiment 1 of the present invention, preferably, in step S3, the mobile terminal positioning method adopts GNSS, UWB or Bluetooth positioning technology, and selects auxiliary gyroscope sensors and accelerometer sensors to sense the dynamics of the mobile terminal.

[0050] In this embodiment, the positioning of the mobile terminal, the component coding, and the latitude and longitude coordinates of the canopy column are combined to achieve high-precision positioning of the defective component and improve the efficiency of subsequent maintenance.

[0051] Example 7: Based on the intelligent inspection method for cross-line steel structure canopies in no-fly zones provided in Embodiment 1 of the present invention, in step S4, the Neo4j knowledge graph is a "disease-cause-treatment" relationship network constructed using Neo4j in the disease repair database. The network contains a variety of disease repair processes and supports similarity matching retrieval.

[0052] In this embodiment, a case reasoning module is established to store typical maintenance cases from previous years and support similarity matching retrieval.

[0053] Example 8: like Figure 2 As shown, an intelligent inspection system for cross-line steel structure canopies used in no-fly zones is described, and a method for intelligent inspection of cross-line steel structure canopies used in no-fly zones is implemented, including the following modules: Modeling Generation and Conversion Module 1: Used to establish a cross-line steel structure canopy model, establish a component coding system, use the latitude and longitude of the canopy columns measured on site as the positioning reference point, obtain the latitude and longitude of each component, correspond one-to-one with the component code, and load the cross-line steel structure canopy model into the GIS platform after lightweighting to form a digital twin model; Disease Data Management Module 2: Used to establish a three-level disease classification system, set thresholds for the early warning module, and create a disease database, a disease maintenance knowledge base, and a disease early warning database; Mobile inspection interaction module 3: It is used to initialize the system, set the inspection route and send it to the mobile terminal. It compares the real-time positioning of the mobile terminal with the inspection route of the inspection personnel holding the mobile terminal. It receives the images of the defective components uploaded by the inspection personnel through the mobile terminal. It performs dual positioning and defect marking on the digital twin model through component code and the latitude and longitude coordinates of the canopy column until the inspection personnel complete the inspection. Disease Analysis and Processing Module 4: Used by the system to generate maintenance work orders based on disease safety levels and call Neo4j knowledge graph to match maintenance solutions; Module 5 for removing defect markers: Used by inspection personnel to remove the corresponding component defect markers on the digital twin model after verification and maintenance; Analysis and Output Report Module 6: This module is used by the system to summarize inspection, maintenance, and disease early warning records based on the disease database, disease maintenance knowledge base, and disease early warning database, and to periodically output inspection reports.

[0054] In this embodiment, the present invention integrates digital twin, GIS positioning, intelligent defect identification and maintenance decision-making for railway overpass steel structure canopies, covering 3D modeling, IoT positioning, database construction and mobile terminal interaction technology.

[0055] II. Application Examples: The embodiments of the present invention will be specifically described below using various application examples.

[0056] Application Example 1: An example of constructing a digital twin model of a cross-line steel structure canopy. 1. BIM Modeling and Data Acquisition Phase Based on existing as-built drawings, an initial BIM model was created using BIM software such as Revit. The model accuracy of the cross-line steel structure canopy reached LOD400 level, including detailed structures such as bolt holes and connecting plates. Point cloud data was collected using a 3D laser scanner, and the point cloud was automatically compared with the BIM model using TrimbleRealWorks software. If there were discrepancies between the point cloud and the drawings, the model was corrected after on-site verification.

[0057] 2. Component Coding and Positioning System The components are marked according to a four-level coding system: "structural classification - structural type - component type - component number".

[0058] Let's take a single component as an example: The main steel structure of the cross-line steel structure canopy is a truss. The component with the upper chord number 05 corresponds to "01" in the main steel structure, "02" in the truss structure, and "04" in the upper chord structural unit. The component with the number "05" is combined with the main steel structure. The code represented by this component is "01-02-04-05".

[0059] The Huace Navigation i70RTK device was used to measure the positioning points of 32 main columns. The latitude and longitude information was recorded using the WGS84 coordinate system, and a spatial index database was established, with spatial coordinates corresponding one-to-one with component codes.

[0060] 3. Lightweight Model and GIS Integration The cross-line steel structure canopy model underwent lightweighting while maintaining the integrity of component attribute information. Using the Cesium 3D geospatial engine, a spatial database was established within a GIS platform. Component coordinates were matched with the city's geographic base map, enabling the canopy model to achieve spatial positioning and orientation overlap with the real world within the geospatial scene. Combined with IoT sensing data collection, real-time inspection and collection of canopy status information was added to the canopy model, allowing the virtual canopy model to reflect the actual state of the steel structure canopy in the real world. This provides maintenance and management personnel with a more intuitive and comprehensive understanding of the canopy's status.

[0061] This application example ultimately forms a digital twin containing 4,862 independent components with a spatial positioning error of less than 15 mm, providing a precise data foundation for full lifecycle management.

[0062] Application Example 2: Example of application of warning levels and handling measures: A steel structure canopy spanning the tracks of a high-speed rail hub station (within the no-fly zone) is 420 meters long and covers 6 tracks.

[0063] Step 1: Based on the completed digital twin model of the cross-line steel structure canopy in Application Example 1, the inspection personnel use a mobile terminal to inspect the canopy components within the platform area according to the inspection route. If defects are found in the components, the corresponding components are selected in the digital twin model, and photos of the defective components are taken and uploaded until the inspection is completed. By matching the digital twin model with the actual components one by one through component codes, the accuracy rate of defect location is 100%.

[0064] Step 2: Based on the established disease identification database, identify the diseases in the captured images. After manual confirmation, obtain the disease results and warning levels for the corresponding components. Step 3: Based on the warning level, the system will issue corresponding alerts. For a yellow warning, the system will immediately push an alert message to administrators and maintenance personnel; for orange and red warnings, the system will issue strong alerts such as ringing on the mobile devices of administrators and maintenance personnel, requiring immediate action. The warning response time can be reduced from 2 hours to less than 15 minutes.

[0065] Step 4: For defective components, after manual confirmation, the defect identification has been completed through the defect database. Based on the specific defect type, the knowledge graph system provides the corresponding repair plan and pushes it to the maintenance personnel. This upgrades from manual dispatch to system push, and includes a repair plan, effectively improving the utilization rate and efficiency of maintenance resources.

[0066] Step 5: After the maintenance personnel arrive at the designated location and complete the maintenance according to the maintenance plan, they upload photos of the repaired components. After the management personnel review and confirm the repairs, the system removes the defective state of the component and restores it to a healthy state, thus completing the closed loop of the inspection process.

[0067] Through the above process, the intelligent inspection solution for railway overpass steel structure canopies has solved the three major problems of "difficulty in locating, slow response, and misallocation of resources" in the treatment of defects in no-fly zones, thus ensuring the safety of railway operations.

[0068] Application Example 3: Creation of disease database 1. Data Acquisition: 20,000 images of steel structure defects across railway lines were obtained through a survey of the railway bureau. The image resolution was no less than 1920×1080, covering different lighting conditions (200-1000 Lux) and viewing angles (top view, side view, and bottom view). If the number of images for some defect types was insufficient, spatial transformation or color perturbation strategies were used for data augmentation, such as random horizontal flipping, rotation (±15°), and brightness adjustment (±15%). Contrast-limited adaptive histogram equalization was performed using OpenCV to enhance local texture features. 2. Disease Image Annotation: Construct a UNet segmentation model to annotate diseased areas (such as rusted areas) at the pixel level and generate JSON format annotation files; the annotation team corrects the boundaries of the pre-annotated results and adds three-level classification labels; 3. Disease data storage: Deploy a Hadoop HDFS cluster to store the original images; store disease metadata through a MySQL 8.0 relational database.

[0069] Application Example 4: Deep learning model training example 1: 1. Dataset partitioning: Representative disease images are extracted from the disease database as training and validation sets. The training set contains no fewer than 10,000 images, which are sampled evenly according to the secondary classification. The validation set contains no fewer than 3,000 images and includes 20% adversarial examples (simulating rain, fog, and partial occlusion scenarios).

[0070] 2. Model Training: Use the training set data to select a deep learning model such as CNN for training, and use cosine annealing scheduling (epoch length = 10 epochs) for a total of 100 training epochs.

[0071] 3. Accuracy Validation: Use validation set data to validate the accuracy of the trained model and verify whether the classification accuracy meets 95%.

[0072] Application Example 5: Deep learning model training example 2: 1. Dataset partitioning: Representative disease images are extracted from the disease database as training and validation sets. The training set contains no fewer than 10,000 images, which are sampled evenly according to the secondary classification. The validation set contains no fewer than 3,000 images and includes 20% adversarial examples (simulating rain, fog, and partial occlusion scenarios).

[0073] 2. Model Training: Use the training set data to select a deep learning model such as CNN for training, and use cosine annealing scheduling (epoch length = 10 epochs) for a total of 100 training epochs.

[0074] 3. Accuracy Validation: Use validation set data to validate the accuracy of the trained model and verify whether the classification accuracy meets 95%.

[0075] Application Example 6: The present invention also provides a computer device comprising: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0076] Application Example 7: The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described method embodiments.

[0077] Application Example 8: The present invention also provides an information data processing terminal, which is used to provide a user input interface to implement the steps in the above method embodiments when executed on an electronic device. The information data processing terminal is not limited to mobile phones, computers, and switches.

[0078] Application Example 9: The present invention also provides a server that, when executed on an electronic device, provides a user input interface to implement the steps described in the above method embodiments.

[0079] Application Example 10: The present invention provides a computer program product that, when run on an electronic device, enables the electronic device to implement the steps described in the various method embodiments above.

[0080] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0081] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention and within the spirit and principles of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent inspection of cross-line steel structure canopies used in no-fly zones, characterized in that, Includes the following steps: S1: Forming a digital twin model A model of the cross-line steel structure canopy was established, and a component coding system was created. The latitude and longitude of the canopy columns measured on site were used as positioning reference points to obtain the latitude and longitude of each component and correspond one-to-one with the component code. The cross-line steel structure canopy model was then lightweighted and loaded into the GIS platform to form a digital twin model. S2: Establish a three-level classification system for diseases Establish a three-level classification system for diseases, set thresholds for early warning modules, and create a disease database, a disease maintenance knowledge base, and a disease early warning database. The disease database collects original images of damaged components of cross-line steel structures from various railway bureaus, performs image enhancement on the damaged component images, uses the UNet segmentation model to pre-annotate the damaged areas of the original damaged component images after image enhancement, generates a JSON format annotation file, and adds three-level classification labels for the diseases: the first level is structural diseases, the second level is technical sub-categories, and the third level is damage features. S3: Dual-location disease marker The system initializes and sets the inspection route and sends it to the mobile terminal. The real-time location of the mobile terminal is then compared with the inspection route of the inspection personnel holding the mobile terminal. The system receives images of damaged components uploaded by inspection personnel via mobile terminals, and uses the component codes and the latitude and longitude coordinates of the canopy columns to perform dual positioning and marking of the defects on the digital twin model until the inspection personnel complete the inspection. S4: Generate a repair work order and match a repair plan The system generates maintenance work orders based on the safety level of the defects and calls the Neo4j knowledge graph to match maintenance solutions; S5: Eliminate disease markers After the maintenance personnel verify the repair, they remove the corresponding component defect markers from the digital twin model. S6: Regularly generate inspection reports The system summarizes inspection, maintenance, and disease early warning records based on the disease database, disease maintenance knowledge base, and disease early warning database, and regularly outputs inspection reports.

2. The intelligent inspection method for cross-line steel structure canopies used in no-fly zone environments according to claim 1, characterized in that: In step S1, the component coding hierarchy is coded according to "structural classification - structural type - component type - component number", and each component code is unique.

3. The intelligent inspection method for cross-line steel structure canopies used in no-fly zone environments according to claim 1, characterized in that: In step S2, the threshold values ​​for the three-level early warning module are set according to the three-level disease classification system, and the early warning levels are as follows: Yellow alert: Defects have occurred in non-critical components; Orange alert: Major components have suffered damage, but structural safety has not yet been affected; Red alert: Major components have suffered damage, affecting structural safety.

4. The intelligent inspection method for cross-line steel structure canopies used in no-fly zone environments according to claim 3, characterized in that: The early warning module includes setting up an early warning push mechanism and message templates. It allows users to set early warning messages based on the early warning level and select the corresponding message template for push notification.

5. The intelligent inspection method for cross-line steel structure canopies used in no-fly zone environments according to claim 1, characterized in that: In step S2, after creating the disease database, a distributed file system is used to store the original disease component images, and a relational database is used to store structured disease metadata information.

6. The intelligent inspection method for cross-line steel structure canopies used in no-fly zone environments according to claim 1, characterized in that: In step S3, the mobile terminal positioning method uses GNSS, UWB or Bluetooth positioning technology, and selects auxiliary gyroscope sensors and accelerometer sensors to sense the dynamics of the mobile terminal.

7. The intelligent inspection method for cross-line steel structure canopies used in no-fly zone environments according to claim 1, characterized in that: In step S4, the Neo4j knowledge graph is a "disease-cause-treatment" relationship network constructed using Neo4j in the disease repair database. The network contains various disease repair processes and supports similarity matching retrieval.

8. An intelligent inspection system for cross-line steel structure canopies used in no-fly zones, comprising an intelligent inspection method for cross-line steel structure canopies used in no-fly zones, characterized in that, Includes the following modules: Modeling Generation and Conversion Module (1): Used to establish a cross-line steel structure canopy model, establish a component coding system, use the latitude and longitude of the canopy columns measured on site as the positioning reference point, obtain the latitude and longitude of each component, correspond one-to-one with the component code, and load the cross-line steel structure canopy model into the GIS platform after lightweighting to form a digital twin model. Disease data management module (2): used to establish a three-level disease classification system, set the threshold of the early warning module, and create a disease database, a disease maintenance knowledge base and a disease early warning database; Mobile inspection interaction module (3): It is used to initialize the system, set the inspection route and send it to the mobile terminal. It compares the real-time positioning of the mobile terminal with the inspection route of the inspector holding the mobile terminal. It receives the images of the defective components uploaded by the inspector through the mobile terminal. It performs dual positioning of the defect marking on the digital twin model through the component code and the latitude and longitude coordinates of the canopy column until the inspector completes the inspection. Disease analysis and processing module (4): used by the system to generate maintenance work orders based on the disease safety level and call Neo4j knowledge graph to match maintenance solutions; Defect Marker Removal Module (5): Used by maintenance personnel to remove corresponding component defect marks on the digital twin model after verification of maintenance. Analysis and output report module (6): The system summarizes inspection, maintenance and disease early warning records based on the disease database, disease maintenance knowledge base and disease early warning database, and outputs inspection reports regularly.

9. A computer device, comprising a memory and a processor, characterized in that: The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the intelligent inspection method for cross-line steel structure canopies for no-fly zone environments as described in any one of claims 1-7.

Images