Methods for Railway Inspection Route Planning and Image Acquisition with Added Multi-Level Classification Labels

By constructing a multi-level classification label tree and a quality assessment mechanism, the problem of image classification and analysis in multi-system scenarios during UAV inspection was solved, achieving an efficient image acquisition and analysis process and improving the accuracy of equipment recognition and the quality of acquisition.

CN123111306APending Publication Date: 2026-11-17SIGNAL & COMM RES INST OF CHINA ACAD OF RAILWAY SCI +3
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611642903.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-22
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing drone inspection technologies lack structured classification and storage methods in multi-system inspection scenarios, resulting in high model training costs, simple waypoint action planning, difficulty in image classification, lack of quality feedback mechanisms for repeated shooting, cumbersome analysis processes, and additional computing power consumed by equipment recognition.

Method used

A multi-level classification label tree is constructed, which is bound to the photo-taking action and pre-associated with a dedicated analysis model. Through quality assessment and correction mechanisms, images can be quickly classified and filtered, ensuring image clarity, and analysis models are automatically assigned according to labels.

Benefits of technology

It enables rapid image classification and filtering in multi-system inspection scenarios, reduces model training costs, improves image acquisition quality and analysis efficiency, reduces repeated shooting and manual inspection, and enhances equipment recognition accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123111306A_ABST
    Figure CN123111306A_ABST
Patent Text Reader

Abstract

This invention discloses a railway inspection route planning and image acquisition method with added multi-level classification labels. It constructs a label tree composed of multi-level classification labels, supporting various inspection scenarios to address the limitations of these scenarios. By binding multi-level classification labels to waypoint actions, the method enables rapid location and editing of photographic actions, solving the problem of difficulty in distinguishing between multiple waypoint shots. After mapping the multi-level classification labels to image files, media routing and image file filtering allow for rapid image file classification, addressing the difficulty of media file classification and facilitating hierarchical display and extraction of image files of different categories. Automatic image file annotation improves annotation efficiency, facilitating subsequent AI model training and addressing the issue of high model training costs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of railway equipment inspection technology, and in particular to a railway inspection route planning and image acquisition method with additional multi-level classification labels. Background Technology

[0002] Railway equipment inspection is a fundamental part of daily railway line maintenance. In the past, inspections were primarily conducted manually, which was inefficient and susceptible to weather and terrain limitations. In recent years, with the development and maturation of drone technology, drone-based inspections have gradually emerged. Drone inspections offer advantages such as faster access to the inspection area, less susceptibility to terrain, and a high-altitude, bird's-eye view that reduces the need for manual climbing of towers. For example, Chinese invention patent application CN121883783A, "A Real-Time Image Enhancement and Exposure Method and System for Drone Inspection," uses an onboard edge computing platform to process video streams in real time, optimizing image quality to improve AI recognition accuracy. Another example is Chinese invention patent application CN121764190A, "An Emergency Inspection Method for Rail Transit Drones Based on Fiber Optic Monitoring and Signaling Systems," which combines fiber optic monitoring and train operation data to direct drones for emergency inspections of abnormal events.

[0003] However, current drone inspections have relatively simple waypoint action planning, mostly limited to simple actions like hovering and taking pictures. When editing waypoints, each photo-taking action can only be distinguished manually through image preview. Furthermore, the image data generated after the inspection task lacks structured classification and storage methods. Besides the solutions disclosed in the two patent applications mentioned above, there are currently two main types of solutions: Option 1: Unmanned Aerial Vehicle (UAV) AI Inspection System. The core technology of this solution is the deep integration of AI (Artificial Intelligence) visual recognition technology, enabling fully automated intelligent inspections of signal boxes, switch machines, turnouts, tracks, and cable route protection zones. Through computer vision and machine learning algorithms, and based on intelligent analysis of massive amounts of inspection data, the system can accurately identify and quickly locate equipment defects, achieving efficient closed-loop management of fault "discovery—alarm—handling."

[0004] However, its shortcomings are: (1) Limited inspection scenarios: The system only performs inspections on signal equipment of a single system (e.g., electrical, power supply, or civil engineering systems), and fails to connect the inspection fields of multiple systems such as electrical, power supply, and civil engineering; (2) Difficult media classification: For the massive number of equipment photos taken during inspection tasks, the classification cost is high, and it is impossible to quickly filter and locate them; (3) High model training cost: The waypoint photo instruction does not have a tag tree attached, and the association between photos and equipment depends on post-event AI recognition or coordinate approximate matching, rather than pre-flight structured injection, which consumes additional computing power.

[0005] Option 2: Drone + AI Low-Altitude Inspection and Control Platform. The core technical solution is: Addressing the linear distribution of subway protection zones, a strip-shaped flight path is adopted. By optimizing flight altitude and camera focal length, a single flight can completely cover the entire protection zone. Furthermore, with the help of AI algorithms, the drone can quickly locate and collect evidence of dangerous sources intruding into the protection zone.

[0006] However, its drawbacks are: (1) The waypoint motion planning is simple: the motion at the waypoint is limited to taking a picture, recording a video, (1) The hovering level does not have a hierarchical classification label field for the equipment ledger on the waypoint, which makes it impossible to quickly classify and locate the captured images; (2) The AI ​​image recognition process is cumbersome: For the equipment images returned by the drone, it is necessary to consume additional computing power to identify and locate the equipment type before identifying whether the equipment is normal; (3) The waypoint shooting action has low distinguishability: For the case of multiple shooting tasks at a single waypoint, it is only possible to distinguish the specific equipment being shot through image preview, and there is no classification label to help quickly identify the image content; (4) The same equipment (shooting position) usually needs to be repeatedly shot in multiple inspection cycles. In the existing solutions, the waypoint acquisition parameters are generally determined once before the flight, and there is a lack of a collection parameter feedback and correction mechanism based on the quality of the acquired images: When a shooting has quality problems such as blurry focus, it can usually only be found by manual inspection after the task is completed, and then the entire route is re-flyed or reworked afterward. It is impossible to make targeted remedies for specific equipment; at the same time, there is a lack of pre-established organizational association between the inspection images and the required analysis models. Post-flight analysis often requires the images to be identified as general equipment first, and then the corresponding analysis models are assigned, which increases the computing power and processing flow. The aforementioned problems restrict the acquisition quality, organizational efficiency, and accuracy of subsequent intelligent analysis of drone inspection images. Summary of the Invention

[0007] The purpose of this invention is to provide a railway inspection route planning and image acquisition method with additional multi-level classification labels. The multi-level classification labels throughout the entire process of "route planning - image acquisition - quality assurance - post-flight analysis" are applicable to various inspection scenarios such as power supply, track maintenance, and signaling. It can also directly complete the basic organization of media classification and image screening at the acquisition end, and ensure clear and comparable images from multiple periods through focus quality assessment and directional correction re-acquisition. Furthermore, it can automatically assign dedicated analysis models according to labels at the analysis end to solve various problems in the background technology mentioned above.

[0008] The objective of this invention is achieved through the following technical solution: A method for railway inspection route planning and image acquisition with added multi-level classification labels, including: Construct a tag tree consisting of multi-level category tags, which cover tags corresponding to different inspection scenarios. For each waypoint, if the waypoint action is a photo-taking action, it is bound to a multi-level classification label on the label tree, and a dedicated analysis model is pre-associated with the multi-level classification label to form a flight mission driven by the multi-level classification label. When a drone performs a flight mission, if the waypoint action is to take pictures, it will automatically capture images, store the correspondence between image files and multi-level classification labels, and temporarily store image files to obtain the first image associated with the multi-level classification labels. A quality assessment is performed on the first batch of images. If the quality assessment requirements are not met, the shooting action is located based on the correspondence between the image file and the multi-level classification label, and the information to be confirmed is output. In response to the confirmation operation of the information to be confirmed, the acquisition parameters of the photo-taking action are revised based on the quality assessment results. When the flight mission is executed again, multi-point continuous shooting is performed in combination with the revised acquisition parameters. The qualified frame is selected as the valid image of the corresponding photo-taking action and fixed through quality assessment. After the flight mission is completed, based on the dedicated analysis model identifier associated with the multi-level classification labels, the fixed image files are routed to the dedicated analysis model according to the multi-level classification labels, and batch-called to execute the analysis task.

[0009] As can be seen from the technical solution provided by the present invention, a tag tree composed of multi-level classification tags is constructed, which can support various inspection scenarios such as power supply, engineering, and electrical engineering, thereby solving the problem of "limited inspection scenarios"; the multi-level classification tags are bound to the photo-taking action, and the photo-taking action can be quickly located and modified and edited through the multi-level classification tags, thereby solving the problem of "simple waypoint action planning and low distinguishability of waypoint photo-taking actions"; the multi-level classification tags are mapped to image files and fixed, and the classification of image files can be quickly completed through media routing and image file filtering, which is convenient for layered display and extraction of image files of different categories, thereby solving the problem of "difficulty in media file classification and cumbersome AI image recognition process"; the focus quality of the first batch of images is evaluated and targeted correction and re-sampling are performed to ensure the clarity and comparability of multiple batches of images under the same tag, thereby solving the problem of "image quality defects need to be manually checked one by one and the entire line is re-flyed and reworked"; the tags are pre-associated and automatically assigned to dedicated analysis models, so that images can be analyzed by the corresponding model without the need for general equipment recognition, and the image file annotation is pre-filled, thereby solving the problem of "high model training and analysis costs". Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating a railway inspection route planning and image acquisition method with additional multi-level classification labels, provided as an embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0013] First, the following explanations are provided for the terms that may be used in this article: The terms “comprising,” “including,” “containing,” “having,” or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, “comprising a technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction conditions, processing conditions, parameter, algorithm, signal, data, product or article of manufacture, etc.)” should be interpreted as including not only the expressly listed technical feature element, but also other technical feature elements that are not expressly listed and are well-known in the art.

[0014] The following is a detailed description of a railway inspection route planning and image acquisition method with additional multi-level classification labels provided by this invention. Contents not described in detail in the embodiments of this invention are prior art known to those skilled in the art. Where specific conditions are not specified in the embodiments of this invention, they shall be performed according to conventional conditions in the art or conditions recommended by the manufacturer. Reagents or instruments used in the embodiments of this invention, unless otherwise specified by the manufacturer, are all conventional products that can be purchased commercially.

[0015] Example 1 like Figure 1 The diagram shown is a flowchart of a railway inspection route planning and image acquisition method with additional multi-level classification labels provided by an embodiment of the present invention, which mainly includes the following steps: Step 1: Construct a tag tree consisting of multi-level category tags.

[0016] In this embodiment of the invention, multi-level classification tags can be obtained from the railway asset management system, equipment ledger, or drawing import system, or multi-level classification tags can be determined according to user configuration, and a tag tree can be constructed using the multi-level classification tags; the multi-level classification tags include: line, specialty, section or mileage, pole, component, and sub-component; wherein: line refers to railway line; specialty is power supply, track maintenance, or signaling; pole is pole number or kilometer marker; component refers to the component of pole.

[0017] Preferably, the tag trees corresponding to different projects are independent of each other, and the tag trees can be modified after creation, including: adding new multi-level category tags or deleting existing multi-level category tags.

[0018] Step 2: Bind tag nodes.

[0019] In this embodiment of the invention, during the route planning stage, waypoints and waypoint actions are configured according to the route conditions. For each waypoint, if the waypoint action is a photo-taking action, it is bound to a multi-level classification label on the label tree, and a dedicated analysis model identifier is pre-associated with the multi-level classification label, forming a flight task driven by the multi-level classification label to complete the route planning. Specifically, one or more waypoint actions are added to each waypoint. If the waypoint action in the waypoint is a photo-taking action, a unique multi-level classification label is explicitly bound, and binding content including trigger rules, label writing template, and target position and attitude constraints is configured, as well as one or more dedicated analysis model identifiers are pre-associated with the multi-level classification label, forming a flight task driven by the multi-level classification label; wherein, the label writing template contains a sequence of label fields set based on the label tree. Thus, the device identity, acquisition method, and output data structure of the image corresponding to each photo-taking action in the task are known before acquisition.

[0020] In this embodiment of the invention, multiple waypoints are configured according to the route conditions, and corresponding actions can be set for each waypoint, such as hovering, setting the flight yaw angle, setting the pitch angle, and various photography actions. The specific waypoint and waypoint action configuration methods can be determined based on existing technologies and specific route conditions, and this invention does not impose specific limitations.

[0021] For each photo-taking action, a unique multi-level category label is explicitly bound: for example, if the first waypoint action at the first waypoint is a photo-taking action, the multi-level category label "xx section / power supply / xx section / tower / insulator / end attachment point" is explicitly bound; if the second photo-taking action at the first waypoint is also a photo-taking action, the multi-level category label "xx section / power supply / xx section / tower / insulator / outer string" is explicitly bound; if the first waypoint action at the second waypoint is a photo-taking action, the multi-level category label "xx section / power supply / xx section / tower / insulator / inner string" is explicitly bound, and so on. This allows for the categorization of equipment such as "insulators" and photos taken at waypoints.

[0022] For waypoint actions that are not photo-taking actions (such as yaw angle settings, pitch angle settings, etc.), multi-level category tags are not bound.

[0023] In this embodiment of the invention, when planning a flight route, the photo-taking action under the waypoint is bound to a multi-level classification label. This means that when the flight route is executed, the image corresponding to the photo-taking action has a corresponding multi-level classification label. Since the image can be automatically classified based on the multi-level classification label, there is no need to call the classification model. The granularity of the classification depends on the granularity of the label tree, thereby avoiding the problem of a large amount of wasted computing resources.

[0024] In this embodiment of the invention, the triggering rule is used to control the automatic image acquisition method, that is: according to the triggering rule, images are acquired by using single-spectrum automatic photography or timed continuous shooting, or by using dual-spectrum automatic photography or timed continuous shooting; the tag writing template contains a tag field sequence based on the tag tree setting. After automatic image acquisition, the image file and the correspondence between multi-level classification tags are stored based on the tag writing template; the target position and attitude constraints are acquisition parameters. The target position includes: waypoint global coordinates or coordinates relative to the tower; the attitude constraints include any one or any combination of the following information: aircraft yaw angle, gimbal pitch angle, zoom magnification.

[0025] Step 3: Initial image acquisition and label mapping.

[0026] In this embodiment of the invention, when the UAV performs a flight mission according to the planned route, if the waypoint action is a photo-taking action, it automatically acquires images, stores the correspondence between image files and multi-level classification labels, and obtains the initial image associated with the multi-level classification labels. Specifically: when performing a flight mission, the UAV flies according to the planned route, and after reaching a waypoint, it performs the corresponding waypoint action; if the waypoint action is a photo-taking action, it automatically acquires images, writes them to a template according to the labels, stores the correspondence between image files and multi-level classification labels in the database, and temporarily stores the image files in the storage medium (local server), thus obtaining the initial image associated with the multi-level classification labels. Since the acquisition parameters, device identity, and output structure of each waypoint and each photo-taking action are already clear during the route planning stage, the acquired photos can be automatically categorized according to different tasks and routes, and the category, location, and specific name can be quickly located based on their multi-level classification labels.

[0027] In this embodiment of the invention, storing the correspondence between image files and multi-level classification labels in the database according to the label writing template includes: the label field sequence based on the label tree set in the label writing template, combined with the multi-level classification labels corresponding to the shooting action to determine the value of each field in the label field sequence and store it in the database, wherein the value of each field represents the correspondence between the image file obtained by automatically acquiring images and the multi-level classification labels.

[0028] In this embodiment of the invention, the image file is temporarily stored in the storage medium according to the tag writing template in the following manner: After automatically acquiring images, temporarily save the image files to the storage medium using any of the following methods: Method 1: Based on the tag field sequence set by the tag tree contained in the tag writing template, multi-level classification tags are combined into the file name of the image file, so that the image file itself carries complete classification identity information; Method 2: Based on the tag field sequence set by the tag tree contained in the tag writing template, the multi-level category tags are converted into a multi-level directory structure and the image files are stored in a nested manner according to the hierarchy; Method 3: Use a flat storage method, that is, store all image files in the same directory; each image file also generates an accompanying file in JSON or XMP format with the same name, which records the multi-level classification label information involved in the image file; where JSON is a text data exchange format and XMP is a metadata format.

[0029] Preferably, in the third method, the flat storage method includes: naming the image files using the first method, and then storing the named image files in a unified directory.

[0030] Step 4: Quality assessment and positioning.

[0031] In this embodiment of the invention, a quality assessment is performed on the initial images. If the quality assessment requirements are met, the corresponding image file is saved. If the quality assessment requirements are not met, the corresponding shooting action is located based on the correspondence between the image file and the multi-level classification labels, and confirmation information is output. Specifically, an image quality detection model is invoked to perform a quality assessment on the initial images to obtain the focus quality judgment result and confidence level of the initial images. The focus quality judgment threshold and confidence level threshold are pre-configured and allow for subsequent adjustments. If the focus quality judgment result is lower than the focus quality judgment threshold (i.e., there is a certain degree of blurring), or the confidence level is lower than the confidence level threshold, the corresponding shooting action is located based on the correspondence between the image file and the multi-level classification labels, and confirmation information is output for operator confirmation. Because the image files carry multi-level classification labels, the attribution of quality problems can be accurately pinpointed to the specific device (label) and its shooting action, eliminating the need for manual searching of images along the entire flight path.

[0032] In this embodiment of the invention, the image quality detection model is an artificial intelligence model, and existing models can be selected to implement it according to the actual situation. This invention does not impose any restrictions.

[0033] Step 5: Correction and re-sampling.

[0034] In this embodiment of the invention, in response to the confirmation operation of the information to be confirmed, the acquisition parameters of the located photographing action are revised based on the quality assessment results. These acquisition parameters are the target position and posture constraints mentioned above. At this time, the goal of clear shooting can be achieved by adjusting the target position and / or posture constraints. Specifically, the revision strategy and magnitude of the acquisition parameters can be judged and adjusted based on the focus quality judgment results (degree of blurriness), confidence level, and the on-site conditions.

[0035] In this embodiment of the invention, when performing a flight mission again, multi-point continuous shooting is performed on the revised photographing action: that is, within a preset neighborhood of the target pose of the photographing action, the gimbal angle is adjusted slightly according to the neighborhood range and adjustment step size set by experience, and multiple frames of images are continuously acquired and written into a template according to the labels. The correspondence between the image files and multi-level classification labels is stored in the database, and the image files are temporarily stored in the storage medium. The focus quality of each frame obtained by multi-point continuous shooting is evaluated, and the frame that passes the re-inspection is determined as the valid image of the photographing action, and the valid image is fixed to the storage medium. Here, any of the methods one to three mentioned above can also be used. If there is still no qualified frame after re-inspection, the photographing action is marked as waiting for manual re-shooting. When a photographing action that has not been located is executed again, it is acquired with the original parameters, without adding extra flight burden.

[0036] Step 6: Automatic post-flight assignment.

[0037] In this embodiment of the invention, after the flight mission is completed, the fixed image files are automatically routed to the corresponding dedicated analysis model according to the dedicated analysis model identifier associated with the multi-level classification labels, and are called in batches to execute the analysis task, so that the image can determine the applicable analysis model without having to go through general equipment recognition.

[0038] In this embodiment of the invention, since the multi-level classification labels indicate the device type and analysis requirements of the image file before flight, the image file does not need to undergo general device identification to determine the applicable analysis model. Each dedicated analysis model only needs to make a normal / abnormal judgment based on the device and defect type corresponding to its label, and a relatively lightweight model can be used, thereby reducing computational overhead and improving analysis accuracy. The aforementioned dedicated analysis models can adopt visual analysis models commonly used in the field, and are selected according to the device type and analysis task corresponding to their multi-level classification labels.

[0039] To more clearly demonstrate the technical solution and its effects provided by the present invention, the method provided by the embodiments of the present invention will be described in detail below with reference to specific examples.

[0040] This invention provides a method for railway inspection route planning and image acquisition with added multi-level classification labels. By adding preset multi-level classification labels to image files acquired at different times along the same route and with the same photographing action, it enables rapid image differentiation and comparison, and can solve the following problems: (1) To solve the problem of “difficulty in distinguishing multiple waypoints” in the existing technology, when planning the route, each waypoint is pre-associated with multi-level classification tags. Each shooting action can be quickly located and quickly modified and edited through the tags.

[0041] (2) To solve the problem of “difficulty in classifying media files” in the existing technology, after the inspection task is completed, the image file is attached with multi-level classification labels. The category of the image can be quickly located through the multi-level classification labels, and the location information, specific name, etc. are also corresponding to the multi-level classification labels. Thus, the location and name of the image file can also be located.

[0042] (3) To solve the problem of "difficulty in horizontal comparison of equipment" in the existing technology, the media folder generated by the task record can be used to retrieve and compare images of the same equipment across tasks through tags, so as to quickly identify the degree of equipment degradation. Among them, the task record is the history of the execution of tasks. Each time a task is executed, a flight path folder is generated, which stores all the image files captured by the flight path.

[0043] (4) Solve the problem of "high cost of recognition model" in existing technologies: By injecting the correspondence between waypoint actions and multi-level classification labels in a structured manner, the correspondence between image files and multi-level classification labels can be obtained after the route is executed. The subsequent artificial intelligence model does not need to classify and recognize the image files, but only needs to identify whether the device is normal.

[0044] The method provided by this invention is mainly as follows: 1. Tag tree construction.

[0045] The system can import data from railway asset management systems, equipment ledgers, or drawings, or users can add configurations directly at the system front end to form multi-level classification tags. For example: Line (e.g., XX power supply section XX section) → Specialty (power supply / engineering / signaling) → Section / mileage → Pole (pole number / kilometer marker) → Component (insulator / cantilever / fittings / disconnecting switch…) → Sub-component (outer string / inner string / end attachment point of No. 1 insulator…).

[0046] Each tag node can carry business attributes such as pole number, mileage, mounting surface (large side / small side), set of design photo positions (relative coordinates / yaw and pitch, expected distance), and historical version pointer.

[0047] The tag tree supports storing and displaying tags by project, and tag trees created for multiple projects do not interfere with each other. Saved tag trees can be modified, including adding child tags, deleting child tags, exporting, and importing.

[0048] It should be noted that the number of levels (6 levels) of the multi-level classification labels provided here, as well as the specific labels for each level, are only examples and can be adjusted according to the actual application.

[0049] 2. Tag node binding.

[0050] In this embodiment of the invention, during the route planning stage, waypoints are configured according to the route conditions, and one or more waypoint actions are added to each waypoint. These waypoint actions include both photographing and non-photographing actions. Each photographing action is bound to a multi-level classification label, and each multi-level classification label corresponds to a node in the label tree. Thus, label nodes are indirectly bound to waypoints, and each waypoint can be indirectly bound to multiple different label nodes. Binding content is configured for each photographing action, and one or more dedicated analysis model identifiers are pre-associated to the multi-level classification labels, forming a label-driven flight mission. Therefore, the device identity, acquisition method, and output data structure of the image corresponding to each photographing action in the mission are known before acquisition.

[0051] Waypoint action bindings must include at least: 1) Target position and attitude constraints: waypoint global coordinates (RTK) / coordinates relative to the tower (based on point cloud or preset offset), aircraft yaw angle, gimbal pitch angle, zoom ratio, etc.

[0052] 2) Triggering rules: The aircraft automatically takes pictures or takes pictures in timed bursts when it reaches the location, or takes pictures in dual-spectrum (visible light + infrared) automatically or in timed bursts.

[0053] 3) Label writing template: Define the fields to be filled in the file / metadata output of waypoint actions (line / specialty / pole number / component / subcomponent / surface / acquisition time / coordinates / task ID / serial number).

[0054] This generates a multi-level classification label-driven flight mission, which essentially reduces the flight control trajectory to the carrier layer and elevates the label node sampling sequence to the scheduling layer.

[0055] 3. Initial image acquisition and label mapping.

[0056] When a drone performs a mission, reaches a waypoint, and meets the image acquisition trigger conditions, it will perform the following actions: Capture images of the preview area: visible light / infrared photographs; Multi-level classification tag information is written synchronously during storage medium writing, for example (selecting one or a combination): ① File naming rules: {task_id}_{timestamp}_{L1}_{L2}_{L3}_{L4}_{L5}_{L6}_raw.jpg; ② Directory-level archiving: {line} / {major} / {section} / {pole} / {component} / {sub} / IMG_xxxx.jpg; ③ Tile the file + accompanying JSON / XMP tags: {line,major,pole,component,sub,side,gps_rtk,timestamp,task_id,shot_index}.

[0057] Where task_id is the flight mission name, timestamp is the timestamp (mission execution time), L1-L6: refers to the ordered hierarchical structure of multi-level classification labels. In actual application, it can contain at least one or more, which is different from the 6-level classification labels mentioned earlier. The 6-level classification labels in this example are as follows: L1 is the line (e.g., XX Passenger Dedicated Line / XX Main Line / Up and Down), L2 is the specialty (e.g., Power Supply Specialty / Engineer Specialty / Electrical Engineering Specialty), L3 is the mileage or tower positioning mark (e.g., XXX km marker / #003 tower / anchor joint ID), L4 is the structural location partition (e.g., A-end cantilever area / B-end cantilever area / positive feeder area), L5 is the component (e.g., rod insulator / suspension insulator string / cantilever support / positioner), L6 is the sub-component (e.g., No. 1 insulator / No. 2 insulator / outer side / inner side). "line" represents the line name; "major" represents the specialty (power supply, electrical engineering, track maintenance); "section" represents the section information; "pole" represents the pole number; "component" represents the component name; and "sub" represents the sub-component name. This section provides several different representations of multi-level classification labels. "raw.jpg", "IMG_xxx.jpg", and "shot_index" are associated with the final image capture and can be understood as the names of the image files. The information preceding them serves as examples of multi-level classification label information.

[0058] 4. Quality assessment and positioning.

[0059] A quality assessment is performed on the initial images. An image quality detection model is invoked to obtain the focus quality judgment result and confidence level for each image. Various thresholds can be pre-configured and adjusted subsequently. Based on the corresponding records of image files, shooting actions, and multi-level classification labels, images deemed unqualified are located to their corresponding shooting actions, and confirmation information is output. Detection results with low confidence levels are also included in the confirmation information for operator verification. Because the images carry multi-level classification labels, the attribution of quality problems can be precisely pinpointed to specific devices (labels) and their shooting actions, eliminating the need for manual searching of every image along the entire flight path. If the image is qualified (meets the quality assessment requirements), the corresponding image file is saved.

[0060] 5. Correction and re-sampling.

[0061] In response to the confirmation operation of the information to be confirmed, the acquisition parameters of the located shooting action are revised. The acquisition parameters include at least one of the shooting position, gimbal attitude, zoom ratio and focus distance. The revision strategy and magnitude can be judged by experienced operators based on the degree of blur and the on-site situation.

[0062] When performing a flight mission again, multi-point continuous shooting is performed on the revised photographic action: within a preset neighborhood of the target pose of the photographic action, the gimbal angle is adjusted slightly according to an empirically set neighborhood range and adjustment step size to continuously acquire multiple frames of images; each frame obtained from the multi-point continuous shooting is evaluated for focus quality, and the frame that passes the re-inspection is determined as the valid image of the photographic action. If no valid frame is found after the re-inspection, the photographic action is marked as requiring manual re-shooting. Photographic actions that are not located are captured with the original parameters when executed again, without adding extra flight burden.

[0063] In this embodiment of the invention, through the closed loop of initial acquisition—quality assessment—directional correction—multi-point continuous re-acquisition, the clarity of images obtained from the same multi-level classification label (i.e., the same shooting position of the same equipment) in different task cycles is guaranteed, and the shooting geometry remains consistent. These images can collectively constitute multiple images under the same multi-level classification label, providing consistent and usable input for subsequent cross-period comparisons and other analyses. It should be noted that the above correction and re-acquisition are not unconditionally repeated automatic cycles: revisions are triggered only after the quality assessment results are confirmed by the operator, avoiding unfounded and repeated modifications to flight path parameters due to the uncertainty of a single model judgment.

[0064] 6. Automatic dispatching after flight.

[0065] After the flight mission is completed, based on the dedicated analysis model identifier associated with the multi-level classification labels, the fixed image files are automatically routed to the corresponding dedicated analysis model according to the multi-level classification labels and called in batches to execute the analysis task, so that the image can determine the applicable analysis model without having to go through general equipment recognition.

[0066] Since the multi-level classification labels specify the device type and analysis requirements of the image files before flight, the applicable analysis model can be determined without first undergoing general device identification. Each specialized analysis model only needs to determine normal / abnormal status based on the device and defect type corresponding to its label, allowing for the use of relatively lightweight models, thereby reducing computational overhead and improving analysis accuracy. The aforementioned specialized analysis models can adopt commonly used visual analysis models in this field, and be selected based on the device type and analysis task corresponding to their respective multi-level classification labels.

[0067] Based on the above scheme, the following can be achieved: (1) Media classification routing: Image files are read from the storage medium and automatically classified and routed according to the multi-level classification labels of the image files. No additional processing is required to route the image to the correct category. (2) Label tree view: Image files of different categories are displayed in the user interface in a collapsible label tree view. By selecting a specified level, the image files of the corresponding level can be filtered out, making image location change from "searching" to "navigation". (3) Label pre-filling: The labeling tool reads the image files from the storage medium, extracts their multi-level classification labels, and automatically fills them into the labeling interface as the initial labeling results. The labelers only need to supplement information such as normal / defect type / severity, thereby improving the labeling efficiency.

[0068] For example: an image of a certain tower component is routed to a bird nest identification model to perform bird nest hazard analysis; multiple valid images across task cycles under the same multi-level classification label are automatically grouped to form a time-series image group for the device / shooting position, which is then handed over to a difference comparison model to perform cross-period analysis; images of components such as insulators can be routed to the corresponding defect identification model.

[0069] The above-described solution provided by the embodiments of the present invention mainly achieves the following beneficial effects: (1) A tag tree composed of multi-level classification tags was constructed, which can support various inspection scenarios such as power supply, engineering, and electrical engineering.

[0070] (2) Bind multi-level category tags to the photo-taking action. Through multi-level category tags, the photo-taking action can be quickly located and quickly modified and edited.

[0071] (3) Map multi-level classification labels to image files and solidify them. Image file classification can be completed quickly through media routing and image file filtering, which facilitates hierarchical display and extraction of image files of different categories.

[0072] (4) Conduct a focus quality assessment on the first phase of images and make targeted corrections and re-sampling to ensure the clarity and comparability of multiple phases of images under the same label.

[0073] (5) Pre-associate with labels and automatically assign dedicated analysis models so that images can be analyzed by the corresponding models without the need for general equipment recognition, and pre-fill with image file annotations.

[0074] Through the above description of the embodiments, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the above embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.), including several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0075] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A method for railway inspection route planning and image acquisition with additional multi-level classification labels, characterized in that, include: Construct a tag tree consisting of multi-level category tags, which cover tags corresponding to different inspection scenarios. During the route planning phase, waypoints and waypoint actions are configured according to the route conditions. For each waypoint, if the waypoint action is a photo-taking action, it is bound to the multi-level classification tag on the tag tree and the multi-level classification tag is pre-associated with a dedicated analysis model identifier, forming a flight mission driven by the multi-level classification tag and completing the route planning. When a drone performs a flight mission, if the waypoint action is to take pictures, it will automatically capture images, store the correspondence between image files and multi-level classification labels, and temporarily store image files to obtain the first image associated with the multi-level classification labels. A quality assessment is performed on the first batch of images. If the quality assessment requirements are met, the corresponding image file is saved. If the quality assessment requirements are not met, the shooting action is located based on the correspondence between the image file and the multi-level classification label, and the information to be confirmed is output. In response to the confirmation operation of the information to be confirmed, the acquisition parameters of the photo-taking action are revised based on the quality assessment results. When the flight mission is executed again, multi-point continuous shooting is performed in combination with the revised acquisition parameters. The qualified frame is selected as the valid image of the corresponding photo-taking action and fixed through quality assessment. After the flight mission is completed, based on the dedicated analysis model identifier associated with the multi-level classification labels, the fixed image files are routed to the dedicated analysis model according to the multi-level classification labels, and batch-called to execute the analysis task.

2. The railway inspection route planning and image acquisition method with additional multi-level classification labels according to claim 1, characterized in that, The construction of the tag tree consisting of multi-level classification tags includes: Obtain multi-level classification labels from railway asset management systems, equipment ledgers, or drawing import systems, or determine multi-level classification labels based on user configuration, and construct a label tree using the multi-level classification labels; The multi-level classification labels include: line, specialty, section or mileage, pole / tower, component and sub-component; where: line refers to railway line; specialty is power supply, track maintenance or signaling; pole / tower is pole number or kilometer marker; component refers to the components of pole / tower.

3. The railway inspection route planning and image acquisition method with additional multi-level classification labels according to claim 1, characterized in that, Also includes: The tag trees for different projects are independent of each other, and the tag trees can be modified after creation, including adding new multi-level category tags or deleting existing multi-level category tags.

4. The railway inspection route planning and image acquisition method with additional multi-level classification labels according to claim 1, characterized in that, If the waypoint action is a photo-taking action, then after binding with the multi-level category tags on the tag tree, it also includes: configuration containing trigger rules, tag writing templates, and binding content for target position and attitude constraints; Among them, the triggering rules are used to control the automatic image acquisition method, that is: according to the triggering rules, images are acquired by using single-spectrum automatic photography or timed continuous shooting, or by using dual-spectrum automatic photography or timed continuous shooting. The tag writing template contains a sequence of tag fields based on the tag tree. After automatically acquiring images, the correspondence between image files and multi-level classification tags is stored in the database based on the tag writing template. The target position and attitude constraints are collected parameters. The target position includes: global coordinates of the waypoint or coordinates relative to the tower; the attitude constraints include any one or any combination of the following information: aircraft yaw angle, gimbal pitch angle, zoom ratio.

5. A method for railway inspection route planning and image acquisition with additional multi-level classification labels as described in claim 4, characterized in that, Based on the label writing template, storing the correspondence between image files and multi-level classification labels in the database includes: determining the values ​​of each field in the label field sequence by combining the multi-level classification labels corresponding to the photo-taking action, and storing them in the database, wherein the value of each field represents the correspondence between the image file obtained by automatically acquiring images and the multi-level classification labels.

6. The railway inspection route planning and image acquisition method with additional multi-level classification labels according to claim 4, characterized in that, Also includes: Based on the labels written into the template, the image files are temporarily stored or permanently saved to the storage medium, as follows: Method 1: Based on the tag field sequence set by the tag tree contained in the tag writing template, multi-level classification tags are combined into the file name of the image file, so that the image file itself carries complete classification identity information; Method 2: Based on the tag field sequence set by the tag tree contained in the tag writing template, the multi-level category tags are converted into a multi-level directory structure and the image files are stored in a nested manner according to the hierarchy; Method 3: Use a flat storage method, that is, store all image files in the same directory; each image file also generates an accompanying file in JSON or XMP format with the same name, which records the multi-level classification label information involved in the image file; where JSON is a text data exchange format and XMP is a metadata format.

7. A method for railway inspection route planning and image acquisition with additional multi-level classification labels as described in claim 6, characterized in that, In the third method, the flat storage method includes: naming the image files using the first method, and then storing the named image files in a unified directory.

8. The railway inspection route planning and image acquisition method with additional multi-level classification labels according to claim 1, characterized in that, A quality assessment is performed on the initial images. If the quality assessment requirements are not met, the capturing action is determined based on the correspondence between the image files and multi-level classification labels. The image quality detection model is invoked to perform a quality assessment on the first image to obtain the focus quality judgment result and confidence level of the first image. If the focus quality judgment result is lower than the preset focus quality judgment threshold, or the confidence level is lower than the preset confidence threshold, the corresponding shooting action is located based on the correspondence between the image file and the multi-level classification label.

9. A method for railway inspection route planning and image acquisition with additional multi-level classification labels as described in claim 1, characterized in that, The revision of the acquisition parameters for the photo-taking action based on the quality assessment results includes: Based on the focus quality judgment and confidence level in the quality assessment results, as well as the on-site conditions, the strategy and magnitude of the acquisition parameters are judged and adjusted; among them, the acquisition parameters include target position and attitude constraints.

Citation Information

Patent Citations

  • Rail traffic unmanned aerial vehicle emergency patrol method based on optical fiber monitoring and signal system

    CN121764190A

  • Real-time image enhancement and intelligent exposure method and system for unmanned aerial vehicle inspection

    CN121883783A