Railway defect multi-level identification method and system based on vehicle-mounted collection and server cooperation

Abstract

The application discloses a vehicle-mounted collection and server cooperative railway defect multi-level identification method and system, relates to the technical field of railway defect detection, and solves the technical problems that the existing 4C detection lacks reasonable division of labor between the vehicle end and the server and an identification mechanism, resulting in large data transmission pressure, insufficient identification capability and result return delay. The application comprises the following steps: collecting inspection videos or images and obtaining metadata; performing pretreatment and quality screening; performing lightweight pre-screening on the image frames to be uploaded or candidate ROIs, and encapsulating the metadata as a task data packet for uploading; scheduling an identification task on the server side, performing multi-level progressive identification according to a railway component hierarchical knowledge graph and an identification chain configuration library until termination; recording the ROI cropping offset, scaling ratio and homogeneous transformation matrix at each level, and mapping the final defect coordinates to the original 4C image coordinate system level by level; associating the identification results at each level, generating a result and returning the result, and the application has the advantages of low transmission load, accurate coordinate mapping, closed-loop traceability and the like.

Description

Technical Field

[0001] This invention belongs to the field of railway defect detection technology, specifically involving a multi-level railway defect identification method and system that combines onboard data acquisition with server collaboration. Background Technology

[0002] The 4C inspection vehicle runs along the railway line and collects high-resolution inspection videos or images of the catenary and related railway components. These images cover fine-grained structures such as catenary wires, contact wires, positioning devices, droppers, connecting hardware, bolts, nuts, pins, clamps, and insulators, providing an important data foundation for the condition inspection and defect verification of railway power supply equipment.

[0003] Existing 4C image defect recognition technologies can be broadly categorized into two types. The first type is a pure vehicle-side recognition solution, where the recognition model is deployed on the inspection vehicle to complete all detection tasks. This type of solution is limited by the computing power, power consumption, heat dissipation, and operational stability of the vehicle-side equipment, making it difficult to deploy complex models on the vehicle to complete all fine-grained defect recognition. Furthermore, the vehicle-side recognition results lack server-side verification. The second type is a pure server-side recognition solution, where all raw video is directly uploaded to a server for centralized processing. This type of solution has advantages in recognition accuracy, but it faces significant communication bandwidth pressure, server storage pressure, and recognition latency due to the massive amounts of data generated by high-resolution 4C images and continuous long-distance acquisition, making it difficult to meet near real-time online detection requirements.

[0004] Furthermore, some existing 4C image defect recognition solutions still rely on offline analysis or manual verification after acquisition. This results in a significant time lag between defect detection and on-site handling. Manual verification requires searching for small defects in a large number of high-resolution images, which is labor-intensive and susceptible to fatigue, lighting, and background interference, leading to missed detections, false detections, or inaccurate defect localization. Existing automatic recognition solutions, such as single-level end-to-end detection, fixed two-level cascaded detection, or fixed-scale detection, struggle to balance efficiency and accuracy for railway components with varying structural complexity. Existing 4C image component recognition solutions often focus on component detection in single-frame images or fixed two-stage recognition, failing to adequately consider the data upload pressure, on-vehicle computing power limitations, and server-side multi-task scheduling issues during online 4C vehicle acquisition.

[0005] Therefore, there is an urgent need to design a method for identifying railway component defects through 4C on-board online data acquisition and server-based collaborative processing to solve the above problems. Summary of the Invention

[0006] This invention discloses a multi-level railway defect identification method and system that combines onboard data acquisition with server collaboration. It aims to solve the technical problems of existing 4C detection, which lacks a reasonable division of labor and identification mechanism between the onboard and server sides, resulting in high data transmission pressure, insufficient identification capability, and delayed result return.

[0007] To solve the aforementioned technical problems, the present invention adopts the following technical solution:

[0008] A multi-level railway defect identification method based on vehicle-mounted data acquisition and server collaboration includes the following steps:

[0009] S1: Collect inspection videos or images along the railway line via the 4C vehicle terminal, and obtain one or more metadata corresponding to the videos or images;

[0010] S2: Preprocess and screen the image quality of the acquired video or images via the 4C vehicle terminal;

[0011] S3: The 4C vehicle end performs lightweight pre-screening on the image frame, video clip or candidate ROI to be uploaded based on the image quality screening results or the lightweight pre-screening model of the 4C vehicle end, and encapsulates it with the corresponding metadata into a detection task data package and uploads it to the server.

[0012] S4: The server receives the detection task data packet and schedules the identification task. It performs configurable multi-level progressive identification based on the railway component hierarchical knowledge graph and the identification chain configuration library until the termination condition is met.

[0013] S5: The server records the offset, scaling ratio and homogeneous transformation matrix of each level of ROI cropping, and maps the final level outlier points, detection boxes or segmentation masks back to the original 4C image coordinate system level by level.

[0014] S6: The server will hierarchically associate the identification results at each level to generate a structured defect result that includes defect type, original map coordinates, line metadata and comprehensive score, and send it back to the 4C vehicle end or the backend front end.

[0015] In this invention, the 4C vehicle-side primarily performs tasks such as data acquisition, preprocessing, quality screening, and data uploading, while the server-side performs complex multi-level progressive identification, coordinate mapping, and result generation. The vehicle-side pre-screening results serve only as input candidates for the server-side fine identification and are not directly used as the final defect judgment result. The server-side can still perform multi-level progressive identification of the original image, candidate ROIs, or video clips based on the identification chain configuration library. The railway component hierarchical knowledge graph describes the parent-child hierarchical relationships, optional identification paths, and geometric constraints between equipment areas, target components, sub-components, and anomaly types. The identification chain configuration library is used to determine the next level of identification task, model type, ROI expansion rules, and termination conditions during server-side runtime. It can be implemented using table structures, graph databases, JSON configuration files, or other structured data methods. In each level of identification, the server-side selects the model type based on the current level's task objective. Different levels can use different models, and detection, segmentation, and classification models can be combined within the same identification chain. Each level of recognition is performed within the current level's ROI image, mask region, or classification interest region. The server records the corresponding cropping offset, scaling ratio, image padding amount, expansion coefficient, image transformation parameters, or homogeneous transformation matrix for each level, forming a cropping parameter chain. Through these steps, an online detection closed loop is established, comprising lightweight pre-screening at the vehicle end, progressive recognition via server-side map routing, reversible ROI parameter chain coordinate mapping, and backhaul of structured defect results. This enables precise recognition of components with varying complexity using recognition chains of different lengths and model types.

[0016] Furthermore, in S1, the metadata includes the acquisition time, line number, line section, kilometer marker, pole number, support number, camera number, image frame number, vehicle speed, positioning coordinates, shooting direction, image width and height, task number, and vehicle-side device number.

[0017] After adopting this technical solution, it should be noted that the metadata is uploaded along with the image frame or ROI and a binding relationship is established with the recognition result on the server side: After receiving the detection task data packet, the server side establishes a binding relationship between the metadata and the image data, candidate ROI and recognition result, so that the defect result not only includes the defect category, but also the line location information that can be used for on-site positioning. Each defect result can be traced back to the original image, vehicle-end equipment, acquisition line location and server-side model version.

[0018] Furthermore, in S2, the image quality screening includes: calculating one or more of the following: brightness index, contrast index, sharpness index, motion blur degree, occlusion ratio, and effective target area ratio; when the image quality is lower than a preset threshold, the corresponding image frame is marked as a low-quality frame, and its upload, recognition, or result output priority is reduced.

[0019] After adopting this technical solution, it should be noted that brightness can be evaluated using grayscale mean and grayscale distribution; sharpness can be evaluated using gradient, Laplacian response, or edge density; motion blur can be evaluated using directional blur index; and occlusion ratio can be estimated using the ratio of the effective target area to the total image area. The vehicle-side calculates an image quality score based on these indicators. When the quality score is below a threshold, the system marks the frame as a low-quality frame, thereby removing invalid or low-quality image frames before data upload and reducing redundant computational load on the server side.

[0020] Furthermore, the lightweight pre-screening described in S3 includes one or more of the following: fixed time interval frame extraction, fixed mileage interval frame extraction, key frame extraction, low-quality frame removal, candidate components based on the lightweight pre-screening model, suspected defect ROI, or suspected defect mask extraction.

[0021] When adopting this technical solution, it's important to note that lightweight pre-screening refers to performing only a preliminary screening with low computational overhead on the vehicle side, without conducting complete fine-grained defect identification. The lightweight pre-screening model can be an object detection model, semantic segmentation model, instance segmentation model, or image classification model. It is only used on the vehicle side to roughly locate candidate parts or suspected abnormal areas, without directly outputting the final defect judgment. This reduces the upload of invalid images and avoids placing all complex fine-grained identification tasks on the vehicle side, while the server side remains responsible for the final multi-level progressive fine identification, coordinate mapping, and result generation.

[0022] Furthermore, the detection task data packet described in S3 includes one or more of the following: task number, frame number, image data or image address, candidate ROI, acquisition time, line number, kilometer marker, pole number, camera number, vehicle speed, image quality score, upload status, and verification information.

[0023] When adopting this technical solution, it should be noted that the detection task data packet is the basic unit of data transmission between the vehicle and the server. Specific detection task data can be combined and configured according to actual communication bandwidth and task requirements. The server establishes a task record for each detection task data packet and writes the result status, transmission status, and verification status to the database after the recognition result is generated. When the communication link is abnormal, the 4C vehicle caches unuploaded or unconfirmed data packets locally and performs breakpoint resumption or retransmission after communication is restored, thereby avoiding image data loss, duplicate recognition, or result omission.

[0024] Furthermore, S4 specifically includes:

[0025] S41: The server schedules identification tasks based on one or more of the following: line segment, upload time, defect risk level, image quality score, vehicle-end device number, or task priority.

[0026] S42: Perform first-level positioning and identification on the received image to locate the device area, target component area, or suspected defect area;

[0027] S43: When there are multiple candidate next-level paths in the current level identification result, calculate the routing score and select the path based on the parent confidence, current level confidence, image quality score, component risk level and parent-child geometric constraint matching degree;

[0028] S44: Based on the current level detection box or segmentation mask bounding box, clip the ROI according to the expansion factor, and record the clipping offset, scaling ratio, expansion factor, parent number, current level category and confidence level;

[0029] S45: Perform the next level of identification on the trimmed ROI and determine whether the termination condition is met; if it is met, stop the progression and output the current level result; if it is not met, return the current level identification result to S43 for route scoring.

[0030] After adopting this technical solution, it should be noted that in scenarios where multiple vehicles simultaneously upload data, the server can establish independent cache queues for different 4C vehicle terminals and perform parallel scheduling based on server-side GPU, CPU, and memory resources and model loading status. The first-level localization and recognition outputs detection boxes, categories, and confidence scores. The routing score is a comprehensive evaluation score for each candidate path. The parent category corresponding to the current-level recognition result may have multiple candidate child nodes in the knowledge graph. The server calculates the routing score based on the parent confidence score, current-level confidence score, image quality score, component risk level, and parent-child geometric constraint matching degree, and selects the next-level detection, segmentation, or classification model that meets preset conditions to continue recognition. The length of the recognition chain is determined based on the structural complexity of the target component and runtime termination conditions, and can be two, three, four, or more levels.

[0031] Preferably, the termination conditions in S45 include: the current level identification result has identified the final defect type; the current level target belongs to a preset terminal sub-component; the current level confidence is lower than the continuing identification threshold; the current level ROI size or mask area is less than the minimum identification threshold; the current identification chain level has reached the preset maximum number of levels; the current level category does not have a corresponding next-level identification model; and the current level identification result is inconsistent with the preset parent-child hierarchy. When one or more of the above conditions are met, the termination condition is reached.

[0032] After adopting this technical solution, it should be noted that the termination judgment checks whether the current level result meets the termination condition after each level of recognition is completed. If the termination condition is not met, the next level ROI is trimmed and the next level model is called; if the termination condition is met, the final level defect recognition result is output. When a certain level of recognition model fails to detect the next level result and the parent target belongs to a preset critical safety component, the server retains the corresponding recognition chain as a suspected defect result, calculates the suspected defect score by combining the parent confidence score, image quality score, component risk level, and line metadata, and outputs it for manual review.

[0033] As a preferred embodiment, the step-by-step mapping in S5 is specifically as follows: the cropping offset, scaling ratio, and homogeneous transformation matrix recorded at each level of ROI are used to step-by-step map the coordinates of the final level outlier, detection box, or segmentation mask from the current level image coordinate system to the previous level image coordinate system; if there is a rotation or perspective transformation at a certain level, the inverse matrix of the homogeneous transformation matrix of that level is used to complete the coordinate inversion mapping.

[0034] When adopting this technical solution, it should be noted that in the multi-level progressive recognition process, each level of ROI is cropped, scaled, and possibly padded or transformed from the previous level image before being input into the next level model. If the k-th level ROI is derived from the (k-1)-th level image with a cropping offset... Cut and scale Scaling and image fill Then, the coordinates of a point mapped from the k-th level image coordinate system to the (k-1)-th level image coordinate system satisfy the following formula:

[0035]

[0036]

[0037] When a certain level involves rotation, flipping, perspective correction, or other image transformations, the corresponding homogeneous transformation matrix is ​​used. inverse matrix The final-level outlier points, detection boxes, or segmentation masks are mapped step by step to the original 4C image coordinate system. Through the above reversible ROI parameter chain, it is ensured that the coordinates of defects after multi-level cropping, scaling, and transformation can be accurately located in the original 4C image.

[0038] Preferably, in S6, the comprehensive score is obtained by weighted summation of the identification confidence scores at each level and adding it to the hierarchical consistency score.

[0039] Overall score meets:

[0040]

[0041] In the formula, For the first The confidence level of the detection, segmentation, or classification results. Here, H represents the confidence weight for the k-th level, and H is the hierarchical consistency score. For hierarchical consistency weights; ≥0, mu≥0, , , The hierarchical consistency score H satisfies: Among them, when the k-th level identification result and the (k-1)-th level identification result conform to the preset parent-child hierarchical relationship in the railway component hierarchical knowledge graph or identification chain configuration library, Take 1, otherwise Take 0, or take a continuous value between 0 and 1 based on the degree of matching between the parent and child levels and the degree of matching between geometric constraints.

[0042] A multi-level railway defect identification system that integrates onboard data acquisition and server collaboration includes:

[0043] The 4C vehicle-mounted acquisition module is used to acquire high-resolution inspection videos or images and obtain metadata.

[0044] The vehicle-mounted preprocessing module is used for preprocessing and quality screening of the acquired images;

[0045] The vehicle-mounted pre-screening module is used to screen image frames, video clips, or candidate ROIs to be uploaded;

[0046] The data encapsulation and upload module is used to encapsulate image data and metadata into detection task data packages and upload them.

[0047] The server task scheduling and identification module is used to receive and parse detection task data packets, establish task queues and schedule identification tasks, and perform multi-level progressive identification based on the railway component hierarchical knowledge graph and identification chain configuration library.

[0048] The coordinate mapping module is used to record the cropping offset, scaling ratio and homogeneous transformation matrix of each level of ROI, and to map the final abnormal coordinates back to the original 4C image coordinate system step by step.

[0049] The result generation and feedback module is used to generate structured defect results and send them back to the 4C vehicle end or the backend front end.

[0050] Preferably, the system further includes a recognition chain configuration library, a dynamic graph routing unit, a model management unit, a result storage database, a front-end display unit, and a manual review unit. The recognition chain configuration library is used to maintain component parent-child relationships, task types at each level, recognition models at each level, ROI expansion rules, routing scoring rules, and termination conditions. The dynamic graph routing unit is used to determine the next recognition level and corresponding model type based on the current level recognition result, parent-child hierarchical relationship, image quality score, component risk level, and geometric constraint matching degree. The model management unit is used to load, switch, and record the versions of the server-side object detection model, semantic segmentation model, instance segmentation model, and image classification model. The result storage database is used to store image data, metadata, recognition results, feedback status, and review records. The front-end display unit is used to display a defect list, detection boxes, segmentation masks, and annotation maps. The manual review unit is used to confirm, modify, or reject defective results and write the review results back to the result storage database.

[0051] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0052] 1. The multi-level railway defect identification method and system provided by the present invention, which combines vehicle-mounted data acquisition and server collaboration, removes low-quality frames and extracts candidate ROIs before data upload by using vehicle-side preprocessing and lightweight pre-screening mechanisms. This effectively reduces the communication bandwidth pressure and server storage burden caused by full data upload, while avoiding the defect that it is difficult to complete complex fine-grained identification under the condition of limited computing power on the vehicle side.

[0053] 2. The multi-level railway defect identification method and system provided by the present invention, which combines vehicle-mounted acquisition and server collaboration, can configure a multi-level progressive identification mechanism through a railway component hierarchical knowledge graph and identification chain configuration library. This enables components with different structural complexities to adopt identification chains of different lengths and model types, thus balancing identification efficiency with the refined detection requirements of complex components.

[0054] 3. The multi-level railway defect identification method provided by the present invention, which combines vehicle-mounted acquisition and server collaboration, achieves accurate mapping of the final-level defect coordinates to the original 4C image coordinate system by fully recording and progressively inversely mapping the ROI cropping offset, scaling ratio, image filling amount and homogeneous transformation matrix at each level, which facilitates manual review and on-site positioning.

[0055] 4. The multi-level railway defect identification method that combines vehicle-mounted data acquisition and server collaboration provided by this invention binds defect results with metadata such as acquisition time, line section, kilometer marker, pole number, camera number, and frame number, enabling the identification results to directly serve defect statistics, maintenance dispatch, and line status tracking.

[0056] 5. The multi-level railway defect identification method that combines vehicle-mounted data acquisition and server collaboration provided by this invention generates a defect list, original map annotation, local ROI magnified map, and structured defect results, and transmits the results back to the 4C vehicle or the backend front end, forming an online closed loop of vehicle-mounted data acquisition, server identification, result transmission, manual review, and model optimization. Attached Figure Description

[0057] The present invention will be described by way of example and with reference to the accompanying drawings, wherein:

[0058] Figure 1 This is an overall flowchart of the present invention;

[0059] Figure 2 This is a framework diagram of the 4C vehicle-side and server collaborative processing system of the present invention;

[0060] Figure 3 This is a flowchart of the data acquisition, preprocessing, and uploading process for the 4C vehicle-mounted terminal of this invention.

[0061] Figure 4 This is a flowchart of the server-side multi-level progressive recognition process of the present invention;

[0062] Figure 5 This is a schematic diagram of the multi-level ROI clipping parameter recording and coordinate step-by-step mapping of the present invention;

[0063] Figure 6 This is a flowchart illustrating the process of defect data feedback, display, and manual review in this invention.

[0064] Figure 7 The data transmission volume and the data reduction rate relative to full upload are for different upload strategies of this invention;

[0065] Figure 8 This is the retention rate and transmission volume trade-off curve formed by the low threshold change in the dual threshold upload strategy of this invention;

[0066] Figure 9 This is a theoretical batch transmission time diagram for each upload strategy under different available uplink bandwidths according to the present invention;

[0067] Figure 10 Histogram and cumulative distribution of coordinate regression error of multi-level parameter chain under final-level integer pixel quantization condition of this invention;

[0068] Figure 11 This is a comparison of the equivalent pixel widths of small target defect features after full-frame scaling and step-by-step ROI cropping in this invention.

[0069] Figure 12 This is a graph showing the final success rate of the task corresponding to different maximum retransmission counts when the single independent failure probability of the present invention is 3%. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the embodiments and accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and marked in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0071] In the description of the embodiments of this application, it should be noted that the terms "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. In addition, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0072] The following is combined Figures 1-12 The present invention will be described in detail below.

[0073] Example 1

[0074] A multi-level railway defect identification method that combines onboard data acquisition with server collaboration, such as... Figure 1 As shown, it includes the following steps:

[0075] S1: Collect inspection videos or images along the railway line via the 4C vehicle terminal, and obtain one or more metadata corresponding to the videos or images;

[0076] S2: Preprocess and screen the image quality of the acquired video or images via the 4C vehicle terminal;

[0077] S3: The 4C vehicle-side performs lightweight pre-screening on the image frames, video clips, or candidate ROIs to be uploaded based on the image quality screening results or the lightweight pre-screening model, and encapsulates them with the corresponding metadata into a detection task data package, which is then uploaded to the server.

[0078] S4: The server receives the detection task data packet and schedules the identification task. It performs configurable multi-level progressive identification based on the railway component hierarchical knowledge graph and the identification chain configuration library until the termination condition is met.

[0079] S5: The server records the offset, scaling ratio and homogeneous transformation matrix of each level of ROI cropping, and maps the final level outlier points, detection boxes or segmentation masks back to the original 4C image coordinate system level by level.

[0080] S6: The server-side performs hierarchical association of the identification results at each level, generating a structured defect result containing defect type, original map coordinates, line metadata, and comprehensive score, and then sends it back to the 4C vehicle terminal or the backend front-end. Specifically in this embodiment: the server-side, based on the parent result number corresponding to each level of identification result, the preset parent-child category relationship in the railway component hierarchical knowledge graph, and the spatial inclusion relationship after coordinate mapping, associates the equipment area, target component, sub-component, and anomaly type into a defect identification chain pointing from the parent node to the child node, generating a structured defect result containing defect type, component level, original map coordinates, line metadata, and comprehensive score, and then sends it back to the 4C vehicle terminal or the backend front-end.

[0081] In this embodiment, the 4C vehicle-side primarily performs tasks such as data acquisition, preprocessing, quality screening, and data uploading, while the server-side performs complex multi-level progressive recognition, coordinate mapping, and result generation. The vehicle-side pre-screening results only serve as input candidates for the server-side fine recognition and are not directly used as the final defect judgment results. The server-side can still perform multi-level progressive recognition on the original image, candidate ROI, or video clips based on the recognition chain configuration library. The railway component hierarchical knowledge graph is used to describe the parent-child hierarchical relationships, optional recognition paths, and geometric constraints between equipment areas, target components, sub-components, and anomaly types. The recognition chain configuration library is used to determine the next level of recognition task, model type, ROI expansion rules, and termination conditions during server-side runtime. It can be implemented using table structures, graph databases, JSON configuration files, or other structured data methods. The server-side selects the model type based on the current level task objective in each level of recognition. Each level of recognition is performed in the current level ROI image, mask region, or classification interest region. The server-side records the corresponding cropping offset, scaling ratio, image fill amount, expansion coefficient, image transformation parameters, or homogeneous transformation matrix for each level, forming a cropping parameter chain. Through the above steps, an online detection closed loop was established, which includes lightweight pre-screening at the vehicle end, progressive identification of the map routing at the server end, reversible ROI parameter chain coordinate mapping, and back transmission of structured defect results. This enabled precise identification of components with different complexities using identification chains of different lengths and model types.

[0082] Example 2

[0083] The difference between this embodiment and embodiment 1 is that in S1, the metadata includes the collection time, line number, line section, kilometer marker, pole number, support number, camera number, image frame number, vehicle speed, positioning coordinates, shooting direction, image width and height, task number, and vehicle-side device number.

[0084] After adopting this technical solution, it should be noted that the 4C vehicle-mounted acquisition module continuously acquires high-resolution inspection images during the operation of the train or inspection vehicle. The vehicle-mounted computing unit generates a frame-level number for each image frame and records the acquisition time, line number, kilometer marker, pole number, camera number, vehicle speed, and positioning information corresponding to that frame. The metadata is uploaded along with the image frame or ROI and a binding relationship is established with the recognition result on the server side. After receiving the inspection task data packet, the server side establishes a binding relationship between the metadata and the image data, candidate ROI, and recognition result, so that the defect result not only includes the defect category, but also the line location information that can be used for on-site positioning. Each defect result can be traced back to the original image, vehicle-mounted equipment, acquisition line location, and server-side model version.

[0085] Example 3

[0086] The difference between this embodiment and embodiment 2 is that, in S2, as... Figure 3 As shown, the image quality screening includes: calculating one or more of the following: brightness index, contrast index, sharpness index, motion blur degree, occlusion ratio, and effective target area ratio; when the image quality is lower than a preset threshold, the corresponding image frame is marked as a low-quality frame, and its upload, recognition, or result output priority is reduced.

[0087] In this embodiment, the comprehensive quality score Q is used as the image quality score. The brightness index, contrast index, sharpness index, motion blur degree (motion stability), occlusion ratio, and effective target ratio are all normalized to 0-1, as shown in the following formula:

[0088]

[0089] In the formula, B is the brightness score, C is the contrast score, S is the sharpness score, and M is the degree of motion blur (i.e., motion stability score). , E represents the occlusion percentage, and E represents the score for the percentage of the effective target area. , , , , and The weights are 1, and the sum of these weights is 1. In this embodiment, the comprehensive quality threshold is 0.6. If Q > 0.6, it is an image that is usable. Otherwise, the corresponding image frame is marked as a low-quality frame, and its priority for uploading, recognition, or result output is reduced. For frames with better image quality, the system generates image frames, video clips, or candidate ROIs to be uploaded according to the key frame strategy or lightweight model pre-screening strategy.

[0090] The vehicle-side preprocessing module first performs basic corrections on the original image. For camera distortion, the system can perform distortion correction based on camera calibration parameters; for uneven lighting, the system can perform brightness correction or contrast enhancement; for images with high noise, the system can perform filtering or noise reduction. These preprocessing steps do not change the binding relationship between the image and metadata, and the preprocessing parameters can also be written into frame-level records. In this embodiment, brightness is evaluated using grayscale mean and grayscale distribution; sharpness is evaluated using gradient, Laplacian response, or edge density; motion blur is evaluated using directional blur index; and the occlusion ratio is estimated by the ratio of the effective target area to the entire frame image area. All of the above utilize existing image processing techniques. Regarding the calculation of the contrast index, considering that complex backgrounds such as the sky and tracks may cause interference, the effective target area is selected for calculation. Internal calculations first convert the image to grayscale. The pixel value range is 0-255. The effective target area is either the fixed contact wire inspection zone determined by camera calibration or the candidate area of ​​railway components quickly extracted from the vehicle end. The calculation formula is as follows:

[0091]

[0092]

[0093] In the formula, In this embodiment, the global normalized root mean square contrast is used. =0.5, thus taking into account the grayscale dispersion of the entire target area, as well as the discernibility of local component textures and edges. For pixels grayscale value, The average gray level of the effective target area. The number of pixels within the effective target area. The value ranges from 0 to 1; a larger value indicates a more distinct grayscale level. For local contrast, in this embodiment, to avoid the influence of local shadows, reflections, or a small number of extreme pixels, the effective target area is divided into 4×4 sub-regions, and the local percentile contrast is calculated using the following formula:

[0094]

[0095] In the formula, , These are the 95th and 5th percentiles of the grayscale value of the m-th sub-region, respectively, from which the local contrast is obtained. for:

[0096]

[0097] In the formula, Based on the number of effective target pixels in each sub-region, in this embodiment, it satisfies the following:

[0098]

[0099] The lightweight pre-screening described in S3 includes one or more of the following: fixed time interval frame extraction, fixed mileage interval frame extraction, key frame extraction, low-quality frame removal, candidate components based on the lightweight pre-screening model, suspected defect ROI, or suspected defect mask extraction.

[0100] In this example, a lightweight pre-screening unit on the vehicle side filters candidate targets from keyframes or images to be uploaded. The upload strategy selection unit determines the type and granularity of the uploaded data based on data transmission conditions and recognition task requirements. The data transmission conditions include at least effective upload bandwidth, transmission load rate, packet loss rate, communication latency, upload queue length, and server confirmation status. The recognition task requirements include at least the risk level of the target component, the allowable processing latency, the input type required by the server-side model, whether the target's surrounding context needs to be preserved, and whether the original high-resolution image needs to be uploaded.

[0101] The vehicle-side measures the actual effective upload rate within a preset statistical time window and calculates the effective upload bandwidth using an exponential moving average. In the formula, This refers to the actual effective upload speed measured during the current statistical period. To achieve smoothed effective upload bandwidth, As a smoothing factor, it is set to 0.2 in this embodiment. The vehicle-side further calculates the transmission load rate: In the formula, This represents the average transmission rate required to upload keyframes or continuous video clips within the current statistical time window.

[0102] In this embodiment, when the transmission load rate When the bandwidth is not higher than 0.60 and the packet loss rate is less than 2%, the current bandwidth is considered sufficient, and the vehicle-side uploads key frames or continuous video clips; when the transmission load rate... If the bandwidth reaches 0.80 and remains so for 5 seconds, or if the estimated waiting time in the upload queue exceeds 2 seconds, it is determined that the current bandwidth is limited. The vehicle only uploads the candidate ROI obtained from the lightweight pre-screening model, its corresponding metadata, and the surrounding context image obtained by expanding the candidate ROI's width and height by 1.2 to 1.5 times (preferred value 1.3), and temporarily stores the corresponding original image on the vehicle. If the packet loss rate is not less than 5%, no server confirmation is received for 3 consecutive seconds, or the transmission load rate is... If the transmission load rate drops below 1.0 for 5 seconds, the communication link is considered unstable. The vehicle-mounted terminal writes ordinary task data packets to its local cache and records the task number, cache location, and upload status. For tasks involving critical safety components or high-risk suspected defects, candidate ROIs, surrounding context images, and line metadata are still uploaded first. After communication is restored, the vehicle-mounted terminal performs breakpoint resume transmission according to the task risk level and acquisition time. When the transmission load rate drops below 0.60 and the data packet loss rate is below 2% for 10 seconds, the upload of key frames or continuous video clips is resumed.

[0103] In this embodiment, lightweight pre-screening refers to performing only a preliminary screening with low computational overhead at the vehicle end, without performing complete fine-grained defect identification. The lightweight pre-screening model can be an object detection model, a semantic segmentation model, an instance segmentation model, or an image classification model. Among them, the object detection model can adopt a YOLO-type model or a DETR-type model, the semantic segmentation model can adopt a U-Net-type model, the instance segmentation model can adopt a Mask R-CNN-type model, and the image segmentation model can adopt a ResNet-type model. These are all existing models, and their structures will not be described in detail here. When the task objective is to locate the location of a device area, component area, or sub-component, the object detection model is preferred. When the task objective is to identify defects with irregular boundaries such as cracks, wear, dirt, foreign object attachment, damaged areas, or defects that require calculation of abnormal area ratios, the semantic segmentation model or the instance segmentation model is preferred. When the task objective is to determine the target status, defect category, or defect level within the ROI, the image classification model is preferred. Different models can be used at different levels, and detection, segmentation, and classification models can be combined within the same recognition chain. It is only used for rough localization of candidate parts or suspected abnormal areas on the vehicle side and does not directly output the final defect judgment. This reduces the upload of invalid images and avoids placing all complex fine-grained recognition tasks on the vehicle side, while the server side is still responsible for the final multi-level progressive fine recognition, coordinate mapping, and result generation.

[0104] The detection task data package described in S3 includes one or more of the following: task number, frame number, image data or image address, candidate ROI, acquisition time, line number, kilometer marker, pole number, camera number, vehicle speed, image quality score, upload status, and verification information.

[0105] In this embodiment, the detection task data packet may include , , , , , , , , , , , , , , , and fields such as, among which, This represents a globally unique identifier for the detection task; Indicates the 4C vehicle-side device number; Indicates the image frame or video frame number; This represents encoded image, video clip, or ROI binary data; This indicates the address or index of the corresponding image in the vehicle-side cache, server storage system, or object storage system; and These represent the width and height of the original image, respectively. Indicates the image acquisition time; Indicates the railway line number; Indicates the line mileage or kilometer marker corresponding to the data collection location; Indicates pole number or support number; Indicates the camera number used for data acquisition; This indicates the vehicle's speed at the time of data collection; Indicates image quality score; This represents the checksum of the data packet or image data used in the detection task.

[0106] This represents one or more candidate ROI records output by the vehicle-side lightweight pre-screening model. Each candidate ROI record includes the ROI number, candidate category, candidate confidence score, coordinates of the candidate ROI in the original 4C image, context spread coefficient, crop offset, scaling ratio, image padding, and ROI image data or storage address. The server uses these candidate ROI records as the initial-level records of the reversible ROI parameter chain to support the mapping of the server-side final-level defect results back to the original 4C image coordinate system.

[0107] This indicates the upload status of the detection task data packet, including one of the following: cached, waiting to upload, uploading, uploaded but not confirmed, confirmed, and upload failed. It can further record the uploaded byte offset, retransmission count, and last confirmation time. The vehicle-mounted device uploads the detection task data packet to the server via a wireless network, dedicated communication link, or local area network.

[0108] Example 4

[0109] The difference between this embodiment and the above embodiments is that, as Figure 4 S4 as shown specifically includes:

[0110] S41: The server determines the task priority based on one or more of the following information: task pre-risk level, line segment, upload time, image quality score, vehicle-end device number, and task time limit, and schedules the identification task according to the task priority.

[0111] S42: Perform first-level positioning and recognition on the received image to locate the device area, target component area, or suspected defect area;

[0112] S43: When there are multiple candidate next-level paths in the current level identification result, calculate the routing score and select the path based on the parent confidence, current level confidence, image quality score, component risk level and parent-child geometric constraint matching degree;

[0113] S44: Based on the current level detection box or segmentation mask bounding box, clip the ROI according to the expansion factor, and record the clipping offset, scaling ratio, expansion factor, parent number, current level category and confidence level;

[0114] S45: Perform the next level of identification on the trimmed ROI and determine whether the termination condition is met; if it is met, stop the progression and output the current level result; if it is not met, return the current level identification result to S43 for route scoring.

[0115] In this embodiment, the pre-risk level of task S41 differs from the final defect risk level obtained after defect identification. Before performing the first-level location identification, the server calculates the pre-risk score based on component importance, vehicle-side suspected anomaly score, line section importance, and historical anomaly data, and determines the task scheduling order accordingly. The pre-risk score satisfies the following:

[0116] In the formula, K is the component importance pre-configured in the knowledge graph, which is retrieved from the component risk configuration table; A is the suspected anomaly score output by the vehicle-side lightweight pre-screening model, which directly adopts the confidence level output by the vehicle-side lightweight model; L is the importance of the line section, which is retrieved from the line section level configuration table; H is the historical anomaly score of the same component or adjacent line location, which is calculated based on historical defect records, and all variables are normalized to 0 to 1.

[0117] when When the value is ≥0.80, the task is classified as a Level 1 risk task; when It was identified as a level-two risk task; when At that time, it was identified as a level three risk task; when When the value is less than 0.40, it is classified as a Level 4 risk task. Level 1 risk tasks take precedence over Level 2, while Level 3 and Level 4 risk tasks enter the server's identification queue.

[0118] After completing multi-level progressive identification, the server updates the final defect risk level based on component importance, defect type severity, defect apparent severity, line segment importance, and historical anomalies. For pre-defined critical component breakage, detachment, or missing critical fasteners, the server is not limited by the weighted scoring results and directly identifies them as Level 1 risk defects.

[0119] In this embodiment, the candidate next-level path in S43 refers to the directed edge from the current node to the candidate child node in the railway component hierarchical knowledge graph, and the identification task configuration bound to the directed edge. For example, when the current node is a hardware component, its candidate next-level path may include a bolt detection path, a pin detection path, and a crack segmentation path; the bolt detection path is bound to a target detection model, the pin detection path is bound to a target detection or classification model, and the crack segmentation path is bound to a semantic segmentation or instance segmentation model. For the j-th candidate path, the server calculates the routing score according to the following formula:

[0120]

[0121]

[0122]

[0123] In the formula: Confidence level of the parent recognition result; The confidence score of the candidate category corresponding to the j-th candidate path, output by the current-level model; Score the image quality of the current ROI; Risk score for the target component or defect type corresponding to the j-th path; This represents the degree of geometric constraint matching between candidate child components and their parent components. This indicates the degree of matching between the actual center location of the candidate sub-component and the preset location range in the knowledge graph; This indicates whether the area ratio between the candidate child component and the parent component is within a preset range; Let $j$ be the degree to which the candidate child component detection box is contained by the parent component detection box in the $j$-th candidate path, i.e., the spatial containment matching degree. The larger the value, the higher the proportion of the candidate child component that is contained within the detection box of the parent component; The candidate sub-component detection box corresponding to the j-th candidate path; The current parent component's detection bounding box; Indicates the area of ​​the region; The area of ​​the intersection of the two detection boxes; The area of ​​the candidate sub-part detection box is used, and all variables are normalized to 0-1. When the current-level model outputs multiple candidate categories, the confidence score of each candidate category is directly used as the area of ​​the detection box. For example, if the model outputs 0.78 for bolts, 0.65 for pins, and 0.42 for suspected crack areas for the current ROI, then the three candidate paths will use 0.78, 0.65, and 0.42 as candidate confidence levels, respectively.

[0124] In this embodiment, after retrieving the image to be identified from the task queue, the server first calls the first-level detection model to locate the equipment area, target component area, or suspected defect area in the original 4C image or the candidate image uploaded by the vehicle. The first-level detection result includes the detection box, category, confidence level, and parent number. For the first-level detection result, the server queries the identification chain configuration library. The identification chain configuration library records the parent-child hierarchical relationship between different equipment areas, target components, sub-components, and anomaly types, as well as the corresponding next-level model, input size, ROI expansion coefficient, confidence threshold, minimum ROI size threshold, and maximum number of levels. For example, a simple component can be configured as a second-level identification chain, that is, anomaly classification is performed directly after component detection; a bolt assembly can be configured as a third-level identification chain, that is, component detection, nut or pin detection, and anomaly identification; a positioning device or complex connecting hardware can be configured as a fourth-level identification chain, that is, equipment area, component, sub-component, and anomaly segmentation.

[0125] The server-side cropped the Region of Interest (ROI) image based on the current-level detection bounding box, segmentation mask bounding box, candidate region, or classification interest region, and input the ROI into the next-level model. The next-level model can be an object detection model, an image classification model, a semantic segmentation model, or an instance segmentation model. Different levels can use different models, and different components can also use different recognition chain configurations. This invention does not limit the specific model type.

[0126] For example, for small components with relatively clear bounding boxes, such as bolts, nuts, and pins, the server can use a target detection model to locate their positions; for anomalies with irregular boundaries, such as broken insulators, cracked wire clamps, contaminated contact network components, or foreign matter attachment, the server can use semantic segmentation or instance segmentation models to obtain anomaly masks; for ROIs that have been cropped to individual sub-components, such as determining whether nuts are loose, pins are missing, or whether the component is in normal condition, an image classification model can be used to output the state category and confidence level.

[0127] Within the same identification chain, model types can be combined hierarchically. For example, level one uses a target detection model to locate the positioning device area, level two uses a target detection model to locate clamps or bolt assemblies, level three uses an instance segmentation model to segment suspected crack or damaged areas, and level four uses a classification model to determine the anomaly level. The above combinations are determined by the identification chain configuration library and can be adjusted according to different lines, different component categories, or different defect types.

[0128] The server can treat the identification chain configuration library as a runtime index of the railway component hierarchical knowledge graph. When the first-level result is a certain equipment area or component category, the server does not automatically enter the preset second-level model. Instead, it first queries the candidate child nodes and candidate task types of that category in the knowledge graph, and then calculates the RouteScore by combining the current image quality, parent confidence, candidate child component risk level, parent-child bounding box position relationship, mask overlap degree, or size ratio constraints. This RouteScore is the same as... If multiple candidate paths meet the threshold, the preferred path can be executed according to the RouteScore, or multiple candidate paths can be retained and filtered through hierarchical consistency scoring at subsequent levels. If no candidate path meets the threshold, a termination condition or a suspected defect retention strategy is triggered. Therefore, this embodiment differs from the fixed secondary fastener inspection process and can form a variable-level, traceable, and reflective server-side precision identification process for multiple types of railway components.

[0129] For example, the bolt assembly can be configured as a three-level identification chain: the first level uses a target detection model to locate the bolt assembly or connecting hardware area; the second level uses a target detection model to locate the nut, pin, or anti-loosening component; the ROI expansion coefficient of the second level can be set to 1.1 to 1.5, and in this embodiment, a value of 1.3 is preferred; the third level selects a classification model or an instance segmentation model according to the sub-component type, outputs the status such as loose nut, missing pin, abnormal anti-loosening component, or suspected occlusion, and uses this status as the terminal result.

[0130] For example, the positioning device can be configured as a four-level identification chain: the first level positions the equipment area, the second level positions the locator, positioning clamp or connecting hardware, the third level positions the pins, bolts, suspected crack areas or suspected wear areas, and the fourth level calls the semantic segmentation model or instance segmentation model to output the abnormal mask according to the abnormal shape, and generates the defect type, abnormal area ratio and original image reflection coordinates after the termination condition is met.

[0131] For example, components with simple structures and stable appearances can be configured into a two-level recognition chain. The first level locates the component's ROI, and the second level uses an image classification model to determine whether it is normal, suspected abnormal, or clearly abnormal. Thus, the recognition chain of this invention is not a fixed cascaded detection, but is dynamically determined by the configuration library based on component complexity, task type, model version, and termination conditions.

[0132] Furthermore, the railway component hierarchical knowledge graph described in this embodiment is a pre-established and stored data set of railway component structural relationships and defect relationships on the server side, and its logical structure can be represented as a directed attribute graph. In this graph, V represents the set of nodes, E represents the set of directed edges, and A represents the set of node attributes and edge attributes. The knowledge graph is used to limit the next level category and recognition task that the current level recognition result is allowed to enter, and does not directly replace the image recognition models at each level. The node set V includes one or more of the following: device region nodes, target component nodes, sub-component nodes, and anomaly type nodes. The directed edge set E includes containment or composition relationship edges between device regions and target components, between target components and sub-components, defect attribution relationship edges between sub-components and anomaly types, and candidate recognition path edges used to determine the next recognition task. The attribute set A includes one or more of the following: category code, category name, node level, component risk level, whether it is an expected component, terminal flag, allowed task type, parent-child relative position range, parent-child size ratio range, spatial containment threshold, and candidate path priority. The direction of each directed edge points from the current level node to the allowed next level node or anomaly type node.

[0133] The railway component hierarchical knowledge graph can be established through the following methods: obtaining one or more of the following: railway catenary equipment structure diagrams, component composition lists, maintenance rules, defect classification rules, historical 4C inspection images, and manual annotation results; standardizing component names, defect names, and category codes from different sources, assigning unique category identifiers to the same equipment area, target component, sub-component, or anomaly type, and establishing a mapping relationship between synonyms and standard names; establishing parent-child directed edges from equipment area to target component and from target component to sub-component according to the actual composition relationship of the equipment, and establishing directed edges from sub-component to anomaly type according to the defect attribution relationship; binding the directed edges to target detection, image classification, semantic segmentation, or instance segmentation task types according to the executable recognition tasks; forming an initial knowledge graph after manual review, and configuring a version number and effective status for the knowledge graph.

[0134] For parent and child components with manually annotated bounding boxes, the parent-child geometric constraints can be further determined based on the annotation results. Let the parent bounding box be... The child detection box is Where x and y are the center coordinates of the detection box, and w and h are the width and height of the detection box, respectively, then the lateral position of the child component relative to the parent component is... Longitudinal position Width ratio Height ratio and spatial inclusion They can be determined according to the following formulas:

[0135]

[0136]

[0137]

[0138]

[0139]

[0140] For the same parent-child category combination, the above parameters of multiple labeled samples can be statistically analyzed. The interval formed by the minimum and maximum values ​​after removing outliers, the mean plus or minus a preset multiple of the standard deviation, or the interval formed by the 5th percentile to the 95th percentile, can be used as the allowable range for the relative position and size ratio of the parent and child. The spatial containment degree can be constrained according to a preset lower limit. When the number of samples is insufficient, an initial range can be set according to the equipment structure dimensions, installation location constraints, or manual experience, and updated according to the results of subsequent manual review. The resulting geometric constraints are used to calculate the degree of matching of parent-child geometric constraints, and are not used as a separate final defect judgment result.

[0141] The railway component hierarchical knowledge graph can be stored using a graph database, or it can be implemented using node tables, relation tables, and attribute tables in a relational database, or node objects, edge objects, and attribute objects in a JSON configuration file. Its storage format is not limited, as long as it allows querying candidate child nodes, candidate anomaly types, parent-child hierarchical relationships, and geometric constraints based on the current category.

[0142] The identification chain configuration library is a set of runtime configurations generated based on executable identification paths in the railway component hierarchical knowledge graph. It is used to convert category relationships in the knowledge graph into server-side executable model calls, ROI pruning, threshold judgment, and termination control rules. One identification chain configuration record corresponds to one state transition from the current level category to the next identification task. The same current level category can correspond to one or more identification chain configuration records.

[0143] In one implementation, the identification chain configuration library may include , , , , , , , , , , , , , etc., among which Configure a unique number for the identification chain to distinguish different levels or different identification branches; The parent category to which the current category belongs; for example, hardware components are the parent category of bolts. The current category configured to trigger this identification chain, such as bolt; For the next identification task corresponding to the current configuration record, such as component location, sub-component detection, defect classification, or anomaly segmentation; The model type or network architecture that performs the task, such as YOLO, DETR, ResNet, U-Net, or Mask R-CNN; The version number, weight version, or model file verification identifier of the called model; Specify the required input image size for the model, such as 640×640, 1280×1280, or 224×224;

[0144] The width and height expansion coefficients of the current ROI before it enters the next model are preferably 1.3; The minimum confidence level required for the current model results to be accepted and allowed to proceed to subsequent recognition levels;

[0145] The minimum ROI area allowed to enter the next level of recognition can be represented by pixel area or as a percentage of the parent image area; This represents the maximum number of levels that the current identification chain is allowed to advance to; This is the terminal flag; a value of 1 indicates that the current result is the final part or defect type and no further progress is allowed, while a value of 0 indicates that continued identification is permitted. Configure one or more next-level identification chains connected to the current configuration record; empty for terminal nodes.

[0146] Should the next level sub-component exist? This indicates the suspected defect type when the sub-component is not detected, such as a missing cotter pin. The server queries the above fields based on the category and parent number of the current identification result to determine whether to continue the process, which model to call, and how to trim the next layer of ROI.

[0147] The identification chain configuration library may also include , , and One or more of them. Geometric constraints used to associate corresponding parent and child categories in a knowledge graph; Used to indicate the risk level of candidate components or anomaly types; Used to indicate whether a next-level sub-component should exist in the normal structure; This indicates the type of suspected defect generated when the expected sub-component is not detected. (From the runtime identification results) Used to associate parent detection instances within the same image with parent categories in the configuration library. Distinguish between them.

[0148] The identification chain configuration library can be obtained as follows: Starting with a device region node or target component node in the knowledge graph, traverse along directed edges to sub-component nodes or abnormal type terminal nodes to obtain one or more candidate hierarchical paths; convert the directed edges between adjacent nodes in each path into identification chain configuration records; and determine the next node's output requirements. When location needs to be output, a target detection task is configured; when irregular abnormal contours need to be output, a semantic segmentation or instance segmentation task is configured; and when the state of a single sub-part needs to be determined, an image classification task is configured. A model matching the task type and category set is selected from the server-side model management unit and written into the configuration. , and .

[0149] for , and Candidate parameters can be set and compared on a pre-labeled set of verification images. You can select a threshold with a lower false positive rate or a higher comprehensive evaluation index from the candidate thresholds that meet the preset minimum recall rate. Within a preset candidate range, an expansion coefficient can be selected that makes the target coverage rate of the next layer reach the preset requirement and has a small redundant background area. For example, 1.3 is preferred within the candidate range of 1.1 to 1.5. It can be determined based on the model input size and the minimum number of effective pixels that the target should retain in the input image. This is determined based on whether the current node is a defect type node or a preset terminal sub-component node. Determined based on the outgoing edges of the current node. The depth of the corresponding candidate level path is determined. Each configuration is verified and manually reviewed before being written to the configuration repository, with the configuration version and effective date recorded.

[0150] When the server runs, it first determines the result based on the current level of recognition. and the parent recognition result Query the knowledge graph to obtain one or more candidate child nodes and candidate anomaly types originating from the current node; then, based on the corresponding directed edges or... The system queries the identification chain configuration library to obtain the task type, model version, input size, ROI expansion coefficient, confidence threshold, geometric constraints, and termination flag for each candidate path. When only one candidate path meets the criteria, the model corresponding to that path is invoked; when multiple candidate paths meet the criteria, they are sorted according to the route score described in S43, or multiple candidate paths are retained. Here, RouteScore and the aforementioned route score RSj are the same metric. 1. To reach No corresponding If no candidate path meets the conditions for continuing identification, stop the progressive identification or implement the suspected defect retention strategy.

[0151] For example, for the connecting hardware region, a directed hierarchical path can be established in the knowledge graph: connecting hardware region—bolt assembly—nut—nut loosening. The identification chain configuration library configures bolt assembly detection tasks for the connecting hardware region, nut detection tasks for the bolt assembly, and loosening status classification tasks for the nuts; the first two levels of configuration records... A value of 0 indicates a loosening status classification record. The value is 1. For example, regarding the cotter pins that should be present in a normal structure, it can be... Set it to 1, and Set to missing cotter pin; when the parent connection component is reliably detected but the next level does not detect the cotter pin, the server does not directly end the identification chain as a normal result, but generates a suspected cotter pin missing result and enters the suspected defect scoring or manual review process.

[0152] Through the above methods, the railway component hierarchical knowledge graph provides stable component structural relationships, anomaly attribution relationships, and candidate path constraints, while the identification chain configuration library provides executable parameters corresponding to each candidate path. Together on the server side, they enable configurable multi-level progressive identification of different railway components using different identification levels, model combinations, and termination conditions. When equipment structure, defect classification rules, or model versions change, adjustments can be made by updating knowledge graph nodes or directed edges and corresponding identification chain configuration records, without altering the on-vehicle data collection process or rewriting the fixed server-side identification process.

[0153] Example 5

[0154] The difference between this embodiment and the above embodiment is that the termination conditions in S45 include: the current level identification result has identified the final defect type; the current level target belongs to the preset terminal sub-component; the current level confidence is lower than the continuing identification threshold; the current level ROI size or mask area is less than the minimum identification threshold; the current identification chain level has reached the preset maximum number of levels; the current level category does not have a corresponding next-level identification model and the current level identification result is inconsistent with the preset parent-child level relationship. When one or more of the above conditions are met, the termination condition is reached.

[0155] In this embodiment, the termination decision checks whether the result of the current level meets the termination condition after each level of identification is completed. If the termination condition is not met, the next level of ROI is trimmed and the next level model is called; if the termination condition is met, the final level defect identification result is output. When a certain level of identification model fails to detect the next level result, the server first queries the identification chain configuration library. .when When the next-level sub-component is not required, the undetected result is treated as a normal termination result; when When indicating that the next level sub-component should exist, according to Generate corresponding suspected defect types and calculate suspected defect scores, wherein the suspected defect scores satisfy:

[0156]

[0157] in, Identify the confidence level for the parent component. The degree of non-detection of the next level of expected sub-components. Score the current ROI image quality. Assess the risk of the parent component. To determine the importance of a line segment based on its line number, section, and kilometer marker, all parameters are normalized to a range of 0 to 1. The degree of non-detection satisfies:

[0158]

[0159] In this embodiment, the suspected defect output threshold is set to 0.60. When the suspected defect score is not lower than 0.60, the server retains the corresponding identification chain as a suspected defect result and outputs it to the manual review queue; when the parent target belongs to a preset critical security component and the next-level sub-component is a necessary sub-component, even if the suspected defect score is lower than 0.60, the corresponding identification chain is retained for manual review.

[0160] Example 6

[0161] The difference between this embodiment and the above embodiments is that: Figure 5As shown, the step-by-step mapping in S5 is as follows: the cropping offset, scaling ratio and homogeneous transformation matrix recorded by each level of ROI are used to map the coordinates of the last-level outlier, detection box or segmentation mask from the current level image coordinate system to the previous level image coordinate system; if there is a rotation or perspective transformation at a certain level, the inverse matrix of the homogeneous transformation matrix of that level is used to complete the coordinate inversion mapping.

[0162] In this embodiment, during the multi-level progressive identification process on the server side, each level of ROI records a parameter chain, which may include... , , , , , , , , , and Fields such as [e.g., ]. Among them, and This represents the cropping offset of the k-th level ROI relative to the top-left corner of the (k-1)-th level image. Indicates the scaling ratio. and Indicates the amount of image fill. Denotes the homogeneous transformation matrix. For the component category, sub-component category, or defect category output by the k-th level model, The k-th level detection box, for the segmentation result, can be represented as the bounding box of the corresponding mask. For the k-th level model pair The identification confidence level, normalized to 0-1, This is a unique identifier for the parent identification result corresponding to the current identification result, used to restore the progressive relationship of "equipment—component—sub-component—defect". This specifies the name, version number, weight version, or model file summary of the actual model called at level k, used for result reproduction and traceability. If the level k ROI is derived from the (k-1)th level image by a cropping offset ( , Crop at ) and scale according to the ratio Scaling and image fill (with) , If the point coordinates are mapped from the k-th level image coordinate system to the (k-1)-th level image coordinate system, then the following conditions must be met:

[0163]

[0164]

[0165] When a certain level involves rotation, flipping, perspective correction, or other image transformations, the corresponding homogeneous transformation matrix is ​​used. inverse matrix The final-level outlier points, detection boxes, or segmentation masks are mapped step by step to the original 4C image coordinate system. If column vectors are used to represent point coordinates, the single-level mapping relationship can be written as:

[0166] If the final-level outlier is obtained after K-level progressive identification, then the mapping from the final-level coordinates back to the original 4C image coordinate system can be expressed as: For ordinary cropping and scaling scenes that do not undergo rotation or perspective transformation, the process can be followed as described above. , , , , The relationship is recursively deduced at each level;

[0167] For rectangular detection boxes, the four corner points can be mapped separately; for segmentation masks, the mask contour points, mask bounding boxes, or mask keypoints can be mapped; for abnormal regions output by the classification model, attention regions, candidate regions, or their bounding rectangles can be used as objects to be mapped. When the final output is a segmentation mask, the server can perform step-by-step mapping of the mask contour points, mask bounding boxes, or mask keypoints, and generate pixel-level or region-level annotations in the original 4C image; when the final output is a classification result, the server binds the classification result to its parent ROI, parent detection box, or parent mask bounding box, and maps the corresponding parent region to the original 4C image as the location region of the classification result in the original image.

[0168] In this embodiment, the homogeneous transformation matrix is ​​defined. Indicates the first Level image coordinate system to the first The forward transformation of the input image coordinate system of the level model, where... This indicates the current recognition level in the server-side multi-level progressive recognition: For cases involving only cropping, proportional scaling, and padding: if the... Level ROI is determined by the first Level image at crop offset Crop the cropped area according to the scaling ratio. Perform proportional scaling and increase padding on the left and top sides of the scaled image. , Then the first Point coordinates in a level image With the Point coordinates in a level image satisfy:

[0169]

[0170]

[0171] Therefore, the corresponding homogeneous transformation matrix is:

[0172]

[0173] When point coordinates are represented by column vectors, the forward transformation relationship is:

[0174]

[0175] exist In this case, It is a non-singular matrix, and its inverse matrix is:

[0176]

[0177] Therefore, the first Level 1 image coordinates are mapped back to the 1st level. When using level image coordinates, the following conditions must be met:

[0178]

[0179]

[0180] The aforementioned reversibility refers to the existence of a reversible geometric mapping between the point coordinates preserved within the ROI, and does not mean that the image content discarded outside the ROI can be recovered through cropping operations.

[0181] For example, when using a center-fill scaling method that maintains the aspect ratio, let the target width and height of the input model be respectively... , The width and height of the ROI after cropping are respectively , Then the scaling ratio can be determined as follows:

[0182]

[0183] The corresponding theoretical filling amount is:

[0184]

[0185] Since padding may involve pixel rounding, the parameter chain should record the integer padding amounts located on the left and top sides after actual execution. If different scaling ratios are used in the width and height directions, then... Record them separately as horizontal scaling ratios and vertical scaling ratio And construct a non-uniform scaling matrix accordingly.

[0186] In cases involving rotation, flipping, or perspective correction: when the first When a ROI undergoes rotation, horizontal or vertical flipping, affine correction, or perspective correction, This is a composite matrix obtained by combining the basic transformation matrices according to the actual image processing order. Taking the sequential execution of cropping, scaling, rotation, flipping, perspective correction, and filling as an example, it can be represented using column vectors as follows:

[0187]

[0188] in, This represents the translation matrix corresponding to the clipping; Represents the scaling matrix; Represents the rotation matrix; Indicates a matrix that can be flipped horizontally or vertically; Represents the affine correction or perspective correction matrix; This indicates that the corresponding translation matrix is ​​filled. If the actual processing order is different, the basic transformation matrices are recombined according to the actual processing order, and the transformation matrix executed later is multiplied by the transformation matrix executed earlier.

[0189] A typical perspective transformation matrix can be represented as:

[0190]

[0191] The matrix elements are determined by the points corresponding to the actual rotation angle, flip direction, scaling ratio, translation amount, or perspective transformation. To ensure reverse coordinate mapping can be performed, the following should be made: It is a non-singular matrix.

[0192] The first Level coordinates are mapped to the first When using level coordinates, first calculate:

[0193]

[0194] Then perform homogeneous coordinate normalization:

[0195]

[0196] when When the matrix is ​​an affine transformation matrix, we usually have ;when When perspective transformations are involved, the normalization process described above, which involves dividing by the third homogeneous component, should be performed. The inverse matrix should be consistently represented as... To distinguish it from the representation of the previous level matrix. .

[0197] Example 7

[0198] The difference between this embodiment and the above embodiment is that, in S6, the comprehensive score is obtained by weighted summation of the identification confidence scores at each level and adding it to the hierarchical consistency score.

[0199] In this embodiment, the comprehensive score satisfies:

[0200]

[0201] In the formula, For the first The confidence level of the detection, segmentation, or classification results. Here, H represents the confidence weight for the k-th level, and H is the hierarchical consistency score. For hierarchical consistency weights; ≥0, mu≥0, , , The hierarchical consistency score H satisfies: Among them, when the k-th level identification result and the (k-1)-th level identification result conform to the preset parent-child hierarchical relationship in the railway component hierarchical knowledge graph or identification chain configuration library, Take 1, otherwise Take 0, or take a continuous value between 0 and 1 based on the degree of matching between the parent and child levels and the degree of matching between geometric constraints.

[0202] The hierarchical consistency score H can be further expressed as: Specifically, when the k-th level identification result and the (k-1)-th level identification result conform to the preset parent-child hierarchical relationship in the railway component hierarchical knowledge graph or identification chain configuration library, Take 1, otherwise H is set to 0. When it is necessary to distinguish the strength of the match, a continuous value between 0 and 1 can also be taken based on the degree of matching between parent and child categories, spatial inclusion relationship, geometric constraint matching degree, or confidence difference. When two adjacent categories both conform to the preset parent-child hierarchical relationship, H takes a higher value; when there is a contradiction in the parent-child relationship, H decreases.

[0203] Example 8

[0204] A multi-level railway defect identification system that integrates onboard data acquisition and server collaboration, such as... Figure 2 As shown, it includes:

[0205] The 4C vehicle-mounted acquisition module is used to acquire high-resolution inspection videos or images and obtain metadata.

[0206] The vehicle-mounted preprocessing module is used for preprocessing and quality screening of the acquired images;

[0207] The vehicle-mounted pre-screening module is used to screen image frames, video clips, or candidate ROIs to be uploaded;

[0208] The data encapsulation and upload module is used to encapsulate image data and metadata into detection task data packages and upload them.

[0209] The server task scheduling and identification module is used to receive and parse detection task data packets, establish task queues and schedule identification tasks, and perform multi-level progressive identification based on the railway component hierarchical knowledge graph and identification chain configuration library.

[0210] The coordinate mapping module is used to record the cropping offset, scaling ratio and homogeneous transformation matrix of each level of ROI, and to map the final abnormal coordinates back to the original 4C image coordinate system step by step.

[0211] The result generation and feedback module is used to generate structured defect results and send them back to the 4C vehicle end or the backend front end.

[0212] The system also includes a recognition chain configuration library, a dynamic graph routing unit, a model management unit, a result storage database, a front-end display unit, and a manual review unit. The recognition chain configuration library maintains component parent-child relationships, task types at each level, recognition models at each level, ROI expansion rules, routing scoring rules, and termination conditions. The dynamic graph routing unit determines the next recognition level and corresponding model type based on the current level recognition result, parent-child hierarchical relationship, image quality score, component risk level, and geometric constraint matching degree. The model management unit loads, switches, and records the versions of the server-side object detection model, semantic segmentation model, instance segmentation model, and image classification model. The result storage database stores image data, metadata, recognition results, feedback status, and review records. The front-end display unit displays a defect list, detection boxes, segmentation masks, and annotation maps. The manual review unit confirms, modifies, or rejects defective results and writes the review results back to the result storage database.

[0213] The annotation image is an image created by drawing defect detection bounding boxes, anomaly masks, or defect labels on the original 4C image. A magnified view of the local ROI is used to display defect details. Structured defect results can be saved and transmitted as JSON files, tabular files, database records, or data returned from an API. Result fields may include... , , , , , , , , , , , , and .

[0214] The result feedback module sends the defect list, annotation diagram, and structured defect results back to the 4C vehicle-side display interface or the backend web frontend. The 4C vehicle-side interface can be used to provide online alerts for high-risk defects found in the current line section; the backend frontend can be used to centrally view, filter, and review defects in each line section. For results requiring maintenance dispatch, the system can push the defect location, defect type, and risk level to the maintenance dispatch system.

[0215] To ensure the reliability of result transmission, the server can assign a transmission number and transmission status to each structured defect result. After receiving the result, the 4C vehicle or backend frontend returns a confirmation message to the server. Upon receiving the confirmation message, the server marks the result as transmitted. If no confirmation message is received within a preset time, the result is kept in the pending transmission state and retransmitted.

[0216] For high-risk defects, the server can adopt a priority return strategy, sending defect results with high risk levels, high comprehensive scores, or located in key line sections to the 4C vehicle terminal and the backend front end first. For ordinary defects, the server can summarize and return them by line section, collection time, or task batch, thus balancing the efficiency of real-time alarms and batch review.

[0217] Human reviewers can view the original map annotations, enlarged views of local ROIs, results at each level of the identification chain, and line metadata on the front end, and can confirm, modify, or reject defective results. The review results are written back to the results storage database, serving as a data source for defect statistics, maintenance closure, and model retraining.

[0218] In one implementation, the server can also adjust the thresholds, expansion coefficients, or model versions in the identification chain configuration library based on the results of manual review. When the false alarm rate of a certain type of defect is high for a long period, its output threshold can be increased or a next-level review model can be added; when the missed detection rate of a certain type of high-risk component is high, the vehicle-side pre-screening threshold can be reduced or a strategy for retaining suspected results can be added. Thus, this invention not only achieves single online detection but also supports a continuously optimized 4C intelligent detection closed loop.

[0219] In maintenance dispatching applications, the system can generate a maintenance list based on confirmed defect results, categorized by line section, pole number, defect type, risk level, and comprehensive score. The original map with annotations and enlarged local area of ​​interest (ROI) maps are provided as attachments to maintenance personnel. After maintenance is completed, the maintenance results, processing time, and on-site verification images can be written back to the server-side database, forming a closed-loop record with the original identification results. This closed-loop record can be used for operation and maintenance tracking, and also serves as a data source for subsequent model training, threshold calibration, and expansion of the defect sample library.

[0220] Example 9

[0221] The difference between this embodiment and the previous embodiments is that reproducible simulation data is used to verify the effects of vehicle-side adaptive uploading, server-side multi-level progressive recognition, and coordinate mapping. The results in this embodiment are simulation results obtained under preset parameter conditions and are not considered as actual on-site measurement results.

[0222] In this embodiment, the total number of simulated image frames is set to 10,000, of which 300 are preset candidate defect frames. The average data size of the original image is 1.50 MB / frame, the average data size of the candidate ROI and its surrounding context image is 0.45 MB / frame, and the average data size of the metadata corresponding to each source image frame is 0.004 MB. A lightweight pre-screening model score is generated using a random seed 20260716, where the scores of candidate defect frames follow a Beta distribution with parameters (5,2), and the scores of ordinary image frames follow a Beta distribution with parameters (1.5,8). The above score distribution is used to simulate the situation where there is partial overlap between the scores of candidate defect frames and ordinary image frames.

[0223] The vehicle-side employs a dual-threshold upload strategy, with a high threshold Thigh set to 0.60 and a low threshold Tlow set to 0.35. When the lightweight pre-screening model score is not lower than Thigh, the candidate ROI and its surrounding context image are uploaded. When the score is not lower than Tlow but lower than Thigh, the corresponding image frame is treated as an uncertain sample, and the original image or keyframe is uploaded. When the score is lower than Tlow, its frame-level metadata is retained, and the upload priority is reduced. For preset critical safety components, forced upload can be performed without being restricted by the low threshold.

[0224] Let the number of original image frames uploaded be The number of frames uploaded for the candidate ROI and its surrounding context image is [number]. The average data size of the original image, candidate ROI, and metadata are respectively , and If the total number of source image frames is N, then the data volume uploaded in a single batch is... satisfy The data volume when uploading all original images and metadata. Based on the baseline, the data reduction rate η satisfies Candidate defect frame retention rate The ratio of the number of candidate defect frames to the total number of preset candidate defect frames for entering the server-side fine identification process.

[0225] like Figure 7Simulation results show that the data volume when uploading all original images is 15040.00 MB; when using a fixed threshold strategy of uploading only ROIs with a score of not less than 0.45, the uploaded data volume is 251.05 MB, and the candidate defect frame retention rate is 90.67%; when using the dual threshold upload strategy of this embodiment, uploading 248 candidate ROIs and 725 uncertain sample original images, the uploaded data volume is 1239.10 MB, which is a data reduction rate of 91.76% compared to uploading all original images, and the candidate defect frame retention rate is 97.33%. This shows that by combining the uploading of high-scoring candidate ROIs with the backtracking of original images in the middle score range, the probability of potential defect information being prematurely eliminated by the vehicle can be reduced while reducing the amount of data transmitted.

[0226] Further changing the low threshold T can yield results such as Figure 8 The curve shown represents the trade-off between the candidate defect frame retention rate and the amount of data transmitted. In practical applications, a minimum candidate retention rate can be set on a pre-labeled validation dataset. and from satisfying Not less than Among the candidate thresholds, the threshold with the smallest uploaded data volume or the shortest expected transmission completion time is selected as the preferred low threshold. The preferred low threshold can be updated according to line segment, component risk level, current available bandwidth, cache utilization, or task time limit.

[0227] like Figure 9 As shown, in the bandwidth transmission simulation, if the amount of data uploaded in a single batch is D megabytes and the available uplink bandwidth is B megabits per second, then the theoretical transmission time T, ignoring protocol overhead, satisfies T = 8D / (60B) minutes. Based on the comparison between the theoretical transmission time and the allowed time limit of the task, the vehicle terminal can switch between continuous video segments, keyframes, candidate ROIs, and local cache waiting for uploading.

[0228] like Figure 10 As shown, in the coordinate mapping simulation, 10,000 coordinate points are randomly generated within the 4096×2160 pixel original image. Each coordinate point undergoes 2 to 4 levels of cropping, scaling, filling, rotation, or perspective transformation, and the actual homogeneous transformation matrix for each level is recorded in the parameter chain. Final coordinates pass The image is mapped back to the original image coordinate system, and normalized according to the third component of the homogeneous coordinates when perspective transformation is present. After rounding the final-level coordinates to integer pixels, the simulation yields an average mapping error of 0.1305 pixels, a 95th percentile error of 0.3067 pixels, and a maximum error of 0.6538 pixels. This demonstrates that recording and inversely combining the actual transformation parameters at each level can maintain the correspondence between the final-level anomaly positions and the original 4C image positions.

[0229] like Figure 11 As shown, in the simulation of effective pixels for small targets, the original image width The defect feature width is 4096 pixels. The model input width is 24 pixels. The candidate ROI width is 640 pixels. The defect is 384 pixels wide. The equivalent width of the defect after scaling the entire frame. Pixels, equivalent width of defects after ROI cropping The latter has 10.67 times the number of pixels as the former. This demonstrates that ROI cropping in multi-level progressive recognition can reduce information loss caused by small target defects during frame scaling.

[0230] like Figure 12 As shown, in the simulation of resuming interrupted transmissions, let p be the probability of each transmission attempt failing independently, and let r be allowed to retransmit at most. Then, the probability of the task finally succeeding in uploading is... satisfy When p=3% and r=3, the theoretical success probability is 99.999919%. For scenarios with continuous network outages, the vehicle-side writes the task data packets to its local cache and records the fragment number, verification information, and upload status. Once communication is restored, it continues uploading without confirming the fragment.

[0231] In this embodiment, the candidate defect frame retention rate is used to evaluate the vehicle-side pre-screening and uploading strategy, and is not equivalent to the final defect identification accuracy on the server side. The final identification effect on the server side can be further verified by calculating the accuracy, recall, F1 score, or mean precision using a real 4C image test set with manually annotated results.

[0232] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-level railway defect identification method based on vehicle-mounted data acquisition and server collaboration, characterized in that, Includes the following steps: S1: Collect inspection videos or images along the railway line via the 4C vehicle terminal, and obtain one or more metadata corresponding to the videos or images; S2: Preprocess and screen the image quality of the acquired video or images via the 4C vehicle terminal; S3: The 4C vehicle-side performs lightweight pre-screening on the image frames, video clips, or candidate ROIs to be uploaded based on the image quality screening results or the lightweight pre-screening model based on the 4C vehicle-side, and encapsulates them with the corresponding metadata into a detection task data package, which is then uploaded to the server. S4: The server receives the detection task data packet and schedules the identification task. It performs configurable multi-level progressive identification based on the railway component hierarchical knowledge graph and the identification chain configuration library until the termination condition is met. S5: The server records the offset, scaling ratio and homogeneous transformation matrix of each level of ROI cropping, and maps the final level outlier points, detection boxes or segmentation masks back to the original 4C image coordinate system level by level. S6: The server will hierarchically associate the identification results at each level to generate a structured defect result that includes defect type, original map coordinates, line metadata and comprehensive score, and send it back to the 4C vehicle end or the backend front end.

2. The multi-level railway defect identification method based on vehicle-mounted data acquisition and server collaboration according to claim 1, characterized in that, In S1, the metadata includes the collection time, line number, line section, kilometer marker, pole number, support number, camera number, image frame number, vehicle speed, positioning coordinates, shooting direction, image width and height, task number, and vehicle-side device number.

3. The multi-level railway defect identification method based on vehicle-mounted data acquisition and server collaboration according to claim 1, characterized in that, In S2, the image quality screening includes: calculating one or more of the following: brightness index, contrast index, sharpness index, motion blur degree, occlusion ratio, and effective target area ratio; when the image quality is lower than a preset threshold, the corresponding image frame is marked as a low-quality frame, and its upload, recognition, or result output priority is reduced.

4. The multi-level railway defect identification method based on vehicle-mounted data acquisition and server collaboration according to claim 1, characterized in that, The lightweight pre-screening described in S3 includes one or more of the following: fixed time interval frame extraction, fixed mileage interval frame extraction, key frame extraction, low-quality frame removal, candidate components based on the lightweight pre-screening model, suspected defect ROI, or suspected defect mask extraction.

5. The multi-level railway defect identification method based on vehicle-mounted data acquisition and server collaboration according to claim 1, characterized in that, The detection task data package described in S3 includes one or more of the following: task number, frame number, image data or image address, candidate ROI, acquisition time, line number, kilometer marker, pole number, camera number, vehicle speed, image quality score, upload status, and verification information.

6. The multi-level railway defect identification method based on vehicle-mounted data acquisition and server collaboration according to claim 1, characterized in that, S4 specifically includes: S41: The server schedules identification tasks based on one or more of the following: line segment, upload time, defect risk level, image quality score, vehicle-end device number, or task priority. S42: Perform first-level positioning and identification on the received image to locate the device area, target component area, or suspected defect area; S43: When there are multiple candidate next-level paths in the current level identification result, calculate the routing score and select the path based on the parent confidence, current level confidence, image quality score, component risk level and parent-child geometric constraint matching degree; S44: Based on the current level detection box or segmentation mask bounding box, clip the ROI according to the expansion factor, and record the clipping offset, scaling ratio, expansion factor, parent number, current level category and confidence level; S45: Perform the next level of identification on the trimmed ROI and determine whether the termination condition is met; if it is met, stop the progression and output the current level result; if it is not met, return the current level identification result to S43 for route scoring.

7. The multi-level railway defect identification method based on vehicle-mounted data acquisition and server collaboration according to claim 6, characterized in that, The termination conditions described in S45 include: the current level identification result has identified the final defect type; the current level target belongs to a preset terminal sub-component; the current level confidence is lower than the continuing identification threshold; the current level ROI size or mask area is less than the minimum identification threshold; the current identification chain level has reached the preset maximum number of levels; the current level category does not have a corresponding next-level identification model; and the current level identification result is inconsistent with the preset parent-child hierarchy. When one or more of the above conditions are met, the termination condition is reached.

8. The multi-level railway defect identification method based on vehicle-mounted data acquisition and server collaboration according to claim 1, characterized in that, In S5, the step-by-step mapping is as follows: the cropping offset, scaling ratio, and homogeneous transformation matrix recorded at each level of ROI are used to map the coordinates of the final level outlier, detection box, or segmentation mask from the current level image coordinate system to the previous level image coordinate system. If there is a rotation or perspective transformation at a certain level, the inverse matrix of the homogeneous transformation matrix of that level is used to complete the coordinate inversion mapping.

9. The multi-level railway defect identification method based on vehicle-mounted data acquisition and server collaboration according to claim 1, characterized in that, In S6, the comprehensive score is obtained by weighted summation of the identification confidence scores at each level and adding it to the hierarchical consistency score.

10. A multi-level railway defect identification system that coordinates onboard data acquisition and server operation, used to implement the identification method as described in any one of claims 1-9, characterized in that, include: The 4C vehicle-mounted acquisition module is used to acquire high-resolution inspection videos or images and obtain metadata. The vehicle-mounted preprocessing module is used for preprocessing and quality screening of the acquired images; The vehicle-mounted pre-screening module is used to screen image frames, video clips, or candidate ROIs to be uploaded; The data encapsulation and upload module is used to encapsulate image data and metadata into detection task data packages and upload them. The server task scheduling and identification module is used to receive and parse detection task data packets, establish task queues and schedule identification tasks, and perform multi-level progressive identification based on the railway component hierarchical knowledge graph and identification chain configuration library. The coordinate mapping module is used to record the cropping offset, scaling ratio and homogeneous transformation matrix of each level of ROI, and to map the final abnormal coordinates back to the original 4C image coordinate system step by step. The result generation and feedback module is used to generate structured defect results and send them back to the 4C vehicle end or the backend front end.

Images