Robot railway along equipment inspection image abnormality identification method and system

Abstract

The application relates to the field of railway inspection image recognition, and discloses a robot railway line equipment inspection image abnormality recognition method and system; the method comprises the following steps: establishing a railway line equipment object archive, recording equipment number, line mileage, equipment reference components, allowed shielding components, a historical normal image set, inspection visual angle visibility records and a normal structure atlas; collecting inspection images and recording inspection visual angle information when a robot inspects, determining a target equipment and selecting a normal image cluster with comparable visual angles; establishing an equipment self-coordinate system according to the equipment reference components, and generating a current component relationship graph; comparing the current component relationship graph with the normal structure atlas in terms of structure consistency, and determining an abnormal candidate area; generating a rephotographing instruction according to an image quality interference state or a component relationship edge abnormality state, and outputting an abnormality recognition result based on the consistency of the initial collection image and the rephotographed image. The application can reduce misjudgments caused by visual angle changes, shielding and imaging interference.

Description

Technical Field

[0001] This invention relates to the field of railway inspection image recognition, specifically to a method and system for anomaly recognition in images of equipment inspection along railway lines by robots. Background Technology

[0002] With the increase in the operating mileage of railway lines and the number of equipment along the lines, the inspection of railway equipment such as trackside boxes, signals, transponders, switch machine housings, cable trough covers, and junction boxes is gradually shifting from manual inspection to robotic inspection and image recognition inspection. By having inspection robots move along the railway line and collect images of the equipment, the frequency of manual entry into the track area can be reduced, and traceable equipment image records can be easily created. Current inspection methods typically involve robots or inspection vehicles collecting images and then using methods such as target detection, semantic segmentation, and image comparison to identify abnormal conditions such as missing parts, external damage, and obstructions.

[0003] However, railway equipment is mostly fixed installations. During actual inspections, robots are affected by factors such as the track environment, trackside space, movement posture, shooting distance, camera orientation, lighting changes, raindrops, dirt, and temporary obstructions. As a result, the image perspectives formed by the same equipment at different inspection times are not entirely consistent. If the current inspection image is directly compared with historical images at the pixel level, or if the presence of a component is determined solely by the recognition model, it is easy to misjudge changes in shooting angle, local shadows, reflections, and obstruction by distant components as equipment abnormalities. At the same time, for structural anomalies such as unlocked latches, abnormal cover plate fit, misalignment of bolts and mounting holes, and offset of transponder installation positions, it is difficult to make a stable judgment based solely on component detection results.

[0004] Therefore, existing methods for identifying anomalies in images of robotic railway equipment inspection still suffer from the following problems: There is a lack of comparability screening between current inspection images and historical normal images, tailored to specific equipment and perspectives; the position of equipment components is easily affected by shooting distance and camera posture; anomaly judgments often rely on differences in image appearance or the identification results of individual components, failing to accurately reflect structural relationships such as connections, fits, inclusions, and occlusions between equipment components; and when suspected anomaly areas are affected by strong reflections, shadows, raindrops, stains, motion blur, or temporary occlusion, there is a lack of a confirmation mechanism combined with the robot's re-shooting action, resulting in a high false alarm rate and insufficient verifiability of anomaly identification results. Based on this, it is necessary to propose a method and system for identifying anomalies in images of robotic railway equipment inspection that can combine equipment object files, the equipment's own coordinate system, component relationship maps, and re-shoot consistency confirmation. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for anomaly identification in images of equipment inspection along railway lines using robots, thereby resolving the technical problems existing in the prior art.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution: An anomaly identification method for images of equipment inspection along railway lines using robots includes the following steps: S1: Establish a railway line equipment object archive. The railway line equipment object archive includes equipment number, line mileage, track side, equipment type, equipment reference components, allowed obstruction components, historical normal image set, inspection view visibility record and normal structure map. The normal structure map includes normal component nodes and normal component relationship edges. S2: When the robot moves along the railway line for inspection, it collects inspection images and records inspection perspective information at the same time. The inspection perspective information includes the robot's mileage position, camera orientation, shooting distance and camera posture. S3: Based on the robot's mileage location and camera orientation, determine the target equipment corresponding to the current inspection image from the railway equipment object archive, and select a cluster of normal images with comparable viewpoints from the historical normal image set of the target equipment based on the shooting distance and camera posture; S4: Extract the target equipment area from the current inspection image, establish the equipment's own coordinate system based on the equipment's reference components, and transform the target equipment area into the equipment's own coordinate system; S5: Perform component identification on the converted target device area and generate a current component relationship graph, which includes current component nodes and current component relationship edges; S6: Compare the current component relationship diagram with the normal structure diagram to determine the structural consistency, and identify the inconsistent content that does not conform to the allowable occlusion component and the inspection view visibility record as abnormal candidate areas. S7: Based on the image quality interference status corresponding to the abnormal candidate region, or based on the inconsistency between the connection relationship, bonding relationship or inclusion relationship corresponding to the abnormal candidate region and the corresponding normal component relationship edge, generate a re-shooting instruction and control the robot to re-acquire the re-shooting image. S8: Extract the target device region from the re-shot image and generate the corresponding current component relationship graph. Based on the consistency of the component nodes or component relationship edges corresponding to the abnormal candidate regions in the initial acquisition image and the re-shot image, output the abnormal identification result.

[0007] Preferably, the establishment of the railway line equipment object file includes: using the equipment number of the railway line equipment as the object index, and writing the line mileage, track side, equipment type, equipment reference components, visible component list, allowed obstruction components, historical normal image set, inspection view visibility record, allowed change range, re-shoot trigger ratio and normal structure map corresponding to the same equipment number into the same object file; Among them, the equipment reference component is one of the following: the edge of the enclosure, the boundary of the cover plate, the mounting hole, the fixing bracket, the outline of the lamp room, and the mounting reference edge of the transponder, which maintain a fixed installation position in the inspection image of the corresponding equipment type.

[0008] Preferably, the establishment of the normal structure map includes: labeling or identifying components in the target device region of the historical normal image set to obtain normal component nodes; Generate normal component relationship edges based on the positional relationships of each normal component node in the device's own coordinate system; Among them, normal component nodes include component category, component center coordinates, component outer region, component direction and component visibility ratio, and normal component relationship edges include adjacent relationship, connection relationship, containment relationship, snapping relationship and occlusion relationship.

[0009] Preferably, in step S3, the determination of the comparability of viewpoints includes: comparing the shooting distance, camera orientation, and camera posture corresponding to the current inspection image with the shooting distance, camera orientation, and camera posture corresponding to each historical normal image in the historical normal image set; When the shooting distance difference, camera orientation difference, and camera posture difference all fall within the allowable difference range recorded in the railway equipment object file for the corresponding equipment type, the corresponding historical normal image will be included in the normal image cluster. If any of the shooting distance difference, camera orientation difference, or camera pose difference does not fall within the allowable difference range, the corresponding historical normal image will not be included in the normal image cluster.

[0010] Preferably, in step S4, the establishment of the device's own coordinate system includes: determining at least two reference points of the device reference component in the target device area, taking the direction of the line connecting the two reference points as the first coordinate axis direction, taking the direction passing through one of the reference points and perpendicular to the first coordinate axis direction as the second coordinate axis direction, and taking the pixel distance between the two reference points as the coordinate normalization scale, and converting the position of each component in the target device area into normalized coordinates relative to the device reference component.

[0011] Preferably, in step S5, the generation of the current component relationship diagram includes: segmenting the transformed target device area into components to obtain the component category, component center coordinates, component outer area, component direction, and component visibility ratio of each current component node; Generate the corresponding current component relationship edge based on the contact state, containment state, edge spacing, center coordinate difference, and occlusion overlap area of ​​any two current component nodes; The current component relationship edges include adjacent relationships, connection relationships, containment relationships, fitting relationships, and occlusion relationships.

[0012] Preferably, in step S6, the structural consistency comparison includes: matching the current component node with the normal component node according to the component category and the position in the device's own coordinate system, and matching the current component relationship edge with the normal component relationship edge according to the relationship type and the connected component node; An abnormal candidate region is generated when the position and visibility ratio of the current component node relative to the equipment reference component do not meet the allowable variation range corresponding to the normal structural map, or when the connection, fitting, inclusion, or occlusion relationship in the current component relationship edge is inconsistent with the normal component relationship edge, and the inconsistent content does not conform to the allowable occlusion component and inspection view visibility record.

[0013] Preferably, the inconsistency content does not conform to the records of allowed obstructed components and inspection view visibility, including: when a normal component node fails to match the current component node, querying the records of allowed obstructed components and inspection view visibility in the railway equipment object archive; If the component category corresponding to the normal component node that failed to match is not recorded in the allowed occlusion components, an abnormal candidate region is generated; If the component category corresponding to the unmatched normal component node is recorded as an allowed occlusion component, but the normal component relationship edge corresponding to the normal component node is recorded as should be visible in the inspection view visibility record, then an abnormal candidate region is generated. The repeat shooting instruction includes the target device number, the position of the abnormal candidate area in the device's own coordinate system, the robot's repeat shooting stop position, the camera rotation angle, and the supplementary lighting parameters; The anomaly identification results include the target equipment number, line mileage, track side identification, equipment type, anomaly component category, the position of the anomaly component in the equipment's own coordinate system, anomaly type, initial image acquisition, re-enactment image, and re-enactment confirmation mark.

[0014] An anomaly recognition system for images from robotic inspections of railway equipment includes: It includes a mobile chassis, an image acquisition device, a positioning device, an attitude acquisition device, a supplementary lighting device, and a processing unit. The image acquisition device, the positioning device, the attitude acquisition device, and the supplementary lighting device are all mounted on the mobile chassis, and the processing unit is connected to the image acquisition device, the positioning device, the attitude acquisition device, and the supplementary lighting device respectively. The image acquisition device is used to acquire inspection images of equipment along the railway line when the robot moves along the railway line for inspection; The positioning device is used to obtain the robot's mileage position; The attitude acquisition device is used to obtain camera orientation, shooting distance, and camera attitude; The supplementary lighting device is used to provide supplementary lighting to the target equipment area during reshoots; The processing unit is used to establish a railway line equipment object archive. The railway line equipment object archive includes equipment number, line mileage, track side, equipment type, equipment reference components, allowed occluded components, historical normal image set, inspection view visibility record and normal structure map. The normal structure map includes normal component nodes and normal component relationship edges. The processing unit is also used to determine the target device corresponding to the current inspection image based on the robot's mileage position and camera orientation, and to select a cluster of normal images that are comparable in viewpoint to the current inspection image from the historical normal image set of the target device based on the shooting distance and camera posture. The processing unit is also used to extract the target equipment area from the current inspection image, establish the equipment's own coordinate system based on the equipment reference components of the target equipment, transform the target equipment area to the equipment's own coordinate system, and perform component identification on the transformed target equipment area to generate the current component relationship graph, which includes the current component node and the current component relationship edge. The processing unit is also used to compare the current component relationship diagram with the normal structure diagram for structural consistency, and to identify inconsistencies that do not conform to the allowable occlusion components and the inspection view visibility record as abnormal candidate areas. The processing unit is also used to generate a reshooting instruction based on the image quality interference status corresponding to the abnormal candidate region, or based on the state that the connection relationship, bonding relationship or inclusion relationship corresponding to the abnormal candidate region is inconsistent with the relationship edge of the corresponding normal component, and to control the robot to re-acquire the reshooting image through the mobile chassis, image acquisition device, posture acquisition device and supplementary lighting device. The processing unit is also used to extract the target device region from the re-shot image and generate the corresponding current component relationship graph. Based on the consistency of the component nodes or component relationship edges corresponding to the abnormal candidate regions in the initial acquisition image and the re-shot image, the abnormal identification result is output.

[0015] Preferably, the processing unit includes an equipment file creation module, a target equipment matching module, an equipment coordinate transformation module, a component relationship diagram generation module, a structural consistency comparison module, a re-shooting control module, and an abnormal result output module; The equipment file creation module is used to create equipment object files along the railway line. The target device matching module is used to determine the target device and normal image clusters based on the robot's mileage position, camera orientation, shooting distance, and camera posture. The equipment coordinate transformation module is used to establish the equipment's own coordinate system based on the equipment's reference components; The component relationship diagram generation module is used to generate the current component node and the current component relationship edge; The structural consistency comparison module is used to compare the current component relationship diagram with the normal structural diagram and generate abnormal candidate regions; The reshoot control module is used to generate a reshoot command based on the image quality interference state corresponding to the abnormal candidate region, or based on the state that the connection relationship, bonding relationship or inclusion relationship corresponding to the abnormal candidate region is inconsistent with the relationship edge of the corresponding normal component. The abnormal result output module is used to output an abnormal identification result when the initial image and the re-captured image correspond to the same target device, the same current component node, or the same current component relationship edge composed of the same group of current component nodes, and the inconsistency type is the same.

[0016] In summary, the present invention has the following main beneficial effects: Compared with existing technologies, this application establishes an object archive for railway equipment along the line, which associates and stores equipment number, line mileage, track side identification, equipment type, equipment reference components, permissible occlusion components, historical normal image sets, inspection view visibility records, and normal structure maps. During inspection, it selects normal image clusters with comparable viewpoints based on robot mileage position, camera orientation, shooting distance, and camera posture. This enables the current inspection image to be compared with the normal structure state of the same equipment and similar viewpoints, avoiding misjudgments caused by direct comparison between different shooting angles, different inspection distances, or different individual equipment, and improving the pertinence and stability of abnormal image recognition for railway equipment along the line.

[0017] By establishing a coordinate system for the equipment itself based on the equipment's reference components, the positions of each component within the target equipment area are converted into normalized coordinates relative to the reference components. Furthermore, a current component relationship graph, including current component nodes and current component relationship edges, is generated, achieving the effect of identifying anomalies from the perspective of equipment structural relationships. Compared to methods that only determine the existence of components or only compare image pixel differences, this application can determine structural states such as the fit between the cover plate and the housing, the connection between the latch and the cover plate, and the inclusion relationship between bolts and mounting holes. This allows for more accurate identification of common anomalies along railway lines, such as missing components, positional offsets, broken fits, and broken connections.

[0018] By jointly judging the anomaly candidate regions obtained from structural consistency comparison with allowable occlusion components, inspection view visibility records, and image quality interference status, and generating a re-shoot command when the anomaly candidate region is affected by strong reflections, shadows, raindrops, stains, motion blur, temporary occlusion, or involves abnormal connections, fits, or inclusion relationships, the robot is controlled to re-acquire the re-shot image, achieving a secondary confirmation of suspected anomalies. The anomaly identification result is output by checking whether the inconsistency type of the same component node or the same component relationship edge remains consistent between the initial acquired image and the re-shot image. This reduces false alarms caused by temporary occlusion, lighting changes, and image quality fluctuations, improving the verifiability of the anomaly identification result and the reliability of its field application. Attached Figure Description

[0019] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] refer to Figure 1 A method for anomaly identification in images of equipment inspection along railway lines using robots includes the following steps: S1: Establish a railway line equipment object archive. The railway line equipment object archive includes equipment number, line mileage, track side, equipment type, equipment reference components, allowed obstruction components, historical normal image set, inspection view visibility record and normal structure map. The normal structure map includes normal component nodes and normal component relationship edges. S2: When the robot moves along the railway line for inspection, it collects inspection images and records inspection perspective information at the same time. The inspection perspective information includes the robot's mileage position, camera orientation, shooting distance and camera posture. S3: Based on the robot's mileage location and camera orientation, determine the target equipment corresponding to the current inspection image from the railway equipment object archive, and select a cluster of normal images with comparable viewpoints from the historical normal image set of the target equipment based on the shooting distance and camera posture; S4: Extract the target equipment area from the current inspection image, establish the equipment's own coordinate system based on the equipment's reference components, and transform the target equipment area into the equipment's own coordinate system; S5: Perform component identification on the converted target device area and generate a current component relationship graph, which includes current component nodes and current component relationship edges; S6: Compare the current component relationship diagram with the normal structure diagram to determine the structural consistency, and identify the inconsistent content that does not conform to the allowable occlusion component and the inspection view visibility record as abnormal candidate areas. S7: Based on the image quality interference status corresponding to the abnormal candidate region, or based on the inconsistency between the connection relationship, bonding relationship or inclusion relationship corresponding to the abnormal candidate region and the corresponding normal component relationship edge, generate a re-shooting instruction and control the robot to re-acquire the re-shooting image. S8: Extract the target device region from the re-shot image and generate the corresponding current component relationship graph. Based on the consistency of the component nodes or component relationship edges corresponding to the abnormal candidate regions in the initial acquisition image and the re-shot image, output the abnormal identification result.

[0022] This application is for the visual inspection of fixed equipment along railway lines. Railway line equipment includes one or more of the following: trackside boxes, signals, transponders, switch machine housings, cable tray covers, junction boxes, choke transformer housings, and identification signs. Identified anomalies include missing parts, misaligned positions, broken fits, broken connections, abnormal obstructions, and foreign parts covering the equipment.

[0023] The system in this embodiment includes a mobile chassis, an image acquisition device, a positioning device, an attitude acquisition device, a supplementary lighting device, and a processing unit. The image acquisition device, positioning device, attitude acquisition device, and supplementary lighting device are all mounted on the mobile chassis, and the processing unit is connected to each of these devices. The mobile chassis is used to move along the railway line; the image acquisition device is used to acquire inspection images of equipment along the railway line; the positioning device is used to obtain the robot's mileage position; the attitude acquisition device is used to obtain the camera orientation, shooting distance, and camera attitude; the supplementary lighting device is used to provide supplementary lighting to the target equipment area during reshoots; the processing unit is used to perform tasks such as establishing equipment object files, target equipment matching, equipment coordinate system transformation, generating component relationship diagrams, structural consistency comparison, reshoot control, and outputting abnormal results.

[0024] The processing unit can be located on the robot itself or in a backend server that communicates with the robot. When the processing unit is located in the backend server, the robot sends the inspection images, robot mileage position, camera orientation, shooting distance, and camera posture to the backend server. The backend server then generates a repeat shooting instruction and sends it to the robot. This setting does not affect the data flow and control flow of each step in this embodiment.

[0025] This embodiment does not directly input the inspection images into a general target detection model and output anomaly categories. Instead, it first establishes equipment object files for fixed equipment along the railway line. During the inspection process, the target equipment is determined based on the robot's mileage position and camera orientation, and normal image clusters with comparable perspectives are selected based on the shooting distance and camera posture. Subsequently, the equipment's own coordinate system is established based on the equipment's reference components, and the target equipment area is transformed into the equipment's own coordinate system to generate the current component relationship diagram. Anomaly candidate regions are generated by comparing the structural consistency between the current component relationship diagram and the normal structure map. Finally, the robot is controlled to retake the image based on the image quality interference state or the inconsistent state of connection, fitting, or inclusion relationships, and the anomaly identification result is output based on the consistency of the same component node or the same component relationship edge in the initial acquisition image and the retaken image.

[0026] Specifically, the steps include the following: S1. Establish an archive of equipment along the railway line.

[0027] Before the robot begins its formal inspection, the processing unit establishes an object file for the railway equipment along the line. The object file uses the equipment number as the object index and writes the line mileage, track side, equipment type, equipment reference components, list of visible components, allowed occluded components, historical normal image set, inspection view visibility record, allowed range of change, repeat shot trigger ratio, and normal structure map corresponding to the same equipment number into the same object file.

[0028] Equipment numbers can be derived from railway equipment ledgers or generated from a combination of line mileage, track side identification, and equipment type. Line mileage is used to determine the equipment's location on the line, track side identification is used to distinguish whether the equipment is located on the left or right side of the line, and equipment type is used to invoke the corresponding component identification rules and structural relationship rules.

[0029] A reference component for an equipment is a component that maintains a fixed installation position in the inspection image of the corresponding equipment type and provides at least two identifiable reference points. For trackside enclosures, the reference component can be the enclosure edge, cover boundary, mounting hole, or fixed bracket; for signals, the reference component can be the outer outline of the signal lamp or the signal mounting bracket; for transponders, the reference component can be the transponder mounting reference edge; for cable trough covers, the reference component can be the cover boundary or the trough edge. The reference component is used to establish the equipment's own coordinate system; therefore, its selection is based on the fact that the component's position relative to the equipment body is fixed in its normal installation state and can be identified from a conventional inspection perspective.

[0030] The visible parts list records the components that should be identified under normal inspection conditions for the target equipment. For example, for a trackside enclosure, the visible parts list includes the enclosure body, cover, latches, hinges, bolts, identification plates, cable interfaces, and protective covers. For a signal, the visible parts list includes the light housing housing, canopy, mounting bracket, junction box, and identification plates. The component names in the visible parts list should remain consistent within the same equipment type.

[0031] The "Permissible Obscured Components" category records components that are permissibly invisible from a specific inspection perspective. The criteria for "Permissible Obscured Components" are determined based on the visibility status of corresponding components in a historical set of normal images under different inspection perspectives, the actual installation structure of the equipment, and manual verification results. For example, from a side inspection perspective, the hinge on the side of the trackside enclosure away from the camera may be obscured by the enclosure edge; this hinge can be recorded as a permissible obscured component from that perspective. However, latches, cover edges, and enclosure edges that should be visible from a frontal inspection perspective are not recorded as permissible obscured components.

[0032] The inspection view visibility record is established according to equipment type, camera orientation range, camera posture range, and shooting distance range. For each view range, the visibility status of normal component nodes and normal component relationship edges is recorded, and the visibility status includes "should be visible" and "allowed to be invisible". Taking a trackside box as an example, in the frontal view, the fit between the cover and the edge of the box, and the connection between the latch and the cover are recorded as "should be visible"; in the lateral view, the far hinge node can be recorded as "allowed to be invisible", but the fit between the outer edge of the cover and the outer edge of the box can still be recorded as "should be visible".

[0033] The historical normal image set consists of images that have been manually reviewed or confirmed to be without abnormalities according to maintenance records. Each historical normal image in the set is associated with its corresponding shooting distance, camera orientation, camera posture, robot mileage position, and acquisition time. Images marked as severely blurred, severely occluded, poorly exposed, or with abnormal equipment status are not included in the historical normal image set as normal samples.

[0034] The permissible range of variation includes the permissible offset range of the component's center coordinates, the permissible range of variation of the area of ​​the component's circumscribed region, the permissible range of variation of the component's orientation, and the permissible range of variation of the component's visible proportion. These permissible ranges are determined by the statistical distribution of corresponding component nodes in a historical normal image set for the same equipment number or type, and are then combined with equipment installation tolerances, on-site calibration results, and manual verification results before being entered into the railway line equipment object archive. For equipment with insufficient historical samples, the permissible range of variation for the same equipment type can be used as the initial configuration, and the permissible range of variation for the corresponding equipment number can be updated after subsequent inspection images have been manually verified.

[0035] The re-shooting trigger ratio is determined based on the overlap between the candidate abnormal region and the image quality interference region in historical inspection images. For suspected abnormal samples that, after manual review, are found to be caused by strong reflections, shadows, raindrops, stains, motion blur, or temporary occlusion, the distribution of the overlap area between the candidate abnormal region and the image quality interference region is statistically analyzed, and the proportion that can cover this type of image quality interference sample is used as the re-shooting trigger ratio for the corresponding equipment type. This re-shooting trigger ratio is written as configuration data into the railway line equipment object file and is not limited to a fixed value in this embodiment.

[0036] A normal structural diagram includes normal component nodes and normal component relationship edges. Normal component nodes represent the constituent parts of the device, while normal component relationship edges represent the structural relationships between two normal component nodes. Normal component nodes include component category, component center coordinates, component circumscribed region, component orientation, and component visibility scale. Normal component relationship edges include adjacent relationships, connection relationships, containment relationships, fitting relationships, and occlusion relationships.

[0037] The process of establishing a normal structure map is as follows: Component annotation or component identification is performed on the target device region in the historical normal image set to obtain normal component nodes; the positions of each normal component node are transformed to the device's own coordinate system; normal component relationship edges are generated based on the contact state, containment state, edge spacing, center coordinate difference, and occlusion overlap area between normal component nodes; the corresponding normal component nodes and normal component relationship edges in multiple historical normal images are summarized to obtain the normal structure map of the device in its normal state.

[0038] The allowable ranges for adjacent distances, edge spacing, and occlusion overlap are determined by the statistical distribution of the corresponding normal component relationship edges in the historical normal image set, and are written into the railway line equipment object file in conjunction with the equipment installation structure and manual review results. The identification threshold for image quality interference areas is determined by image quality anomaly samples in historical inspection images and manual review results, and is written into the equipment object file of the corresponding equipment type; the identification thresholds include the brightness saturation threshold for strongly reflective areas, the brightness change threshold for shadow areas, the morphological feature threshold for raindrop areas, the texture change threshold for stain areas, the edge sharpness threshold for motion-blurred areas, and the non-equipment component identification threshold for temporarily occluded areas.

[0039] The aforementioned inspection view visibility records, allowed occluded components, allowed range of variation, repeat shot trigger ratio, adjacent distance range, allowed edge spacing range, allowed occlusion overlap range, and image quality interference area recognition thresholds are all saved as configuration data in the railway line equipment object archive. The configuration data can be updated based on subsequent manual review results and equipment maintenance records, but the updated data is written into the corresponding equipment object archive before being used for subsequent inspection identification.

[0040] S2. The robot collects inspection images and records inspection perspective information.

[0041] When the robot moves along the railway line for inspection, the image acquisition device captures inspection images, the positioning device simultaneously records the robot's mileage and position, and the attitude acquisition device simultaneously records the camera orientation, shooting distance, and camera attitude. Camera attitude includes the camera pitch angle and camera roll angle; when the image acquisition device is mounted on a gimbal, the camera attitude also includes the gimbal rotation angle. The shooting distance can be obtained through laser ranging, binocular ranging, structured light ranging, or calculated from the known installation position of the device and the robot's current position.

[0042] Inspection images, robot mileage position, camera orientation, shooting distance, and camera pose are bound together at the same timestamp to form an inspection image record. If image exposure failure, severe image blurring, or inability to locate the target device area occurs during acquisition, the inspection image record is marked as unusable for structural comparison, triggering the robot to reacquire the image or marking it as an image quality anomaly in the inspection results.

[0043] S3. Determine the target device and select the normal image cluster.

[0044] The processing unit determines the target device corresponding to the current inspection image from the railway line equipment object file based on the robot's mileage location and camera orientation. Specifically, it first searches for candidate devices within the line mileage range based on the robot's mileage location, and then determines the target device from the candidate devices based on the camera orientation and track side. If multiple devices exist within the same mileage range, they are confirmed based on the device's candidate area location in the image, shooting distance, and device type identification results.

[0045] After identifying the target device, the processing unit selects a cluster of normal images from the target device's historical normal image set that are visually comparable to the current inspection image, based on the shooting distance and camera orientation corresponding to the current inspection image. Visual comparability is used to determine whether the current inspection image and the historical normal images are under shooting conditions that allow for structural consistency comparison, rather than requiring the two images to be completely identical.

[0046] Specifically, the shooting distance, camera orientation, and camera attitude of the current inspection image are compared with the shooting distance, camera orientation, and camera attitude of each historical normal image in the historical normal image set. When the differences in shooting distance, camera orientation, and camera attitude all fall within the allowable difference range recorded in the corresponding equipment type in the railway equipment object file, the corresponding historical normal image is included in the normal image cluster; when any one of the differences in shooting distance, camera orientation, and camera attitude falls outside the allowable difference range, the corresponding historical normal image is not included in the normal image cluster.

[0047] The allowable differences in shooting distance, camera orientation, and camera pose are determined by a historical set of normal images from the same device number or type, and by on-site calibration results. For devices with stable installation postures and primarily shooting from the front, the allowable difference in camera orientation can be set relatively more strictly; for sections of the robot trajectory with lateral fluctuations, the allowable difference in shooting distance can be determined in conjunction with on-site calibration results. This process avoids incorrectly comparing parts that are permissible to be invisible from a side view with parts that should be visible from a front view.

[0048] S4. Extract the target device area and establish the device's own coordinate system.

[0049] The processing unit extracts the target equipment region from the current inspection image. The target equipment region can be determined through the equipment detection model, the image region corresponding to a preset mileage, template matching, or manually calibrated regions. The equipment detection model is only used to locate the target equipment region and is not the sole basis for anomaly judgment.

[0050] The processing unit determines at least two reference points for the reference components of the equipment within the target equipment area. For example, for a trackside enclosure, the two reference points can be the two endpoints of the upper edge of the cover plate; for a transponder, the two reference points can be the two endpoints of the transponder's mounting reference edge; for a signal, the two reference points can be the two fixed corner points of the signal light's external outline. If multiple reference points exist for the equipment reference component, the two reference points with higher clarity and lower obstruction ratio under the current inspection view are preferentially selected.

[0051] Let the first reference point be The second benchmark is Any component point to be converted is The pixel distance between the two reference points is Using the line connecting the two reference points as the first coordinate axis direction, and the direction passing through the first reference point and perpendicular to the first coordinate axis direction as the second coordinate axis direction, any component point within the target equipment area is considered. Convert to normalized coordinates in the device's own coordinate system : ; ; in, Represents the normalized coordinates of the component point along the first coordinate axis; Represents the normalized coordinates of the component point along the direction of the second coordinate axis; Indicates the first reference point; Indicates the component point to be converted; This represents the pixel distance between two reference points; The unit vector representing the direction of the first coordinate axis; The unit vector representing the direction of the second coordinate axis.

[0052] Through the coordinate transformation described above, the positions of each component in the current inspection image no longer directly use the pixel coordinates of the entire image, but instead use normalized coordinates relative to the reference component of the equipment. If there are slight changes in the robot's shooting distance, camera height, or camera angle, as long as the structure of the target equipment is not abnormal, the positional relationship of each component in the equipment's own coordinate system will remain within the allowable range of change.

[0053] If two reference points for the equipment's reference component cannot be determined in the current inspection image, structural consistency comparison will not be performed, and the inspection image will be marked as an image where the reference component is unavailable. For such images, the robot can be controlled to retake the image, or it can be marked as an image quality anomaly in the inspection results.

[0054] S5. Generate the current component relationship diagram.

[0055] The processing unit identifies components within the converted target device area to obtain the current component node. The current component node includes the component category, component center coordinates, component perimeter, component orientation, and component visibility ratio. The component category matches the name in the visible component list.

[0056] Component identification can be achieved using instance segmentation models, keypoint detection models, template matching methods, or edge contour-based component localization methods. In this embodiment, the model's role is to obtain the component's location and its circumscribed region. Anomaly detection does not directly depend on the anomaly category output by the model, but is accomplished by comparing the structural consistency between the current component relationship diagram and the normal structure diagram.

[0057] After obtaining the current component node, the processing unit generates the corresponding current component relationship edge based on the contact state, containment state, edge spacing, center coordinate difference, and occlusion overlap area of ​​any two current component nodes. The current component relationship edge includes at least one of the following: adjacency relationship, connection relationship, containment relationship, snapping relationship, and occlusion relationship.

[0058] The adjacency relationship indicates that the edge distance between two current component nodes in the device's own coordinate system falls within the adjacent distance range recorded for the corresponding device type, and there are no other component nodes between them. The connection relationship indicates that there is a structural connection between two current component nodes, such as the connection between a latch and a cover, a hinge and a housing, a bolt and a mounting hole, or a cable interface and a housing. The containment relationship indicates that the outer region of one current component node is located within the outer region of another current component node, such as a bolt being located within a mounting hole area or a sign being located within the housing surface area. The mating relationship indicates that the corresponding edges of two current component nodes remain in contact or close to each other in the device's own coordinate system, such as the edge of a cover fitting against the edge of a housing. The occlusion relationship indicates that one current component node occludes another current component node, resulting in a reduced visibility of the occluded component.

[0059] For trackside enclosures, if the current component node includes the enclosure body, cover plate, and latch, the processing unit generates an edge indicating the fit between the cover plate and the enclosure body, and an edge indicating the connection between the latch plate and the cover plate. For transponders, if the current component node includes the transponder body and an installation reference edge, the processing unit generates an edge indicating the fit between the transponder body and the installation reference edge. For cable trough covers, if the current component node includes the trough body and the cover plate, the processing unit generates an edge indicating either the containment relationship between the cover plate and the trough body, or an edge indicating the fit between them.

[0060] The conditions for a successful match between a current component node and a normal component node are that the component categories are the same and the position of the current component node in the device's own coordinate system falls within the allowed range of variation of the corresponding normal component node. The conditions for a successful match between a current component relationship edge and a normal component relationship edge are that the relationship types are the same and both connected component nodes are successfully matched.

[0061] S6. Perform structural consistency comparison and identify candidate regions for anomalies.

[0062] The processing unit compares the current component relationship diagram with the normal structure diagram to determine the structural consistency, and identifies inconsistencies that do not conform to the allowable occlusion components and the inspection view visibility record as abnormal candidate areas.

[0063] Specifically, the processing unit matches the current component node with the normal component node according to the component category and the position in the device's own coordinate system, and matches the current component relationship edge with the normal component relationship edge according to the relationship type and the component node it is connected to.

[0064] For a successfully matched current component node and a normal component node, compare the component center coordinate offset, the difference in the area of ​​the component's outer region, the difference in the component's direction, and the difference in the component's visibility ratio. If any of these differences exceeds the allowable variation range recorded in the railway equipment object file for the corresponding equipment type, the area where the current component node is located is identified as an abnormal candidate area.

[0065] The permissible range of variation is determined as follows: For a set of historical normal images of the same equipment number or type, the center coordinates of each normal component node in the equipment's own coordinate system, the area of ​​the component's outer region, the component's direction, and the visible ratio of the component are statistically analyzed. After removing samples that have been manually reviewed and marked as having abnormal image quality or abnormal equipment status, the permissible range of variation corresponding to that component node is formed. When the on-site equipment maintenance record shows that the equipment has had its components replaced or reinstalled, the historical normal images of the equipment are re-acquired and reviewed, and the corresponding permissible range of variation is updated.

[0066] When comparing the current component relationship edges with normal component relationship edges, first confirm whether they connect to the same group of component nodes, and then confirm whether the relationship types are consistent. For example, in a normal structural diagram, a fitting relationship is recorded between the cover plate and the housing. If this fitting relationship does not exist in the current component relationship diagram, or if a gap exceeding the allowable variation range is formed between them, then the area near the edge of the cover plate is identified as an abnormal candidate area. Similarly, in a normal structural diagram, a connection relationship is recorded between the latch and the cover plate. If the latch does not form a connection relationship with the cover plate in the current component relationship diagram, then the area where the latch is located is identified as an abnormal candidate area.

[0067] For normal component nodes that fail to match, the processing unit queries the railway equipment object archive for records of allowed occlusion components and inspection view visibility. If the component category corresponding to the unmatched normal component node is not recorded in allowed occlusion components, an abnormal candidate region is generated. If the component category corresponding to the unmatched normal component node is recorded in allowed occlusion components, but the normal component relationship edge corresponding to the normal component node is recorded as "should be visible" in the inspection view visibility record, an abnormal candidate region is still generated.

[0068] For example, from a certain inspection perspective, the hinge on the far side of the box can be recorded as a component that can be obscured. If this hinge is not identified, an anomaly candidate region is not directly generated. However, if the connection between the latch and the cover is recorded as something that should be visible from this inspection perspective, but this connection does not exist in the current inspection image, an anomaly candidate region is generated. In this way, we can avoid mistaking normal viewpoint obstruction for missing components, while retaining the identification of anomalies in key structural relationships.

[0069] In this embodiment, the structural consistency comparison is not a typical image difference test. Typical image difference tests usually compare pixel differences or regional feature differences between two images, which are easily affected by lighting, dirt, shadows, and changes in shooting angle. This embodiment compares component nodes and component relationship edges in the device's own coordinate system; therefore, the anomaly detection object is the device's structural relationships, rather than the overall appearance differences of the image.

[0070] S7. Generate a repeat shooting instruction and control the robot to re-acquire the repeat shooting image.

[0071] After generating abnormal candidate regions, the processing unit generates a re-shooting instruction based on the image quality interference status corresponding to the abnormal candidate regions, or based on the inconsistency between the connection relationship, bonding relationship, or inclusion relationship corresponding to the abnormal candidate regions and the corresponding normal component relationship edge, and controls the robot to re-acquire the re-shooting image.

[0072] Image quality interference states include strong reflections, shadows, raindrops, stains, motion blur, and temporary occlusions. Strongly reflective areas can be identified based on local brightness saturation and color channel distribution; shadow areas can be identified based on local brightness being lower than adjacent areas and continuous edges; raindrop areas can be identified based on local transparent spots, edge highlights, and morphological features; stain areas can be identified based on low-texture patches in fixed locations; motion-blurred areas can be identified based on edge direction consistency and high-frequency information reduction; and temporary occlusion areas can be identified based on human figures, tools, temporary markers, swaying vegetation, or other non-equipment component areas.

[0073] The processing unit calculates the overlap area between the anomaly candidate region and the strong reflection region, shadow region, raindrop region, stain region, motion blur region, and temporary occlusion region, and calculates the overlap area ratio based on the overlap area and the area of ​​the anomaly candidate region. ; in, Indicates the percentage of overlapping area; Indicates an abnormal candidate region; Indicates areas of image quality interference; This represents the area of ​​overlap between the abnormal candidate region and the image quality interference region. This represents the area of ​​the abnormal candidate region.

[0074] When the overlap area ratio reaches the re-shooting trigger ratio recorded in the railway equipment object file for the corresponding equipment type, the processing unit generates a re-shooting instruction. The re-shooting trigger ratio is determined by suspected abnormal samples caused by image quality interference in historical inspection images and is written into the equipment object file after manual verification. For samples that are manually verified as image quality interference, their overlap area ratio distribution is statistically analyzed, and the proportion that can cover this type of sample is used as the re-shooting trigger ratio for the corresponding equipment type. Samples that are confirmed to be genuine equipment abnormalities after re-shooting are not used as a basis for reducing the re-shooting trigger ratio.

[0075] In addition to image quality interference, the processing unit also generates a re-shot instruction when the connection, fit, or containment relationship corresponding to the abnormal candidate region is inconsistent with the relationship edge of the corresponding normal component. The connection, fit, and containment relationships usually correspond to structural states such as cover closure, latch connection, bolt installation, and transponder installation position. Even if the abnormal candidate region does not have obvious image quality interference, it is confirmed whether it is a real abnormality by re-shotting.

[0076] The re-shooting instruction includes the target device number, the position of the candidate abnormal area in the device's own coordinate system, the robot's re-shooting stop position, the camera rotation angle, and the supplementary lighting parameters. The robot moves to the corresponding position according to the re-shooting stop position, adjusts the camera posture according to the camera rotation angle, and acquires the re-shot image according to the supplementary lighting parameters.

[0077] The stopping position for reshooting is determined based on the target equipment's route mileage, the side of the equipment installation, and the location of the candidate anomaly area in the equipment's own coordinate system. If the candidate anomaly area is located on the upper right side of the target equipment, the robot adjusts to a position that facilitates shooting the area, while maintaining a safe distance on site. If the candidate anomaly area is affected by strong reflections, the robot adjusts the camera posture or reduces the intensity of the supplementary light. If the candidate anomaly area is affected by shadows, the robot turns on the supplementary light device or changes the angle of the supplementary light. If the candidate anomaly area is affected by raindrops or stains, the robot confirms the condition of the area by changing the camera angle or re-acquiring images.

[0078] S8. Verify the consistency of the re-shot images and output the anomaly identification results.

[0079] After the robot acquires repeated images, the processing unit extracts the target device region from the repeated images and generates a corresponding current component relationship diagram. Specifically, the processing unit repeatedly performs target device region extraction, device coordinate system transformation, and current component relationship diagram generation on the repeated images, and matches the current component relationship diagram corresponding to the repeated images with the current component relationship diagram corresponding to the initially acquired images.

[0080] During the re-image confirmation process, the initial image and the re-image are first checked to see if they correspond to the same target device number. If the target device numbers are inconsistent, no anomaly confirmation is performed, and the re-image is marked as having an inconsistent re-image object. If the target device numbers are consistent, the process continues to determine whether the anomaly candidate region corresponds to the same current component node, or to the same current component relationship edge composed of the same group of current component nodes.

[0081] The criteria for determining if a node is the same as a current component include: consistent component category, positional difference within the allowable range in the device's own coordinate system, and consistent relative position between the component's external region and the target device. The criteria for determining if an edge is a relationship of the same current component include: the two component nodes connected by the edge are the same, the relationship type is the same, and the edges correspond in the same position in the device's own coordinate system. For example, if the connection between the latch and cover plate is broken in the initial image, and the connection between the same latch and cover plate is still broken in the re-image, then the edges are considered to be the same current component relationship edge composed of the same set of current component nodes.

[0082] Inconsistency types include one of the following: missing component, positional offset, broken bonding relationship, broken connection relationship, abnormal occlusion relationship, and coverage by external component. If the abnormal candidate regions in the initial acquisition image and the re-captured image both correspond to the same target device number and the same current component node, or correspond to the same current component relationship edge composed of the same group of current component nodes, and the inconsistency type corresponding to the two abnormal candidate regions is the same, the processing unit outputs the anomaly identification result.

[0083] The anomaly identification results include the target equipment number, line mileage, track side identification, equipment type, anomaly component category, location of the anomaly component in the equipment's own coordinate system, anomaly type, initial image acquisition, re-enhanced image, and re-enhanced confirmation marker. The location of the anomaly component in the equipment's own coordinate system is used by subsequent maintenance personnel to locate the anomaly area. The re-enhanced confirmation marker is used to distinguish between anomalies that have been confirmed by re-enhanced images, images whose image quality needs to be verified, and inconsistencies in the re-enhanced objects.

[0084] If an abnormal candidate region exists in the initial image acquisition, but the same inconsistent content does not reappear in the re-captured image for the same current component node or the same current component relationship edge, then no confirmed abnormality will be output, and the result will be marked as suspected image interference. This processing method is used to avoid directly reporting suspected abnormalities caused by raindrops, reflections, short-term occlusion, or motion blur as equipment abnormalities.

[0085] Example 2 This embodiment provides an anomaly recognition system for images of equipment inspection along railway lines using a robot. The system includes a mobile chassis, an image acquisition device, a positioning device, an attitude acquisition device, a supplementary lighting device, and a processing unit. The image acquisition device, positioning device, attitude acquisition device, and supplementary lighting device are all mounted on the mobile chassis, and the processing unit is connected to the image acquisition device, positioning device, attitude acquisition device, and supplementary lighting device respectively.

[0086] The processing unit includes an equipment file creation module, a target equipment matching module, an equipment coordinate transformation module, a component relationship diagram generation module, a structural consistency comparison module, a re-shooting control module, and an abnormal result output module.

[0087] The equipment file creation module is used to create equipment object files along the railway line. This module writes equipment number, line mileage, track side identification, equipment type, equipment reference components, permissible obstruction components, historical normal image sets, inspection view visibility records, and normal structure diagrams into the same object file.

[0088] The target device matching module is used to determine the target device and normal image clusters based on the robot's odometer position, camera orientation, shooting distance, and camera pose. The module first searches for candidate devices based on the robot's odometer position, then determines the target device based on the camera orientation and track orientation, and finally selects a normal image cluster with comparable viewpoints from the historical normal image set based on the shooting distance and camera pose.

[0089] The equipment coordinate transformation module is used to establish the equipment's own coordinate system based on the equipment reference component. The equipment coordinate transformation module identifies at least two reference points of the equipment reference component in the target equipment area, and establishes a first coordinate axis direction, a second coordinate axis direction, and a coordinate normalization scale based on the two reference points, converting the positions of each component in the target equipment area into normalized coordinates relative to the equipment reference component.

[0090] The component relationship graph generation module is used to generate the current component nodes and the current component relationship edges. The component relationship graph generation module performs component segmentation on the transformed target device area to obtain the component category, component center coordinates, component outer region, component direction, and component visibility ratio. It also generates the current component relationship edges based on the contact state, containment state, edge spacing, center coordinate difference, and occlusion overlap area of ​​any two current component nodes' outer regions.

[0091] The structural consistency comparison module compares the current component relationship diagram with the normal structural diagram and generates candidate regions for anomalies. The module matches the current component nodes with normal component nodes according to component category and position in the equipment's own coordinate system, and matches the current component relationship edges with normal component relationship edges according to relationship type and the connected component nodes. Inconsistencies that do not conform to the allowed occlusion of components and the visibility records of the inspection view are identified as candidate regions for anomalies.

[0092] The repeat shot control module generates repeat shot instructions based on the image quality interference status corresponding to the abnormal candidate region, or based on the inconsistency between the connection, bonding, or inclusion relationships of the abnormal candidate region and the corresponding normal component's relationship edge. The repeat shot instructions include the target device number, the position of the abnormal candidate region in the device's own coordinate system, the robot's repeat shot stop position, the camera rotation angle, and the supplementary lighting parameters. The mobile chassis moves according to the robot's repeat shot stop position, the attitude acquisition device adjusts the camera attitude, the supplementary lighting device operates according to the supplementary lighting parameters, and the image acquisition device re-acquires the repeat shot image.

[0093] The abnormal result output module is used to output the abnormal identification result when the initial image and the re-captured image correspond to the same target device, the same current component node, or the same current component relationship edge composed of the same group of current component nodes, and the inconsistency type is the same.

[0094] Example 3 This embodiment uses a trackside box as an example to illustrate the specific application of the above method and system.

[0095] Establish an equipment object file for each trackside enclosure. The equipment object file records the equipment number, track mileage, track side, and equipment type for that trackside enclosure. The reference component is selected as the upper edge of the enclosure's front cover, and the reference points are the left and right endpoints of the upper edge of the cover. The list of visible components includes the enclosure, cover, latches, hinges, bolts, identification plates, and cable interfaces. Permissible obstruction components are determined based on the inspection perspective. For example, from a side inspection perspective, the far hinge can be recorded as a permissible obstruction component, but from a front inspection perspective, latches, cover edges, and enclosure edges are all recorded as should be visible.

[0096] When the robot moves along the railway line to the vicinity of the trackside box, it acquires the current inspection image and records the robot's mileage position, camera orientation, shooting distance, and camera posture. The processing unit determines the target device corresponding to the current inspection image as the trackside box based on the robot's mileage position and camera orientation, and selects a cluster of normal images from the historical normal image set of the trackside box that are comparable in viewpoint to the current inspection image.

[0097] The processing unit extracts the trackside enclosure area from the current inspection image, establishes the equipment's own coordinate system using the left and right endpoints of the top edge of the cover plate, and converts the positions of the latches, hinges, cover plate edges, and bolts into normalized coordinates within the equipment's own coordinate system. The processing unit then performs component identification on the converted trackside enclosure area, generating a current component relationship diagram. Specifically, connection edges are generated between the latches and the cover plate, fitting edges between the cover plate and the enclosure body, and containment edges between the bolts and mounting holes.

[0098] If a latch node exists in the current component relationship diagram, but the connection edge between the latch and the cover plate does not exist, and this connection edge is recorded as "should be visible" in the inspection view visibility record, then the processing unit generates an abnormal candidate region in the area where the latch is located. If the overlap ratio between this abnormal candidate region and the strongly reflective area reaches the repeat-shot trigger ratio, or if the abnormal candidate region corresponds to an abnormal connection relationship between the latch and the cover plate, then the processing unit generates a repeat-shot command. The robot adjusts the camera posture and supplementary lighting parameters according to the repeat-shot command and then acquires the repeat-shot image.

[0099] In the re-enhanced image, the processing unit re-establishes the device's own coordinate system and generates the current component relationship diagram. If the re-enhanced image still shows a broken connection between the same latch and the same cover plate, an anomaly identification result is output, with the anomaly type being "connection broken." If the connection between the latch and the cover plate is normal in the re-enhanced image, the anomaly candidate area in the initial image is marked as suspected image interference, and no confirmed anomaly is output.

[0100] Example 4 This embodiment uses a cable trough cover as an example to illustrate another application of the above method and system.

[0101] When creating an equipment object file for cable trough covers, the equipment reference component is selected as the edge of the cable trough or the boundary of the cover. The visible component list includes the trough body, cover, cover edge, fasteners, and identification components. The normal structural diagram records the fit between the cover and the trough body, the adjacency relationship between the cover edge and the trough edge, and the containment relationship between the fasteners and the cover.

[0102] After the robot acquires the current inspection image, the processing unit determines the target equipment as the corresponding cable trough cover based on the line mileage and camera orientation. The processing unit establishes the equipment's own coordinate system using two reference points on the edge of the trough and transforms the positions of the cover edge, the trough edge, and the fixing components to the equipment's own coordinate system. If the fit between the cover and the trough is broken in the current component relationship diagram, manifested as the positional offset of the cover edge relative to the trough edge exceeding the allowable range, an abnormal candidate region is generated in the cover edge area.

[0103] If the overlap ratio between the candidate abnormal region and the temporary occlusion region reaches the re-shooting trigger ratio, the robot adjusts the shooting distance and re-acquires the re-shot image. If the bonding relationship between the same cover plate and the same groove is still broken in the re-shot image, an abnormality identification result is output, and the abnormality type is "bonding relationship broken". If the bonding relationship is normal in the re-shot image, no confirmation abnormality is output.

[0104] In this embodiment, the determination of cable trough cover anomalies does not rely on pixel differences between the entire image and historical images, but rather on whether the fit between the cover and the trough remains inconsistent in the device's own coordinate system. Therefore, even if the ambient lighting or the image background changes, it will not directly affect the determination of the cover's structural condition.

[0105] Example 5 This embodiment uses a transponder as an example to illustrate another application of the above method and system.

[0106] When creating a device object file for a transponder, the device reference component is selected as the transponder mounting reference edge. The visible component list includes the transponder body, mounting reference edge, fasteners, and identification area. The normal structural diagram records the fit between the transponder body and the mounting reference edge, as well as the containment relationship between the fasteners and the transponder body.

[0107] After the robot acquires the current inspection image, the processing unit determines the target transponder corresponding to the current inspection image based on the robot's mileage position and camera orientation, and selects normal image clusters based on the shooting distance and camera attitude. The processing unit establishes the device's own coordinate system using the transponder's mounting reference edge, converting the positions of the transponder body and the fixing component into normalized coordinates. If the position offset of the transponder body relative to the mounting reference edge in the current component node exceeds the allowable range, or if the containment relationship between the fixing component and the transponder body does not exist, an abnormal candidate region is generated.

[0108] For transponders, the abnormal candidate area corresponds to a positional offset or a missing fastener. Since the transponder's installation position is a structural state that needs to be confirmed, a re-shot command is generated even if there is no obvious strong reflection, shadow, or raindrop interference when the corresponding fit or inclusion relationship of the abnormal candidate area is abnormal. If the re-shot image still shows a positional offset of the same transponder body relative to the same installation reference edge, or an abnormal inclusion relationship between the same fastener and the transponder body, then the abnormality identification result is output.

[0109] In the above embodiments, the anomaly identification process is composed of equipment object files, inspection view visibility records, equipment coordinate system, current component relationship diagram, normal structure diagram, allowed occluded components, allowed range of change, re-shoot trigger ratio, and re-shoot consistency confirmation. This process differs from methods that only identify whether equipment components are missing, methods that only compare the current image with historical images at the pixel level, and methods that only center the target equipment by adjusting the camera with a robot. This embodiment limits the anomaly judgment objects to component nodes and component relationship edges in the equipment's own coordinate system, and verifies whether the same component node or the same component relationship edge is continuously abnormal through re-shoot images, so that the anomaly identification process of railway equipment inspection images has a clear data source, processing steps, and confirmation logic.

[0110] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for anomaly recognition in images of equipment inspection along railway lines by a robot, characterized in that, Includes the following steps: S1: Establish a railway line equipment object archive. The railway line equipment object archive includes equipment number, line mileage, track side, equipment type, equipment reference components, allowed obstruction components, historical normal image set, inspection view visibility record and normal structure map. The normal structure map includes normal component nodes and normal component relationship edges. S2: When the robot moves along the railway line for inspection, it collects inspection images and records inspection perspective information at the same time. The inspection perspective information includes the robot's mileage position, camera orientation, shooting distance and camera posture. S3: Based on the robot's mileage location and camera orientation, determine the target equipment corresponding to the current inspection image from the railway equipment object archive, and select a cluster of normal images with comparable viewpoints from the historical normal image set of the target equipment based on the shooting distance and camera posture; S4: Extract the target equipment area from the current inspection image, establish the equipment's own coordinate system based on the equipment reference components of the target equipment, and transform the target equipment area into the equipment's own coordinate system; S5: Perform component identification on the converted target device area and generate a current component relationship graph, which includes current component nodes and current component relationship edges; S6: Compare the current component relationship diagram with the normal structure diagram to determine the structural consistency, and identify the inconsistent content that does not conform to the allowable occlusion component and the inspection view visibility record as abnormal candidate areas. S7: Based on the image quality interference status corresponding to the abnormal candidate region, or based on the inconsistency between the connection relationship, bonding relationship or inclusion relationship corresponding to the abnormal candidate region and the corresponding normal component relationship edge, generate a re-shooting instruction and control the robot to re-acquire the re-shooting image. S8: Extract the target device region from the re-shot image and generate the corresponding current component relationship graph. Based on the consistency of the component nodes or component relationship edges corresponding to the abnormal candidate regions in the initial acquisition image and the re-shot image, output the abnormal identification result.

2. The method for anomaly identification in images of robotic railway equipment inspection according to claim 1, characterized in that, The establishment of the railway line equipment object file includes: using the equipment number of the railway line equipment as the object index, and writing the line mileage, track side, equipment type, equipment reference components, visible component list, allowed obstruction components, historical normal image set, inspection view visibility record, allowed change range, re-shoot trigger ratio and normal structure map corresponding to the same equipment number into the same object file; Among them, the equipment reference component is one of the following: the edge of the enclosure, the boundary of the cover plate, the mounting hole, the fixing bracket, the outline of the lamp room, and the mounting reference edge of the transponder, which maintain a fixed installation position in the inspection image of the corresponding equipment type.

3. The anomaly recognition method for robot-generated railway equipment inspection images according to claim 2, characterized in that, The establishment of the normal structure map includes: labeling or identifying components in the target device region of the historical normal image set to obtain normal component nodes; Generate normal component relationship edges based on the positional relationships of each normal component node in the device's own coordinate system; Among them, normal component nodes include component category, component center coordinates, component outer region, component direction and component visibility ratio, and normal component relationship edges include adjacent relationship, connection relationship, containment relationship, snapping relationship and occlusion relationship.

4. The anomaly recognition method for robot-generated railway equipment inspection images according to claim 3, characterized in that, In step S3, the determination of the comparability of the viewpoints includes: comparing the shooting distance, camera orientation, and camera posture corresponding to the current inspection image with the shooting distance, camera orientation, and camera posture corresponding to each historical normal image in the historical normal image set; When the shooting distance difference, camera orientation difference, and camera posture difference all fall within the allowable difference range recorded in the railway equipment object file for the corresponding equipment type, the corresponding historical normal image will be included in the normal image cluster. If any of the shooting distance difference, camera orientation difference, or camera pose difference does not fall within the allowable difference range, the corresponding historical normal image will not be included in the normal image cluster.

5. The anomaly recognition method for robot-generated railway equipment inspection images according to claim 4, characterized in that, In step S4, the establishment of the device's own coordinate system includes: determining at least two reference points of the device reference component in the target device area, taking the direction of the line connecting the two reference points as the first coordinate axis direction, taking the direction passing through one of the reference points and perpendicular to the first coordinate axis direction as the second coordinate axis direction, and taking the pixel distance between the two reference points as the coordinate normalization scale, and converting the position of each component in the target device area into normalized coordinates relative to the device reference component.

6. The anomaly recognition method for robot-generated railway equipment inspection images according to claim 5, characterized in that, In step S5, the generation of the current component relationship diagram includes: segmenting the transformed target device area into components to obtain the component category, component center coordinates, component outer area, component direction, and component visibility ratio of each current component node; Generate the corresponding current component relationship edge based on the contact state, containment state, edge spacing, center coordinate difference, and occlusion overlap area of ​​any two current component nodes; The current component relationship edges include adjacent relationships, connection relationships, containment relationships, fitting relationships, and occlusion relationships.

7. The method for anomaly recognition in robot-generated railway equipment inspection images according to claim 6, characterized in that, In step S6, the structural consistency comparison includes: matching the current component node with the normal component node according to the component category and the position in the device's own coordinate system, and matching the current component relationship edge with the normal component relationship edge according to the relationship type and the connected component node; An abnormal candidate region is generated when the position and visibility ratio of the current component node relative to the equipment reference component do not meet the allowable variation range corresponding to the normal structural map, or when the connection, fitting, inclusion, or occlusion relationship in the current component relationship edge is inconsistent with the normal component relationship edge, and the inconsistent content does not conform to the allowable occlusion component and inspection view visibility record.

8. The method for anomaly recognition in robot-generated railway equipment inspection images according to claim 7, characterized in that, The inconsistencies mentioned above do not conform to the records of allowed obstructed components and inspection view visibility, including: when a normal component node fails to match the current component node, query the records of allowed obstructed components and inspection view visibility in the railway equipment object archive; If the component category corresponding to the normal component node that failed to match is not recorded in the allowed occlusion components, an abnormal candidate region is generated; If the component category corresponding to the unmatched normal component node is recorded as an allowed occlusion component, but the normal component relationship edge corresponding to the normal component node is recorded as should be visible in the inspection view visibility record, then an abnormal candidate region is generated. The repeat shooting instruction includes the target device number, the position of the abnormal candidate area in the device's own coordinate system, the robot's repeat shooting stop position, the camera rotation angle, and the supplementary lighting parameters; The anomaly identification results include the target equipment number, line mileage, track side identification, equipment type, anomaly component category, the position of the anomaly component in the equipment's own coordinate system, anomaly type, initial image acquisition, re-enactment image, and re-enactment confirmation mark.

9. An anomaly recognition system for robot-generated railway equipment inspection images, applicable to the anomaly recognition method for robot-generated railway equipment inspection images as described in any one of claims 1-8, characterized in that, include: It includes a mobile chassis, an image acquisition device, a positioning device, an attitude acquisition device, a supplementary lighting device, and a processing unit. The image acquisition device, the positioning device, the attitude acquisition device, and the supplementary lighting device are all mounted on the mobile chassis, and the processing unit is connected to the image acquisition device, the positioning device, the attitude acquisition device, and the supplementary lighting device respectively. The image acquisition device is used to acquire inspection images of equipment along the railway line when the robot moves along the railway line for inspection; The positioning device is used to obtain the robot's mileage position; The attitude acquisition device is used to obtain camera orientation, shooting distance, and camera attitude; The supplementary lighting device is used to provide supplementary lighting to the target equipment area during reshoots; The processing unit is used to establish a railway line equipment object archive. The railway line equipment object archive includes equipment number, line mileage, track side, equipment type, equipment reference components, allowed occluded components, historical normal image set, inspection view visibility record and normal structure map. The normal structure map includes normal component nodes and normal component relationship edges. The processing unit is also used to determine the target device corresponding to the current inspection image based on the robot's mileage position and camera orientation, and to select a cluster of normal images that are comparable in viewpoint to the current inspection image from the historical normal image set of the target device based on the shooting distance and camera posture. The processing unit is also used to extract the target equipment area from the current inspection image, establish the equipment's own coordinate system based on the equipment reference components of the target equipment, transform the target equipment area to the equipment's own coordinate system, and perform component identification on the transformed target equipment area to generate the current component relationship graph, which includes the current component node and the current component relationship edge. The processing unit is also used to compare the current component relationship diagram with the normal structure diagram for structural consistency, and to identify inconsistencies that do not conform to the allowable occlusion components and the inspection view visibility record as abnormal candidate areas. The processing unit is also used to generate a reshooting instruction based on the image quality interference status corresponding to the abnormal candidate region, or based on the state that the connection relationship, bonding relationship or inclusion relationship corresponding to the abnormal candidate region is inconsistent with the relationship edge of the corresponding normal component, and to control the robot to re-acquire the reshooting image through the mobile chassis, image acquisition device, posture acquisition device and supplementary lighting device. The processing unit is also used to extract the target device region from the re-shot image and generate the corresponding current component relationship graph. Based on the consistency of the component nodes or component relationship edges corresponding to the abnormal candidate regions in the initial acquisition image and the re-shot image, the abnormal identification result is output.

10. The anomaly recognition system for robot-generated railway equipment inspection images according to claim 9, characterized in that, The processing unit includes an equipment file creation module, a target equipment matching module, an equipment coordinate transformation module, a component relationship diagram generation module, a structural consistency comparison module, a re-shooting control module, and an abnormal result output module. The equipment file creation module is used to create equipment object files along the railway line. The target device matching module is used to determine the target device and normal image clusters based on the robot's mileage position, camera orientation, shooting distance, and camera posture. The equipment coordinate transformation module is used to establish the equipment's own coordinate system based on the equipment's reference components; The component relationship diagram generation module is used to generate the current component node and the current component relationship edge; The structural consistency comparison module is used to compare the current component relationship diagram with the normal structural diagram and generate abnormal candidate regions; The reshoot control module is used to generate a reshoot command based on the image quality interference state corresponding to the abnormal candidate region, or based on the state that the connection relationship, bonding relationship or inclusion relationship corresponding to the abnormal candidate region is inconsistent with the relationship edge of the corresponding normal component. The abnormal result output module is used to output an abnormal identification result when the initial image and the re-captured image correspond to the same target device, the same current component node, or the same current component relationship edge composed of the same group of current component nodes, and the inconsistency type is the same.

Images