Railway limit intrusion detection method and related equipment

Abstract

The embodiment of the invention provides a railway limit intrusion detection method and related equipment, and belongs to the technical field of railway detection. The method comprises the following steps: acquiring a track detection result of track detection and a railway line image; carrying out time difference matching according to the rail inspection result and the railway line image to obtain an image extrinsic parameter; performing image correction processing on the railway line image to obtain a distorted image; performing target detection according to the distorted image to obtain target information of the to-be-detected railway equipment; based on two different feature extraction algorithms, performing feature matching on the distorted image according to the target information to obtain a target feature set; performing triangulation on the target feature sets of the images along the different railways according to the image extrinsic parameters to obtain boundary position coordinates of the railway equipment to be measured in a track coordinate system; and according to the boundary position coordinates and the track detection result, carrying out intrusion detection to obtain an intrusion result. According to the invention, automatic detection can be realized, the accuracy and operation efficiency of limit invasion detection are improved, and data support is provided for track adjustment.

Description

Technical Field

[0001] This application relates to the field of railway inspection technology, and in particular to a railway encroachment detection method and related equipment. Background Technology

[0002] With the rapid development of the railway industry, the railway operating mileage is constantly increasing, and the train speed and density are continuously improving, which puts forward higher requirements for the safety of railway transportation.

[0003] To ensure the normal operation of trains on railway lines and prevent collisions with adjacent structures and equipment, it is necessary to restrict the outlines of trains, railway structures, and equipment. The prescribed dimensions that trains and adjacent structures and equipment are not permitted to exceed are called clearances. Exceeding these safety clearances and encroaching on the train's operating space poses a potential threat to traffic safety. Therefore, railway authorities conduct annual centralized maintenance on railways, inspecting track displacement and smoothness, and checking whether equipment exceeds prescribed limits.

[0004] However, current clearance detection methods involve manually using a measuring tape or a handheld laser rangefinder to record the mileage and the offset of the object from the track. This manual detection method is not only inefficient but also prone to errors, and it cannot be quickly and directly verified when new clearance risks arise. Summary of the Invention

[0005] The main objective of this application is to propose a railway encroachment detection method and related equipment, aiming to achieve automated detection, improve the accuracy and efficiency of encroachment detection, and provide scientific data support for track adjustment.

[0006] To achieve the above objectives, one aspect of this application proposes a railway encroachment detection method, the method comprising: The track inspection results and images along the railway line are obtained, wherein the track inspection results include track coordinates, vehicle attitude information and vehicle position information; Based on the track inspection results and the railway line images, time difference matching is performed to obtain image extrinsic parameters; The images along the railway line are subjected to image correction processing to obtain distortion-free images; Target detection is performed based on the distortion-free image to obtain target information of the railway equipment under test, wherein the target information includes bounding box coordinates and category labels; Based on two different feature extraction algorithms, feature matching is performed on the distortion-reduced image according to the target information to obtain a target feature set, wherein the target feature set includes several visual feature pairs, and each visual feature pair includes two visual feature points pointing to the same device structure; Based on the image extrinsic parameters, triangulation is performed on the target feature sets of different railway line images to obtain the boundary position coordinates of the railway equipment under test in the track coordinate system; Entry limit detection is performed based on the boundary position coordinates and the track inspection results to obtain the entry limit result.

[0007] In some embodiments, the step of performing time difference matching based on the track inspection results and the railway line images to obtain image extrinsics includes the following steps: Obtain the track inspection timestamp of the track inspection results and the photograph timestamp of the railway line image; The track inspection timestamp is queried and matched with the photo timestamp to obtain a track inspection result whose time difference with the railway line image does not exceed a preset difference value, wherein the time difference is the difference between the photo timestamp and the track inspection timestamp; The vehicle attitude information and vehicle position information from the track inspection results are determined as image extrinsic parameters.

[0008] In some embodiments, the image correction processing of the railway line image to obtain a distortion-free image includes the following steps: The pixels in the railway line image are normalized to obtain the normalized coordinates of the pixels; Based on the pre-calibrated camera intrinsic parameter matrix and distortion coefficients, the normalized coordinates are corrected for deviation to obtain the distortion-free coordinates; Pixel mapping is performed based on the distortion-corrected coordinates of all pixels to obtain the distortion-corrected image.

[0009] In some embodiments, the step of performing target detection based on the distortion-reduced image to obtain target information of the railway equipment under test includes the following steps: The model is trained based on a sample image set of pre-set railway equipment labels to obtain a trained equipment detection model; The distortion-reduced image is input into the equipment detection model for target detection to obtain the target information of the railway equipment under test.

[0010] In some embodiments, the step of performing feature matching on the distortion-removed image based on the target information using two different feature extraction algorithms to obtain a target feature set includes the following steps: Image extraction is performed based on the category label and the bounding box coordinates to obtain a local image of the railway equipment under test. Based on the scale-invariant feature transform algorithm, the local image is subjected to a first feature extraction process to obtain a first visual feature set, wherein the first visual feature set includes several first visual feature points. Based on the accelerated robust feature algorithm, the local image is subjected to second feature extraction processing to obtain a second visual feature set, wherein the second feature set includes several second visual feature points; Based on the first visual feature set and the second visual feature set, same-name matching is performed to determine the first visual feature point and the second visual feature point pointing to the same device structure as visual feature pairs, thus obtaining the target feature set.

[0011] In some embodiments, the step of performing triangulation on the target feature sets of different railway line images based on the image extrinsic parameters to obtain the boundary position coordinates of the railway equipment under test in the track coordinate system includes the following steps: Based on the target feature sets of different images along the railway line, an association feature set is obtained by performing association matching. Based on the vehicle attitude information corresponding to each of the railway line images, the coordinate axes are rotated to obtain the rotation matrix of each of the railway line images from the camera coordinate system to the track coordinate system; Based on the vehicle position information and rotation matrix corresponding to each of the railway line images, a translation transformation is performed to obtain the transformation matrix of each of the railway line images from the camera coordinate system to the track coordinate system; Based on the transformation matrix corresponding to different railway line images, an error equation is constructed for the associated feature set by minimizing the square root method, and the error is minimized to obtain the boundary position coordinates of the railway equipment under test in the track coordinate system.

[0012] In some embodiments, the step of performing encroachment detection based on the boundary position coordinates and the track inspection results to obtain the encroachment result includes the following steps: The specification limit value is determined based on the category label corresponding to the boundary position coordinates; Distance detection is performed based on the boundary position coordinates and the track coordinates to obtain the true distance from the railway equipment under test to the track. The intrusion limit result is obtained by comparing the specified limit value with the actual distance.

[0013] To achieve the above objectives, another aspect of this application proposes a railway encroachment detection system, the system comprising: The data acquisition module is used to acquire track inspection results and images along the railway line, wherein the track inspection results include track coordinates, vehicle attitude information and vehicle position information; The time difference matching module is used to perform time difference matching based on the track inspection results and the railway line images to obtain image extrinsic parameters. An image correction module is used to perform image correction processing on the railway line images to obtain distortion-free images; The target detection module is used to perform target detection based on the distortion-free image to obtain target information of the railway equipment under test, wherein the target information includes bounding box coordinates and category labels; The feature matching module is used to perform feature matching on the distortion-removed image based on the target information according to two different feature extraction algorithms to obtain a target feature set, wherein the target feature set includes several visual feature pairs, and each visual feature pair includes two visual feature points pointing to the same device structure; The triangulation module is used to perform triangulation on the target feature set of different railway line images based on the image extrinsic parameters, so as to obtain the boundary position coordinates of the railway equipment under test in the track coordinate system; The intrusion detection module is used to perform intrusion detection based on the boundary position coordinates and the track inspection results to obtain the intrusion result.

[0014] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0015] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0016] To achieve the above objectives, another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0017] The embodiments of this application include at least the following beneficial effects: This application provides a railway encroachment detection method and related equipment. This scheme acquires track inspection results and railway line images, performs time difference matching based on the track inspection results and railway line images to obtain image extrinsic parameters, and enables linked analysis of track inspection and encroachment detection, breaking down the barriers of data isolation. By performing image correction processing on railway line images to obtain distortion-free images, target detection is performed based on the distortion-free images to obtain target information of the railway equipment under test, enabling intelligent identification and classification of various types of equipment. By using two different feature extraction algorithms, feature matching is performed on the distortion-free images based on the target information to obtain target feature sets, which can effectively overcome difficulties such as changes in railway ambient lighting and weak textures on equipment surfaces, improving the accuracy and robustness of matching. By performing triangulation on the target feature sets of different railway line images based on image extrinsic parameters, the boundary position coordinates of the railway equipment under test in the track coordinate system are obtained; encroachment detection is performed based on the boundary position coordinates and track inspection results to obtain encroachment results, enabling automated detection, improving the accuracy of encroachment detection, and providing scientific data support for track adjustment. Attached Figure Description

[0018] Figure 1 This is a flowchart of the railway encroachment detection method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the on-site clearance detection operation provided in the embodiments of this application; Figure 3 This is a schematic diagram of the track inspection trolley provided in the embodiments of this application; Figure 4 This is a schematic diagram of the signal box boundary provided in an embodiment of this application; Figure 5 This is a schematic diagram of the track inspection trolley being pushed and photographed according to an embodiment of this application; Figure 6 This is a schematic diagram of the three-camera coordinate system provided in an embodiment of this application; Figure 7 This is a flowchart of a railway encroachment detection method provided in another embodiment of this application; Figure 8 This is a schematic diagram of the railway encroachment detection system provided in the embodiments of this application; Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of systems and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0021] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0022] (1) LKJ equipment: LKJ equipment is short for train safety operation monitoring device. It is a safety protection device for railway locomotives, used to monitor, control and adjust the train's running speed and braking mode according to the objective conditions and actual situation of the train running on the railway line.

[0023] (2) Overhead contact network: The overhead contact network in railways is a special type of power transmission line that provides power to electric locomotives. It consists of several parts, including contact suspension, support device, positioning device, support column and foundation.

[0024] (3) Station area: The area centered on the station, used for passenger boarding and alighting and train stopping operations.

[0025] (4) Equipment encroachment: Equipment encroachment refers to the situation where railway equipment exceeds the prescribed safety limit for various reasons, enters the train operating space, and poses a threat to the safety of train operation.

[0026] Next, some related technologies involved in the embodiments of this application will be described.

[0027] In related technologies, to prevent equipment from exceeding limits, railway authorities are required to conduct centralized maintenance on passenger dedicated line equipment every year, and to test track displacement and smoothness, and whether railway equipment exceeds the prescribed range.

[0028] However, the current track and clearance inspections are conducted separately. The clearance inspection method involves assigning additional personnel to manually measure the distance and the offset of the object from the track using a measuring tape or a handheld laser rangefinder. Due to the large workload of clearance inspections, only a portion of the equipment is often inspected, making it difficult to cover all railway equipment. Uninspected railway equipment may become the cause of the next accident, leaving behind safety hazards.

[0029] Moreover, even if the on-site inspection is qualified, the data from track inspection and equipment encroachment detection are isolated and lack systematicity. When there is a large deviation in the track, such as after the tamping / lifting operation of a large machine, the track alignment changes and there is a risk of new encroachment. Current methods cannot quickly and directly verify this.

[0030] In view of this, this application provides a railway encroachment detection method and related equipment. This method acquires track inspection results and railway line images, performs time difference matching based on the track inspection results and railway line images to obtain image extrinsic parameters, and enables linked analysis of track inspection and encroachment detection, breaking down data silos. By performing image correction processing on the railway line images to obtain distortion-free images, target detection is performed based on the distortion-free images to obtain target information of the railway equipment under test, enabling intelligent identification and classification of various types of equipment. By using two different feature extraction algorithms, feature matching is performed on the distortion-free images based on the target information to obtain a target feature set, effectively overcoming challenges such as changes in railway ambient lighting and weak surface texture of equipment, improving the accuracy and robustness of matching. By performing triangulation on the target feature sets of different railway line images based on the image extrinsic parameters, the boundary position coordinates of the railway equipment under test in the track coordinate system are obtained; encroachment detection is performed based on the boundary position coordinates and track inspection results to obtain the encroachment result, enabling automated detection, improving the accuracy of encroachment detection, and providing scientific data support for track adjustment.

[0031] The railway encroachment detection method provided in this application relates to the field of railway inspection. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the railway encroachment detection method, but is not limited to the above forms.

[0032] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0033] Figure 1 This is an optional flowchart of the railway encroachment detection method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S107.

[0034] Step S101: Obtain the track inspection results and railway line images, wherein the track inspection results include track coordinates, vehicle attitude information and vehicle position information.

[0035] Step S102: Based on the track inspection results and the images along the railway line, perform time difference matching to obtain image extrinsic parameters.

[0036] Step S103: Perform image correction processing on the railway line images to obtain distortion-free images.

[0037] Step S104: Target detection is performed based on the distortion-free image to obtain target information of the railway equipment to be tested, including bounding box coordinates and category labels.

[0038] Step S105: Based on two different feature extraction algorithms, feature matching is performed on the distortion-removed image according to the target information to obtain the target feature set. The target feature set includes several visual feature pairs, and each visual feature pair includes two visual feature points pointing to the same device structure.

[0039] Step S106: Triangulation is performed on the target feature sets of different railway line images based on the image extrinsic parameters to obtain the boundary position coordinates of the railway equipment under test in the track coordinate system.

[0040] Step S107: Perform intrusion detection based on the boundary position coordinates and track inspection results to obtain the intrusion result.

[0041] In this embodiment, unlike... Figure 2 The manual on-site clearance inspection operation shown requires two people to conduct track inspection using a track inspection trolley, and another two people to conduct clearance inspection of ground features using a measuring tape. In this embodiment, three cameras are installed on the track inspection trolley to simultaneously conduct track inspection and equipment clearance inspection.

[0042] Specifically, such as Figure 3 As shown, the track inspection trolley is equipped with a first camera 100 facing the left side to collect images of the railway line on the left, a second camera 200 facing the top to collect images of the railway line at the top, and a third camera 300 facing the right side to collect images of the railway line on the right.

[0043] Alternatively, lighting devices can be installed on the three cameras to facilitate lighting inside the tunnel or at night.

[0044] The track inspection trolley measures relevant data such as the track centerline and track surface elevation using a GNSS / IMU mounted on the vehicle to obtain track inspection results. The track inspection results include, but are not limited to, track coordinates, track gauge, vehicle attitude information, vehicle position information, and time information.

[0045] During the track inspection trolley's operation, three cameras continuously photograph the environment surrounding the railway to capture railway equipment such as signal boxes, electrical boxes, and overhead contact lines on both sides of the railway. While the cameras are taking pictures, GNSS or IMU time information is simultaneously recorded for each image along the railway line.

[0046] As the track inspection trolley is pushed along the railway, a large number of track inspection results and railway images at different times and locations can be collected, eliminating the need for manual inspection and recording of encroachment data using a measuring tape or handheld laser rangefinder.

[0047] After data acquisition is completed, the track inspection results and railway line images are read. Based on the time of the railway line images, the data stream of the track inspection results is searched and matched to find the vehicle attitude information and vehicle position information that are closest to the shooting time. Each railway line image is assigned the spatial pose corresponding to the shooting moment, thereby obtaining image extrinsic parameters used to describe the relative position and orientation relationship between the camera coordinate system and the track coordinate system.

[0048] Furthermore, in order to eliminate the impact of geometric distortion introduced by the lens optical characteristics on the measurement accuracy, distortion correction processing is performed on the railway line images to obtain distortion-corrected images that are closer to the real spatial relationships.

[0049] Next, to effectively improve computational efficiency, a deep learning object detection method is used to detect specific objects in the distortion-free image. This identifies pre-defined railway equipment to be detected from the distortion-free image, such as signal boxes, overhead contact lines, signal lights, platforms, and guardrails. Because railway equipment is diverse, and different equipment or structures have different encroachment limits—for example, the distance from a platform to the track is very short, while the distance from a fence to the track is relatively long—object detection not only needs to detect the bounding box coordinates of the railway equipment in the distortion-free image but also needs to output the category label of the railway equipment to obtain target information.

[0050] For example, such as Figure 4 As shown, the railway equipment to be tested captured by the railway line image is a signal box. When the boundary outline of the signal box is identified in the railway line image through target detection, a rectangular frame is located in the railway line image. This rectangular frame is the smallest bounding rectangle of the boundary outline of the signal box. The purpose of the rectangular frame is to enclose the boundary outline of the signal box, but it does not describe the precise shape of the object itself.

[0051] Optionally, the coordinates of the rectangle can be taken from the coordinates of the four vertices or from the coordinates of a pair of opposite diagonal vertices.

[0052] After defining the boundary outline, the railway equipment to be tested within the rectangular area is classified. A category label is determined from a variety of preset railway equipment as the identification result. For example, through target detection, the category label is determined to be signal box from preset labels such as signal box and signal light.

[0053] It is understandable that if there are multiple railway devices to be tested captured by images along the railway line, then each railway device to be tested will be identified with a corresponding target information.

[0054] After accurately locating the target information of the railway equipment under test, feature extraction is performed only on the image within the detected boundary of the railway equipment under test.

[0055] Specifically, in the feature extraction process, to avoid the situation where a single feature point extraction algorithm cannot cope with the low light of tunnels and the strong light of bridges in railway scenes, which would lead to subsequent feature point matching failures or mismatches, this embodiment adopts a dual feature redundancy verification mechanism. Two different feature extraction algorithms are used to extract features from the same image separately. Only when the visual feature point is identified under both different feature extraction methods is it considered to be a real visual feature point.

[0056] For example, if the first feature extraction algorithm identifies screws and rust spots in an image, while the second feature extraction algorithm identifies screws and nameplates in the image, then the screws identified by both feature extraction algorithms are considered a visual feature pair.

[0057] Rust spots identified only by the first feature extraction algorithm and nameplates identified only by the second feature extraction algorithm are considered as non-existent visual feature points because they lack visual feature points pointing to the same equipment structure.

[0058] By matching and cross-validating the visual feature points extracted from the same image, a target feature set containing several visual feature pairs is finally obtained.

[0059] For the target feature set that has been matched, based on the target feature set and image extrinsic parameters corresponding to railway images at different times and from different perspectives, the three-dimensional coordinates of each visual feature point in the track coordinate system are calculated using the principle of triangulation. By aggregating a large number of three-dimensional coordinates representing the outline or key parts of the railway equipment to be tested, the boundary position coordinates of the railway equipment to be tested in the track coordinate system are obtained. The track coordinate system is determined by railway design data, and the track-related data are known data designed in the early stage of railway construction.

[0060] Finally, in the intrusion detection stage, the distance from the railway equipment under test to the track is calculated using the calculated boundary position coordinates and the track coordinates obtained from the track inspection results. Based on the calculated distance, the equipment is checked to see if it exceeds the specified range, and the intrusion result is obtained.

[0061] For example, such as Figure 3 As shown, there is a signal box on the left side of the track, a contact wire support rod on the right side of the track, and a contact wire support rod at the top of the track. Calculate the distance from any point on the side of each piece of equipment facing the track to the track coordinates.

[0062] For a signal box, the side facing the track is the right side. The horizontal distance is calculated by taking the boundary position coordinates of any point on the right side and the track coordinates of the center line on the left side of the track.

[0063] For the overhead contact wires, the side facing the track is the bottom. The vertical distance between the boundary coordinates of any point on the bottom and the track coordinates is calculated to obtain the vertical distance.

[0064] The horizontal distance of the signal box is detected according to the clearance range specified in the standard, and the vertical distance of the contact wire is detected according to the clearance range of the contact wire, so as to obtain the encroachment result.

[0065] This embodiment integrates three cameras on a GNSS / IMU track inspection trolley, synchronizing with the GNSS / IMU time using the time provided by the cameras. It then collects and stores real-time image data from both sides and the top, eliminating the need for manual inspection and recording, thus achieving automated data acquisition for encroachment detection. Furthermore, by fusing position and attitude information provided by the GNSS / IMU, real-time phase center and attitude information are provided to the moving cameras, enabling a transition from static to dynamic image measurement. Finally, data collected from multiple photographs of the same feature by the mobile track inspection trolley cameras is used to implement railway equipment measurement functions through algorithms. This achieves automated detection, improves the accuracy of encroachment detection, and provides scientific data support for track adjustment.

[0066] In some embodiments, step S102 may include, but is not limited to, steps S201 to S203.

[0067] Step S201: Obtain the track inspection timestamp of the track inspection results and the photo capture timestamp of the railway line images.

[0068] Step S202: Query and match the track inspection timestamp based on the photo timestamp to obtain the track inspection result with a time difference of no more than a preset difference from the railway line image, where the time difference is the difference between the photo timestamp and the track inspection timestamp.

[0069] Step S203: Determine the vehicle attitude information and vehicle position information from the track inspection results as image extrinsic parameters.

[0070] In this embodiment, in order to achieve the fusion of railway line images and spatial pose, the time synchronization of measurement data must first be completed.

[0071] Specifically, during the operation of the track inspection trolley, the system receives and buffers two data streams in parallel. The first is the track inspection result generated by the GNSS / IMU combination. The track inspection timestamp of the track inspection result usually uses GNSS time or relative time. The 0th moment of the relative time refers to the power-on time. The second is the railway line image collected by the track inspection trolley. The system also records the time stamp of the camera when the railway line image is captured.

[0072] The time difference matching uses the timestamp of each railway line image as the query condition to search and match the track inspection results of the GNSS / IMU track inspection vehicle. The specific matching strategy is to traverse the track inspection results, calculate the time difference by comparing the image timestamp with each track inspection timestamp, and then filter out one or more sets of track inspection results whose time differences are less than or equal to a preset difference.

[0073] Preferably, when there are multiple sets of track inspection results, the track inspection result with the smallest time difference is selected to improve the accuracy of spatiotemporal correlation.

[0074] Once the most timely track inspection result is found for each railway line image, the three-dimensional coordinates and orientation of the railway line image at the time of capture can be obtained, providing image extrinsic parameters for each railway line image.

[0075] Specifically, the vehicle attitude information and vehicle location information in the track inspection results are extracted and bound to the railway line images for data binding, thus combining the vehicle attitude information of this set of track inspection results ( ψ , θ , φ ) and vehicle location information ( x v , y v , z v This serves as the image extrinsic parameter for the frame, thus providing a geometric transformation basis for the subsequent conversion of two-dimensional pixel information to three-dimensional orbital space.

[0076] In some embodiments, step S103 may include, but is not limited to, steps S301 to S303.

[0077] Step S301: Normalize the pixels in the railway line image to obtain the normalized coordinates of the pixels.

[0078] Step S302: Based on the pre-calibrated camera intrinsic parameter matrix and distortion coefficients, the normalized coordinates are corrected for deviation to obtain the distortion-free coordinates.

[0079] Step S303: Perform pixel mapping based on the distortion coordinates of all pixels to obtain the distortion-free image.

[0080] In this embodiment, distortion correction processing is performed on each captured railway line image. The railway line image is an image composed of pixels. Each original captured image contains n pixel coordinates. To ensure that the pixel positions in the image can reflect the real scene, each pixel is first normalized according to the row and column coordinates of the pixel.

[0081] Specifically, the pixels in the railway line images are normalized, and the pixel coordinates of the pixels are normalized. u o , v o Transform the coordinates to a camera coordinate system with the camera's optical center as the origin to obtain the normalized coordinates of the pixels. x d , y d ).

[0082] Next, each camera has different camera intrinsic parameters and distortion coefficients. Using a pre-calibrated camera intrinsic parameter matrix K and distortion coefficients [k1k2p1p2], the normalized coordinates are subjected to distortion correction to correct pixel coordinate deviations. This more accurately reflects the ideal spatial position of the object, resulting in distorted coordinates. u d , v d The calculation formula for distortion correction is as follows: (1); in,( f x , f y ( ) represents the focal length of the camera in the x and y directions. C x , C y ) represents the coordinates of the camera's optical center on the image plane.

[0083] By backtracking the original railway line images using the distortion-reduced coordinates, the pixel values ​​in the railway line images are mapped back to the pixel coordinate system of the pixel points to obtain the distortion-reduced images, thereby eliminating the influence of lens distortion and providing reliable visual data for subsequent feature extraction and accurate measurement.

[0084] In some embodiments, step S104 may include, but is not limited to, steps S401 to S402.

[0085] Step S401: Train the model based on the sample image set of the preset railway equipment labels to obtain the trained equipment detection model.

[0086] Step S402: Input the distortion-reduced image into the device detection model to perform target detection and obtain the target information of the railway equipment under test.

[0087] In this embodiment, in order to effectively improve computational efficiency, the target detection model is first trained by supervised deep learning using a large set of sample images labeled with railway equipment tags, and a trained equipment detection model is obtained.

[0088] Specifically, the sample image set contains a large number of labeled railway line images. Each railway equipment appearing in each railway line image is labeled with precise bounding box information and corresponding railway equipment labels. These railway line images cover various lighting conditions, weather conditions, shooting angles, etc., to ensure the generalization ability of the model.

[0089] By iteratively optimizing and continuously adjusting the model parameters within the target monitoring model, the target detection model can eventually locate various railway equipment from complex backgrounds and correctly output the category label of the railway equipment.

[0090] The distortion-reduced image obtained in step S103 is used as input data and fed into the pre-trained equipment detection model. The equipment detection model performs target detection on specific objects. The equipment detection model can intelligently identify and classify various railway equipment and output the bounding box coordinates of the detected specific objects. x min , y min , x max , y max The system uses data from the railway equipment under test and category labels to obtain target information.

[0091] In some embodiments, step S105 may include, but is not limited to, steps S501 to S504.

[0092] Step S501: Extract images based on category labels and bounding box coordinates to obtain local images of the railway equipment under test.

[0093] Step S502: Based on the scale-invariant feature transformation algorithm, the local image is subjected to first feature extraction processing to obtain a first visual feature set, wherein the first visual feature set includes several first visual feature points.

[0094] Step S503: Based on the accelerated robust feature algorithm, perform second feature extraction processing on the local image to obtain a second visual feature set, wherein the second feature set includes several second visual feature points.

[0095] Step S504: Perform same-name matching based on the first visual feature set and the second visual feature set, and determine the first visual feature point and the second visual feature point pointing to the same device structure as a visual feature pair to obtain the target feature set.

[0096] In this embodiment, image extraction is performed based on category labels and bounding box coordinates. Local images of a single railway equipment under test are extracted from the distortion-free image, reducing the amount of data required for subsequent processing. The target of interest is measured directly, avoiding the processing of a large amount of useless environmental information, improving computational efficiency, and demonstrating strong purposefulness.

[0097] After acquiring the local image, two different feature extraction algorithms are used in parallel to extract features from the same local image.

[0098] Specifically, SIFT (Scale Invariant Feature Transform) is used to perform first feature extraction processing on the local image to obtain a first visual feature set including several first visual feature points and their descriptors. At the same time, SURF (Speed-Up Robust Feature Transform) is used to perform second feature extraction processing on the local image to obtain a second visual feature set including several second visual feature points and their descriptors.

[0099] Based on the results of the two algorithms, a matching process is performed. If the error between the first visual feature point and the second visual feature point is less than the preset matching threshold, it can be considered that although the two visual feature points are generated based on different mathematical principles, they actually point to the same device structure in the local image. Therefore, the first visual feature point and the second visual feature point pointing to the same device structure are determined as a visual feature pair. Finally, all visual feature pairs that pass the matching process are gathered to form the target feature set.

[0100] It should be noted that the target feature set must include at least two visual feature pairs. The more visual feature pairs in the target feature set, the higher the accuracy of the coordinate values ​​calculated by subsequent triangulation.

[0101] It is understood that, in addition to SIFT and SURF, other feature extraction algorithms such as HOG (Histogram of Oriented Gradients) and LBP (Local Binary Pattern) can also be used. This application does not impose specific limitations on these algorithms.

[0102] This embodiment effectively overcomes challenges such as changes in railway ambient lighting and weak textures on equipment surfaces by employing a dual-feature algorithm redundancy verification mechanism of SIFT and SURF, significantly improving the accuracy and robustness of feature matching.

[0103] In some embodiments, step S106 may include, but is not limited to, steps S601 to S604.

[0104] Step S601: Perform association matching based on the target feature sets of different railway line images to obtain the associated feature set.

[0105] Step S602: Rotate the coordinate axes according to the vehicle attitude information corresponding to each railway line image to obtain the rotation matrix of each railway line image from the camera coordinate system to the track coordinate system.

[0106] Step S603: Perform translation transformation based on the vehicle position information and rotation matrix corresponding to each railway line image to obtain the transformation matrix of each railway line image from the camera coordinate system to the track coordinate system.

[0107] Step S604: Based on the transformation matrix corresponding to different railway line images, construct an error equation for the associated feature set by minimizing the squares method, and minimize the error to obtain the boundary position coordinates of the railway equipment under test in the track coordinate system.

[0108] In this embodiment, the core of 3D reconstruction and coordinate calculation lies in converting pixel information in a 2D image into 3D coordinates, such as... Figure 5 As shown, this embodiment uses a track inspection trolley to acquire multiple railway line images of the same target object at different positions and orientations, and performs triangulation based on the multiple railway line images to calculate the three-dimensional coordinates of the railway equipment under test in the track coordinate system.

[0109] Specifically, firstly, target feature sets are collected from different railway line images. Each image's target feature set contains multiple salient visual feature points on the equipment surface. By performing cross-image association matching, it is determined which visual feature points in different railway line images are projections pointing to the same visual structure. This associates the pixel coordinates of the same visual structure across multiple different railway line images, resulting in an associated feature pair. If the same visual structure appears in N images, an associated feature pair containing N visual feature points is formed. The visual feature pairs of different visual structures are then aggregated to obtain the associated feature set.

[0110] Understandably, the more perspectives used, the more visual feature points are in the associated feature pairs, and the more accurate and reliable the calculated coordinates become.

[0111] Furthermore, after establishing cross-image associations, a mathematical transformation relationship between the camera coordinate system and the track coordinate system is constructed for each railway line image involved in coordinate calculation.

[0112] Specifically, such as Figure 6 As shown, the track inspection trolley is equipped with three cameras, each pointing in a different direction. The railway line images captured by different cameras each have their own corresponding camera coordinate system. In order to unify the conversion from the camera coordinate system to the track coordinate system, the vehicle attitude information is first matched based on the time difference of each railway line image. ψ , θ , φ Construct the rotation matrix R from the camera coordinate system to the orbit coordinate system: R=R z ( ψ )R y ( θ )R x ( φ (2).

[0113] For example, the vehicle attitude information describes the orientation of the track inspection trolley at the moment of taking a picture. Based on the vehicle attitude information, a three-dimensional rotation matrix is ​​constructed through the rotation coordinate axis. This matrix defines the orientation transformation relationship between each camera coordinate system and the fixed track coordinate system, thereby correcting the influence of the camera's attitude change caused by its orientation.

[0114] For the camera on the left, the Z-axis of its camera coordinate system is in the same direction as the Z-axis of the orbital coordinate system. The X and Y axes of its camera coordinate system are -90° different from the X and Y axes of the orbital coordinate system. Therefore, to transform from the camera coordinate system to the orbital coordinate system, it is necessary to rotate 90° clockwise around the Z-axis.

[0115] For the camera at the top, the direction of its camera coordinate system Z-axis differs from that of the orbit coordinate system Z-axis by 90°, and the direction of its camera coordinate system X-axis and Y-axis differs from that of the orbit coordinate system X-axis and Y-axis by -90°. Therefore, to transform from the camera coordinate system to the orbit coordinate system, it is necessary to first rotate 90° clockwise around the Z-axis and then rotate 90° counterclockwise around the X-axis.

[0116] For the camera on the right, the Z-axis of its camera coordinate system is in the same direction as the Z-axis of the orbital coordinate system. The X and Y axes of its camera coordinate system are 90° different from the X and Y axes of the orbital coordinate system. Therefore, to transform from the camera coordinate system to the orbital coordinate system, it is necessary to rotate 90° counterclockwise around the Z-axis.

[0117] Next, based on the determined rotational relationship, the position information of the track inspection trolley is combined ( x v , y v , z v Construct the transformation matrix from the camera coordinate system to the orbit coordinate system. T c ： (3); in, t = ( x v , y v , z v ) T .

[0118] Specifically, the vehicle position information provides the position information of the track inspection trolley in the track coordinate system when capturing images along the railway line. By fusing the aforementioned rotation matrix R of each camera and the vehicle position information, the translation amount of the coordinate system transformation is calculated to eliminate the deviation between the origin of the camera coordinate system and the origin of the track coordinate system, thus obtaining the transformation matrix. T c .

[0119] Finally, using the principle of triangulation, combined with the camera intrinsic parameter matrix K and the transformation matrix... T c Calculate the boundary position coordinates of the railway equipment under test in the track coordinate system.

[0120] Specifically, suppose that in the set of associated features, the normalized coordinates of the feature points contained in an associated feature pair in different images are respectively (x... i ,y i ), (x i+1 ,y i+1 )……(x) n ,y n The corresponding camera projection matrices are P i P i+1 ...P n The transformation matrices from the camera coordinate system to the orbit coordinate system are respectively T c_i , T c_i+1 , T c_n The three-dimensional coordinates P of the railway equipment under test can then be calculated using the following formula. xyz : (4); Among them, P i + It is the pseudo-inverse of the camera projection matrix. The camera projection matrix P is 3*4 in dimensions (intrinsic and extrinsic parameters), and its pseudo-inverse is 4*3 in dimensions. Therefore, (P) i + *P i+1 ) is a 4x4 dimension matrix.

[0121] By rearranging formula (4), we obtain: (5).

[0122] By using the least squares method to construct the error equation for formula (5) and minimizing the error, the optimal coordinate value of the object's position can be obtained. This optimal coordinate value can then be used as the final boundary position coordinate of the device under test in the orbital coordinate system.

[0123] In some embodiments, step S107 may include, but is not limited to, steps S701 to S703.

[0124] Step S701: Determine the specification limit value based on the category label corresponding to the boundary position coordinates.

[0125] Step S702: Distance detection is performed based on the boundary position coordinates and track coordinates to obtain the true distance from the railway equipment to be tested to the track.

[0126] Step S703: Compare the standard limit value and the actual distance to obtain the intrusion result.

[0127] In this embodiment, the purpose of the intrusion detection is to compare the boundary coordinates of the railway equipment under test with the standard specifications for safe railway operation. After obtaining the boundary coordinates of the railway equipment under test, the system queries a pre-set railway safety clearance database based on the category label of the railway equipment under test. This database integrates the standards specified for various types of railway equipment, such as the "Standard Track Railway Locomotive and Rolling Stock Clearance" and the "Railway Station and Hub Design Specification". By querying the database according to the category label, the system will determine a standard clearance value that strictly corresponds to the category label for the current railway equipment under test. This value is used as a safety distance to determine whether the position of the railway equipment exceeds the specified range.

[0128] For example, when the railway equipment to be tested is a platform, by consulting the "Technical Specifications for Rail Transit", it is found that the distance between the edge of the platform and the center line of the track should not encroach on the vehicle clearance. On straight sections, the horizontal gap between the edge of the platform and the floor of the carriage should not be greater than 100mm, and on curved sections, the horizontal gap between the edge of the platform and the floor of the carriage should not be greater than 180mm.

[0129] After defining the safety standard limits, the system randomly performs spatial geometric relationship calculations to detect the actual distance between the railway equipment under test and the track.

[0130] Specifically, the distance detection algorithm takes boundary position coordinates and track coordinates as inputs. The boundary position coordinates and track coordinates define the three-dimensional spatial positions of any point on the surface of the railway equipment under test facing the track and the center line of the track in the track coordinate system, respectively. By calculating the shortest straight-line distance between the boundary position coordinates and the track coordinates, the true distance from the railway equipment under test to the track is obtained. The output result is a quantitative intrusion limit based on absolute coordinates, which can be directly integrated with track geometric state data (such as track orientation, track gauge, level, and elevation) to provide accurate and scientific data support and adjustment scheme suggestions for track maintenance and equipment improvement, forming a closed loop of detection, adjustment, and analysis.

[0131] Finally, an automated decision-making process is performed based on the standard clearance values ​​and the actual distance. The calculated actual distance is compared in real time with the pre-determined standard clearance values ​​to obtain the violation result. The comparison logic is to check whether the actual distance exceeds the standard clearance values. If the actual distance does not exceed the standard clearance values, the railway equipment under test is determined to be in a safe state; otherwise, if the actual distance exceeds the standard clearance values, the railway equipment under test is determined to have violated the clearance values.

[0132] The following is a detailed description and explanation of the solutions in the embodiments of the present invention, using specific application examples: This application embodiment uses a track inspection trolley equipped with a camera to automatically complete data collection, processing, and judgment while traveling at high speed, which greatly improves the work efficiency and realizes automated dynamic detection of intrusion detection, enabling a comprehensive survey of all railway equipment on the line.

[0133] Specifically, such as Figure 7 As shown, during the data acquisition phase, three cameras continuously photograph the object while the track inspection trolley is operating, and simultaneously record and store the IMU time information.

[0134] During the data preprocessing stage, the synchronization information of the data acquisition process is read, and the coordinates of the GNSS / IMU track inspection vehicle are matched by the time difference. x v , y v , z v ) and posture ( ψ , θ , φ This provides external references for each image along the railway line.

[0135] In the image distortion processing stage, each camera has different camera intrinsic parameters K and distortion coefficients [k1k2p1p2]. Using the pre-calibrated camera intrinsic parameter matrix and distortion coefficients, distortion removal processing is performed on each captured image.

[0136] In the equipment identification phase, the equipment categories to be identified, such as signal boxes, overhead contact lines, traffic lights, platforms, and guardrails, are pre-defined. A deep learning-based object detection method is used to detect specific objects in the distortion-free image. The image is then input into a trained object detection model, which outputs the bounding box coordinates and category labels of the detected objects.

[0137] In the feature extraction and same-name matching stage, only for images within the boundaries of the detected specific objects, both SIFT and SURF feature extraction algorithms are used to extract feature points and their descriptors. For the results of the two algorithms, only matching point pairs that meet the error less than a set threshold are retained to form the final matched visual feature pairs.

[0138] In the coordinate system transformation and attitude matrix construction stage, based on the attitude information of the GNSS / IMU track inspection vehicle ( ψ , θ , φ ), construct the rotation matrix R from the camera coordinate system to the orbit coordinate system. Then, combine the position information of the track inspection trolley ( x v , y v , z v Construct the transformation matrix from the camera coordinate system to the orbit coordinate system. Tc .

[0139] In the coordinate calculation stage, for the visual feature pairs that have been matched, the boundary position coordinates (X, Y, Z) of the railway equipment under test in the track coordinate system are calculated by using the triangulation principle, combining different railway line images with the camera intrinsic parameter matrix K, and the transformation matrix Tc from the camera coordinate system to the track coordinate system.

[0140] In the stage of outputting clearance results, the distance from the railway equipment under test to the track is calculated using the boundary position coordinates (the track coordinates are obtained by measuring the track inspection trolley). This distance, along with the identified category label of the railway equipment under test, is used as the clearance result for the railway equipment under test. Based on the clearance result, the standard clearance value of the equipment is queried, and the distance calculated by the test is compared with the standard distance to determine whether it exceeds the safety clearance.

[0141] The clearance data obtained in this embodiment can assist in the formulation of high-quality track adjustment technical solutions. This application achieves absolute coordinate measurement by linking track detection and clearance detection through deep fusion of high-precision GNSS / IMU pose information during track detection, thus eliminating cumulative errors. At the same time, by integrating track and railway equipment into the same system for measurement, when the track adjustment alignment changes, the clearance detection method provided by this application can quickly and directly verify the tilt status of each piece of equipment, promptly identify new tilt risks, and provide scientific and quantitative decision support for track adjustment.

[0142] Reference Figure 8 This application also provides a railway encroachment detection system that can implement the above-described method. The system includes: The data acquisition module is used to acquire track inspection results and images along the railway line. The track inspection results include track coordinates, vehicle attitude information, and vehicle position information.

[0143] The time difference matching module is used to perform time difference matching based on track inspection results and images along the railway line to obtain image extrinsic parameters.

[0144] The image correction module is used to perform image correction processing on images along the railway line to obtain distortion-free images.

[0145] The target detection module is used to perform target detection based on the distortion-free image to obtain the target information of the railway equipment under test. The target information includes bounding box coordinates and category labels.

[0146] The feature matching module is used to perform feature matching on the distortion-removed image based on two different feature extraction algorithms and target information to obtain a target feature set. The target feature set includes several visual feature pairs, and each visual feature pair includes two visual feature points pointing to the same device structure.

[0147] The triangulation module is used to perform triangulation on target feature sets of different railway line images based on image extrinsic parameters, so as to obtain the boundary position coordinates of the railway equipment under test in the track coordinate system.

[0148] The intrusion detection module is used to perform intrusion detection based on the boundary position coordinates and track inspection results, and obtain the intrusion result.

[0149] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0150] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0151] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0152] Reference Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0153] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the methods described in the embodiments of this application.

[0154] The input / output interface 903 is used to implement information input and output.

[0155] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0156] Bus 905 transmits information between various components of the device, such as processor 901, memory 902, input / output interface 903, and communication interface 904.

[0157] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0158] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0159] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0160] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0161] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0162] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0163] The railway encroachment detection method and related equipment provided in this application acquire track inspection results and railway line images. By performing time difference matching based on the track inspection results and railway line images, image extrinsic parameters are obtained, enabling linked analysis of track inspection and encroachment detection, breaking down data silos. Image correction processing is performed on the railway line images to obtain distortion-free images. Target detection is then performed on the distortion-free images to obtain target information of the railway equipment under test, enabling intelligent identification and classification of various types of equipment. Based on two different feature extraction algorithms, feature matching is performed on the distortion-free images according to the target information to obtain target feature sets. This effectively overcomes challenges such as changes in railway ambient lighting and weak surface textures of equipment, improving the accuracy and robustness of matching. By performing triangulation on the target feature sets of different railway line images based on image extrinsic parameters, the boundary position coordinates of the railway equipment under test in the track coordinate system are obtained. Encroachment detection is then performed based on the boundary position coordinates and track inspection results to obtain encroachment results. This enables automated detection, improves the accuracy of encroachment detection, and provides scientific data support for track adjustment.

[0164] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0165] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0166] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0167] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0168] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0169] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0170] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0171] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0172] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0173] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0174] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method of railway gauge corner detection, the method comprising: The method comprises the following steps: Obtaining track detection results and railway line images, wherein the track detection results comprise track coordinates, carrier attitude information and carrier position information; Performing time difference matching according to the track detection results and the railway line images to obtain image external parameters; Performing image correction processing on the railway line images to obtain distortion-free images; Performing target detection according to the distortion-free images to obtain target information of the railway equipment to be detected, wherein the target information comprises boundary box coordinates and category labels; Performing feature matching on the distortion-free images according to the target information based on two different feature extraction algorithms to obtain a target feature set, wherein the target feature set comprises a plurality of visual feature pairs, and each visual feature pair comprises two visual feature points pointing to the same device structure; Performing triangulation on the target feature set of different railway line images according to the image external parameters to obtain boundary position coordinates of the railway equipment to be detected in a track coordinate system; Performing intrusion detection according to the boundary position coordinates and the track detection results to obtain an intrusion result.

2. The method of claim 1, wherein, The time difference matching according to the track detection results and the railway line images to obtain image external parameters comprises the following steps: Obtaining track detection timestamps of the track detection results and photographing timestamps of the railway line images; Performing query matching on the track detection timestamps according to the photographing timestamps to obtain track detection results with a time difference of no more than a preset difference value from the railway line images, wherein the time difference is the difference between the photographing timestamp and the track detection timestamp; Determining the carrier attitude information and the carrier position information of the track detection results as image external parameters.

3. The method of claim 1, wherein, The image correction processing on the railway line images to obtain distortion-free images comprises the following steps: Performing normalization processing on pixel points in the railway line images to obtain normalized coordinates of the pixel points; Performing deviation correction on the normalized coordinates based on a pre-calibrated camera internal parameter matrix and distortion coefficients to obtain distortion-free coordinates; Performing pixel mapping according to the distortion-free coordinates of all pixel points to obtain distortion-free images.

4. The method of claim 1, wherein, The target detection according to the distortion-free images to obtain target information of the railway equipment to be detected comprises the following steps: Performing model training according to a sample image set of preset railway equipment labels to obtain a trained equipment detection model; Inputting the distortion-free images into the equipment detection model for target detection to obtain target information of the railway equipment to be detected.

5. The method of claim 1, wherein, The feature matching on the distortion-free images according to the target information based on two different feature extraction algorithms to obtain a target feature set comprises the following steps: Performing image extraction according to the category labels and the boundary box coordinates to obtain local images of the railway equipment to be detected; Performing first feature extraction processing on the local images based on a scale-invariant feature transform algorithm to obtain a first visual feature set, wherein the first visual feature set comprises a plurality of first visual feature points; Performing second feature extraction processing on the local images based on a histogram of oriented gradients algorithm to obtain a second visual feature set, wherein the second visual feature set comprises a plurality of second visual feature points; performing second feature extraction processing on the local image based on an accelerated robust features algorithm to obtain a second visual feature set, wherein the second feature set includes a plurality of second visual feature points; performing same-name matching according to the first visual feature set and the second visual feature set, determining the first visual feature point and the second visual feature point pointing to the same device structure as a visual feature pair, and obtaining a target feature set.

6. The method of claim 2, wherein, performing triangulation on the target feature set of different railway images according to the image extrinsic parameters to obtain boundary position coordinates of the railway device to be detected in the track coordinate system, including the following steps: performing correlation matching according to the target feature set of different railway images to obtain a correlation feature set; performing coordinate axis rotation according to the carrier attitude information corresponding to each of the railway images to obtain a rotation matrix of each of the railway images from a camera coordinate system to a track coordinate system; performing translation transformation according to the carrier position information corresponding to each of the railway images and the rotation matrix to obtain a transformation matrix of each of the railway images from the camera coordinate system to the track coordinate system; constructing an error equation for the correlation feature set by a least square method according to the transformation matrix corresponding to different railway images, and minimizing the error to obtain the boundary position coordinates of the railway device to be detected in the track coordinate system.

7. The method of claim 1, wherein, performing intrusion detection according to the boundary position coordinates and the track detection result to obtain an intrusion result, including the following steps: determining a standard limit value according to the category label corresponding to the boundary position coordinates; performing distance detection according to the boundary position coordinates and the track coordinates to obtain the real distance of the railway device to be detected to the track; comparing the standard limit value and the real distance to obtain the intrusion result.

8. A railway gauge detection system, characterized in that, The system comprises: a data acquisition module configured to acquire a track detection result of track detection and railway images along a railway line, wherein the track detection result comprises track coordinates, carrier attitude information, and carrier position information; a time difference matching module configured to perform time difference matching according to the track detection result and the railway images along the railway line to obtain image extrinsic parameters; an image correction module configured to perform image correction processing on the railway images along the railway line to obtain distortion-free images; a target detection module configured to perform target detection according to the distortion-free images to obtain target information of a railway device to be detected, wherein the target information comprises boundary box coordinates and a category label; a feature matching module configured to perform feature matching on the distortion-free images according to the target information based on two different feature extraction algorithms to obtain a target feature set, wherein the target feature set includes a plurality of visual feature pairs, and each visual feature pair includes two visual feature points pointing to the same device structure; a triangulation module configured to perform triangulation on the target feature set of different railway images according to the image extrinsic parameters to obtain boundary position coordinates of the railway device to be detected in the track coordinate system; an intrusion detection module configured to perform intrusion detection according to the boundary position coordinates and the track detection result to obtain an intrusion result.

9. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the method in any one of claims 1 to 7.

Images