Railway track geometric measurement system and method

Abstract

Methods and systems are provided for detecting and measuring physical conditions of railway tracks based on distance measurements of images. In an embodiment, at least one object related to a railway track condition is detected in at least one image. A first point and a second point on at least one object are detected. A pixel distance between the first point and the second point is measured, and a railway track condition measurement based on the physical distance is determined using a transformation model based on the pixel distance between the first point and the second point. An alarm is issued when the measured value of the railway track condition based on the physical distance exceeds a predetermined threshold value.

Description

Technical Field

[0001] The present invention generally relates to the state detection of physical components, and more particularly to a technique for measuring the state of railway tracks based on image analysis. Background Art

[0002] Railways play an important role in freight transportation. A train typically consists of a locomotive and one or more carriages, and travels along railway tracks to transport goods. A typical track may contain a combination of various objects. For example, a track can include two tracks laid parallel to each other, which can provide a surface on which the wheels of a train carriage can move and be guided. The parallel track lines can be connected to sleepers perpendicular to the tracks to provide structural support for the railway track. Each rail of the railway track can be composed of multiple rail segments. For example, each of the multiple track segments can be connected end-to-end to another track segment to form the track of the railway track. A track joint can be used to connect one track segment to another. In this case, the end of one track segment can be abutted against the end of another track segment, and the joint rod of the track joint can be attached to each of the two track segments to connect the two track segments. The gap between the two ends of the track segments connected to each other at each joint segment can be referred to as a track gap. The combination of the various components of the railway track enables a train to travel along the railway track.

[0003] Given our dependence on railways, it is very important to determine the condition of railway track components. Currently, the condition of railway track components can be determined through manual inspection. However, since railway tracks are very long, such a process is very cumbersome and in some cases too costly, because this means that thousands of components may need to be inspected. This is why current systems may also use computer vision to determine the condition of railway track components. Specifically, when using computer vision, images of various parts of the railway track can be captured and processed to determine the condition of the various components of the railway track detected in the captured images.

[0004] However, current object detection systems generally lack the function of determining the condition of railway track components based on distance measurement. For example, the condition of a railway track joint segment (e.g., the part where one track segment is connected to another using a track joint) can be determined by the size of the track gap between the two track segments, but current systems lack the function of measuring the track gap. In another example, the condition of a component can be determined by the size of a defect on the component. For example, the size of a crack on the track surface can determine the condition of the track, because a small crack may not indicate a problem, while a larger crack may indicate a defect on the track surface and may require maintenance. Other examples of conditions based on distance measurement may include how the spacing between sleepers indicates the condition of the railway track, or how the distance between an anchor and a sleeper indicates the condition. SUMMARY OF THE INVENTION

[0005] The present disclosure, as a system, method, and computer-readable storage medium, realizes technical advantages by providing a function for detecting and measuring the physical condition of a railway track based on image-based distance measurement. The present disclosure provides a system integrated into a practical application with meaningful limitations, which may include acquiring an image of at least a portion of a railway track, detecting at least one object associated with the condition of the railway track in the image; detecting a first point and a second point within the at least one object, measuring the pixel distance between the first point and the second point, using a conversion model to determine a physical-distance-based measurement result of the railway track condition based on the pixel distance between the first point and the second point, and generating an alert when the physical-distance-based measurement result of the railway track condition exceeds a predetermined threshold. For example, in an embodiment, a discontinuity may be detected in an image of a rail joint portion. The discontinuity may represent a track gap between a first track segment and a second track segment, and the size of the discontinuity may represent the condition of the railway track. In an embodiment, based on the detected discontinuity, the edge of the end of the first track segment and the edge of the end of the second track segment may be detected, and a first point in the edge of the end of the first track segment and a second point in the edge of the end of the second track segment may be identified. In an embodiment, the pixel distance between the first point and the second point may be measured, and the physical size of the track gap may be determined using a conversion model based on the pixel distance between the first point and the second point. In some embodiments, an alert may be generated when the physical size of the track gap exceeds a predetermined threshold.

[0006] The present disclosure solves the technical problem of the lack of technical functions in current systems for determining the condition of railway track components based on image-based distance measurement by providing a method and system that provide a novel and creative mechanism for determining the condition of a railway track based on distance-based measurement. The technical solution provided herein, which is missing in traditional systems, is not merely the simple application of a manual process to a computer environment, but also includes the function of implementing a technical process to complement the current manual solutions for inspecting railway tracks and enhancing the current object detection system by providing a mechanism for determining the condition of a railway track based on distance measurement of the detected objects. Therefore, the present disclosure goes far beyond the simple application of manual processes to a computer. For example, the solution provided by the present disclosure includes the function of implementing a pixel-based measurement applied to image-based detection to determine the physical size of the condition existing in a railway track. An alert may be generated depending on whether the physical size of the railway track condition exceeds a predetermined threshold.

[0007] Accordingly, the present disclosure discloses concepts that are inseparable from computer technology, such that the present disclosure provides technical advantages for implementing functions to determine the condition of a railroad track based on distance-based measurements. The systems and techniques of the embodiments provide an improved system by providing the ability to perform functions currently performed manually and functions currently not possible to implement.

[0008] An object of the present invention is to provide a system for determining the condition of a railroad track. Another object of the present invention is to provide a method for determining the condition of a railroad track. Another object of the present invention is to provide a computer-based tool for determining the condition of a railroad track. The present disclosure provides the above and other objects, including at least the following embodiments.

[0009] In a particular embodiment, a method for determining the condition of a railroad track is provided. The method includes detecting at least one object in at least one image of at least a portion of a railroad track, the at least one object being associated with at least one condition of the railroad track, identifying a first point and a second point on the at least one object detected in the at least one image of at least a portion of the railroad track, and measuring a pixel distance between the first point and the second point. In an embodiment, the pixel distance between the first point and the second point may indicate the number of pixels in the line between the first point and the second point. The method further includes using a conversion model to determine a physical dimension of at least one condition of the railroad track associated with the at least one object based on the pixel distance between the first point and the second point, and generating an alert when the physical dimension of at least one condition of the railroad track exceeds a predetermined threshold.

[0010] In another embodiment, a method for determining the condition of a railroad track is provided. The method includes: detecting a discontinuity in a joint portion of a railroad track in an image of at least a portion of the railroad track, the discontinuity representing a track gap between a first track segment and a second track segment; detecting an edge of an end of the first track segment and an edge of an end of the second track segment based on the detected discontinuity; identifying a first point in the edge of the end of the first track segment and a second point in the edge of the end of the second track segment; and measuring a pixel distance between the first point and the second point. In an embodiment, the pixel distance between the first point and the second point may indicate the number of pixels in the line between the first point and the second point. The method further includes using a conversion model to determine a physical dimension of the track gap based on the pixel distance between the first point and the second point, and generating an alert when the physical dimension of the track gap exceeds a predetermined threshold.

[0011] In another embodiment, a computer-based tool for determining the condition of a railway track is provided. The computer-based tool may include a non-transitory computer-readable medium having computer code stored thereon that, when executed by a processor, causes a computing device to perform operations. The operations include detecting at least one object in at least one image of at least a portion of a railway track, the at least one object being associated with at least one condition of the railway track, identifying a first point and a second point on the at least one object detected in the at least one image of at least a portion of the railway track, and measuring a pixel distance between the first point and the second point. In an embodiment, the pixel distance between the first point and the second point may indicate the number of pixels in the line between the first point and the second point. The operations further include using a conversion model to determine a physical dimension of at least one condition of the railway track associated with the at least one object based on the pixel distance between the first point and the second point, and generating an alert when the physical dimension of at least one condition of the railway track exceeds a predetermined threshold.

[0012] In another embodiment, a computer-based tool for determining the condition of a railway track is provided. The computer-based tool may include a non-transitory computer-readable medium having computer code stored thereon that, when executed by a processor, causes a computing device to perform operations. The operations include detecting a discontinuity in a joint portion of a track in an image of at least a portion of the railway track, the discontinuity representing a track gap between a first track portion and a second track portion; detecting an edge of an end of the first track portion and an edge of an end of the second track portion based on the detected discontinuity; identifying a first point in the edge of the end of the first track portion and a second point in the edge of the end of the second track portion; and measuring a pixel distance between the first point and the second point. In an embodiment, the pixel distance between the first point and the second point may indicate the number of pixels in the line between the first point and the second point. The operations further include using a conversion model to determine a physical dimension of the track gap based on the pixel distance between the first point and the second point, and generating an alert when the physical dimension of the track gap exceeds a predetermined threshold.

[0013] In another embodiment, a system for determining the condition of a railway track is provided. The system includes at least one processor and a memory operably coupled to the processor and storing processor-readable code that, when executed by the processor, is configured to perform operations. The operations include detecting at least one object in at least one image of at least a portion of the railway track, the at least one object being associated with at least one condition of the railway track, identifying a first point and a second point on the at least one object detected in the at least one image of at least a portion of the railway track, and measuring a pixel distance between the first point and the second point. In an embodiment, the pixel distance between the first point and the second point may indicate the number of pixels in a line between the first point and the second point. The operations further include using a conversion model to determine a physical dimension of at least one condition of the railway track associated with the at least one object based on the pixel distance between the first point and the second point, and generating an alert when the physical dimension of at least one condition of the railway track exceeds a predetermined threshold.

[0014] In another embodiment, a system for determining the condition of a railway track is provided. In certain embodiments, the system includes at least one processor and a memory operably coupled to the processor and storing processor-readable code that, when executed by the processor, configures the system to perform operations. The operations include detecting a discontinuity in a joint portion of a track in an image of at least a portion of the railway track, the discontinuity representing a track gap between a first track portion and a second track portion; detecting an edge of an end of the first track portion and an edge of an end of the second track portion based on the detected discontinuity; identifying a first point in the edge of the end of the first track portion and a second point in the edge of the end of the second track portion; and measuring a pixel distance between the first point and the second point. In an embodiment, the pixel distance between the first point and the second point may indicate the number of pixels in a line between the first point and the second point. The operations further include using a conversion model to determine a physical dimension of the track gap based on the pixel distance between the first point and the second point, and generating an alert when the physical dimension of the track gap exceeds a predetermined threshold.

[0015] The features and technical advantages of the present invention have been outlined above rather extensively so as to enable a better understanding of the detailed description of the present invention hereinafter. Additional features and advantages of the present invention will be described hereinafter, which form the subject matter of the claims of the present invention. Those skilled in the art should understand that the disclosed conceptions and specific embodiments can be readily used as a basis for modifying or designing other structures to achieve the same purposes of the present invention. Those skilled in the art should also recognize that such equivalent constructions do not depart from the spirit and scope of the present invention as set forth in the appended claims. The novel features regarding its organization and method of operation, as well as further purposes and advantages, which are considered to be the features of the present invention, can be better understood from the following description in conjunction with the accompanying drawings. It should be clearly understood, however, that each drawing is for illustrative and descriptive purposes only and does not delimit the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To more fully understand the present invention, the following is described in conjunction with the accompanying drawings:

[0017] Figure 1 is a block diagram of an exemplary system configured with the ability and functionality to detect and measure the condition of a railway track and / or railway track components based on image distance measurement, according to an embodiment of the present disclosure.

[0018] Figure 2A shows an example of image data captured according to an embodiment of the present disclosure.

[0019] Figure 2B shows an example of image data captured from different angles according to an embodiment of the present disclosure.

[0020] Figure 3 shows an example of edge detection and distance-based measurement of a track gap according to an embodiment of the present disclosure.

[0021] Figure 4 shows a high-level flowchart of the operation of a system configured according to an embodiment of the present disclosure for providing image-based distance measurement to detect and measure the condition of a railway track and / or railway track components.

[0022] Figure 5 shows a high-level flowchart of the operation of a system configured according to an embodiment of the present disclosure for providing image-based distance measurement to detect and measure the condition of a railway track and / or railway track components.

[0023] It should be understood that the drawings are not necessarily drawn to scale and that the disclosed embodiments are sometimes shown in diagrammatic and partial view forms. In some cases, details that are unnecessary for understanding the disclosed methods and apparatuses or that make other details difficult to understand may be omitted. Of course, it should be understood that the present disclosure is not limited to the specific embodiments shown herein. Detailed implementation manners

[0024] The disclosure presented in the following written description, along with its various features and advantageous details, will be more fully explained with reference to the non-limiting examples included in the accompanying drawings and the content detailed in the specification. Descriptions of well-known components are omitted herein to avoid unnecessarily obscuring the main features described herein. The following examples are intended to facilitate understanding of the implementation and practice of the present disclosure. A person of ordinary skill in the art will understand the present disclosure, meaning that any suitable combination of the following functions or exemplary embodiments can be combined to implement the claimed subject matter. The present disclosure includes a representative number of species within the genus, or structural features common to the members of the genus, so that a person of ordinary skill in the art can identify the members of the genus. Therefore, these examples should not be construed as limiting the scope of the claims.

[0025] A person of ordinary skill in the art will understand that any system claim presented herein encompasses all elements and limitations disclosed therein, and thus each system claim is to be regarded as a whole. Any reasonably foreseeable item having a functional relationship with the claim also falls within the relevant scope. After thoroughly understanding the disclosure and claims in the presented application, the examiner searched for prior art disclosed in patents and other published documents (i.e., non-patent literature). Therefore, as evidenced by the issuance of this patent, the prior art failed to disclose or teach the elements and limitations set forth in the claims supported by the specification and drawings, and thus the claimed claims are patentable under the applicable laws and rules of this jurisdiction.

[0026] Various embodiments of the present disclosure are directed to systems and techniques for providing functionality to detect and measure the condition of a railroad track and / or railroad track components based on image-based distance measurement. In a particular embodiment, an image of at least a portion of a railroad track can be obtained. For example, an image of a railroad track portion that includes a rail joint portion can be obtained. In an embodiment, at least one object related to the railroad track condition can be detected in the image, and a first point and a second point within the at least one object can be identified. For example, a discontinuity in a rail joint portion can be detected in an image of a railroad track. In this example, the discontinuity may represent a track gap between a first track segment and a second track segment, and the physical size of the track gap may represent the condition of the railroad track. In this example, based on the detected discontinuity, the edge of the end of the first track segment and the edge of the end of the second track segment can be detected, and a first point in the edge of the end of the first track segment and a second point in the edge of the end of the second track segment can be identified. In an embodiment, the pixel distance between the first point and the second point can be measured. The pixel distance between the first point and the second point can indicate the number of pixels in the line between the first point and the second point. For example, the number of pixels in the line between the first point and the second point can be measured and / or obtained. In an embodiment, based on the pixel distance between the first point and the second point, a conversion model can be used to determine a physical-distance-based measurement of the railroad track condition. For example, based on the pixel distance between the first point and the second point, a conversion model can be used to determine the physical size of the track gap. In an embodiment, an alert can be generated when the physical-distance-based measurement of the railroad track condition exceeds a predetermined threshold. For example, an alert may be generated when the physical size of the track gap exceeds a predetermined threshold.

[0027] It should be noted that although the following disclosure focuses on one example application where the condition of a railroad track is described as the size of a track gap in a rail joint portion, this is for illustrative purposes only and is not intended to be limiting in any way. In fact, in some embodiments, the condition of a railroad track can include (as a non-limiting and non-exhaustive list) the size of the track surface condition (e.g., the size of cracks, metal fatigue, and / or other defects in the track surface), the spacing or distance between railroad ties in the railroad track, the separation between tie anchors and ties, etc. These examples illustrate conditions that can be measured or quantified by the techniques disclosed in embodiments of the present disclosure, but other examples can also be envisioned.

[0028] Figure 1 is a block diagram of an exemplary system 100 according to an embodiment of the present disclosure, the system being configured with the ability and functionality to detect and measure the condition of a railroad track and / or railroad track components based on image-based distance measurement. As Figure 1As shown, system 100 may include a server 110, an image collector 130, a network 145, and a user interface 140. These components and their respective components may operate in concert to provide the functions discussed herein. For example, in an operation according to an embodiment, the image collector 130 may operate to obtain image data related to at least a portion of a railway track (e.g., railway track 150). In an embodiment, the image data obtained by the image collector 130 may include at least one object related to the condition of the railway track 150. The functions of the server 110 may be used to process the image data obtained by the image collector 130 to detect at least one object related to the condition of the railway track 150 based on the image data and determine a physical distance measurement related to the condition. For example, in some embodiments, the physical dimensions (e.g., length, width, area, volume, etc.) of the condition of the railway track 150 may be obtained through the functions of the server 110. The functions of the server 110 may operate to generate an alert in response to determining that the physical distance measurement of the condition of the railway track 150 exceeds a predetermined threshold. In an embodiment, the alert may be sent and received through a user interface (e.g., user interface 140). In response to the alert, corrective measures may be taken for the condition of the railway track 150, such as in-depth inspection, replacement, repair, etc.

[0029] It should be noted that the functional blocks and their components of the system 100 according to the embodiments of the present invention may be implemented using a processor, an electronic device, a hardware device, an electronic component, a logic circuit, a memory, software code, firmware code, etc., or any combination thereof. For example, one or more functional modules, or some of their functional modules, may be implemented as discrete gate or transistor logic, discrete hardware components, or a combination thereof, configured to provide the logic for performing the functions described herein. Additionally or alternatively, when implemented in software, one or more functional blocks or some of their parts may include code segments operable on a processor to provide the logic for performing the functions described herein.

[0030] It should also be noted that the various components of the system 100 are shown as single and independent components. However, it can be understood that each of the various shown components may be implemented as a single component (e.g., a single application, a server module, etc.), may be a functional component of a single component, or the functions of these various components may be distributed over multiple devices / components. In such an embodiment, the functions of each corresponding component may be aggregated from the functions of multiple modules located in a single or multiple devices.

[0031] It should also be noted that the functions described with reference to each different functional block of the system 100 described herein are for illustrative purposes only and not for limiting purposes, and the functions provided by different functional blocks may be combined into a single component, or may be provided by computing resources located in a cloud-based environment accessible through a network (e.g., one of the networks 145).

[0032] The user interface 140 can be implemented as a mobile device, smartphone, tablet computing device, personal computing device, notebook computing device, desktop computing device, computer system of a vehicle, personal digital assistant (PDA), smartwatch, other types of wired and / or wireless computing devices, or any part thereof. In some embodiments, the user interface 140 can be configured to provide an interface (e.g., a graphical user interface (GUI)) for an operator to interact with the system 100 via the network 145, so as to execute and utilize the functions provided by the server 110. In an embodiment, the operator can be enabled to provide configuration parameters usable by the system 100, for example, through the functions of the user terminal 140, to provide the function of identifying and / or measuring the condition of the railway track. In an embodiment, the user terminal 140 can be configured to communicate with other components of the system 100. In an embodiment, the functions of the user terminal 140 can include receiving alerts (e.g., alerts generated using the functions of the server 110). In an embodiment, the alerts can be presented to the operator through the GIU of the user terminal 140.

[0033] The image collector 130 can be configured to acquire image data related to at least a part of a railway track (e.g., the railway track 150), and transmit or deliver the image data to the server 110 for further processing. For example, in some embodiments, the image collector 130 can include one or more cameras configured to capture images of at least a part of the railway track 150. In some embodiments, one or more cameras can be physically mounted on a train car (not shown). When the train car travels on a railway track (e.g., the railway track 150), one or more cameras can capture or collect images of different parts of the railway track. In alternative or additional embodiments, the image collector 130 can be configured to receive image data from an external system. For example, in some embodiments, the external system can capture image data related to at least a part of the railway track and can provide the image data to the image collector 130, and the image collector 130 can provide the image data to the server 110 for further processing. In some cases, the function of the image collector 130 receiving image data from an external system can be integrated into the server 110 (although it may be shown as a separate function in Figure 1 it).

[0034] In an embodiment, image data collection and processing can be performed in real time or near real time. For example, image collector 130 can obtain image data related to at least a portion of a railroad track when capturing the image data, and can provide the image data to server 110 for processing when the image data is obtained. In some embodiments, image data can be captured, but may not be immediately provided to server 110 for processing when the image data is obtained, but rather the image data can be stored for subsequent processing by server 110.

[0035] In an embodiment, the image data obtained by image collector 130 can include image data related to multiple portions of railroad track 150. For example, the image data can include images of different portions of railroad track 150. The portions of railroad track 150 for which image data can be obtained can be determined according to the configuration of system 100. For example, in some embodiments, image data of a target portion of railroad track 150 can be obtained. In certain cases, the target portion of railroad track 150 can include a specific portion of railroad track 150. For example, in some embodiments, image data of each rail joint portion, image data of each rail portion, etc. can be captured. In these cases, the image data can represent image data of a target railroad track section (e.g., a rail joint section, a rail section, a tie section, etc.). In some embodiments, image data related to railroad track 150 can be captured based on a calibrated pulse encoder. For example, the pulse encoder can be located on the wheels of a train car, image detector 130 can be mounted on the wheels of the train car, and the pulse encoder can be used to trigger the capture of railroad track images. It is noted that in these embodiments, the pixel size of the image data (e.g., an image related to at least a portion of the railroad track) can be highly correlated with the physical size of at least a portion of the railroad track. In these cases, the image size of the first portion of the railroad track can be highly correlated with the physical size of the first portion. Thus, the size of the pixels in the image data obtained by image collector 130 can be highly correlated with the physical size. In some embodiments, image data related to railroad track 150 can be captured according to a schedule. For example, when a train car configured to capture image data travels on railroad track 150, the image data can be captured every few seconds, minutes, hours, etc. In some embodiments, image data of railroad track 150 can be captured continuously. For example, when a train car configured to capture image data travels on railroad track 150, the image data can be captured continuously.

[0036] In a particular embodiment, for example when the image data obtained by image collector 130 represents image data related to a rail joint portion, the image data can include various railroad track components (e.g., rails, ties, spikes, anchors, rail joints, rail gaps, etc.).Figure 2A An example of image data captured according to an embodiment of the present disclosure is shown. As Figure 2A shown, the image data 200 may include image data of a railway track portion 210. In this example, the railway track portion 210 may include a track joint portion 215, which may be a place where the track portion 220 and the track portion 222 are interconnected. For example, the track segment end 221 of the track segment 220 may be connected to the track segment end 223 of the track segment 222 to form a track joint. As Figure 2A shown, the track segment 220 and the track segment 222 are connected together by a connecting rod 225. In an embodiment, at the track joint, there may be a track gap between the track segment 220 and the track segment 222. Specifically, the track gap 230 may exist between the track segment end 221 and the track segment end 223. In an embodiment, the track gap 230 may be represented in the image data 200 by a discontinuity in the track surface formed by the track segment 220 and the track segment 222.

[0037] It is noted that the inventors have found that the characteristics of the track gap may affect the performance of the railway track. Specifically, the size of the track gap may affect the performance of the railway track where the track gap exists. For example, an overly wide track gap may indicate a problem with the connecting rod used to connect the track segments forming the track gap. For example, the connecting rod may be incorrectly installed, overly worn, bolts or nails may be loose, etc. Any problem may lead to problems and even catastrophic failures. In addition, an overly large track gap may also cause damage to the wheels of train cars or locomotives traveling on the railway or result in derailment. Therefore, providing a mechanism to ensure that the size of the track gap is within acceptable parameters (such as the mechanism provided by the embodiments of the present disclosure) can significantly improve the railway system and avoid catastrophic failures.

[0038] In an embodiment, the image data obtained by the image collector 130 may include images of the same portion of the railway track captured from different angles. Figure 2B An example of image data captured from different angles according to an embodiment of the present disclosure is shown. For example, Figure 2B shows image data 200 captured from an angle different from the angle shown in Figure 2A , which may include image data of a railway track portion 210, including a rail joint portion 215 and a rail gap 230. In particular, Figure 2B shows image data 200 captured from the field side of the rail portion 210, while Figure 2AImage data 200 captured from the gauge side of track section 210 is shown. In an embodiment, due to the different angles from which the image data 200 can be captured, the discontinuity between track segment 220 and track segment 222 (which may represent track gap 230) may exhibit different characteristics. In fact, the inventors have found that, in some cases, the discontinuity between track segment 220 and track segment 222 (which may represent track gap 230) may appear as different sizes. Embodiments of the present disclosure provide mechanisms to accommodate the different characteristics presented in the different angle image data of the track gap, and can utilize the different angle views of the track gap by providing a mechanism to measure the size of the track gap using different angle views and combine the various results (e.g., by taking an average, selecting a maximum value, etc.) into a single track gap measurement value, as will be described in more detail below.

[0039] Referring again to Figure 1 , server 110, user interface 140, and image collector 130 may be communicatively coupled via network 145. Network 145 may include a wired network, a wireless communication network, a cellular network, a cable transmission system, a local area network (LAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), the Internet, a public switched telephone network (PSTN), etc. In some embodiments, image collector 130 and server 110 may be directly communicatively coupled without being routed through network 145, e.g., via a direct connection between sensor 130 and server 110. In some embodiments, image collector 130 may be integrated into server 110 such that image collector 130 may be part of the functionality of server 110, even though Figure 1 image collector 130 is shown separately from server 110 in

[0040] Server 110 may be configured to facilitate the following operations: detect at least one object associated with a railway track condition in image data provided by image collector 130 that is associated with a portion of a railway track, detect at least two points on the at least one object, determine a physically distance-based measurement of the railway track condition, and generate an alert in response to determining that the physically distance-based measurement of the railway track condition exceeds a predetermined threshold, in accordance with embodiments of the present disclosure. The functionality of server 110 may be provided by the various components of server 110 cooperating, as described in more detail below. Although Figure 1 a single server 110 is shown, it will be understood that server 110 and its various functional blocks may be implemented as a single device or may be distributed across multiple devices having their own processing resources, the overall functionality of which may be configured to perform operations in accordance with the present disclosure. Additionally, those skilled in the art will recognize that although Figure 1The components of server 110 are shown as single and independent blocks. Each of the various components of server 110 can be a single component (e.g., a single application, a server module, etc.), can be a functional component of the same component, or the functionality can be distributed across multiple devices / components. In such an embodiment, the functionality of each respective component can be aggregated from the functionality of multiple modules located in a single or multiple devices. Additionally, the specific functionality described for a particular component of server 110 may actually be part of a different component of server 110. Thus, the description of the specific functionality described for a particular component of server 110 is for illustrative purposes only and is not limiting in any way.

[0041] It should be noted that, in an embodiment, the measurement of railway track conditions based on physical distance can include dimensions, lengths, widths, areas, volumes, and / or any other dimensional measurements of the physical condition of the railway track. For example, the measurement of railway track conditions based on physical distance can include the width or length of the track gap at the track joint of the railway track. In another example, the measurement of railway track conditions based on physical distance can include the size of the track surface condition (e.g., the size of cracks, the size of metal fatigue areas, and / or the size of other defects in the track surface), the length of the spacing or distance between sleepers in the railway track, the separation length between sleeper anchors and sleepers, etc. It should be understood that these examples are provided for illustrative purposes only and are not limiting.

[0042] As Figure 1 shown, server 110 includes a processor 111, a memory 112, a database 126, an object detector 120, a point detector 121, distance-based measurement logic 123, and an alarm generator 124. The processor 111 can include a processor, a microprocessor, a controller, a microcontroller, multiple microprocessors, an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), or any combination thereof, and can be configured to execute instructions to perform operations in accordance with the disclosure herein. In some embodiments, the implementation of the processor 111 can include code segments (e.g., software, firmware, and / or hardware logic) executable in hardware (e.g., a processor) to perform the tasks and functions described herein. In other embodiments, the processor 111 can be implemented as a combination of hardware and software. The processor 111 can be communicatively coupled to the memory 112.

[0043] The memory 112 may include one or more semiconductor memory devices, read-only memory (ROM) devices, random access memory (RAM) devices, one or more hard disk drives (HDDs), flash memory devices, solid state drives (SSDs), erasable ROM (EROM), compact disc ROM (CD-ROM), optical discs, other devices configured to store data in a persistent or non-persistent state, network memory, cloud memory, local memory, or a combination of different memory devices. The memory 112 may include a processor-readable medium configured to store one or more instruction sets (e.g., software, firmware, etc.), which when executed by a processor (e.g., one or more processors of the processor 111), perform the tasks and functions described herein.

[0044] The memory 112 may also be configured to facilitate storage operations. For example, the memory 112 may contain a database 126 for storing various information related to the operation of the system 100. In some embodiments, the database 126 may store configuration parameters related to the operation of the system 100, such as user information, predefined thresholds, etc. In some embodiments, the database 126 may store machine learning models, mathematical models, rule models, algorithms, and / or other models that can be used by components of the server 110 to analyze and process image data related to the condition of the railway track to determine distance-based measurements related to the condition of the railway track. For example, in some embodiments, the database 126 may store a conversion model that can be used to obtain a physical distance-based measurement related to the condition of the railway track based on a pixel distance measurement, as will be described in more detail below. In some embodiments, the conversion model may be trained to map a pixel distance-based measurement to a physical distance-based measurement. In an embodiment, the database 126 may also store a threshold that can be compared with the physical distance-based measurement value to determine whether an alert can be generated. The database 126 is shown integrated in the memory 112, but in certain embodiments, the database 126 may be provided as a separate storage module, or may be provided as a cloud-based storage module. Additionally, or alternatively, the database 126 may be a single database or a distributed database implemented through multiple database modules.

[0045] The object detector 120 may be configured to detect at least one object in the image data provided by the image collector 130. As described above, the image data provided by the image collector 130 may include image data related to at least a portion of the railway track 150, such as at least one image of a railway track portion. In an embodiment, at least one image of a railway track section may include at least one object, and the at least one object may be associated with the condition of the railway track 150.

[0046] For example, referring toFigure 2A , the image data 200 may include an image of a railway track portion 210, which may include a track joint portion 215 where a track segment 220 and a track segment 222 are connected together, and track segment ends 223 to track segment end 221. As Figure 2A , the image data 200 includes at least one object (e.g., a rail gap 230) related to the condition of the railway track 150 (e.g., the width of the rail gap 230). As described above, the width of the track gap 230 may indicate a serious problem with the railway track 150 or be a serious problem itself. In this example, the object detector 120 may be configured to detect the rail gap 230.

[0047] In some embodiments, detecting the rail gap 230 may include detecting a discontinuity in the track formed by the track segment 220 and the track segment 222 in the track joint portion 215. The discontinuity in the track formed by the track segment 220 and the track segment 222 can be detected by applying a classification model trained to identify and / or label objects in the image data. In this case, the classification model can be applied to the image data 200 to detect the discontinuity in the track formed by the track segment 220 and the track segment 222. The discontinuity in the track formed by the track segment 220 and the track segment 222 in the track joint portion 215 can be determined as the rail gap between the track segment 220 and the track segment 222 in the track joint portion 215. In some embodiments, detecting the discontinuity in the track formed by the track segment 220 and the track segment 222 in the track joint portion 215 may include applying a classification model trained to detect the track segment 220 and the track segment 222. In this case, the rail gap can be determined as the spacing or gap between the track segment 220 and the track segment 222.

[0048] In some embodiments, the object detector 120 can be used to detect cracks or metal fatigue indicators (e.g., at least one object) on the surface of a track section (e.g., track section 220 and / or track section 222) or any other component of the railway track 150, which have physical dimensions (e.g., conditions) associated with the railway track 150. Cracks or metal fatigue indicators on the surface of railway track components may indicate a serious problem with the railway track 150 or may itself be a serious problem. In some embodiments, the object detector 120 can detect at least two sleepers (e.g., at least one object) separated by a certain distance (e.g., a condition associated with the railway track). The separation distance between the at least two sleepers may indicate a serious problem with the railway track or may itself be a serious problem. In some embodiments, the object detector 120 can operate to detect at least one sleeper anchor and crosstie (e.g., at least one object) separated by a certain distance (e.g., a condition associated with the railway track). The separation distance between the sleeper anchoring device and the crosstie may indicate a serious problem with the railway track or may itself be a serious problem. Again, note that these examples are for illustrative purposes only and not for limiting purposes.

[0049] In an embodiment, the function of the object detector 120 to detect at least one object in the image data provided by the image collector 130 can be integrated into the server 110. For example, as Figure 1 shown in the example shown, the object detector 120 can be part of the server 110 and can perform operations to detect at least one object in the image data provided by the image collector 130.

[0050] In alternative or additional embodiments, the object detector 120 may be configured to receive object detection data associated with the image data from an external system. For example, in some embodiments, image data related to at least a portion of the railway track 150 may be processed by an external system to detect objects present in the image data. The objects detected in the image data may be labeled by the external system, and the labels may be provided to the object detector 120. In these embodiments, the object detector 120 may pass the labeled image data to the point detector 121 for further processing. In some embodiments, the object detector 120 may further process the labeled data to extract a target portion from the labeled image data. For example, in certain cases, the labeled image data may include image data with labels for various detected objects within the image data. In such cases, the object detector 120 may extract the target portion of the image data by cropping or the like. For example, the object detector 120 may extract a target portion of the image data that includes a detected discontinuity, a target portion that includes a detected crack or metal fatigue indicator, a target portion that includes a detected sleeper, a target portion that includes an anchor and a sleeper, etc. In these cases, the object detector 120 may provide the extracted or cropped portion of the image data to the point detector 121 for further processing.

[0051] The point detector 121 may be configured to detect or identify at least two points on at least one object detected by the object detector 120. In an embodiment, the point detector 121 may be configured to detect at least two points detected on at least one object to facilitate performing distance-based measurements. For example, in some embodiments, the point detector 121 may detect at least two points on an edge of at least one object. By detecting points at the edge of at least one object, the point detector 121 may conveniently perform distance-based measurements of at least one object. For example, at least one object may include a discontinuity representing a track gap. In this example, the point detector 121 may be configured to detect at least two points on the discontinuity, where the at least two points are located on the edge of the discontinuity. In some embodiments, a first of the at least two points may be set on a first edge of the discontinuity, and a second of the at least two points may be set on a second edge of the discontinuity that is different from the first edge. In this way, two-dimensional measurements (such as length or width) related to the discontinuity may be obtained based on these two points.

[0052] In some embodiments, the number of points among at least two points may depend on the type of target measurement. For example, two points may be sufficient to measure length or width. However, measuring area or volume may require at least three points and, in some cases, even more points. In some embodiments, the number of points identified on at least one object may be based on the type of at least one object. For example, at least one object may include a discontinuity representing a track gap. In this example, since at least one object is a track gap discontinuity, at least two points may include two points located at opposite edges of the track gap discontinuity (e.g., the track gap discontinuity edges near the ends of the track segments forming the track gap) to facilitate determining the physical width of the track gap. In another example, at least one object may include a metal fatigue indicator on the track surface. In this example, since at least one object is a metal fatigue indicator, at least two points may include at least two points located at both ends of the metal fatigue indicator to facilitate determining the physical length of the metal fatigue indicator, or may include at least three points to facilitate determining the physical area of the metal fatigue indicator on the track surface.

[0053] In an embodiment, detecting an edge of at least one object may include applying an edge detection model to image data associated with the at least one object. In an embodiment, the point detector 121 may include an edge detector 122. The edge detector 122 may be configured to detect at least one edge of at least one object detected by the object detector 120. In an embodiment, the edge detector 122 may be configured to apply an edge detection model to image data including object detection to detect at least one edge of the detected object. In an embodiment, the edge detector 122 may be configured to analyze the gray-scale values between individual pixels of the image data to detect at least one edge of the detected object. Figure 3 An example of edge detection is shown.

[0054] Figure 3 An example of edge detection and distance-based measurement of a track gap according to an embodiment of the present disclosure is shown. As Figure 3As shown, the track gap 230 can be detected as a discontinuity in the track joint section, including the track joint between track section 220 and track section 222. In this example, the edge detector 122 can apply an edge detection model to image detection (e.g., the track gap 230), and can detect at least edge 340 and edge 342 of the track gap 230. In an embodiment, the edge detection model applied by the edge detector 122 to identify the edges of the track gap 230 can include identifying the inner surfaces of the ends of track portion 220 and track portion 222, and determining the edges as the planes where the inner surfaces are located. In this case, the edge detection model applied by the edge detector 122 can analyze the differences between pixels in the detected discontinuity to identify the edges of the discontinuity.

[0055] In an embodiment, after identifying the edges of the discontinuity associated with the rail gap 230, the point detector 121 can detect at least two points on the identified edges of the discontinuity associated with the rail gap 230. For example, in an embodiment, the point detector 121 can apply a point recognition model to generate point data associated with at least one object (e.g., the discontinuity associated with the track gap 230). In this example, the point detector 121 can identify point 320 on edge 340 of the discontinuity associated with the rail gap 230, and can identify point 322 on edge 342 of the discontinuity associated with the rail gap 230. In an embodiment, the point recognition model can be trained to identify points on the edge of at least one object based on the expected contrast in the image data at a specific location within the at least one object. For example, the position of point 320 is expected to contain a high-contrast region because the position of point 320 within the track gap 230 is distinguished by the intersection of the top inner flange of the connecting rod connecting track section 220 and track section 222 and the discontinuity at the point where track section 220 and / or track section 222 may cast a shadow on the track gap 230. The contrast between the shaded portion of the discontinuity and the non-shaded portion of the discontinuity along the edge (e.g., edge 340) provides a mechanism for the point recognition model to consistently identify at least two points on the edge of the discontinuity associated with the rail gap 230.

[0056] In some embodiments, the point detector 121 may provide the identification of at least two points to the distance-based measurement logic 123 for further processing. In an embodiment, the identification of at least two points may be provided by the point detector 121 as a data structure. The data structure may identify each point as coordinates within the image data. For example, the track gap 230 may be identified as a focus box of a given size and defined by x and y coordinates. In an embodiment, the point detector 121 may identify points 320 and 322 by the x and y coordinates within the image data of the detected track gap 230. In some embodiments, the x and y coordinates may refer to pixel positions, where the x coordinate provides the x position within the pixel and the y coordinate provides the y position within the pixel.

[0057] Referring again to Figure 1 、the distance-based measurement logic 123 may be configured to determine a physical distance-based measurement of the condition of a railway track associated with at least one object based on at least two points on the at least one object identified by the point detector 121. In an embodiment, the result of the physical distance-based measurement of the condition of the railway track associated with at least one object may include measuring the pixel distance between at least two points on the at least one object identified by the point detector 121, and using a conversion model to determine the result of the physical distance-based measurement of the condition of the railway track associated with at least one object based on the pixel distance between at least two points on the at least one object.

[0058] In an embodiment, measuring the pixel distance between at least two points on at least one object may include determining the number of pixels in the line between the at least two points. For example, referring to Figure 3 , the pixel distance 325 between points 320 and 322 may include a plurality of pixels. In an embodiment, the pixel distance 325 may be measured by counting the pixels between points 320 and 322 along the pixel distance 325. In some embodiments, the pixel distance 325 may be measured by determining the difference between the positions of points 320 and 322. For example, the position of each of points 320 and 322 may be identified by the x and y coordinates within the image data of the detected track gap 230. In this case, the absolute difference between the y coordinate of point 320 and the y coordinate of point 322 may be obtained, which may result in a plurality of pixels (e.g., pixel distance 325) along a straight line between points 320 and 322.

[0059] It should be noted that the pixel distance between two points on the edge of an object detected in the image data may precisely be the pixel distance between the two points in the image data. However, the distance between two points on the edge of an object detected in the image data may also be related to the physical distance between the corresponding two points of the real-world points represented in the image data. For example, points 320 and 322 on the edge of the discontinuity associated with the rail gap 230 may correspond to real points on the edge of the physical rail gap represented by the rail gap 230. In an embodiment, the distance-based measurement logic 123 may provide a function of mapping the pixel distance to a measurement based on the physical distance.

[0060] For example, the distance-based measurement logic 123 may determine the physical width of the physical rail gap represented by the rail gap 230 based on the pixel distance between point 320 and point 322. For example, the pixel distance 325 may be mapped to a measurement based on the physical distance, which in this example may be the physical width of the physical rail gap represented by the rail gap 230. In an embodiment, the physical width of the physical rail gap represented by the rail gap 230 may be expressed in inches.

[0061] In an embodiment, the distance-based measurement logic 123 may include a conversion model that may be configured to map the pixel distance to a measurement based on the physical distance. In an embodiment, the conversion model may receive as input the pixel distance between at least two points on the edge of at least one object detected in the image data and output a measurement based on the physical distance corresponding to the physical measurement between the real-world points corresponding to the at least two points in the image data. In this way, the conversion model of the distance-based measurement logic 123 can be used to provide a measurement based on the physical distance of the railway track condition associated with at least one object detected in the image data related to at least a portion of the railway track.

[0062] In some embodiments, the conversion model of the distance-based measurement logic 123 may include a conversion formula that converts a plurality of pixels (e.g., pixel distances) into a physical distance (e.g., a distance measurement in inches, millimeters, centimeters, feet, etc.). For example, the pixel distance may be divided by or multiplied by a conversion factor to generate a physical distance. In some embodiments, the conversion factor may define the pixels per inch value, which can be used to convert a pixel distance (e.g., the number of pixels between two points) into a physical distance value (e.g., the distance in inches). In some embodiments, the pixels per inch value may be a value between 30 and 60 pixels per inch. In an embodiment, the conversion factor may vary according to the resolution of the image data obtained by the image collector 130. In some embodiments, as the resolution of the image data increases, the pixels per inch value may also increase, and as the resolution of the image data decreases, the pixels per inch value may also decrease. In some embodiments, the conversion factor may be at least partially based on the distance from the camera that captures the image data to at least one object, the angle of the camera that captures the image data relative to at least one object, the type of the camera, etc.

[0063] In an embodiment, the conversion factor may be based on a calibration process that can be used to train the conversion model of the distance-based measurement logic 123. In an embodiment, the calibration process that can be used to train the conversion model of the distance-based measurement logic 123 may include using a training image of a known physical distance between two points obtained using a configuration that is substantially equivalent to the configuration of the system 100 and measuring the pixel distance between the two points in the training image. For example, an image containing two points whose physical distance in the real world is known may be processed, and the pixel distance between the two points may be measured. In this example, the conversion model of the distance-based measurement logic 123 may learn to map the measured pixel distance to the known physical distance between the two points. In an embodiment, a configuration that is substantially equivalent to the configuration of the system 100 may include a configuration in which the image detector used to capture the training image may be mounted on the train car in a similar orientation to the image detector 130 installed during the operation of the system 100.

[0064] In an embodiment, the distance-based measurement logic 123 may be configured to operate as an integrated application executable by another module (e.g., the point detector 121 or the edge detector 122). In these embodiments, the distance-based measurement logic 123 may be executed to determine a physical distance-based measurement of the condition of a railroad track associated with at least one object based on at least two points on the at least one object identified by the point detector 121, and may be operated to return the result (e.g., the physical distance-based measurement) in a specific data structure, which in some embodiments may be fed into an external system.

[0065] In some embodiments, as described above, the image data related to at least a portion of a railway track that detects at least one object related to the track condition may include multiple images of at least a portion of the railway track captured from different perspectives or angles. For example, as described above, in some embodiments, the image data may include multiple images of the same portion of the railway track, but captured from different angles. In some embodiments, multiple cameras may be used to capture the image data. For example, multiple cameras may be mounted on a train car at different angles, and the multiple cameras may be mounted on the train car. In some embodiments, at least one camera may be mounted to capture at least a portion of the railway track from the gauge side of the railway track, and at least one camera may be mounted to capture at least a portion of the railway track from the field side of the railway track. In an embodiment, the image data including images of at least a portion of the railway track captured from different angles may be processed by an object detector 120, a point detector 121, and a distance-based measurement logic 123, such that for each image of at least a portion of the railway track, at least one object associated with the condition of the railway track may be detected, at least two points on at least one object may be identified, and a physical distance-based measurement of the condition of the railway track associated with at least one object may be determined based on at least two points on at least one object.

[0066] In an embodiment, the distance-based measurement logic 123 may be configured to combine the results obtained from each image of at least a portion of the railway track captured from different angles (e.g., the physical distance-based measurement of the condition of the railway track associated with at least one object) to generate an overall physical distance-based measurement of the condition of the railway track associated with at least one object. For example, the physical distance-based measurement of the condition of the railway track associated with at least one object detected in a first image captured from a first angle (e.g., the gauge side angle) may be combined with the physical distance-based measurement of the condition of the railway track associated with at least one object detected in a second image captured from a second angle (e.g., the field side angle) to obtain an overall physical distance-based measurement of the condition of the railway track associated with at least one object. In some embodiments, more than two images may be used to determine the overall physical distance-based measurement result of the condition of the railway track associated with at least one object.

[0067] In an embodiment, the result obtained by combining each image of at least a portion of a railway track captured from different angles may include averaging the physically - distance - based measurements of the railway track condition associated with at least one object detected in all the images. In an embodiment, the result obtained by combining each image of at least a portion of a railway track captured from different angles may include selecting the maximum physically - distance - based measurement of the railway track condition associated with at least one object from the physically - distance - based measurements of the railway track condition associated with at least one object in all the images.

[0068] Referring again to Figure 1 , the alert generator 124 may be configured to generate an alert in response to determining that the physically - distance - based measurement of the condition of the railway track associated with at least one object exceeds a predetermined threshold. In an embodiment, the alert may be communicated to an operator, for example, via the user interface 140. In an embodiment, the alert generated by the alert generator 124 may include alerts of different severity levels. For example, an alert of the first severity level may indicate to the operator that the condition of the railway track associated with at least one object is severe and immediate action must be taken to avoid a potential catastrophic failure. For example, referring to Figure 3 , the distance - measurement logic 123 may determine that the physical width of the track gap 230 may be greater than 2 inches. In this case, the alert generator 124 may generate an alert indicating that immediate action must be taken to reduce the physical width of the track gap 230 to avoid a catastrophic failure. An alert of the second severity level may indicate to the operator that the condition of the railway track associated with at least one object is moderate and measures can be taken at a later time to address the condition of the railway track. For example, the distance - based measurement logic 123 may determine that the physical width of the track gap 230 may be greater than 1.5 inches and less than 2 inches. In this case, the alert generator 124 may generate a moderate alert indicating that action can be taken at a later time to reduce the physical width of the track gap 230 and a catastrophic failure is not necessarily imminent.

[0069] An alert of the third severity level may indicate to the operator that the condition of the railway track associated with at least one object is good and no action is required at this time. For example, the distance - based measurement logic 123 may determine that the physical width of the track gap 230 may be less than 1.5 inches. In this case, the alert generator 124 may generate a low alert indicating that no action is required for the track gap 230 at this time. In some embodiments, when the condition of the railway track associated with at least one object is satisfactory, an alert of the third severity level may not be generated, but rather no alert is generated.

[0070] It should be noted that the above discussion of the three severity levels is for illustrative purposes only and should not be construed as a limitation in any way. In fact, in embodiments, more or fewer than three severity levels can be used to classify alerts, and in these cases, the severity of an alert can correspond to a physical dimension of the railroad track condition associated with at least one object.

[0071] Figure 4 FIG. 400 is a high-level flowchart showing the operation of a system configured according to an embodiment of the present disclosure for providing a function of detecting and measuring the condition of a railroad track and / or railroad track components based on image-based distance measurement. For example, Figure 4 the functions shown in the example boxes in Figure 1 can be performed by the system 100 in

[0072] depending on the embodiments in this specification. In an embodiment, the operations of method 400 can be stored as instructions that, when executed by one or more processors, cause the one or more processors to perform the operations of method 400.

[0072] At block 402, at least one object can be detected in an image of at least a portion of a railroad track. In an embodiment, the at least one object can be associated with at least one condition of the railroad track. In some embodiments, the function of an object detector (e.g., Figure 1 the object detector 120 in Figures 1-3 can be used to detect at least one object in an image of at least a portion of a railroad track. In some embodiments, the object detector can perform operations to detect at least one object in an image of at least a portion of a railroad track according to the description of the object detector 120 above and Figures 1-3 the operations and functions shown in

[0073] At block 404, a first point and a second point can be identified on at least one object detected in an image of at least a portion of a railroad track. In some embodiments, the function of a point detector (e.g., Figure 1 the point detector 121 in Figures 1-3 can be used to identify a first point and a second point on at least one object detected in an image of at least a portion of a railroad track. In some embodiments, the point detector can perform operations to identify a first point and a second point on at least one object detected in an image of at least a portion of a railroad track according to the description of the point detector 121 above and Figures 1-3 the operations and functions shown in

[0074] At block 406, the pixel distance between the first point and the second point can be measured. In an embodiment, the pixel distance between the first point and the second point can indicate the number of pixels in the line between the first point and the second point. In some embodiments, distance-based measurement logic (e.g., Figure 1The functionality of the distance-based measurement logic 123) in can be used to measure the pixel distance between a first point and a second point. In certain embodiments, the distance-based measurement logic can perform operations to measure the pixel distance between a first point and a second point in accordance with the description of the distance-based measurement logic 123 above and Figures 1-3 the operations and functionality shown.

[0075] At block 408, based on the pixel distance between the first point and the second point, a transformation model can be used to determine the physical dimensions of at least one condition of a railroad track associated with at least one object. In certain embodiments, the functionality of the distance-based measurement logic (e.g., Figure 1 the distance-based measurement logic 123) in can be used to determine the physical dimensions of at least one condition of a railroad track associated with at least one object using a transformation model based on the pixel distance between a first point and a second point. In certain embodiments, the distance-based measurement logic can use a transformation model to determine the physical dimensions of at least one condition associated with at least one object of a railroad track based on the pixel distance between a first point and a second point in accordance with the description of the distance-based measurement logic 123 above and Figures 1-3 the operations and functionality shown.

[0076] At block 410, an alert can be generated when the physical dimensions of at least one condition of a railroad track exceed a predetermined threshold. In certain embodiments, when the physical dimensions of at least one condition of a railroad track exceed a predetermined threshold, the functionality of an alert generator (e.g., Figure 1 the alert generator 124) in can be used to generate an alert. In certain embodiments, when the physical dimensions of at least one condition of a railroad track exceed a predetermined threshold, the alert generator (e.g., Figure 1 the alert generator 124) in can perform operations to generate an alert in accordance with the description of the alert generator 124 above and Figures 1-3 the operations and functionality shown.

[0077] Figure 5 FIG. 500 is a high-level flowchart showing the operation of a system configured in accordance with an embodiment of the present disclosure for providing functionality to detect and measure conditions of railroad tracks and / or railroad track components based on image-based distance measurements. For example, Figure 5 the functionality shown in the example boxes in can be performed by Figure 1 the system 100 in, depending on the embodiments in this specification. In certain embodiments, the operations of method 500 can be stored as instructions that, when executed by one or more processors, cause the one or more processors to perform the operations of method 500.

[0078] At block 502, a discontinuity in a rail joint portion can be detected in an image of at least a portion of a railroad track. In an embodiment, the discontinuity can represent a track gap between a first track segment and a second track segment. In some embodiments, the functionality of an object detector (e.g., Figure 1 the object detector 120 therein) can be used to detect a discontinuity in a rail joint portion in an image of at least a portion of a railroad track. In some embodiments, the object detector can perform operations to detect a discontinuity in a rail joint portion in an image of at least a portion of a railroad track, based on the description of the object detector 120 above and Figures 1-3 the operations and functionality shown.

[0079] At block 504, a first edge and a second edge of a discontinuity representing a track gap can be detected. In some embodiments, the functionality of an edge detector (e.g., Figure 1 the edge detector 122 therein) can be used to detect a first edge and a second edge of a discontinuity representing a track gap. In some embodiments, the edge detector can perform operations to detect a first edge and a second edge of a discontinuity representing a track gap, based on the description of the edge detector 122 above and Figures 1-3 the operations and functionality shown.

[0080] At block 506, a first point in a first edge of a discontinuity representing a rail gap and a second point in a second edge of a discontinuity representing a rail gap can be identified. In some embodiments, the functionality of a point detector (e.g., Figure 1 the point detector 121 therein) can be used to identify a first point in a first edge of a discontinuity representing a track gap and a second point in a second edge of a discontinuity representing a track gap. In some embodiments, the point detector can perform operations to identify a first point in a first edge of a discontinuity representing a track gap and a second point in a second edge of a discontinuity representing a track gap, based on the description of the point detector 121 above and Figures 1-3 the operations and functionality shown.

[0081] At block 508, a pixel distance between the first point and the second point can be measured. In an embodiment, the pixel distance between the first point and the second point can indicate the number of pixels in a line between the first point and the second point. In some embodiments, the functionality of distance-based measurement logic (e.g., Figure 1 the distance-based measurement logic 123 therein) can be used to measure a pixel distance between the first point and the second point. In some embodiments, the distance-based measurement logic can perform operations to measure a pixel distance between the first point and the second point, based on the description of the distance-based measurement logic 123 above and Figures 1-3 the operations and functionality shown.

[0082] At block 510, based on the pixel distance between the first point and the second point, a conversion model can be used to determine the physical size of the track gap. In some embodiments, the functionality of distance-based measurement logic (e.g., Figure 1 the distance-based measurement logic 123 in Figures 1-3 ) can be used to determine the physical size of the track gap using the conversion model, based on the pixel distance between the first point and the second point. In some embodiments, the distance-based measurement logic can use the conversion model to determine the physical size of the track gap, based on the pixel distance between the first point and the second point, according to the description of the distance-based measurement logic 123 above and the

[0083] operations and functionality shown. At block 512, an alert can be generated when the physical size of the track gap exceeds a predetermined threshold. In some embodiments, the functionality of an alert generator (e.g., Figure 1 the alert generator 124 in Figures 1-3 ) can be used to generate an alert when the physical size of the track gap exceeds a predetermined threshold. In some embodiments, when the physical size of the track gap exceeds a predetermined threshold, the alert generator can perform operations to generate an alert according to the description of the alert generator 124 above and the

[0084] Those skilled in the art will readily understand that the above advantages and objectives would not be possible without the specific combination of computer hardware and other structural components and mechanisms assembled in the system of the present invention and described herein. Additionally, the algorithms, methods, and processes disclosed herein transform and convert any general-purpose computer or processor disclosed in the specification and drawings into a special-purpose computer programmed to execute the disclosed algorithms, methods, and processes to achieve the above functions, advantages, and objectives. It should also be understood that for those skilled in the art, there are a variety of programming tools available for generating and implementing the functions and operations described above. Additionally, the choice of a specific programming tool may be determined by the specific goals and constraints for implementing these concepts, which are selected to implement the concepts set forth in this specification and the appended claims.

[0085] The description in this patent document should not be construed as implying that any particular element, step, or function is an essential or critical element that must be included within the scope of the claims. Additionally, no claim can be intended to invoke 35 U.S.C. § 112(f) for any additional claim or claim element, unless the exact words "means for" or "step for" are expressly used in a particular claim, followed by a participle phrase identifying the function. Terms used in the claims (such as, but not limited to, "mechanism", "module", "device", "unit", "component", "element", "member", "means", "machine", "system", "processor", "processing device", or "controller") should be understood as and intended to refer to structures known to those of ordinary skill in the relevant art, further modified or enhanced by the features of the claim itself, and not intended to invoke 35 U.S.C. § 112(f). For example, the terms "processor" and "controller" can be a class of structures rather than a specific structure, and can be defined in functional terms, but this does not mean that it is a means-plus-function. Even under the broadest reasonable interpretation, in the absence of the above specific language, claims are not intended to invoke 35 U.S.C. § 112(f) based on this section of the specification.

[0086] The present disclosure may be embodied in other specific forms without departing from its spirit or essential characteristics. For example, each new structure described herein can be modified to accommodate specific local variations or requirements while retaining their basic configuration or structural relationships to one another, or while performing the same or similar functions described herein. Accordingly, the present embodiments are to be considered in all respects as illustrative and not restrictive. Thus, the scope of the invention is determined by the appended claims rather than by the foregoing description. Accordingly, all variations within the meaning and scope of the claims are to be embraced within the claims. Additionally, the individual elements of the claims are not well-known, conventional, or traditional. Rather, the claims are directed to the non-conventional inventive concepts described in the specification.

[0087] Those skilled in the art should also understand that the various illustrative logical blocks, modules, circuits, and algorithmic steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been described generally in terms of their functionality above. Whether such functionality is implemented as hardware or software depends upon the particular application and the design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be construed as causing a departure from the scope of the present disclosure. Those skilled in the art will also readily recognize that the order or combination of the components, methods, or interactions described herein is merely exemplary, and the components, methods, or interactions of the various embodiments of the present disclosure can be combined or performed in ways other than those shown and described herein.

[0088] The functional blocks and modules shown in Figures 1-5 may include a processor, an electronic device, a hardware device, an electronic component, a logic circuit, a memory, software code, firmware code, etc., or any combination thereof. Consistent with the foregoing, the various illustrative logical blocks, modules, and circuits described in connection with the disclosure herein can be implemented or executed using a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but, alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0089] The steps of a method or algorithm described in connection with the disclosure herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. The exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal, a base station, a sensor, or any other communication device. Alternatively, the processor and the storage medium may reside as discrete components in a user terminal.

[0090] In one or more exemplary designs, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted via a computer-readable medium as one or more instructions or code. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. The computer-readable storage media may be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and that can be accessed by a general purpose or special purpose computer or a general purpose or special purpose processor. Also, a connection may be properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL), then the coaxial cable, fiber optic cable, twisted pair, or DSL are included in the definition of medium. The terms "disk" and "disc" can include CD's, laser disks, optical disks, DVD's, floppy disks, and Blu-ray disks where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0091] Although the invention has been described in detail with respect to its advantages, it is to be understood that various changes, substitutions and alterations can be made without departing from the spirit and scope of the invention as defined by the appended claims. Furthermore, the scope of the present application is not limited to the specific embodiments of the processes, machines, manufactures, compositions of matter, means, methods and steps described in the specification. It will be readily apparent to those of ordinary skill in the art that the present disclosure, the processes, machines, manufacture, compositions of matter, means, methods or steps, whether existing or to be developed, that perform substantially the same function, in substantially the same manner, or achieve substantially the same result as the corresponding embodiments described herein, can be used in accordance with the present invention. Accordingly, the appended claims are intended to embrace such processes, machines, manufactures, compositions of matter, means, methods or steps within their scope.

Claims

1. A method for determining the condition of a railway track, comprising: detecting at least one object in at least one image of at least a portion of a railway track, the at least one object being associated with at least one condition of the railway track; identifying a first point and a second point on the at least one object detected in at least one image of at least a portion of the railway track; measuring a pixel distance between the first point and the second point, wherein the pixel distance between the first point and the second point represents the number of pixels on the straight line between the first point and the second point; using a conversion model to determine a physical dimension of at least one condition of the railway track associated with the at least one object based on the pixel distance between the first point and the second point; and generating an alarm when the physical dimension of at least one condition of the railway track exceeds a predetermined threshold.

2. The method according to claim 1, wherein identifying a first point and a second point on the at least one object detected in at least one image of at least a portion of the railway track comprises: inputting at least one object detection into an edge detection model, wherein the edge detection model is configured to detect at least one edge of the at least one object; and identifying a first point on a first edge of the at least one edge of the at least one object and a second point on a second edge of the at least one edge of the at least one object.

3. The method according to claim 1, further comprising acquiring at least one image of at least a portion of the railway track while traveling over the railway track.

4. The method according to claim 1, wherein the conversion model defines a conversion formula for converting the number of pixels into a physical distance.

5. The method according to claim 4, wherein the conversion formula defines a pixels per inch value to convert the pixel distance into a physical distance, wherein the pixels per inch value is between 30 and 60 pixels per inch.

6. The method according to claim 1, wherein at least one condition of the railway track associated with the at least one object includes one or more of the following: the width of the rail gap between a first rail section and a second rail section within a rail joint section of the railway track; the surface condition of a track section of the railway track, wherein, the surface condition includes one or more of defects, cracks, and metal fatigue indicators; the spacing between two or more crossbeams; and the separation between an anchor and a crossbeam.

7. The method according to claim 1, wherein the at least one image includes a plurality of images, each of the plurality of images being captured from a different angle, wherein identifying a first point and a second point on the at least one object includes identifying a first point and a second point on the at least one object in each of the plurality of images captured from different angles.

8. The method according to claim 7, wherein measuring the pixel distance between the first point and the second point includes measuring the pixel distance between the first point and the second point in each of a plurality of images captured from different angles, and determining the physical dimension of at least one condition of the railway track associated with at least one object includes determining the physical dimension of at least one condition of the railway track associated with at least one object based on the pixel distance between the first point and the second point measured in each of a plurality of images captured from different angles.

9. The method according to claim 8, wherein, determining the physical dimension of at least one condition of the railway track associated with at least one object based on the pixel distance between the first point and the second point measured in each of a plurality of images captured from different angles includes one of the following: averaging the physical dimension of at least one condition of the railway track associated with at least one object based on the pixel distance between the first point and the second point measured in all of the plurality of images captured from different angles; or selecting the maximum physical distance-based measurement value from the physical dimension of at least one condition of the railway track associated with at least one object based on the pixel distance between the first point and the second point measured in all of the plurality of images captured from different angles.

10. A method for determining the condition of a railway track, comprising: detecting a discontinuity in a rail joint portion in at least one image of at least a portion of the railway track, the discontinuity representing a track gap between a first track portion and a second track portion; detecting a first edge and a second edge of the discontinuity representing the track gap; identifying a first point in the first edge of the discontinuity representing the rail gap and a second point in the second edge of the discontinuity representing the rail gap; measuring the pixel distance between the first point and the second point, wherein the pixel distance between the first point and the second point represents the number of pixels on the straight line between the first point and the second point; determining the physical dimension of the track gap based on the pixel distance between the first point and the second point using a conversion model; and issuing an alarm when the physical dimension of the track gap exceeds a predetermined threshold.

11. The method according to claim 10, wherein detecting the first edge and the second edge of the discontinuity representing the track gap comprises: inputting the discontinuity detection into an edge detection model, wherein the edge detection model is configured to detect the first edge and the second edge of the discontinuity representing the track gap.

12. The method according to claim 1, further comprising acquiring at least one image of at least a portion of the railway track while traveling over the railway track.

13. The method according to claim 1, wherein the conversion model defines a conversion formula for converting the number of pixels to a physical distance.

14. The method according to claim 13, wherein the conversion formula defines the number of pixels per inch to convert the pixel distance to a physical distance, and the number of pixels per inch is between 30 and 60 pixels per inch.

15. The method according to claim 1, wherein at least one image comprises a plurality of images, each of the plurality of images being captured from a different angle, and wherein identifying a first point on a first edge of a discontinuity and a second point on a second edge of the discontinuity comprises identifying, in each of the plurality of images captured from different angles, the first point on the first edge of the discontinuity and the second point on the second edge of the discontinuity.

16. The method according to claim 15, wherein measuring the pixel distance between the first point and the second point comprises measuring the pixel distance between the first point and the second point in each of the plurality of images captured from different angles, and determining the physical size of the track gap comprises determining the physical size of the track gap based on the pixel distance between the first point and the second point measured in each of the plurality of images captured from different angles.

17. The method according to claim 16, wherein, determining the physical size of the track gap based on the pixel distance between the first point and the second point measured in each of the plurality of images captured from different angles comprises one of the following: averaging the physical size of the track gap according to the pixel distance between the first point and the second point measured in all of the plurality of images captured from different angles; or selecting the largest physical distance-based measurement value from the physical size of the track gap according to the pixel distance between the first point and the second point measured in all of the plurality of images captured from different angles.

18. A system for determining the condition of a railway track, the system comprising: at least one processor; and a memory operably coupled to the at least one processor and storing processor-readable code that, when executed by the at least one processor, is configured to perform operations including: detecting at least one object in at least one image of at least a portion of a railway track, the at least one object being associated with at least one condition of the railway track; identifying a first point and a second point on the at least one object detected in at least one image of at least a portion of the railway track; measuring the pixel distance between the first point and the second point, wherein the pixel distance between the first point and the second point represents the number of pixels on the straight line between the first point and the second point; using a conversion model to determine the physical size of at least one condition of the railway track associated with the at least one object based on the pixel distance between the first point and the second point; and generating an alert when the physical size of at least one condition of the railway track exceeds a predetermined threshold.

19. The system according to claim 18, wherein identifying a first point and a second point on the at least one object detected in at least one image of at least a portion of the railway track comprises: inputting at least one object detection into an edge detection model, wherein the edge detection model is configured to detect at least one edge of the at least one object; and identifying a first point on a first edge of the at least one edge of the at least one object and a second point on a second edge of the at least one edge of the at least one object.

20. The system according to claim 18, wherein, The conversion model defines a conversion formula for converting the number of pixels into a physical distance, and wherein the conversion formula defines the pixels per inch value for converting a pixel distance into a physical distance, and wherein the pixels per inch value is between 30 and 60 pixels per inch.

Images