Systems and methods for reconstructing objects using transitional images

Abstract

To reconstruct an image from consecutively captured images of objects such as railroad tracks taken from a railway vehicle, the image achieving both identification speed and identification accuracy.SOLUTION: A method includes, by an object reconstruction module, a step of receiving a first image and a second image. The first image includes a first region of an object, and the second image includes a second region of the object. The method also includes, by the object reconstruction module, a step of identifying a transitional image. The transitional image includes the first region of the object and the second region of the object. The method further includes, by the object reconstruction module, a step of determining that the first region of the object in the transitional image and the first region of the object in the first image are equivalent regions, and a step of generating a reconstruction of the object by using the first image and the transitional image. The reconstruction of the object includes the first region of the object and the second region of the object and excludes the equivalent regions.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE This disclosure relates generally to reconstructing objects, and more particularly to a system and method for reconstructing objects using transitional images. [Background technology]

[0002] Object detection models process large amounts of image data and infer each image independently. Deep learning models inherently have a tradeoff between speed and accuracy. This tradeoff is amplified when models are trained to detect objects that vary dramatically in size and / or portions of objects that represent a very small proportion of the total pixel data. While object detection models can be trained to accurately identify objects within multiple images, certain systems cannot tolerate the latency penalty associated with accurately detecting very small portions of an object. Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment, a method includes receiving, by an object reconstruction module, a first image and a second image. The first image includes a first region of an object, and the second image includes a second region of the object. The method also includes identifying, by the object reconstruction module, a transition image. The transition image includes the first region of the object and the second region of the object. The method also includes determining, by the object reconstruction module, that the first region of the object in the transition image and the first region of the object in the first image are first equivalent regions, and generating, by the object reconstruction module, a reconstruction of the object using the first image and the transition image. The reconstruction of the object includes the first region of the object and the second region of the object and excludes the first equivalent region.

[0004] In certain embodiments, the first image further includes a third region of the object, a fourth region of the object, and a fifth region of the object, and the transition image includes the third region of the object and the fourth region of the object. In some embodiments, the method includes determining, by an object reconstruction module, the third region of the object in the transition image and the third region of the object in the first image. The transition image is a third equivalent region, and the object reconstruction module determines that the fourth region of the object in the transition image and the fourth region of the object in the first image are fourth equivalent regions. In certain embodiments, the reconstruction of the object excludes the third and fourth regions. In some embodiments, the method includes concatenating, by the object reconstruction module, the first region of the object, the second region of the object, the third region of the object, the fourth region of the object, and the fifth region of the object to generate a reconstruction of the object, where the reconstruction of the object represents the entire object.

[0005] In certain embodiments, the method includes identifying, by an object reconstruction module, a first portion of an object in the first image; and constructing, by the object reconstruction module, a first bounding box around the first portion of the object in the first image. The first bounding box may include a first region of the object. In some embodiments, the method includes identifying, by an object reconstruction module, a second portion of the object in the transitional image; and constructing, by the object reconstruction module, a second bounding box around the second portion of the object in the transitional image. The second bounding box may include the first region of the object and the second region of the object. In particular embodiments, the method includes using Intersection over Union (IoU) to determine that the first region of the object in the transitional image and the first region of the object in the first image are equivalent regions. In some embodiments, the transitional image includes half of the first image and half of the second image.

[0006] According to another embodiment, an apparatus includes one or more processors and one or more computer-readable non-transitory storage media coupled to the one or more processors. The one or more computer-readable non-transitory storage media include instructions that, when executed by the one or more processors, cause the apparatus to perform a process including receiving a first image and a second image. The first image includes a first region of an object, and the second image includes a second region of the object. The process also includes identifying a transition image. The transition image includes the first region of the object and the second region of the object. The process also includes determining that the first region of the object in the transition image and the first region of the object in the first image are first equivalent regions, and generating a reconstruction of the object using the first image and the transition image. The reconstruction of the object includes the first region of the object and the second region of the object and excludes the first equivalent region.

[0007] According to yet another embodiment, one or more computer-readable storage media embody instructions that, when executed by a processor, cause the processor to perform a process including receiving, by an object reconstruction module, a first image and a second image. The first image includes a first region of an object, and the second image includes a second region of the object. The process also includes identifying, by the object reconstruction module, a transition image. The transition image includes the first region of the object and the second region of the object. The process also includes determining, by the object reconstruction module, that the first region of the object in the transition image and the first region of the object in the first image are first equivalent regions, and generating, by the object reconstruction module, a reconstruction of the object using the first image and the transition image. The reconstruction of the object includes the first region of the object and the second region of the object and excludes the first equivalent region.

[0008] Technical advantages of certain embodiments of the present disclosure may include one or more of the following: The present disclosure describes systems and methods for reconstructing an object using transitional images. In certain embodiments, the bounding box reduction algorithm used to reconstruct the object may be used in combination with standard non-maximum suppression algorithms that operate on a per-image basis to reduce the bounding box contained within a single image and a single analogy result. In certain embodiments, the transitional image is generated by overlapping two images at a predetermined percentage. In certain embodiments of the present disclosure, a predetermined percentage of 50% is used, although the present disclosure contemplates any suitable predetermined percentage (e.g., 25 percent, 75 percent, etc.). In certain embodiments, the systems and methods disclosed herein may use reduction criteria such as an IoU score between two or more boxes, equality of detected labels, and / or a confidence score threshold.

[0009] In certain embodiments of the present disclosure, the systems and methods disclosed herein may be used across several object classes within the same image. For example, in some embodiments, bounding boxes belonging to the same class may be logically shrunk, and the As a result, several fully constructed objects may be generated. In certain embodiments, IoU may be used to calculate equivalent regions of bounding boxes and / or other regions of interest, similar to how standard non-maximum suppression algorithms are used to calculate equivalents. The systems and methods described in this disclosure may be generalized to a variety of transportation infrastructures, including rail, road, and waterways.

[0010] Other technical advantages will be readily apparent to those skilled in the art from the following figures, description, and claims. Additionally, while certain advantages have been enumerated above, various embodiments may include all, some, or none of the enumerated advantages.

[0011] For an improved understanding of the present disclosure, reference is made to the following description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 illustrates an exemplary system for reconstructing an object using transitional images. [Figure 2] FIG. 2 illustrates an exemplary image that may be used by the system of FIG. 1. [Figure 3] 2A and 2B illustrate exemplary transition images that may be used by the system of FIG. 1. [Figure 4] 2A and 2B illustrate exemplary equivalent regions that may be determined by the system of FIG. 1. [Figure 5] FIG. 2 illustrates an example of an object reconstruction that may be generated by the system of FIG. 1. [Figure 6] FIG. 1 illustrates an exemplary method for reconstructing an object using transitional images. [Figure 7] FIG. 1 illustrates an exemplary computer system that may be used by the systems and methods described herein. DETAILED DESCRIPTION OF THE INVENTION

[0013] Figures 1-7 illustrate exemplary systems and methods for reconstructing an object using transition images. Figure 1 illustrates an exemplary system for reconstructing an object using transition images, and Figure 2 illustrates an exemplary image that may be used by the system of Figure 1. Figure 3 illustrates an exemplary transition image that may be used by the system of Figure 1, and Figure 4 illustrates an exemplary equivalent region that may be determined by the system of Figure 1. Figure 5 illustrates an example object reconstruction that may be generated by the system of Figure 1. Figure 6 illustrates an exemplary method for reconstructing an object using transition images. Figure 7 illustrates an exemplary computer system that may be used by the systems and methods described herein.

[0014] FIG. 1 illustrates an exemplary system 100 for reconstructing an object using transitional images. System 100, or portions thereof, may be associated with an entity, including any organization, such as a business, company (e.g., a railroad company, a transportation company, etc.), or government agency (e.g., a transportation department, a public safety department, etc.), that reconstructs an object using transitional images. Elements of system 100 may be implemented using any suitable combination of hardware, firmware, and software. For example, elements of system 100 may be implemented using one or more components of the computer system of FIG. 7. System 100 includes a network 110, a railway environment 120, railroad tracks 130, a railcar 140, an image capture module 150, an object 160, an object reconstruction module 170, an image 172, a transitional image 174, a bounding box 176, an equivalent region, and an object reconstruction 180.

[0015] Network 110 may be any type of network that facilitates communication between components of system 100. One or more portions of network 110 may be an ad hoc network. The network 110 may include a network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), a wireless wide area network (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the public switched telephone network (PSTN), a cellular network, a 3G network, a 4G network, a 5G network, a Long Term Evolution (LTE) cellular network, a combination of two or more thereof, or any other suitable type of network. One or more portions of the network 110 may include one or more access (e.g., mobile access), core, and / or edge networks. The network 110 may be any communication network, such as a private network, a public network, a connection via the Internet, a mobile network, a Wi-Fi network, a Bluetooth network, etc. One or more components of the system 100 may communicate over the network 110. For example, the object reconstruction module 170 may communicate over the network 110, including receiving information from the image capture module 150.

[0016] The rail environment 120 of the system 100 is an area that includes one or more railroad tracks 130. The railroad environment 120 may be associated with divisions and / or subdivisions. A division is a portion of a railroad under the supervision of a supervisor. A subdivision is a smaller portion of a division. A subdivision may be a crew district and / or a branch line. In the embodiment shown in FIG. 1, the railroad environment 120 includes railroad tracks 130, railcars 140, image capture modules 150, and objects 160.

[0017] The railroad track 130 of the system 100 is a structure that allows the railroad vehicles 140 to move by providing a surface for the wheels of the railroad vehicles 140 to roll on. In certain embodiments, the railroad track 130 includes rails, fasteners, sleepers, ballast, etc. The railroad vehicles 140 of the system 100 are vehicles that can move along the railroad track 130. The railroad vehicles 140 may be geometry cars, locomotives, passenger cars, freight cars, box cars, flat cars, tank cars, etc. In certain embodiments, the railroad vehicles 140 may be associated with one or more image capture modules 150.

[0018] The image capture module 150 of the system 100 is a component that captures images 172 of objects 160. The objects 160 of the system 100 are transportation infrastructure components such as roads, railroads, airways, waterways, canals, pipelines, and terminal components. The objects 160 may include components within the railway environment 120, such as railroad tracks 130, debris (e.g., rubble, wreckage, ruins, litter, trash, brush, etc.), pedestrians (e.g., trespassers), animals, vegetation, ballast, etc. The objects 160 may include components of the railroad track 130, such as joints, switches, frogs, rail heads, anchors, fasteners, gauge plates, ballast, and sleepers (e.g., concrete sleepers and wooden sleepers). The images 172 of the system 100 show the physical form of one or more of the objects 160. The images 172 may include digital images, photographs, etc. The images 172 include image 172a, image 172b, etc., up to image 172n, where 172n represents any suitable number.

[0019] Image capture module 150 may include one or more cameras, lenses, sensors, optical devices, lighting elements, etc. For example, image capture module 150 may capture an image of object 1 The image capture module 150 may include one or more cameras (e.g., high-resolution cameras, line-scan cameras, stereo cameras, etc.) that automatically capture images 172 of the railroad environment 120. In particular embodiments, the image capture module 150 is mounted to the railroad vehicle 140. The image capture module 150 may be mounted to the railroad vehicle 140 in any suitable location that provides a clear view of the railroad environment 120. For example, the image capture module 150 may be mounted to the front end (e.g., the front bumper) of the railroad vehicle 140 to provide a downward-facing view of the railroad track 130. As another example, the image capture module 150 may be mounted to the front end (e.g., the windshield) of the railroad vehicle 140 to provide a forward view of the railroad track 130. As yet another example, the image capture module 150 may be mounted to the rear end (e.g., the rear windshield) of the railroad vehicle 140 to provide a rearward view of the railroad track 130.

[0020] Image capture module 150 may automatically capture still or video images 172 while railcar 140 travels along railroad track 130. Image capture module 150 may automatically capture any suitable number of still or video images 172. For example, image capture module 150 may automatically capture images 172 as a function of distance using an encoder (e.g., a position encoder) to trigger a camera (e.g., a line scan camera). As another example, image capture module 150 may automatically capture a predetermined number of still images 172 every second, every minute, every hour, etc. In some embodiments, image capture module 150 may inspect images 172 for object 160. For example, image capture module 150 may detect object 160 in image 172 using one or more models. The models may include one or more object detection models, machine learning models, machine vision models, deep learning models, etc. The image capture module 150 may communicate one or more images 172 of the one or more objects 160 to the object reconstruction module 170 .

[0021] The object reconstruction module 170 of the system 100 is a component that recreates the object 160 from one or more images 172 and / or one or more transition images 174. In particular embodiments, the object reconstruction module 170 receives the images 172 from one or more components of the system 100. For example, the object reconstruction module 170 may receive the image 172a and the image 172b from the image capture module 150. The multiple images 172 may include one or more portions of a single object 160. For example, the image 172a may include a first portion of the object 160, and the image 172b may include a second portion of the object 160.

[0022] Each object 160 may be represented as a region. For example, object 160 (e.g., a rail joint) may have five regions, such that image 172a includes four contiguous regions of object 160 and image 172b includes the remaining fifth region of object 160. As another example, object 160 may have five regions, such that image 172a includes three contiguous regions of object 160 and image 172b includes the remaining fourth and fifth regions of object 160.

[0023] In particular embodiments, object reconstruction module 170 performs inference on one or more images 172 and / or transition images 174. Object reconstruction module 170 may perform inference by executing an object detection algorithm to detect objects 160 in images 172 and / or transition images 174. In response to performing the inference, object reconstruction module 170 may generate one or more bounding boxes 176 around one or more objects 160 (or portions thereof). Bounding boxes 176 are contours that identify objects 160. In particular embodiments, each bounding box 176 is represented by x and y coordinates that identify the four corners of a rectangle. , which is a rectangular box around object 160. Bounding boxes 176 include bounding box 176a, bounding box 176b, bounding box 176c, etc., up to bounding box 176n, where 174n represents any suitable number.

[0024] In particular embodiments, image capture module 150 constructs a bounding box 176 around object 160 (or a portion thereof) in image 172. For example, object reconstruction module 170 may identify a first portion of object 160 in image 172a and construct a bounding box 176a around the first portion of object 160 in image 172a. As another example, object reconstruction module 170 may identify a second portion of object 160 in image 172b and construct a bounding box 176b around the second portion of object 160 in image 172b. As yet another example, object reconstruction module 170 may identify a third portion of object 160 in transitional image 174a and construct a bounding box 176c around the third portion of object 160 in transitional image 176a.

[0025] In some embodiments, the regions of object 160 are defined by bounding boxes 176. For example, for an object having five regions, a first portion of object 160 bounded by bounding box 176a in image 172a may define the first, second, third, and fourth regions of object 160, a second portion of object 160 bounded by bounding box 176b in image 172b may define the fifth region of object 160, and a third portion of object 160 bounded by bounding box 176c in transition image 174a may define the second, third, fourth, and fifth regions of object 160.

[0026] In particular embodiments, object reconstruction module 170 generates transition images 174 from images 172. A transition image 174 is an image that shares the same pixels as one or more other images 172. For example, image 172a and transition image 174a may share a predetermined percentage of the same pixels (e.g., 25, 50, or 75%). Transition images 174 include transition image 174a, transition image 174b, and up to transition image 174n, where 176n represents any suitable number.

[0027] In some embodiments, object reconstruction module 170 generates transition image 174 by overlapping two or more images 172. For example, object reconstruction module 170 may generate transition image 174a by overlapping a portion (e.g., 50%) of image 172a with a portion (e.g., 50%) of image 172b such that transition image 174a includes overlapping portions of image 172a and image 172b. In some embodiments, image capture module 150 captures transition image 174. For example, image capture module 150 may be configured to capture subsequent images 172 that include a predetermined amount of overlap (e.g., 25, 50, or 75 percent overlap).

[0028] In certain embodiments, each transition image 174 includes one or more regions of object 160. For example, for an object having five regions, image 172a may include a first region, a second region, a third region, and a fourth region, image 172b may include a fifth region, and transition image 174a may include the second, third, and fourth regions of image 172a and the fifth region of image 172b.

[0029] In certain embodiments, the object reconstruction module 170 determines one or more equivalent regions 178. The equivalent regions 178 are regions of the image 172 and the transition image 173 that share the same pixels. 174. Equivalent regions 178 include equivalent regions 178a, 178b, etc., up to equivalent region 178n, where 178n represents any suitable number. Object reconstruction module 170 may determine equivalent regions 178 between one or more images 172 and one or more transition images 174. For example, object reconstruction module 170 may determine that both image 172a and transition image 174a contain second, third, and fourth regions of object 160. In particular embodiments, object reconstruction module 170 excludes equivalent regions 178 from object reconstruction 180. In some embodiments, object reconstruction module 170 uses IoU, equivalence of detection labels, and / or a confidence score threshold to determine that a first region of object 160 in transition image 174 and a first region of object 160 in image 172a are equivalent regions 178.

[0030] In certain embodiments, object reconstruction module 170 generates object reconstructions 180. Each object reconstruction 180 is a representation of the entire object 160. For example, object reconstruction 180 may include first, second, third, and fourth regions of image 172a and a fifth region of image 172b. Object reconstruction 180 includes object reconstruction 180a, object reconstruction 180b, etc., up to object reconstruction 180n, where 180n represents any suitable number. In certain embodiments, object reconstruction module 170 generates object reconstruction 180 using one or more images 172 and one or more transition images 174. For example, object reconstruction module 170 may generate object reconstruction 180a by concatenating the first, second, third, and fourth regions from image 172a with the fifth region from transition image 174a. In certain embodiments, object reconstruction module 170 excludes equivalent regions 178 to avoid overlapping regions of object 160 when generating object reconstruction 180. For example, if both image 172a and transition image 174a include the second, third, and fourth regions of object 160, object reconstruction module 170 may exclude the second, third, and fourth regions of transition image 174a to avoid overlapping these regions in object reconstruction 180a.

[0031] In operation, the image capture module 150 of the system 100 is mounted on the rail vehicle 140 .

[0032] The image capture module captures images 172a and 172b of an object 160 while the rail vehicle 140 moves along a railroad track 130 in the railroad environment 120. An object reconstruction module 170 of the system 100 receives images 172a and 172b of the object 160 (e.g., a rail joint bar) from the image capture module 150 via the network 110. The object reconstruction module 170 uses one or more object detection models to detect a first portion of the object 160 in the image 172a. The object reconstruction module 170 constructs a bounding box 176a around the first portion of the object 160 in the image 172a. The object 160 is represented as five regions, and the first portion of the image 172a enclosed by the bounding box 176a includes the first, second, third, and fourth regions of the object 160. The image 172b includes the second and fifth portions of the object 160. However, object reconstruction module 170 may not detect the second portion of object 160 due to the size of the second portion relative to the overall size of image 172b.

[0033] The object reconstruction module 170 generates the transition image 174a by overlapping the image 172a and the image 172b such that the transition image 174a includes half of the image 172a and half of the image 172b. The object reconstruction module 170 detects the third portion of the object 160 in the transition image 174a and generates the transition image 174a by overlapping the image 172a and the image 172b. The object reconstruction module 170 then ... detects the third portion of the object 160 in the transition image 174a and generates the transition image 174a by overlapping the image 172a and the image 172b. The object reconstruction module 170 constructs a bounding box 176c around the object 160. The bounding box 176c includes the second, third, fourth, and fifth regions of the object 160. The object reconstruction module 170 determines that the second, third, and fourth regions of the image 172a and the transition image 174a are equivalent regions 178. The object reconstruction module 170 excludes the equivalent regions 178 (e.g., the third, fourth, and fifth regions of the transition image 174a) in the object reconstruction 180 to avoid overlapping regions of the object 160. The object reconstruction module 170 generates the object reconstruction 180 of the object 160 by concatenating the first, second, third, and fourth regions of the image 172a with the fifth region of the transition image 174a. In this manner, the system 100 can be used to accurately reconstruct the entire object 160 even if a small portion of the object 160 is not detected in the image 172b.

[0034] 1 illustrates a particular arrangement of network 110, railroad environment 120, railroad tracks 130, railroad vehicles 140, image capture module 150, object 160, object reconstruction module 170, image 172, bounding box 176, transition image 174, equivalent region 178, and object reconstruction 180. This specification contemplates any suitable arrangement of network 110, railroad environment 120, railroad tracks 130, railroad vehicles 140, image capture module 150, object 160, object reconstruction module 170, image 172, bounding box 176, transition image 174, equivalent region 178, and object reconstruction 180.

[0035] For example, the image capture module 150 and the object reconstruction module 170 may be combined into a single module.

[0036] 1 illustrates a particular number of networks 110, rail environments 120, rail tracks 130, rail vehicles 140, image capture modules 150, objects 160, object reconstruction modules 170, images 172, bounding boxes 176, transition images 174, equivalent regions 178, and object reconstructions 180. This specification contemplates any suitable number of networks 110, rail environments 120, rail tracks 130, rail vehicles 140, image capture modules 150, objects 160, object reconstruction modules 170, images 172, transition images 174, bounding boxes 176, equivalent regions 178, and object reconstructions 180. For example, system 100 may include multiple rail environments 120, multiple image capture modules 150, and / or multiple object reconstruction modules 170.

[0037] Figure 2 shows an example image 172 that may be used by the system of Figure 1. In the embodiment shown in Figure 2, image 172 includes image 172a and image 172b. Image 172a includes a portion of an object (e.g., object 160 of Figure 1), and image 172b includes the remaining portion of the same object. The portion of the object in image 172a is surrounded by bounding box 176a, and the portion of the object in image 172b is surrounded by bounding box 176b.

[0038] To aid in understanding the entire object in image 172, the object is divided into five regions. The portion of the object in image 172a includes regions 1, 2, 3, and 4, while the portion of the object in image 172b includes region 5. If the object is represented as two independent images 172, the reconstruction of the entire object is bounding box 176a (including regions 1 through 4) from image 172a, followed by bounding box 176b (including region 5) from image 172b. Concatenation of bounding box 176a and bounding box 176b generates the complete set of regions 1 through 5.

[0039] As shown in FIG. 2, image 172b contains a relatively small portion of the object, a fifth of the object. While the object reconstruction module of FIG. 1 can be trained to accurately detect the relatively small portion of the object in image 172b, this process can significantly slow down object detection speed. In certain embodiments, the object reconstruction module may not be able to absorb the delay penalty of detecting region 5 in image 172b. However, the object reconstruction module needs to capture the entire object. To solve this problem, the object reconstruction module can utilize transition images, as described in FIG. 3 below, to accurately detect the entire object without incurring the full delay penalty.

[0040] 3 illustrates a transition image 174a that may be used by the system of FIG. 1. Images 172a and 172b are each divided into two portions, a first portion and a second portion. The second portion of image 172a includes regions 2, 3, and 4 of bounding box 176a, and the first portion of image 172b includes region 5 of bounding box 176b. In particular embodiments, the object reconstruction module may generate transition image 174a by overlaying the second portion of image 172a with the first portion of image 172b, such that transition image 174a includes the second portion of image 172a and the first portion of image 172b. In some embodiments, transition image 174a is generated by an image capture module.

[0041] Independent inference is performed on each of image 172a, image 172b, and transition image 174a of Figure 3. In response to performing the inference, bounding box 176a is constructed on image 172a, bounding box 176b is constructed on image 172b, and bounding box 176c is constructed on transition image 174a. Image 172a includes regions 1, 2, 3, and 4, image 172b includes region 5, and transition image 174a includes regions 2, 3, 4, and 5.

[0042] The presence of transition image 174a allows an object reconstruction module that sacrifices accuracy for speed to accurately detect the entire object in images 172a and 172b. Even if the object reconstruction module cannot detect region 5 of the object in image 172b, the pixels originally contained in image 172b are also contained in transition image 174a. Thus, the detection of the object in image 172a and transition image 174a is a perfect detection. The analogy to an additional image (e.g., transition image 174a) incurs a delay penalty, but in certain embodiments, this delay penalty may be less than the delay penalty incurred by accurately detecting region 5 of the object in image 172b. Figures 4 and 5 below illustrate additional processing applied to the images to avoid overlap of regions 1 through 5.

[0043] 4 shows an exemplary equivalent region 178a that may be determined by the system of FIG. 1. Image 172a includes bounding box 176a (enclosing region 1, region 2, region 3, and region 4 of the object), and image 172b includes bounding box 176b (enclosing region 5 of the object). In particular embodiments, each region may be enclosed by its own bounding box 176. For example, region 1 of image 172a may be enclosed by a first bounding box 176, region 2 of image 172a may be enclosed by a second bounding box 176, and so on. In particular embodiments, bounding boxes may be nested within each other. For example, if two other overlapping objects are detected, region 1 may be enclosed by a first bounding box 176 that may be nested within a larger bounding box 176.

[0044] As shown in FIG. 4, bounding box 176a of image 172a and bounding box 176c of transition image 174a are not equivalent. Bounding boxes 176a and 176c occupy different coordinate spaces and contain different regions of the object. However, the pixels contained in the second portion of image 172a are the same pixels contained in the first portion of transition image 174a. A simple concatenation of bounding box 176a of image 172a and bounding box 176c of transition image 174a produces region 12342345, which is an inexact overlap of regions 2, 3, and 4 of the entire object.

[0045] To accurately reconstruct the entire object without overlapping regions of the object, image 172a and / or transition image 174a are scaled down to account for overlapping pixels. In certain embodiments, the object reconstruction module of FIG. 1 determines equivalent regions 178a in image 172a and transition image 174a. Equivalent regions 178a are regions in two or more images that share the same pixels. The object reconstruction module 170 may use IoU, equality of detection labels, and / or a confidence score threshold to determine equivalent regions 178a. For example, the object reconstruction module 170 may use IoU to determine that a bounding box 176 enclosing regions 2, 3, and 4 in image 172a and regions 2, 3, and 4 in transition image 174a are equivalent regions 178a. In certain embodiments, the object reconstruction module calculates the IoU of the detections in image 172a and transition image 174a and determines whether the IoU is greater than a predetermined threshold. If the IoU is greater than a predetermined threshold, the object reconstruction module 170 determines that the overlap between the image 172a and the transition image 174a defines an equivalent region 178a, which may be excluded from the final object reconstruction, as described in FIG.

[0046] FIG. 5 illustrates an exemplary object reconstruction 180a that may be generated by the system of FIG. 1. As shown in FIG. 4 above, regions 2, 3, and 4 of image 172a and transition image 174a are equivalent regions 178a. Equivalent regions 178a are excluded from object reconstruction 180a to avoid overlapping of such regions (i.e., regions 2, 3, and 4 are included only once in object reconstruction 180a). The object reconstruction module of FIG. 1 generates object reconstruction 180a by concatenating regions 1, 2, 3, 4, and 5. The concatenation of regions 1 through 5 may be represented as 12345 such that object reconstruction 180a represents an object (e.g., object 160 of FIG. 1 collectively). If region 5 of the object in image 172b is detected by the object reconstruction module, the same process may be applied to avoid overlapping of equivalent regions 178b.

[0047] FIG. 6 shows an example method 600 for reconstructing an object using transitional images (e.g., transitional image 174a in FIG. 1 ). Method 600 begins at step 610. At step 620, an object reconstruction module (e.g., object reconstruction module 170 in FIG. 1 ) receives two or more images of an object (e.g., object 160 in FIG. 1 ) over a network (e.g., network 110 in FIG. 1 ). For example, the object reconstruction module may receive a first image (e.g., image 172a in FIG. 1 ) and a second image (e.g., image 172b in FIG. 1 ) of the object from an image capture module (e.g., image capture module 150 in FIG. 1 ). The image capture module may be attached to a rail vehicle (e.g., rail vehicle 140 in FIG. 1 ). In certain embodiments, the image capture module captures images of the object while the rail vehicle travels along railroad tracks (e.g., railroad track 130) in a railroad environment (e.g., railroad environment 120 in FIG. 1 ). Next, the method 600 moves from step 620 to step 630 .

[0048] At step 630 of method 600, the object reconstruction module determines whether to use a transition image (e.g., transition image 174a in FIG. 3) for object reconstruction. For example, the object reconstruction module may decide to use a transition image if the object reconstruction module prioritizes object detection speed over object detection accuracy (e.g., detecting a relatively small portion of the object in the received image). If the object reconstruction module determines not to use a transition image for object reconstruction, the object reconstruction module proceeds from step 630 to step 670, where the object reconstruction module generates a reconstruction of the object from the received images. For example, a first image received by the object reconstruction module may include a first region, a second region, and a third region of the object, and a second image received by the object reconstruction module may include a fourth region and a fifth region of the object.

[0049] The object reconstruction module may detect a first region, a second region, and a third region of the object in the first image. The object reconstruction module may detect a fourth region and a fifth region of the object in the second image. The object reconstruction module may generate a reconstruction of the object by concatenating the first region, the second region, and the third region detected in the first image with the fourth region and the fifth region detected in the second image, such that the object reconstruction represents the entire object. Next, method 600 moves from step 670 to step 680, where method 600 ends.

[0050] If, in step 630, the object reconstruction module determines to utilize a transition image for object reconstruction, the object reconstruction module may create the transition image by overlapping the first and second images such that the transition image includes overlapping portions of each image. For example, the first image received by the object reconstruction module may include a first region, a second region, a third region, and a fourth region of the object, and the second image received by the object reconstruction module may include a fifth region of the object. The object reconstruction module may create the transition image by overlapping the first and second images such that the transition image includes the second region, the third region, the fourth region, and the fifth region of the object. In certain embodiments, the object reconstruction module receives the transition image from the image capture module. Next, method 600 moves from step 630 to step 640.

[0051] At step 640 of method 600, the object reconstruction module performs inference on each image. For example, the object reconstruction module may perform inference on the first image, the second image, and the transition image by running an object detection algorithm to detect objects in the first image, the second image, and the transition image. In response to performing inference and detecting objects, the object reconstruction module may generate one or more bounding boxes around the detected objects (or portions thereof). Each bounding box may include one or more regions of the object. Method 600 then moves from step 640 to step 650.

[0052] At step 650 of method 600, the object reconstruction module determines equivalent regions in the received images (e.g., equivalent regions 178a in FIG. 4). Equivalent regions are regions that share the same pixels. In particular embodiments, the object reconstruction module performs an intersection of the bounding boxes and overlap regions of the first image, the second image, and the transition image to normalize bounding boxes that may extend beyond the overlap region before calculating the IoU of two or more boxes. In some embodiments, the object reconstruction module determines the overlap region using IoU, detection label equivalence, and / or a confidence score threshold. For example, the object reconstruction module may use IoU, detection label equivalence, and / or a confidence score threshold to determine that the second region, the third region, and the fourth region overlap in the first image and the transition image. As another example, the object reconstruction The module may use IoU, equality of the detected labels, and / or a confidence score threshold to determine that the fifth region overlaps in the second image and the transition image. Next, method 600 moves from step 650 to step 660.

[0053] At step 660 of method 600, the object reconstruction module excludes equivalent regions from the reconstruction of the object. For example, if the object reconstruction module determines that the second, third, and fourth regions from the first image and the transition image are equivalent regions, the object reconstruction module may exclude the equivalent regions in the object reconstruction such that the object reconstruction includes only one copy of the second, third, and fourth regions. As another example, if the object reconstruction module determines that the fifth region from the second image and the transition image are equivalent regions, the object reconstruction module may exclude the equivalent region in the object reconstruction such that the object reconstruction includes only one copy of the fifth region. Method 600 then moves from step 660 to step 670.

[0054] At step 670 of method 600, the object reconstruction module generates a reconstruction of the object (e.g., object reconstruction 180a of FIG. 5) from the one or more received images and one or more transition images. For example, a first image received by the object reconstruction module may include a first region, a second region, a third region, and a fourth region of the object, and the transition image generated at step 630 of method 600 may include a second region, a third region, a fourth region, and a fifth region of the object. The object reconstruction module may generate the reconstruction of the object by concatenating the first region, the second region, the third region, and the fourth region from the first image with the fifth region of the transition image such that the reconstruction of the object represents the entire object. Method 600 then moves from step 670 to step 680, where method 600 ends. In this manner, method 600 may be used to globally reconstruct an object even if a specific region of the object in the received image is not detected.

[0055] Modifications, additions, or omissions may be made to the method 600 shown in FIG. 4 . Method 600 may include more, fewer, or other steps. For example, method 600 may include utilizing a second transition image. Steps may be performed in parallel or in any suitable order. Although discussed as specific components completing steps of method 600, any suitable component may perform any step of method 600. For example, one or more steps of method 600 may be performed by an image capture module.

[0056] 7 illustrates an exemplary computer system that may be used by the systems and methods described herein. For example, one or more components of system 100 of FIG. 1 (e.g., image capture module 150 and / or object reconstruction module 170) may include one or more interfaces 710, processing circuits 720, memory 730, and / or other suitable elements. Interface 710 receives input, sends output, processes the input and / or output, and / or performs other suitable operations. Interface 710 may include hardware and / or software.

[0057] Processing circuitry 720 performs or manages the processing of the components. Processing circuitry 720 may include hardware and / or software. Examples of processing circuitry include one or more computers, one or more microprocessors, one or more applications, etc. In particular embodiments, processing circuitry 720 executes logic (e.g., instructions) to perform actions (e.g., processing), such as generating output from input. The logic executed by processing circuitry 720 may be encoded in one or more tangible, non-transitory computer-readable media (e.g., memory 730). For example, the logic may be stored in a computer program, software, a computer readable medium, or a memory device. The computer program may include computer-executable instructions and / or instructions that are executable by a computer. In certain embodiments, the processes of the embodiments may be performed by one or more computer-readable media having computer programs stored, embodied, and / or encoded thereon and / or having computer programs stored and / or encoded thereon.

[0058] The memory 730 (or memory unit) stores information. The memory 730 may include one or more non-transitory, tangible, computer-readable, and / or computer-executable storage media. Examples of the memory 730 include computer memory (e.g., RAM or ROM), mass storage media (e.g., hard disks), removable storage media (e.g., compact discs (CDs) or digital video discs (DVDs)), databases and / or network storage (e.g., servers), and / or other computer-readable media.

[0059] As used herein, a computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other integrated circuits (ICs) (e.g., field programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy disks, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM drives, Secure Digital (SD) cards or drives, or other suitable computer-readable non-transitory storage media, or any suitable combination of two or more thereof, as appropriate. A computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, as appropriate.

[0060] As used herein, "or" is inclusive and not exclusive, unless expressly indicated otherwise or dictated otherwise by context. Thus, as used herein, "A or B" means "A, B, or both," unless expressly indicated otherwise or dictated otherwise by context. Additionally, "and" means jointly and plurally, unless expressly indicated otherwise or dictated otherwise by context. Thus, as used herein, "A and B" means "A and B, jointly or severally," unless expressly indicated otherwise or dictated otherwise by context. The scope of the present disclosure includes all changes, substitutions, variations, modifications, and alterations to the exemplary embodiments described or illustrated herein that can be understood by those skilled in the art. The scope of the present disclosure is not limited to the exemplary embodiments described or illustrated herein. Furthermore, although this disclosure describes and illustrates each embodiment herein as including particular components, elements, features, functions, processes, or steps, any of these embodiments may include any combination or permutation of the components, elements, features, functions, processes, or steps described or illustrated anywhere herein that would be understood by one of ordinary skill in the art. Furthermore, references in the appended claims to an apparatus or system, or components thereof, that are suitably configured and arranged, and operative or operable to perform a particular function, include that system or component of the apparatus, regardless of whether that apparatus, system, or component is suitably configured and arranged, and operative or operable, or whether the particular function is activated or deactivated. Furthermore, although this disclosure describes or illustrates particular embodiments as providing certain advantages, a particular embodiment may provide none, some, or all of those advantages.

Claims

1. 1. An apparatus comprising: one or more processors; one or more computer-readable non-transitory storage media coupled to the one or more processors, the one or more computer-readable non-transitory storage media containing instructions that, when executed by the one or more processors, cause the device to perform a process; Including, The process comprises: a process for identifying a transition image via a processor, the transition image comprising: a first region of an object in a first image of the transportation infrastructure component; and a second region of the object within a second image of the transportation infrastructure component; and generating a reconstruction of the object using the transition image, the reconstruction of the object including a first region of the object and a second region of the object; identifying a first portion of the object in the first image; constructing a first bounding box around a first portion of the object in the first image, the first bounding box including a first region of the object; identifying a second portion of the object in the transition image; constructing a second bounding box around a second portion of the object in the transitional image, the second bounding box including a first region of the object and a second region of the object in the transitional image; Including, Device.

2. the first image further includes a third region of the object, a fourth region of the object, and a fifth region of the object; The apparatus of claim 1 , wherein the transition image includes a third region of the object and a fourth region of the object.

3. The process comprises: determining that a third region of the object in the transition image and a third region of the object in the first image are third equivalent regions; determining a fourth region of the object in the transition image and a fourth region of the object in the first image as a fourth equivalent region; Including, The apparatus of claim 2 , wherein the reconstruction of the object excludes the third and fourth equivalent regions.

4. 3. The apparatus of claim 2, wherein the processing further comprises concatenating the first region of the object, the second region of the object, the third region of the object, the fourth region of the object, and the fifth region of the object to generate a reconstruction of the object, wherein the reconstruction of the object represents the entire object.

5. 2. The apparatus of claim 1, wherein the processing further comprises using Intersection over Union (IoU) to determine that a first region of the object in the transition image and a first region of the object in the first image are the first equivalent region.

6. The apparatus of claim 1 , wherein the one or more processors include an object reconstruction module.

7. The apparatus of claim 1 , wherein the transition image comprises half of the first image and half of the second image.

8. 1. A method comprising: Identifying, via a processor, a transition image, the transition image comprising: a first region of an object in a first image of the transportation infrastructure component; and a second region of the object within a second image of the transportation infrastructure component; and generating a reconstruction of the object using the transition image, the reconstruction of the object including a first region of the object and a second region of the object; identifying a first portion of the object in the first image; constructing a first bounding box around a first portion of the object in the first image, the first bounding box including a first region of the object; identifying a second portion of the object in the transition image; constructing a second bounding box around a second portion of the object in the transitional image, the second bounding box including a first region of the object and a second region of the object in the transitional image; Including, method.

9. The first image is further comprising a third region of the object, a fourth region of the object, and a fifth region of the object; The transition image is a third region of the object, and a fourth region of the object; The method of claim 8.

10. determining, by the object reconstruction module, a third region of the object in the transition image and a third region of the object in the first image as a third equivalent region; determining, by the object reconstruction module, a fourth region of the object in the transition image and a fourth region of the object in the first image as a fourth equivalent region; further comprising the reconstruction of the object excludes the third and fourth equivalent regions; 10. The method of claim 9.

11. The method comprises: further comprising concatenating, by the object reconstruction module, the first region of the object, the second region of the object, the third region of the object, the fourth region of the object, and the fifth region of the object to generate a reconstruction of the object; the reconstruction of the object shows the object in its entirety; 10. The method of claim 9.

12. 9. The method of claim 8, further comprising using Intersection over Union (IoU) to determine that a first region of the object in the transition image and a first region of the object in the first image are the first equivalent region.

13. The method of claim 8 , wherein the processor includes an object reconstruction module.

14. The method of claim 8 , wherein the transition image comprises half of the first image and half of the second image.

15. one or more computer-readable non-transitory storage media embodying instructions that, when executed by a processor, cause the processor to perform a process; The process comprises: a process for identifying a transition image via a processor, the transition image comprising: a first region of an object in a first image of the transportation infrastructure component; and a second region of the object within a second image of the transportation infrastructure component; and generating a reconstruction of the object using the transition image, the reconstruction of the object including a first region of the object and a second region of the object; identifying a first portion of the object in the first image; constructing a first bounding box around a first portion of the object in the first image, the first bounding box including a first region of the object; identifying a second portion of the object in the transition image; constructing a second bounding box around a second portion of the object in the transitional image, the second bounding box including the first region of the object and the second region of the object in the transitional image; Including, A computer-readable non-transitory storage medium.

16. the first image further includes a third region of the object, a fourth region of the object, and a fifth region of the object; the transition image includes a third region of the object and a fourth region of the object.

16. The computer-readable non-transitory storage medium of claim 15.

17. The process comprises: determining, by the object reconstruction module, a third region of the object in the transition image and a third region of the object in the first image as a third equivalent region; determining, by the object reconstruction module, a fourth region of the object in the transition image and a fourth region of the object in the first image as a fourth equivalent region; further comprising the reconstruction of the object excludes the third and fourth equivalent regions; 17. The computer-readable non-transitory storage medium of claim 16.

18. The process comprises: further comprising concatenating, by the object reconstruction module, the first region of the object, the second region of the object, the third region of the object, the fourth region of the object, and the fifth region of the object to generate a reconstruction of the object; the reconstruction of the object shows the entire object; 17. The computer-readable non-transitory storage medium of claim 16.

19. The process comprises:

16. The computer-readable non-transitory storage medium of claim 15, further comprising using Intersection over Union (IoU) to determine that a first region of the object in the transition image and a first region of the object in the first image are the first equivalent region.

20. The computer-readable non-transitory storage medium of claim 15 , wherein the one or more processors include an object reconstruction module.

Images