Traffic location measurement system and method using LIDAR

Abstract

Traffic location systems and methods using LIDAR are provided. In some embodiments, a computer-implemented method includes receiving a 3D image captured from a vehicle on a path; transforming the 3D image into a first 2D image; and determining the location of the vehicle along the path, which includes: comparing the first 2D image with a plurality of second 2D images, each of which was captured at a corresponding known location along the path, selecting one or more of the second 2D images based on the comparison, and determining the location of the vehicle along the path based on the known location at which the selected one or more of the second 2D images were captured. The 3D image may be captured by capturing LIDAR data using a LIDAR unit mounted on the vehicle.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 924,017, filed on October 21, 2019, entitled “LIDAR RTLS PROJECT,” and U.S. Provisional Patent Application No. 62 / 945,785, filed on December 9, 2019, entitled “TRANSIT LOCATION SYSTEMS AND METHODS USING LIDAR,” the disclosures of which are incorporated herein by reference in their entireties. Technical Field

[0003] The present disclosure relates generally to transportation intelligence and, more particularly, to systems and methods for identifying transportation locations. Background Art

[0004] In conventional methods, the captured 3D images are compared to reference 3D images previously captured at known locations along the track. However, the computational effort required to quickly compare 3D images is so intensive that the power required by a computer to perform the comparison exceeds the power available on a standard subway train. Summary of the invention

[0005] Embodiments of the present disclosure provide systems and methods for tracking information related to transportation vehicles. For example, some embodiments of the present disclosure provide a system for enhancing location and data collection from a vehicle traveling along a vehicle path. For example, the vehicle can be a light rail train, a commuter train, a freight train, a car, an airplane, a ship, a space vehicle, a ski lift, a cable car, or other forms of transportation known in the art. The transportation path can be any path along which the corresponding vehicle moves, such as a train track, a road, a canal or a waterway, a runway, an airway, or other vehicle paths known in the art.

[0006] The method may include capturing 3D images from a vehicle on a path that the vehicle is traveling. The vehicle may employ light detection and ranging (LIDAR) technology to generate a 3D image of the path. In one embodiment, a LIDAR unit is mounted on a subway train to capture images of a subway tunnel.

[0007] A computer may be used to process the collected LIDAR data to determine the position of the vehicle. The computer may be located on the vehicle. For each 3D image, the computer receives the LIDAR data and transforms the 3D image into a 2D image. The 2D image may be compared to a reference 2D image previously captured at a known position along the path. Based on the comparison, the computer may select one or more reference 2D images that are most similar to the captured 2D image. The position of the vehicle on the path may be determined based on the position at which the selected reference 2D image was captured.

[0008] In some embodiments, the 2D image is a color image in a color space. For example, the color space may be an HSV color space. However, other color spaces may be used. In these embodiments, transforming the 3D image into a 2D image may include converting the captured data of each point in the 3D image into color data in the color space of the corresponding point in the 2D image. In the case of LIDAR data, the captured data of each point includes a distance value and a reflectivity value, and converting the captured data of the point may include mapping the distance value of the point to the value of a first component of the color space, and mapping the reflectivity value of the point to the value of a second component of the color space. In the example of the HSV color space, the distance value may be mapped to an H value, and the reflectivity value may be mapped to a V value.

[0009] Some embodiments include eliminating false positive matches between captured 2D images in reference 2D images. In these embodiments, one or more of the matching 2D images are deselected as false positive matches before determining the position of the vehicle along the path.

[0010] In some embodiments, comparing the captured 2D image to the reference 2D image employs the use of key points and descriptors. In these embodiments, a plurality of key points are extracted from each of the captured 2D image and the reference 2D image, and corresponding descriptors are generated for each of the 2D images based on the key points.

[0011] These and other objects, features, and characteristics of the systems and / or methods disclosed herein, as well as methods of operation and function of the related elements of the structures and combinations of components and manufacturing economies, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures. However, it should be expressly understood that the drawings are for illustration and description purposes only and are not intended as a definition of limitations of the present invention. As used in the specification and claims, the singular forms of "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The technology disclosed herein according to one or more various embodiments is described in detail with reference to the following drawings. The drawings are provided for illustrative purposes only and depict only typical or example embodiments of the disclosed technology. These drawings are provided to facilitate the reader's understanding of the disclosed technology and should not be considered as limitations on its breadth, scope, or applicability. It should be noted that for clarity and ease of description, these figures are not necessarily drawn to scale.

[0013] Figure 1 A location system for a train on a track according to an embodiment of the disclosed technology is illustrated.

[0014] Figure 2 A process for surveying a train track according to an embodiment of the disclosed technology is illustrated.

[0015] Figure 3 A process for determining the position of a vehicle on a path is illustrated in accordance with an embodiment of the disclosed technology.

[0016] Figure 4 Illustration of a 3D LIDAR image captured by a subway train inside a subway tunnel.

[0017] Figure 5 An example 2D image generated according to an embodiment of the disclosed technology is shown.

[0018] Figure 6 Shown is an example outdoor image that has been processed using the SIFT algorithm.

[0019] Figure 7 A pair of similar images are shown, showing matching keypoints.

[0020] Figure 8 A graph illustrating the matching quality of several similar images to the captured image is shown.

[0021] Fig. 9 Shows that after applying the transformation matrix Figure 8 of charts.

[0022] Fig.10 Shows the ordered Figure 8 A chart of matching images.

[0023] Fig.11 An example computing system is illustrated that can be used to implement various features of embodiments of the disclosed technology.

[0024] The drawings are not intended to be exhaustive or to limit the invention to the precise forms disclosed. It should be understood that the invention can be practiced with modification and alteration, and that the disclosed technology is limited only by the claims and their equivalents. DETAILED DESCRIPTION

[0025] Embodiments of the present disclosure provide systems and methods for positioning a vehicle along a path using LIDAR technology. For clarity and brevity, in the following description, the vehicle and path are described as a train and a track. However, the disclosed technology can be used to position any vehicle along any path. In addition, the disclosed technology is not limited to LIDAR and can be used with any technology that captures 3D data in the form of distance and reflectivity.

[0026] According to the disclosed embodiments, a LIDAR unit mounted on a train captures 3D images and uses the 3D images to determine the train's position along its track. In conventional methods, the captured 3D images are compared to reference 3D images previously captured at known locations along the track. However, the computational effort required to quickly compare 3D images is so large that the power required by a computer to perform the comparison exceeds the power available on a standard subway train. The disclosed embodiments take a different approach.

[0027] In the disclosed embodiment, 3D images are converted to 2D images, and the 2D images are compared. The power required to compare the 2D images is well within the capabilities of a standard subway train. Therefore, the computer that performs the techniques described herein can be located on the train.

[0028] Embodiments of the disclosed technology may have additional advantages. In particular, the disclosed positioning system does not require wayside equipment. That is, all equipment required to determine the position of a train may be located on or within the train. Since wayside equipment involves significant installation and maintenance costs, this feature greatly reduces the cost of operating a positioning system.

[0029] Figure 1 A positioning system 100 according to an embodiment of the disclosed technology is shown for use with a train 102 on a track 104. However, as described above, the technology may also be used to position other vehicles on other paths.

[0030] refer to Figure 1 , the train 102 may be located on the track 104. The train may be moving or stationary. For example, the positioning system 100 may have been used to determine the position of the stationary train 102 when the train 102 is first powered on. As another example, the positioning system 100 may be used to determine the position of the train 102 while the train 102 is moving along the track 104.

[0031] The positioning system 100 may include a LIDAR sensor 106. The LIDAR sensor 106 may be installed at any location outside the train 102, such as the front or rear of the train 102. The LIDAR sensor 106 may collect 3D images, for example, according to conventional techniques. For example, the LIDAR sensor 106 may include a rotating sensor with multiple beams arranged at multiple angles. In other examples, the LIDAR sensor 106 may include a fixed sensor with a rotating mirror to form multiple beams. However, the LIDAR sensor 106 may be implemented using any LIDAR technology.

[0032] The positioning system 100 may include a computer 108. In some embodiments, the computer 108 may be implemented as a special purpose computer, for example optimized to perform the calculations described herein, and / or ruggedized to withstand the operating environment of the train 102. In other embodiments, the computer 108 may be implemented as a general purpose computer.

[0033] The positioning system 100 may include a descriptor / position database 110. The database 110 may include descriptors generated from LIDAR images previously captured at known locations along the track 104. For example, in a mapping mode, the positioning system 100 may be employed to generate the database 110. In the mapping mode, the system 100 may include an orientation sensor (not shown) to provide the location of each 3D image captured by the LIDAR sensor 106. The system 100 transforms each 3D image into a 2D image, generates a descriptor for the 2D image, and stores the descriptor and the location in the database 110. Then, after the database 110 is populated, the positioning system 100 may operate in the positioning mode to determine the location of the train 102 based on the images captured by the LIDAR sensor 106 and the descriptors and locations stored in the database 110, for example, as described in detail below.

[0034] The location generated by the positioning system 100 may be used in any manner. For example, the positioning system 100 may include an onboard display 112 to display the location of the operator of the train 102. The display 112 may show an interactive map of the track system, as well as an indicator of the current location of the train 102. As another example, the positioning system may include an onboard transmitter 114 to transmit the location to a system external to the train 102. For example, the location may be transmitted to a central control system for use in controlling multiple trains traveling on the track 104.

[0035] Figure 2 A process 200 for mapping a train track 104 in accordance with an embodiment of the disclosed technology is illustrated. Although elements of the process 200 are described in a particular order, it should be understood that in various embodiments, steps may be omitted, performed in other orders, performed simultaneously, etc.

[0036] During the survey, a train 102 travels along a track 104. The train may include a LIDAR sensor 106, such as Figure 1 As shown. At 202, as the train 102 travels along the track 104, the LIDAR sensor 106 can collect 3D images and positions. For example, the train 102 can travel along the track 104 at a constant speed rate while the LIDAR sensor 106 collects 3D images at regular intervals. The train 102 can include a dedicated positioning system to determine its position along the track 104. Any positioning system can be used for this purpose. The positioning system can determine the position of the train 102 along the track 104 for each 3D image collected.

[0037] At 204, the computer 108 may transform each 3D image into a 2D image, for example, as described in detail below. At 206, the computer 108 may generate a descriptor for each 2D image, also as described in detail below. At 208, the computer 108 may store the descriptors and the locations in the database 110. For example, the computer 108 may store the descriptors and the locations as a table, where each descriptor is associated with its location. Once the mapping is complete, the database 110 may be used to determine the location of the train 102 along the track 104.

[0038] Figure 3 A process 300 for determining the position of a vehicle on a path is illustrated in accordance with an embodiment of the disclosed technology. Although elements of process 300 are described in a particular order, it should be understood that in various embodiments, steps may be omitted, performed in other orders, performed simultaneously, etc.

[0039] For example, process 300 may be used to determine the position of a train 102 traveling along track 104, such as Figure 1 The process 300 may be implemented according to Figure 2 The example generated database 110. Figure 3 , process 300 may include capturing a 3D image from a vehicle on the path at 302. Figure 1 In the example of FIG. 1 , the LIDAR sensor 106 can capture 3D images from the train 102. The 3D images can include the track 104, the surroundings of the train 102, etc. For example, for a ground train 102, the surroundings can include trees, buildings, signs, etc. For a subway train 102, the surroundings can include tunnel walls, subway infrastructure, subway platforms, etc.

[0040] Figure 4 Figure 3 shows a 3D LIDAR image captured by a subway train inside a subway tunnel. Figure 4In the figure, the white square represents the vertical plane passing through the LIDAR sensor and parallel to the front of the train. The data collected by the reflection of each LIDAR beam can be viewed as Figure 4 Individual curves in .

[0041] Reference again Figure 3 , process 300 may include transforming the 3D image into a 2D image at 304. In some embodiments, the 2D image is a color image in a color space. In other embodiments, other 2D images may be used.

[0042] Each point in the 3D LIDAR image can contain 4 pieces of data. The data can include an angle θ around the vertical axis of the LIDAR sensor. In some embodiments, the angle θ can be expressed in a range of 0 to 360 with a resolution of 0.2°. The data can include the number of beams b. In some embodiments, the LIDAR sensor includes 32 beams, and the number of beams b can be expressed as an integer value in the range of 0 to 31. The data can include the distance δ from the unit. In some embodiments, the distance can be expressed as a 16-bit number with an increment of 1 cm. A distance δ=0 can indicate that no reflection of the beam was detected. This can occur when the reflective surface is out of range of the sensor, the reflective surface completely absorbs the LIDAR beam, the reflective surface completely reflects the LIDAR beam away from the LIDAR sensor, etc. The data can include the returned signal strength R, also known as "reflectivity". The reflectivity R can be expressed as an 8-bit number in arbitrary units.

[0043] In some embodiments, each LIDAR image is generated using 2 LIDAR scans. The first scan is used to determine the minimum non-zero distance δ min and the maximum non-zero distance δ max , and the minimum non-zero reflectivity R min and the maximum non-zero reflectivity R max In the second scan, each distance and reflectivity is normalized within the minimum and maximum values ​​determined in the first scan and clamped to the range of 0-1 (inclusive), for example according to equations (1) and (2).

[0044] δ norm = clamp((δ-δ min ) / (δ max -δ min ),0,1) (1)

[0045] R norm =clamp((RR min ) / (R max -R min ), 0, 1) (2)

[0046] This method represents a form of linear dynamic ranging. In some cases, employing nonlinear dynamic ranging can also be helpful. For example, this technique can be used to enhance differential distances within a certain range, exaggerating features that are helpful for image matching purposes.

[0047] Next, the normalized distance and reflectivity can be converted to colors in a color space. For example, one of the most common color spaces is RGB with three 8-bit color channels, each with a value range of 0-255. Due to the popularity of this color space, there are many image processing software libraries, especially including many feature extraction algorithms that can be used in embodiments of the disclosed technology. In LIDAR data, it can be said that distance is far more important than reflectivity. But in the RGB color space, distance is mapped to multiple color components. This mapping may represent similar distances as very different colors, resulting in abrupt discontinuities in the resulting image.

[0048] Another common color space is HSV. One advantage of using the HSV color space is that distance and reflectivity can be mapped to separate color space components. In some embodiments, the normalized distance and normalized reflectivity can be converted to component values ​​of the HSV color space, for example according to equations (3) to (5).

[0049] H=360xδ norm (3)

[0050] S=1 (4)

[0051] V=R norm (5)

[0052] In some embodiments, the HSV color space data may be processed directly to obtain key points and descriptors. In other embodiments, the HSV color space data may be converted to RGB color space data prior to the process, for example to utilize an existing library of algorithms for feature recognition in the RGB color space. In some of these embodiments, the low V values ​​may be enhanced according to equation (6) prior to conversion from the HSV color space to the RGB color space.

[0053] V=R norm 0.2 (6)

[0054] Using this technique, very close objects appear red, and as distance increases, appear orange, then yellow, then green, then blue, purple, and then back to red, proportional to distance. The brightness of each pixel indicates reflectivity, with brighter pixels indicating higher reflectivity than darker pixels. Sudden changes in distance appear as sudden changes in hue, while smooth changes in distance appear as smooth changes in hue. Objects with significantly less reflectivity than their nearby surroundings retain their hue but appear darker.

[0055] As described above, for certain locations in the image, data may not be collected, resulting in undesirable single-pixel "holes." In some embodiments, a filter may be applied to those pixels to "despeckle" the image. For example, a simple median filter may be applied to the pixels.

[0056] Figure 5 An example 2D image generated according to an embodiment of the disclosed technology is shown. The example image is 32 pixels high, with one scan line for each LIDAR beam. The example image is 595 pixels wide, representing a horizontal range of approximately 119 degrees. The image represents a section of a subway tunnel that curves to the left. The dark spots near the center of the image show that the tunnel recedes into the distance. Yellow spots near the dark spots represent roadside structures such as signals, cables, and even debris. Looking closely, two subway train tracks can be seen, with the train on the left track. It can also be seen that the walls of the tunnel are curved and that there is a raised sidewalk on the right side of the tunnel. The image includes horizontal lines, which are artifacts that can be corrected by better calibrating the LIDAR sensor.

[0057] Reference again Figure 3 , process 300 may include comparing the 2D image to a plurality of reference 2D images, each reference 2D image being captured at a respective known position along the path, at 306. According to the technique, the position of the reference image(s) most similar to the captured image indicates the position of the vehicle along the path.

[0058] In the disclosed embodiments, rather than comparing 2D images directly, extracted features of the 2D images are compared. In some embodiments, multiple key points are extracted from each 2D image. A descriptor is then generated for each 2D image based on the extracted key points. The descriptor may include the key points and may also include additional information about the image surrounding each key point. For example, the information may describe the gradient of the grayscale value in eight different directions around each key point. By comparing their descriptors, the captured 2D image may be compared to a reference 2D image.

[0059] There are several popular algorithms for extracting keypoints and descriptors. The open source software package OpenCV includes several such algorithms. Example feature extraction algorithms include the Scale Invariant Feature Transform (SIFT), the Speeded Up Robust Features (SURF) algorithm, the Oriented Fast and Rotated Brief (ORB) algorithm, and the KAZE, AKAZE, and BRISK algorithms. These algorithms have different behaviors and focus on different image features. For example, BRISK is a corner detector, while SIFT, SURF, and KAZE are blob detectors. Most of these algorithms have parameters that can be adjusted to "fine tune" these behaviors.

[0060] These algorithms also have different execution times. Table 1 lists the execution time for processing 9021 outdoor images for several of these algorithms, including the total execution time and the average time per image.

[0061] algorithm Total time (seconds) Average time per image (seconds) BRISK 60.64187288284302 0.0067223005 SIFT 54.89843988418579 0.00608562685 SURF 17.822772979736328 0.00197569814 KAZE 140.67809510231018 0.01559451226

[0062] Table 1

[0063] Figure 6 shows an example outdoor image that has been processed using the SIFT algorithm. Figure 6 In the figure, the bright circles indicate the key points extracted by the SIFT algorithm.

[0064] Reference again Figure 3 , process 300 may include selecting one or more reference 2D images based on the comparison at 308. For example, the comparison and matching may be performed using one or more of the algorithms described above. These algorithms may generate a value indicating the quality of the match.

[0065] In some cases, the captured image can be well matched with multiple pre-processed images. In this case, the multiple matched images can be used to determine the position of the train. For example, if the first two matches are adjacent in position, one matches 75% and the other matches 50%, then the position of the train is between the positions of these images and may be closer to the position of the previous image. Techniques such as weighting functions, interpolation, etc. can be used to determine the position of the train based on these multiple images. As another example, the orientation of the train can be calculated by generating a transformation matrix from the matched key points.

[0066] Figure 7 A pair of similar images showing matching keypoints are shown. Note that one image is shown directly above the other and that matching keypoints are shown in the same color and connected with lines of that color. Figure 7 The matches in are generated using the SIFT algorithm with a Brute Force K Nearest Neighbor (BFKNN) matcher.

[0067] Figure 8A graph illustrating the quality of matches of several similar images to the captured image is shown. The quality of each match is indicated by a distance in pixels, with fewer pixels indicating a better match. The captured image has image ID = 8360, and therefore has a distance of zero pixels, as Figure 8 As shown. The image ID indicates the order in which the images were taken. Therefore, the nearest neighbors to the captured images are images 8359 and 8361. Figure 8 It can be seen that while these are good matches, they are not optimal matches. Figure 8 In this case, the best match is actually a false positive.

[0068] Reference again Figure 3 , process 300 may include deselecting one of the selected reference 2D images as a false positive match before determining the position of the vehicle at 310. As described in detail below, many techniques may be used alone or in combination to eliminate false positive matches. These and other techniques may be used alone or in combination to reduce the set of selected reference 2D images.

[0069] Process 300 may include determining the position of the vehicle along the path based on the known position(s) at which the selected reference 2D image(s) were captured at 312. For example, if only one 2D reference image is selected to match the captured image, the position of that 2D image may be used as the position of the vehicle. If multiple reference 2D images are selected to match the captured image, the positions of the selected reference 2D images may be used to determine the position of the vehicle, such as using interpolation or other similar techniques (e.g., as described herein).

[0070] One technique for eliminating false positives is to use a transformation matrix, as described in detail below. Fig. 9 Shows the result after applying the transformation matrix Figure 8 . Fig. 9 As can be seen, the only matches left are the two closest to the captured image, namely 8359 and 8361. All false positive matches have been eliminated. Fig. 9 As can be seen, image 8361 matches much better than image 8359.

[0071] Another technique for eliminating false positives is to arrange the reference images in order and find the global minimum of the matching indices. Fig.10 Shows the ordered Figure 8 A chart of the matching images. Fig.10 In the , dark gray bars indicate matching images, and light gray bars indicate no match at all. Fig.10As shown, there is a global minimum near the captured image 8360. Another minimum exists at 8350, but can be discarded because the nearby images do not match well or not at all. Fig.10 Another feature that can be seen in is that all images within the four images of captured image 8360 show some degree of matching. This feature can be used to eliminate false positives, for example by deselecting images outside a window of ±n matching images at the global minimum.

[0072] Another technique for eliminating false positive matches is to simply deselect matches in locations where trains are unlikely to be. This technique is aided by the fact that vehicles such as trains have highly constrained behavior. They are limited in speed and acceleration, cannot spontaneously switch tracks, cannot make instantaneous U-turns, etc. These techniques can be used alone or in combination to eliminate multiple false positive matches.

[0073] As mentioned above, a transformation matrix can be used to eliminate false positive matches. The technique is now described in detail. The points that match between the two images can be used to generate a transformation matrix that mathematically encodes the distortion of one image relative to the other. For example, a large picture of a bird can be matched to an overall image of similar size (but with a smaller image of the bird embedded in it). The resulting transformation matrix will reflect the scaling of the bird.

[0074] But in the current example, since the captured image and the reference image are taken from the same position on the train and should be very similar for the same position on the track, the transformation matrix should be very close to the identity matrix. In practice, it is unlikely that the transformation matrix is ​​exactly the identity matrix because the train is unlikely to be in exactly the same position and attitude for both images. That is, the train may sway from side to side, bounce slightly, etc.

[0075] Open source libraries such as OpenCV provide algorithms for generating these transformation matrices. Quantification and thresholding of the difference between the transformation matrix and the identity matrix can be used to eliminate erroneous images. One such technique is to use the SSD (square sum of distances) between the transformation matrix and the identity matrix. Table 2 shows the results of extracting SSD from a sample of 1000 images for two different types of transformation matrices (i.e., homography matrix and essential matrix).

[0076] method Homography matrix Basic Matrix Avg.Adj 1.51 1.27 Not matched 17 0 Error Percentage 9.76％ 5.10％ Time (seconds) 46.89 54.16

[0077] Table 2

[0078] In Table 2, Avg.Adj represents the average SSD of the image ID from the ground truth (i.e., the correct image). For the homography matrix example, the closest match is about 1.51 images away from the ground truth on average. For the basic matrix example, the closest match is about 1.27 images away from the ground truth on average.

[0079] "Unmatched" is the number of images received that were not matched. For the homography example, out of the 1000 sample images, 17 did not have a closest match. For the fundamental matrix example, every sample image had a match.

[0080] The "Error Percent" is the number of images where the closest match is ±2 images away from the ground truth. For the homography example, 9.76% of the 1000 sample images are ±2 images away from the ground truth. For the fundamental matrix example, 5.10% of the sample images are ±2 images away from the ground truth.

[0081] Some of the data in Table 2 can be better understood with an understanding of the difference between the fundamental matrix and the homography matrix. The homography matrix requires at least 4 key points to calculate, and these key points must be coplanar with each other. For example, the four points of a postcard exist on the same plane and they are coplanar no matter how the paper is oriented. Therefore, the homography matrix method can easily identify postcards in a continuous array of photos, even if the camera of each photo is different.

[0082] The fundamental matrix is ​​a more generalized form of the homography matrix. While the homography matrix relates coplanar image space points, the fundamental matrix relates any set of points in an image to points in another image taken by the same camera. Continuing with the postcard example, the fundamental matrix approach can easily identify which photos are identical (with respect to the orientation of the postcard and its surroundings) as long as the camera used is the same.

[0083] For these reasons, the basic matrix method is more suitable for use with the disclosed technology. The results in Table 2 confirm this conclusion. Although it takes 7.27ms more time per image (because Table 2 describes the time to process 1000 images), the basic matrix method has a lower Avg.Adj number, no missed matches, and a 4.66% lower error percentage.

[0084] If the mapping images were captured at a high enough density, images near the correct location should closely match the test image, but less so for images further away. If a strong match is found to an image, but neighboring images do not match well or at all, the quality of the match may be questionable. As described above, these situations can be resolved using a transformation matrix.

[0085] In addition to eliminating false positive matches, it is also desirable to prevent false negative matches. Techniques for preventing false negative matches include collecting alternative images and removing outdated images. There are several possible reasons why an image may not match well or at all. These can include the presence of another train, temporary construction, new debris, weather conditions, seasonal appearance of trees, newly constructed or now non-existent buildings, etc.

[0086] To address these situations, the described technology can save images along with information that can help determine the image's location. This information can include a best guess location based on the last time the system calculated a valid location, the next time the system calculated a valid location, and the time and speed between those times and the time the image was acquired. Such images can then be processed offline and added as additional possible candidates for their location. If there are multiple images for a location, the system can collect information about which images performed best. If an image does not perform well over a set period of time, it can be eliminated as obsolete. Multiple images can be maintained for a location.

[0087] The system may employ several additional techniques, either alone or in combination, to improve the results of the system. One such technique employs an additional positioning system that only needs to provide coarse, low-quality position information. These approximate positions can be used to eliminate all but a small subset of reference images for comparison to the captured image, thereby greatly improving the performance of the system.

[0088] Another technique uses knowledge of the approximate location of the train to select appropriate algorithms and / or parameters to match the images. The approximate location of the train may be determined based on the last known position of the train, by the additional positioning system described above, etc. For example, certain algorithms and / or parameters may be more effective in a tunnel than outdoors. These algorithms and / or parameters may be selected when it is known that the train is in a tunnel. These determinations may be made using offline processing, such as by simulated annealing of parameters to determine the best settings for each algorithm for each track area.

[0089] In some cases, the approximate location of the train may be unknown, such as when the train is first powered on after a catastrophic power outage. To address these situations, multiple descriptors may be stored for each image, including, for example, a first descriptor that is good for providing a highly accurate location and a second descriptor that does not provide as high an accuracy but is good at providing an approximate location. At power on, the system may employ the second descriptor to determine the approximate location of the train. The system may then use the approximate location to select algorithms and / or parameters, which may then be used with the first descriptor to obtain the accurate location of the train.

[0090] Another technique for preventing false positive matches involves masking out portions of the image. In practice, some portions of an image may generate keypoints that are not useful for matching. For example, railroad ties may generate many keypoints. But since railroad ties appear the same in almost every image, these keypoints may not be useful for image matching. Therefore, masking out the railroad ties in the image before generating keypoints may improve the performance of the system.

[0091] The positions generated by the disclosed technology have many uses. For example, the positions of vehicles on a path can be used to manage the schedule of these vehicles. These positions can be used to warn drivers of conditions on the path or even the presence of workers on the path. Many other applications are envisioned.

[0092] Appendix A shows the results of processing a set of images taken inside a tunnel.

[0093] Appendix B shows the results of processing a set of images taken outdoors.

[0094] As will be appreciated, the methods described herein may be performed using a computing system having machine-executable instructions stored on a tangible medium. The instructions may be executable autonomously or with the assistance of input from an operator to perform each portion of the method.

[0095] Those skilled in the art will appreciate that the disclosed embodiments described herein are intended only as examples and that there will be many variations. The present invention is limited only by the claims, which cover the embodiments described herein as well as variations that are obvious to those skilled in the art. In addition, it should be understood that the structural features or method steps shown or described in any embodiment herein may also be used in other embodiments.

[0096] As used herein, the terms logic circuit and component can describe a given functional unit that can be performed according to one or more embodiments of the technology disclosed herein. As used herein, any form of hardware, software or combination thereof can be used to implement a logic circuit or component. For example, one or more processors, controllers, ASICs, PLAs, PALs, CPLDs, FPGAs, logic components, software routines or other mechanisms can be implemented to form components. In an embodiment, the various components described herein can be implemented as discrete components, or the functions and features described can be partially or completely shared between one or more components. In other words, as will be apparent to those of ordinary skill in the art after reading this specification, the various features and functions described herein can be implemented in any given application, and can be implemented in one or more separate or shared components in various combinations and arrangements. Even if the various features or elements of a function can be described or required as separate components separately, those of ordinary skill in the art will understand that these features and functions can be shared between one or more common software and hardware elements, and such a description should not require or imply the use of separate hardware or software components to implement such features or functions.

[0097] Where components, logic circuits or components of the technology are implemented in whole or in part using software, in one embodiment, these software elements may be implemented to operate with a computing or logic circuit capable of performing the functions described with respect thereto. Various embodiments are described in terms of this example logic circuit 1100. After reading this description, it will be clear to a person skilled in the relevant art how to implement the technology using other logic circuits or architectures.

[0098] Reference now Fig.11 , computing system 1100 may represent computing or processing capabilities such as found in desktop, laptop, and notebook computers; handheld computing devices (PDAs, smart phones, cell phones, PDAs, etc.); mainframes, supercomputers, workstations, or servers; or any other type of special-purpose or general-purpose computing device as may be desirable or appropriate for a given application or environment. Logic circuitry 1100 may also represent computing capabilities embedded within or otherwise available to a given device. For example, logic circuitry may be found in other electronic devices such as digital cameras, navigation systems, cellular phones, portable computing devices, modems, routers, WAPs, terminals, and other electronic devices that may include some form of processing capabilities.

[0099] The computing system 1100 may include, for example, one or more processors, controllers, control components, or other processing devices, such as a processor 1104. The processor 1104 may be implemented using a general or special processing component, such as a microprocessor, controller, or other control logic. In the example shown, the processor 1104 is connected to a bus 1102, although any communication medium may be used to facilitate interaction with other components of the logic circuit 1100 or to communicate with the outside.

[0100] The computing system 1100 may also include one or more memory components, referred to herein as main memory 1108. For example, preferably, a random access memory (RAM) or other dynamic memory may be used to store information and instructions to be executed by the processor 1104. The main memory 1108 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processor 1104. The logic circuit 1100 may also include a read-only memory ("ROM") or other static storage device coupled to the bus 1102 for storing static information and instructions for the processor 1104.

[0101] The computing system 1100 may also include one or more information storage mechanisms 1110 in various forms, which may include, for example, a media drive 1112 and a storage unit interface 1120. The media drive 1112 may include a drive or other mechanism that supports fixed or removable storage media 1114. For example, a hard drive, a floppy disk drive, a tape drive, an optical drive, a CD or DVD drive (R or RW), or other removable or fixed media drive may be provided. Thus, the storage media 1114 may include, for example, a hard disk, a floppy disk, a tape, a box, an optical disk, a CD or DVD, or other fixed or removable media that is read, written, or accessed by the media drive 1112. As shown in these examples, the storage media 1114 may include a computer-usable storage medium having computer software or data stored therein.

[0102] In alternative embodiments, the information storage mechanism 1110 may include other similar means for allowing computer programs or other instructions or data to be loaded into the logic circuit 1100. Such means may include, for example, a fixed or removable storage unit 1122 and an interface 1120. Examples of such storage units 1122 and interfaces 1120 may include program cartridges and cartridge interfaces, removable memory (e.g., flash memory or other removable memory components) and memory slots, PCMCIA slots and cards, and other fixed or removable storage units 1122 and interfaces 1120 that allow software and data to be transferred from the storage unit 1122 to the logic circuit 1100.

[0103] The logic circuit 1100 may also include a communication interface 1124. The communication interface 1124 may be used to allow software and data to be transferred between the logic circuit 1100 and external devices. Examples of the communication interface 1124 may include a modem or soft modem, a network interface (such as Ethernet, a network interface card, WiMedia, IEEE 802.XX or other interface), a communication port (e.g., a USB port, an IR port, an RS232 port, etc.), a 32-bit RF interface, a 4 ... The software and data transmitted via the communication interface 1124 may generally be carried on a signal, which may be electronic, electromagnetic (including optical), or other signal capable of being exchanged by a given communication interface 1124. These signals may be provided to the communication interface 1124 via a channel 1128. The channel 1128 may carry signals and may be implemented using a wired or wireless communication medium. Some examples of a channel may include a telephone line, a cellular link, an RF link, an optical link, a network interface, a local or wide area network, and other wired or wireless communication channels.

[0104] In this document, the terms "computer program medium" and "computer-usable medium" are generally used to refer to media such as memory 1108, storage unit 1120, medium 1114, and channel 1128. These and other various forms of computer program media or computer-usable media may involve transmitting one or more sequences of one or more instructions to a processing device for execution. Such instructions embodied on the medium are generally referred to as "computer program code" or "computer program product" (which may be grouped in the form of a computer program or other grouping). When executed, such instructions may enable logic circuit 1100 to perform features or functions of the disclosed technology discussed herein.

[0105] although Fig.11 A computer network is depicted, but it should be understood that the present disclosure is not limited to operation using a computer network, but rather the present disclosure may be practiced in any suitable electronic device. Fig.11 The computer network depicted in FIG. 5 is for illustration purposes only and is therefore not meant to limit the present disclosure in any respect.

[0106] Although various embodiments of the disclosed technology have been described above, it should be understood that they are presented only as examples rather than in a restrictive manner. Similarly, various figures can depict example architectures or other configurations of the disclosed technology, which is done to help understand the features and functions that can be included in the disclosed technology. The disclosed technology is not limited to the illustrated example architectures or configurations, but various alternative architectures and configurations can be used to implement the desired features. In fact, it will be apparent to those skilled in the art how alternative functions, logical or physical partitions and configurations can be implemented to implement the desired features of the technology disclosed herein. In addition, many different component names can be applied to various partitions in addition to those described herein.

[0107] Furthermore, with respect to flow diagrams, operational descriptions, and method claims, the order in which the steps are presented herein shall not obligate the various embodiments to perform the recited functions in the same order unless the context dictates otherwise.

[0108] Although the disclosed technology is described above according to various exemplary embodiments and implementations, it should be understood that the various features, aspects, and functions described in one or more individual embodiments are not limited to their applicability to the specific embodiments used to describe them, but can be applied alone or in various combinations to one or more of the other embodiments of the disclosed technology, whether or not such embodiments are described and whether or not such features are presented as part of the described embodiments. Therefore, the breadth and scope of the technology disclosed herein should not be limited by any of the above-described exemplary embodiments.

[0109] Unless expressly stated otherwise, the terms and phrases used in this document, and variations thereof, should be interpreted as open-ended and not limiting. As examples of the foregoing: the term "including" should be understood to mean "including but not limited to," etc.; the term "example" is used to provide an illustrative example of the items discussed, rather than an exhaustive or limiting list thereof; the terms "a" or "an" should be understood to mean "at least one," "one or more," etc.; and adjectives such as "conventional," "traditional," "normal," "standard," "known," and terms of similar meaning should not be interpreted as limiting the items described to a given time period or items available at a given time, but should be understood to cover conventional, traditional, normal, or standard technologies that may be available or known now or at any time in the future. Similarly, where this document refers to technology that is obvious or known to a person of ordinary skill in the art, such technology covers technology that is obvious or known to a person of ordinary skill in the art at any time now or in the future.

[0110] In some cases, the presence of broadening words and phrases (such as "one or more," "at least," "but not limited to," or other similar phrases) should not be interpreted as meaning that the narrower case is intended or required in the absence of such broadening words. Use of the term "component" does not imply that the components or functions described or claimed as part of the component are all configured in a common package. In fact, any or all of the various components of the component, whether control logic or other components, can be combined in a single package or maintained separately, and can be further distributed in multiple groups or packages or across multiple locations.

[0111] In addition, the various embodiments set forth herein are described in terms of exemplary block diagrams, flow charts, and other illustrations. As will become apparent to one of ordinary skill in the art after reading this document, the illustrated embodiments and their various alternatives may be implemented without limitation to the illustrated examples. For example, the block diagrams and their accompanying descriptions should not be construed as mandating a particular architecture or configuration.

Claims

1. A system for identifying a traffic location, comprising: Hardware processor; as well as A non-transitory machine-readable storage medium encoded with instructions executable by the hardware processor to perform a method comprising: receiving a 3D image comprising LIDAR data, the 3D image being captured from a vehicle on a path; and transforming the 3D image into the first 2D image by converting the LIDAR data of a point in the 3D image into color data in a color space of a corresponding point in the first 2D image, wherein the LIDAR data of the point includes a distance value and a reflectivity value, wherein converting the LIDAR data of the point includes: Mapping the distance value of the point to a value of a first component of the color space; and mapping the reflectance value of the point to a value of a second component of the color space; and Determining the position of the vehicle along the path comprises: comparing the first 2D image to a plurality of second 2D images, each of the plurality of second 2D images being captured at a respective known position along the path, wherein each of the second 2D images comprises a plurality of points, each of the plurality of points having color data in the color space, selecting one or more of the second 2D images based on the comparison, and The position of the vehicle along the path is determined based on the known locations at which the selected one or more of the second 2D images were captured.

2. The system according to claim 1, the method further comprising: The 3D image is captured.

3. The system of claim 2, wherein capturing the 3D image comprises: The LIDAR data is captured using a LIDAR unit mounted on the vehicle.

4. The system of claim 1, wherein the method further comprises: Prior to determining the position of the vehicle, one or more of the selected second 2D images are deselected as false positive matches.

5. The system of claim 1 , wherein comparing the first 2D image to the plurality of second 2D images comprises: extracting a plurality of key points from each of the first 2D image and the second 2D image; generating a corresponding descriptor for each of the first 2D image and the second 2D image based on the corresponding key points; as well as The descriptor of the first 2D image is compared with the descriptor of each of the second 2D images.

6. The system of claim 1, wherein the vehicle on the path is a train on a track.

7. A non-transitory machine-readable storage medium encoded with instructions executable by a hardware processor of a computing component, the machine-readable storage medium comprising instructions for causing the hardware processor to perform a method comprising: receiving a 3D image comprising LIDAR data, the 3D image captured from a vehicle on a path; as well as transforming the 3D image into the first 2D image by converting the LIDAR data of a point in the 3D image into color data in a color space of a corresponding point in the first 2D image, wherein the LIDAR data of the point includes a distance value and a reflectivity value, wherein converting the LIDAR data of the point includes: Mapping the distance value of the point to a value of a first component of the color space; and mapping the reflectivity values ​​of the points to values ​​of a second component of the color space; and determining the position of the vehicle along the path, comprising: comparing the first 2D image to a plurality of second 2D images, each of the plurality of second 2D images being captured at a respective known position along the path, wherein each of the second 2D images comprises a plurality of points, each of the plurality of points having color data in the color space, selecting one or more of the second 2D images based on the comparison, and The position of the vehicle along the path is determined based on the known locations at which the selected one or more of the second 2D images were captured.

8. The non-transitory machine-readable storage medium of claim 7, the method further comprising: The 3D image is captured.

9. The non-transitory machine-readable storage medium of claim 8, wherein capturing the 3D image comprises: The LIDAR data is captured using a LIDAR unit mounted on the vehicle.

10. The non-transitory machine-readable storage medium of claim 7, the method further comprising: Prior to determining the position of the vehicle, one or more of the selected second 2D images are deselected as false positive matches.

11. The non-transitory machine-readable storage medium of claim 7, wherein comparing the first 2D image to the plurality of second 2D images comprises: extracting a plurality of key points from each of the first 2D image and the second 2D image; generating a corresponding descriptor for each of the first 2D image and the second 2D image based on the corresponding key points; as well as The descriptor of the first 2D image is compared with the descriptor of each of the second 2D images.

12. The non-transitory machine-readable storage medium of claim 7, wherein the vehicle on the path is a train on a track.

13. A computer-implemented method comprising: receiving a 3D image comprising LIDAR data, the 3D image captured from a vehicle on a path; as well as transforming the 3D image into the first 2D image by converting the LIDAR data of a point in the 3D image into color data in a color space of a corresponding point in the first 2D image, wherein the LIDAR data of the point includes a distance value and a reflectivity value, wherein converting the LIDAR data of the point includes: Mapping the distance value of the point to a value of a first component of the color space; and mapping the reflectivity values ​​of the points to values ​​of a second component of the color space; and determining the position of the vehicle along the path, comprising: comparing the first 2D image to a plurality of second 2D images, each of the plurality of second 2D images being captured at a respective known position along the path, wherein each of the second 2D images comprises a plurality of points, each of the plurality of points having color data in the color space, selecting one or more of the second 2D images based on the comparison, and The position of the vehicle along the path is determined based on the known locations at which the selected one or more of the second 2D images were captured.

14. The computer-implemented method of claim 13, further comprising: The 3D image is captured.

15. The computer-implemented method of claim 14, wherein capturing the 3D image comprises: The LIDAR data is captured using a LIDAR unit mounted on the vehicle.

16. The computer-implemented method of claim 13, further comprising: Prior to determining the position of the vehicle, one or more of the selected second 2D images are deselected as false positive matches.

17. The computer-implemented method of claim 13, wherein comparing the first 2D image to the plurality of second 2D images comprises: extracting a plurality of key points from each of the first 2D image and the second 2D image; generating a corresponding descriptor for each of the first 2D image and the second 2D image based on the corresponding key points; as well as The descriptor of the first 2D image is compared with the descriptor of each of the second 2D images.

18. The computer-implemented method of claim 13, wherein the vehicle on the path is a train on a track.

Images