A train real-time positioning method based on artificial intelligence video features

Abstract

The application discloses a train real-time positioning method based on artificial intelligence video features, which does not need to use an inertial navigation device, and only relies on collected images to enable the train to acquire accurate position information, avoids common cumulative drift errors of the inertial navigation device, and solves problems such as complex device use and maintenance; the application can infer the position of trains in high-speed, medium-speed and low-speed motion, and is a very suitable method for a future track transportation driving system (including a driver or an automatic driving vehicle).

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a real-time train positioning method based on artificial intelligence video features. Background Technology

[0002] During train operation, train positioning results affect the safety of each train, and accurate positioning can assist in train scheduling and operation control. Currently, in my country's high-speed rail operation, for accurate positioning, trackside equipment such as track circuits and transponders are commonly used, or integrated train positioning technology combining BeiDou satellite navigation is employed. A typical solution uses INS / GNSS (Inertial Navigation System / Global Navigation Satellite System) combined navigation. However, current methods suffer from problems such as signal blind spots, cumulative drift errors, and complex equipment use and maintenance.

[0003] In view of this, the present invention is hereby proposed. Summary of the Invention

[0004] The purpose of this invention is to provide a real-time train positioning method based on artificial intelligence video features, which can locate the train's running position with meter-level accuracy.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A real-time train localization method based on artificial intelligence video features includes:

[0007] The collected query images are input into the train location network to obtain the image representation corresponding to the query image. A similarity search based on the image representation is then performed in the image database to retrieve several images. Each image in the image database has corresponding location information.

[0008] All obtained images are sorted according to their similarity to the image representation of the query image;

[0009] The location information of the query image is determined by using the location information in the sorted images.

[0010] As can be seen from the technical solution provided by the present invention, it is not necessary to use inertial navigation equipment. The train can obtain accurate position information simply by relying on the collected images, avoiding the problems of cumulative drift error and complex use and maintenance of equipment that are common with inertial navigation. The present invention can infer the position of trains moving at high, medium and low speeds, and is a very suitable method for future rail transit driving systems (including driverless or autonomous vehicles). Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating a real-time train positioning method based on artificial intelligence video features, provided as an embodiment of the present invention;

[0013] Figure 2 A schematic diagram of the training phase provided in an embodiment of the present invention;

[0014] Figure 3 A schematic diagram of the reasoning stage provided in an embodiment of the present invention;

[0015] Figure 4 This is a schematic diagram illustrating the train position calculation concept provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0017] First, the following explanations are provided for the terms that may be used in this article:

[0018] The terms “including,” “comprising,” “containing,” “having,” or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, “including a technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction conditions, processing conditions, parameter, algorithm, signal, data, product or article of manufacture, etc.)” should be interpreted as including not only the expressly listed technical feature element, but also other technical feature elements that are not expressly listed and are well-known in the art.

[0019] The term "composed of" excludes any technical features not expressly listed. When used in a claim, it closes the claim to exclude all technical features other than those expressly listed, except for associated conventional impurities. If the term appears only in a clause of a claim, it limits the claim to the elements expressly listed in that clause; elements recited in other clauses are not excluded from the overall claim.

[0020] The following is a detailed description of a real-time train positioning method based on artificial intelligence video features provided by this invention. Contents not described in detail in the embodiments of this invention are prior art known to those skilled in the art. Where specific conditions are not specified in the embodiments of this invention, they are performed according to conventional conditions in the art or conditions recommended by the manufacturer. Where the manufacturers of the instruments used in the embodiments of this invention are not specified, they are all conventional products that can be purchased commercially.

[0021] This invention provides a real-time train positioning method based on artificial intelligence video features. Essentially, it's a deep learning method. First, a camera captures roadside images during the train's journey. Then, a deep network (train position network) extracts features from each frame and associates them with the image's BeiDou or GPS location. After training, the deep network can infer the closest image from the image database and its location. When the input is video (continuous images) and the inference is continuous, it can continuously provide the train's position, thus achieving train positioning. This invention differs from most robot SLAM (Simultaneous Localization and Mapping) methods. It does not use inertial navigation equipment, relying solely on video images for positioning. It also differs from pure video methods in SLAM by not calculating and updating the state frame by frame, nor does it require prior knowledge of the positional differences between pairs of input images. The deep network learns using features from three frames (the three images mentioned later). After learning, it retrieves the most similar images from the image database based on the image representation. Then, it refines the position of these images to obtain the global position.

[0022] like Figure 1 The diagram shows a flowchart of a real-time train positioning method based on artificial intelligence video features provided by an embodiment of the present invention, which mainly includes the following steps:

[0023] Step 1: Input the collected query image into the train location network to obtain the image representation corresponding to the query image, and perform a similarity search based on the image representation in the image database to retrieve several images from the image database.

[0024] In this embodiment of the invention, each image in the image database has corresponding location information.

[0025] In this embodiment of the invention, the query image is captured by a camera that travels alongside the train; when the camera captures video, each video frame in the video is used as the query image to achieve real-time positioning during train operation.

[0026] In this embodiment of the invention, the step of performing a similarity search based on image representation in the image database to search for a number of images from the image database includes: calculating the similarity between the image representation of the query image and the image representation of each image in the image database; if the similarity between the image representation of a certain image A in the image database and the image representation of the query image exceeds a threshold, then image A is selected as the searched image, and finally, a number of images are searched from the image database.

[0027] The image database is built using images from different sections of the railway line. Based on the region corresponding to the query image, the appropriate image database is selected for a similarity search. This effectively shortens the retrieval time, thus providing real-time location information even for trains traveling at higher speeds.

[0028] Step 2: Sort all the obtained images according to their similarity to the image representation of the query image.

[0029] In this embodiment of the invention, the corresponding images are sorted in descending order according to their similarity, that is, images with higher similarity to the query image are sorted first, and vice versa.

[0030] Step 3: Use the positional information in the sorted images to determine the positional information of the query image.

[0031] In this embodiment of the invention, the position information of the query image can be obtained by extracting the position of the sorted images. Specifically, any of the following methods can be used: (1) The position information of the first sorted image (i.e. the image with the highest similarity to the image representation of the query image) can be used as the position information of the query image; (2) The position information of the query image can be extracted by using the position information of the first sorted images, including: combining the position information of the first sorted images, after filtering out abnormal points, extracting the position information of the query image by calculating the mean, interpolation or center point.

[0032] Those skilled in the art will understand that the mean, interpolation, and center point can all be understood as calculation methods that deduce or estimate other data points from known data points, and can all be calculated in a conventional way, so they will not be elaborated on. In practical applications, users can choose the appropriate calculation method to extract the location information of the query image based on the actual situation or experience.

[0033] The above-mentioned solutions provided by the embodiments of the present invention mainly achieve the following beneficial effects:

[0034] (1) The train positioning method provided by the present invention is different from the SLAM method. It does not rely on inertial navigation equipment, but only on video equipment, and the purchase, installation and maintenance costs are very low.

[0035] (2) This invention does not require the large and complex state space update and optimization of SLAM. It can obtain the current position by searching and filtering after obtaining the image representation by the deep network. There will be no situation where the trajectory is interrupted due to discontinuous state and the system needs to be repositioned and initialized. Therefore, the trajectory tracking of this invention is more continuous.

[0036] (3) The input data of this invention is a single image, not a pair of images. It is not necessary to know the position difference of the input pair of images in advance. The reasoning result is not a relative position, but a direct global position.

[0037] (4) The positioning of this invention requires image retrieval. The larger the content of the image database, the longer the retrieval time. Considering that the train routes are fixed, this invention adopts the method of building a database between station areas and searching the image database between stations separately. Compared with the continuous database construction from the starting station to the terminal station and global search, it can effectively shorten the query time, thus providing real-time location positioning for trains with higher speeds.

[0038] (5) The present invention can still effectively determine the current location in the event of certain abnormal situations, such as interference or interruption of Beidou GPS signal, or special geographical environment. It is not affected by electromagnetic interference and can be directly verified and compared by naked eye without relying on additional verification equipment. It is a very applicable positioning method for future rail transit systems to work effectively in any environment.

[0039] To more clearly demonstrate the technical solution and its effects provided by the present invention, the method provided by the embodiments of the present invention will be described in detail below with reference to specific examples.

[0040] I. Introduction to the Solution Process

[0041] 1. Data source for the train location network.

[0042] In this embodiment of the invention, the train location network is a deep network, divided into a learning and training phase and an inference phase. The purpose of training is to enable the network to learn the correspondence between images and locations, so that the trained network can be used to infer the location of the currently queried image during the inference phase.

[0043] Before training, data needs to be prepared and preprocessed. The image data used during training is obtained from videos stored by cameras that follow the train. The locations are obtained from devices that are time-synchronized with the cameras. Each video frame has corresponding location information. For example, the video contains 25 frames per second, and the time is synchronized once every second. Each frame corresponds to a spatial location coordinate every 40 milliseconds. The video is converted into images, each image is named with time information (timestamp), and the image is bound to the corresponding location information.

[0044] 2. Training phase.

[0045] like Figure 2 As shown, during training, a sequence of three images is input, and the image representation corresponding to each image is obtained through the train position network. The three-image loss is calculated, and the parameters of the train position network are optimized using the three-image loss. Here, the three images refer to the given current image, the image closest to the current image, and the image least close to the current image. The closeness and distance are defined by the similarity of the features extracted from the images.

[0046] In this embodiment of the invention, the process of obtaining an image representation through a train position network is described as follows: After the image is input into the train position network, the feature map is extracted by the backbone network of the train position network. The feature aggregation module of the train position network then outputs a feature descriptor, i.e., the image representation. The backbone network is a feature extraction network, which can be implemented using conventional networks, such as VGG (Convolutional Neural Network) or ResNet (Residual Network). Feature extraction can employ algorithms such as SIFT (Scale Invariant Feature Transform), and the feature aggregation module can employ aggregation algorithms such as NetVLAD.

[0047] In this embodiment of the invention, the specific implementation process of optimizing the parameters of the train position network using three-graph loss can be implemented with reference to conventional techniques, and therefore will not be described in detail.

[0048] 3. Reasoning stage.

[0049] In this embodiment of the invention, the inference stage utilizes the trained train position network to obtain image representations, which are then used for subsequent image search and localization. For example... Figure 3 As shown, the query image is input into the train location network to obtain the corresponding image representation. This process is the same as the training phase, i.e., feature maps are first extracted through the backbone network, and then feature descriptors are output through the feature aggregation module. Similarly, each image in the image database also obtains its corresponding image representation through the train location network (this process can be completed offline). The image representation mentioned here mainly refers to the feature descriptors obtained after extracting feature maps through the train location network. Afterwards, several images are retrieved from the image database through similarity search, and then the location information of the query image is determined by location extraction from the retrieved images. For details, please refer to the descriptions of steps 1 to 3 above.

[0050] II. Explanation of the Principles and Conclusions of the Scheme

[0051] 1. The concept of location calculation.

[0052] When a train or a moving object is in motion, the scene seen from the outside will show that two consecutive frames are necessarily close in position. According to the technical concept of robot SLAM, after the initial position is obtained, each subsequent frame must be compared with the previous frame to extract feature points and calculate feature state associations. As long as a large number of feature points are not lost between the later and previous frames, the calculation can continue frame by frame. The result is a relative position. (See [link to documentation]). Figure 4 Part (a) of this invention utilizes the principle that "two very similar images must be close in location." The image comparison uses historical images taken at close locations, without needing to retain the state of frame-by-frame calculations. The position of each image is obtained by comparing it with the most similar image in a historical database. See [link to relevant documentation]. Figure 4 Part (b).

[0053] The most similar image result obtained from the retrieval (i.e. the image obtained in step 1 above) can be 1, 5, 10, 20, etc., because the image features are very similar and the positions are also very similar. By extracting the position, the position of the first image can be directly selected, or the positions of several images above can be combined and filtered out by outliers, and then the mean, interpolation or center point can be used to obtain the positioning result.

[0054] 2. Positioning accuracy.

[0055] As discussed above, the image query retrieves several images from the database whose features are closest. The image location information in the image database is obtained through spatial coordinate conversion. The coordinate accuracy of the images in the database is obtained synchronously with BeiDou or GPS during image acquisition. Therefore, the accuracy of BeiDou or GPS during acquisition determines the accuracy of location positioning, and civilian-grade BeiDou and GPS can achieve accuracy down to 10 meters. Furthermore, this invention can further optimize positioning accuracy by employing data correction methods during location extraction. The data correction method described here is the one introduced earlier, which involves first filtering out outliers and then using the mean, interpolation, or center point to obtain the positioning result.

[0056] 3. Real-time location tracking.

[0057] The location positioning of this invention is obtained by searching an image database. The retrieval time of this image database is clearly the most important factor affecting real-time performance. While using a high-performance CPU or GPU can certainly improve real-time performance under normal circumstances, this invention considers that when a train travels across a large area, the operation time can be tens of hours, and 25 frames of high-definition image data per second need to be stored in the database. Performing a global search on such a massive database is extremely difficult even with data center-level servers, and it is not cost-effective for moving objects such as trains. Therefore, this invention proposes a station-by-station database segmentation approach for retrieval. The positioning of each segment is only searched in the segmented inter-station database, and the retrieval can be performed in parallel to further reduce search latency.

[0058] 4. The relationship between location and train speed.

[0059] In summary, the location positioning of this invention is obtained by searching an image database. The query image must share common feature points with images in the database for positioning to be successful. Assuming extreme cases, when the train travels at 600 km / h (166.7 meters per second) and 400 km / h (111.2 meters per second), and assuming a one-second difference between the query image and an image in the database, the distance difference is over 100 meters. Dividing this by the camera frame rate (25 frames per second), the difference is approximately 6.69 meters and 4.45 meters respectively. Because the images are more likely to differ as the distance increases, ensuring a sufficient number of common feature points between the images becomes difficult. Fortunately, there are two ways to solve this problem: one is to use a high-speed camera, increasing the frame rate from 25 to 60 frames per second, thus making the distance difference between images much smaller and acceptable; the other is to collect images multiple times. Since each train run is different, repeated collections gradually reduce the physical distance between the images in the database at the time of collection.

[0060] Through the above description of the embodiments, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the above embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.), including several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0061] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A train real-time positioning method based on artificial intelligence video features, characterized in that, include: The collected query images are input into the train position network to obtain the image representation corresponding to the query image. A similarity search based on the image representation is then performed in the image database to retrieve several images. Each image in the image database has corresponding location information. The train position network is a deep network and requires pre-training. During training, a sequence of three images is input, and the train position network obtains the image representation corresponding to each image and calculates the three-image loss. The parameters of the train position network are then optimized using the three-image loss. The three images refer to the given current image, the image closest to the current image, and the image least close to the current image. All obtained images are sorted according to their similarity to the image representation of the query image; The location information of the query image is determined by using the location information in the sorted images.

2. The train real-time positioning method based on artificial intelligence video features according to claim 1, characterized in that, The query image is captured by a camera that travels alongside the train; when the camera captures video, each video frame is used as the query image to achieve real-time positioning during train operation. 3.The train real-time positioning method based on artificial intelligence video features according to claim 1, characterized in that, The image data used during training was obtained from videos stored by cameras that travel alongside the train. The location was obtained from a device that is time-synchronized with the camera. Each video frame has corresponding location information. The video was converted into images, each image was named with time information, and the images were bound to the corresponding location information. 4.The train real-time positioning method based on artificial intelligence video features according to claim 1, characterized in that, The step of performing a similarity search based on image representation in the image database, and retrieving a number of images from the image database, includes: Calculate the similarity between the image representation of the query image and the image representation of each image in the image database; If the similarity between the image representation of image A in the image database and the image representation of the query image exceeds a threshold, then image A is selected as the searched image. Finally, several images are retrieved from the image database.

5. The real-time train positioning method based on artificial intelligence video features according to claim 1, characterized in that, The image database is a database built using images of railway line sections. Based on the region corresponding to the query image, the corresponding image database is selected for similarity search. 6.The train real-time positioning method based on artificial intelligence video features according to claim 1, characterized in that, The step of determining the location information of the query image using the location information in the sorted images includes: extracting the location from the sorted images to obtain the location information of the query image.

7. The train real-time positioning method based on artificial intelligence video features according to claim 1, characterized in that, The location information of the query image is obtained by extracting the location from the sorted images, including: Use the location information of the first sorted image as the location information of the query image; Alternatively, the location information of the query image can be extracted from the locations of the top-ranked images. This includes: combining the location information from the top-ranked images, filtering out outliers, and then extracting the location information of the query image by calculating the mean, interpolation, or center point.

Images