Train track line pairing detection method

Abstract

The embodiment of the invention discloses a train track line pairing detection method, and relates to the technical field of intelligent driving. According to the scheme, the method comprises the steps that features are extracted through a convolutional neural network to obtain a feature map, track features in the feature map are extracted through paired anchors, track lines are generated through the track features, the track line detection efficiency is improved, and when the track lines are shielded, the track lines are detected through the position relation between left tracks and right tracks in paired tracks. And the shielded track line can be accurately determined.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular, to a method for paired detection of train track lines. Background Art

[0002] With the continuous development of intelligent driving technology, convenience and safety have been greatly improved. Among them, intelligent trains use advanced sensors and computer vision algorithms to assist or replace train drivers in perceiving and analyzing the train operation environment, effectively reducing the risk of accidents caused by human factors. As a part of intelligent driving, track line detection can provide strong guarantee for the safe operation of trains.

[0003] In related technologies, the detection method using deep learning is becoming increasingly mature. The detection method based on the segmentation deep neural network makes pixel-by-pixel predictions on the entire acquired image, resulting in low detection efficiency. And when the track line is blocked, accurate detection results cannot be obtained. Summary of the Invention

[0004] Embodiments of this specification provide a method for paired detection of train track lines to solve the problems of low detection efficiency and inability to obtain accurate detection results when the track line is blocked in the prior art.

[0005] To solve the above technical problems, the embodiments of this specification are implemented as follows: In a first aspect, a method for paired detection of train track lines provided by an embodiment of this specification includes: Obtain a track line image to be detected; Use a convolutional neural network to extract features from the track line image to obtain a feature map containing track features; Use multiple pairs of paired anchors to respectively extract paired track features in the feature map to obtain multiple pairs of paired track feature vectors; the paired anchors are set according to the positional relationship between the left track and the right track in the paired tracks; Input all the paired track feature vectors into a track line detection model to obtain the position information of multiple pairs of paired track lines; Generate target paired track lines from the position information of a pair of the paired track lines that meet the preset conditions.

[0006] An embodiment of this specification achieves the following beneficial effects: extracting features using a convolutional neural network to obtain a feature map, extracting track features in the feature map using paired anchors, and generating track lines through the track features, improving the detection efficiency of track lines. And when the track line is blocked, the positional relationship between the left track and the right track in the paired tracks can be used to accurately determine the blocked track line. Description of the Drawings

[0007] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0008] Figure 1 It is a schematic flowchart of a method for paired detection of train track lines provided by an embodiment of this specification; Figure 2 It is a schematic diagram of obtaining high-level and low-level feature maps provided by an embodiment of this specification; Figure 3 It is a schematic diagram of an application scenario of a method for paired detection of train track lines provided by an embodiment of this specification; Figure 4 It is a schematic diagram of another application scenario of a method for paired detection of train track lines provided by an embodiment of this specification; Figure 5 It is a schematic structural diagram of a device for paired detection of train track lines provided by an embodiment of this specification; Figure 6 It is a schematic structural diagram of a device for paired detection of train track lines provided by an embodiment of this specification. Specific embodiments

[0009] To make the objectives, technical solutions, and advantages of one or more embodiments of this specification clearer, the following will clearly and completely describe the technical solutions of one or more embodiments of this specification in conjunction with the specific embodiments and corresponding drawings of this specification. Obviously, the described embodiments are only some embodiments of this specification, rather than all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope protected by one or more embodiments of this specification.

[0010] The following will detail the technical solutions provided by each embodiment of this specification in conjunction with the drawings.

[0011] Specifically explain a method for paired detection of train track lines provided by an embodiment of the specification in conjunction with the drawings.

[0012] Figure 1 It is a schematic flowchart of a method for paired detection of train track lines provided by an embodiment of this specification. From a program perspective, the execution subject of the process can be a program or an application client running on an application server. On the other hand, from a hardware perspective, the execution subject of the process can be a terminal device or a detection platform, etc. This embodiment does not make special limitations in this regard.

[0013] As Figure 1 shown, the process may include the following steps: Step 110: Obtain the track line image to be detected.

[0014] In the embodiments of this specification, an image containing a track line collected by a camera or other image acquisition device in front of the train is obtained.

[0015] Step 120: Use a convolutional neural network to extract features from the track line image to obtain a feature map containing track features.

[0016] In the embodiments of this specification, a pre-trained backbone convolutional neural network such as ResNet18 is used as the feature extraction network. The track line image is input into the backbone convolutional neural network, and the network will output a series of feature maps, which contain feature information at different levels in the image, from low-level edge and texture features to high-level shape and semantic features.

[0017] In practical applications, the track line image can be pre-processed, such as resizing the image, normalizing, etc., to meet the input requirements of the convolutional neural network.

[0018] Step 130: Use multiple pairs of paired anchors to respectively extract paired track features in the feature map to obtain multiple pairs of paired track feature vectors; the paired anchors are set according to the positional relationship between the left track and the right track in the paired tracks.

[0019] In the embodiments of this specification, an anchor is an auxiliary ray used when extracting features and calculating the lane position. The paired anchors include the starting points and angle information of the left and right anchors. Each pair of paired anchors contains the positional relationship between the two tracks in the paired tracks. The setting of the paired anchors can capture different forms and positional changes of the paired track lines. The paired anchors are set according to the positional relationship between the left track and the right track in the paired tracks. Using the positional relationship between the left track and the right track in the paired tracks, an occluded track line can be predicted based on an unoccluded track line, and the track line can also be accurately determined when the track line is occluded. The track features may include the edges, textures, shapes, etc. of the track line.

[0020] By respectively using each pair of paired anchors to extract the paired track features in the feature map, paired track feature vectors equal in number to the paired anchors can be obtained, and the track feature vectors can comprehensively reflect the feature information of the track line.

[0021] Step 140: Input all the paired track feature vectors into the track line detection model to obtain the position information of multiple pairs of paired track lines.

[0022] In the embodiments of this specification, multiple pairs of paired orbital feature vectors are input into a trained orbital line detection model, and the orbital line detection model will perform inference based on the input feature vectors to calculate the position information of the orbital line. The position information may include the starting point, ending point, direction, sampling point coordinates, etc. of the orbital line.

[0023] In practical applications, a pair of paired orbital feature vectors can correspond to obtaining the position information of a pair of paired orbital lines. For example, when 112 pairs of paired orbital feature vectors are input into the orbital line detection model, 112 pairs of paired orbital line position information can be obtained.

[0024] Step 150: Generate a target paired orbital line from the position information of a pair of the paired orbital lines that meet the preset conditions.

[0025] In the embodiments of this specification, the position information of each pair of orbital lines is matched with the preset conditions, which usually involves calculating and comparing the parameters of the orbital lines to determine which orbital lines meet the screening conditions. According to the matching results, the position information of a pair of orbital lines that meet the preset conditions is screened out, and the screened position information of the paired orbital lines is integrated to form a complete target paired orbital line.

[0026] It should be understood that the order of some steps in the methods described in one or more embodiments of this specification can be interchanged according to actual needs, or some of the steps can also be omitted or deleted.

[0027] In the embodiments of this specification, a convolutional neural network is used to extract features to obtain a feature map, paired anchors are used to extract orbital features in the feature map, and orbital lines are generated through the orbital features, which improves the detection efficiency of the orbital lines. Moreover, when the orbital lines are occluded, the position relationship between the left orbital line and the right orbital line in the paired orbitals can be used to accurately determine the occluded orbital line.

[0028] Based on Figure 1 the method in, the embodiments of this specification also provide some specific implementation schemes of this method, which will be described below.

[0029] Optionally, in the embodiments of this specification, the feature map includes a high-level feature map and a low-level feature map. The use of multiple pairs of paired anchors to extract paired orbital features in the feature map to obtain multiple pairs of paired orbital feature vectors may specifically include: Using multiple pairs of paired anchors to extract paired orbital features in the high-level feature map to obtain multiple pairs of first paired orbital feature vectors; Performing convolutional processing on each pair of first paired orbital feature vectors to obtain multiple pairs of first paired orbital local feature vectors; Inputting all the first paired orbital feature vectors into the orbital line detection model to obtain the position information of multiple pairs of paired orbital lines to be corrected; According to the position information of all the pairs of orbital lines to be corrected, extract pairs of orbital features in the low-level feature map respectively to obtain multiple pairs of second paired orbital feature vectors; Perform convolution processing on each pair of the second paired orbital feature vectors to obtain multiple pairs of second paired orbital local feature vectors.

[0030] In the embodiments of this specification, the feature map may include a high-level feature map and a low-level feature map. ResNet18 is used as the feature extraction network to obtain the top-down feature maps 、feature map 、feature map , whose sizes are (7, 7, 512), (14, 14, 256), (28, 28, 128) respectively. Comparatively speaking, the feature map has the highest resolution and the lowest-level semantic features, such as orbital edges and textures, while has the lowest resolution and the highest-level semantic features, such as the overall shape of the orbit and more abstract expressions.

[0031] Figure 2 is a schematic diagram for obtaining the high-level feature map and the low-level feature map provided by the embodiments of this specification.

[0032] As Figure 2 shown, after performing 1x1 convolution on , the obtained size is (7, 7, 64) of ; after performing 1x1 convolution on and adding it to the upsampled , the obtained size is (14, 14, 64) of ; after performing 1x1 convolution on and adding it to the upsampled , the obtained size is (28, 28, 64) of . After performing 1x1 convolution on , the high-level feature map is obtained. After performing 1x1 convolution on , the low-level feature map is obtained.

[0033] The high-level feature map has a lower resolution but contains rich semantic information, which helps to capture the overall shape, direction, and positional relationship of the orbital lines.

[0034] The low-level feature map has a higher resolution and detailed texture information, which helps to capture the detailed features of the orbital lines, such as edges, texture changes, etc.

[0035] Extract paired orbital features at the feature extraction positions of the paired anchors in the high-level feature map to obtain the first paired orbital feature vectors, and correct the feature extraction positions, where the feature extraction positions are represented as the sampling point coordinates on the orbit.

[0036] Perform a convolution on each pair of the first paired orbital feature vectors to obtain multiple pairs of first paired orbital local feature vectors; Input the first paired orbital feature vectors into the orbital line detection model to obtain the position information of the paired orbital lines to be corrected. Use the corrected feature extraction positions to extract paired orbital features at the same positions in the low-level feature map to obtain the second paired orbital feature vectors.

[0037] Perform a convolution on each pair of the second paired orbital feature vectors, splice them with the first paired orbital local feature vectors, and then perform another convolution to obtain the second paired orbital local feature vectors.

[0038] The paired orbital local feature vectors contain the overall shape and position information of the left and right orbits, can mark the regions of interest in the feature map, and also contain detailed texture and edge features, providing a comprehensive feature description for subsequent orbital line detection.

[0039] To improve the detection accuracy, the second paired orbital feature vectors can also be input into the orbital line detection model to obtain the position information of the new paired orbital lines to be corrected. Then, extract paired orbital features at the same positions in the low-level feature map, after convolution, splice them with the first paired orbital local feature vectors and the second paired orbital local feature vectors, and then perform another convolution to obtain the third paired local orbital feature vectors for obtaining more candidate orbits subsequently.

[0040] Furthermore, optionally, the method described in the embodiments of this specification may further include: Calculate the similarity between each pair of the first paired orbital local feature vectors and the high-level feature map respectively to obtain the first similarity matrix corresponding to each of the first paired orbital local feature vectors; Adopt the attention mechanism to obtain the first paired orbital global feature vectors based on the first similarity matrix; Calculate the similarity between each pair of the second paired orbital local feature vectors and the low-level feature map respectively to obtain the second similarity matrix corresponding to each of the second paired orbital local feature vectors; Adopt the attention mechanism to obtain the second paired orbital global feature vectors based on the second similarity matrix.

[0041] In the embodiments of this specification, the attention mechanism is a deep learning technology that mimics human visual attention. It enables the model to focus more on important parts when processing information, thereby improving the performance and accuracy of the model. In the track line detection task, the attention mechanism can be used to evaluate the degree of association between paired track local feature vectors and low-level feature maps, so as to extract more global and important features.

[0042] By calculating the similarity between each pair of first paired track local feature vectors and each position in the high-level feature map respectively, a first similarity matrix corresponding to each first paired track local feature vector can be constructed. By calculating the similarity between each pair of second paired track local feature vectors and each position in the low-level feature map respectively, a second similarity matrix corresponding to each second paired track local feature vector can be constructed.

[0043] Based on the similarity matrix, attention weights are calculated through the softmax function or other normalization methods, and these weights reflect the degree of correlation between each position in the feature map and the paired track local features.

[0044] Using the calculated attention weights, weighted summation is performed on the feature vectors in the feature map, and the result of the weighted summation is the paired track global feature.

[0045] Performing the attention weighted summation step on each pair of paired track local feature vectors can obtain multiple pairs of paired track global feature vectors.

[0046] Specifically, the process of obtaining the first paired track local feature vector and the first paired track global feature vector is called the first stage, the process of obtaining the second paired track local feature vector and the second paired track global feature vector is called the second stage, and the process of obtaining the third paired track local feature vector and the third paired track global feature vector is called the third stage. Then, in the t-th stage, calculating the paired track feature vector and the paired track global feature vector can be generally represented by the following formula:

[0047] where, T is the transpose of the matrix, C is the number of channels of , is the paired track feature extracted from using the position information of the paired track line to be corrected in the t-th stage, is the feature map used in the t-th stage. Specifically, is the high-level feature map, are all low-level feature maps. is the similarity matrix in the t-th stage, is the vector concatenation operation.

[0048] Inputting the global feature vectors of paired orbits in each stage into the orbit line detection model can obtain the position information of the corresponding paired orbit lines in each stage. Inputting all the paired orbit feature vectors into the orbit line detection model can obtain multiple pairs of candidate paired orbit lines, and then the target paired orbit lines can be selected from the multiple pairs of candidate paired orbit lines.

[0049] Since the orbit line is slender in shape and the pixel area it occupies only accounts for a small part in the feature map, in order to enhance the importance of the features in the area near the orbit, the feature map is weighted using the extracted local features. On the one hand, the attention to the region of interest is increased, and on the other hand, the information near the orbit is obtained to expand the receptive field.

[0050] Since the global features simultaneously enhance the attention to the left orbit and the right orbit, the relationship between the left orbit and the right orbit can be learned during the network training process, and then the occlusion problem can be correctly handled. For example, when the left orbit is severely occluded, the position of the left orbit can be inferred based on the position of the unoccluded right orbit, thus solving the problem that the position of the orbit line cannot be accurately determined when the orbit line is occluded.

[0051] To better extract the paired orbit features, optionally, before using multiple pairs of paired anchors to respectively extract the paired orbit features in the feature map to obtain multiple pairs of paired orbit feature vectors in the embodiments of this specification, the method may further include: Selecting the starting points of the preset number of paired anchors at the same horizontal interval at the bottom of the orbit line image to obtain multiple pairs of paired anchors evenly distributed at the bottom of the orbit line image.

[0052] In the embodiments of this specification, using the positional relationship between the left orbit and the right orbit, the starting points and directions of the paired anchors are determined in the image. At the bottom of the orbit line image, the starting points of multiple pairs of paired anchors are selected at the same horizontal interval. The selection of the starting points should ensure that the paired anchors are evenly distributed in the image to cover the entire orbit line area. According to the complexity of the orbit line and the detection accuracy requirements, the number of paired anchors to be used is determined, and the paired anchors can be organized into a set for subsequent extraction and processing in the feature map.

[0053] Specifically, a paired anchor consists of two parts: (1) the abscissa of the starting point of the left anchor , the ordinate and the angle ; (2) the abscissa of the starting point of the right anchor , the ordinate and the angle ; Select the starting points of 26 pairs of paired anchors at the bottom edge of the track line image at the same horizontal interval. For each pair of starting points, 4 pairs of paired anchors are constructed at an interval angle of 0.2π. Thus, 104 pairs of paired anchors evenly distributed at the bottom of the track line image are obtained to cover the entire track line area.

[0054] To increase the number of anchors near the positions where the tracks appear with a higher probability and improve the detection accuracy, a preset number of paired anchors are obtained using a clustering algorithm. Specifically, calculate the ground truth of the paired anchors. The ground truth coordinates of each track at the sampling points are , and the average value of the angles from the ground truth of the sampling points to the ground truth of the track starting point is used as the angle ground truth of each anchor . The calculation formula for the angle ground truth is:

[0055] Calculate the angle ground truth of the left and right anchors respectively 、 , and compare them with the starting point ground truth of the left and right anchors 、 read, to construct the ground truth of the paired anchors.

[0056] Perform K-means clustering with K = 8 on the ground truth of the paired anchors. The 8 obtained cluster centers reflect the overall distribution and central tendency of all data points within the cluster, representing 8 pairs of paired anchors at high-frequency positions and shapes.

[0057] Merge the 104 pairs of paired anchors and the 8 pairs of paired anchors after clustering into a set of 112 pairs of paired anchors, so as to subsequently use the 112 pairs of paired anchors to extract paired track features in the feature map.

[0058] Optionally, before inputting all the paired track feature vectors into the track line detection model in the embodiments of this specification, the method may further include: Construct the track line detection model according to the classification branch and the regression branch; Establish a loss function including the correlation relationship between the classification branch and the regression branch; Use the loss function to optimize the track line detection model to obtain the target track line detection model.

[0059] In the embodiments of this specification, a track line detection model may be pre-constructed. The track line detection model includes two parallel branches, namely the classification branch and the regression branch. Among them, the classification branch includes three linear layers, and the output result represents the probability that the track detected according to each pair of anchors is a real track, that is, the track confidence ; the regression branch includes three linear layers, and the output result includes four parts: (1) the starting coordinates of the left track , Inclination angle , Length ; (2) The starting coordinates of the right track , Inclination angle , Length ; (3) The horizontal coordinate offset of the left track at the sampling point ; (4) The horizontal coordinate offset of the right track at the sampling point .

[0060] After splicing the detection results of the classification branch and the regression branch, the detection result of the track line detection model is obtained. The first 2 columns in the detection result are the output results of the classification branch, and the last 152 columns are the output results of the regression branch. The left and right tracks are represented as a series of horizontal coordinates at the sampling positions, that is and . Since the output values of the regression branch are all normalized physical quantities, and their value range is , when calculating the track position coordinates, the position of the track in the image should be calculated according to the height and width of the image. The calculation formulas for the horizontal coordinates of the left and right tracks at the sampling point are:

[0061] Among them, is the vertical coordinate of the fixed sampling point, is the width of the picture.

[0062] In practical applications, for the problem that there is no explicit connection between the confidence level and the regression effect, the loss function can be set to optimize the track line detection model. The loss function includes segmentation loss, classification loss, and regression loss, and the relationship between the classification task and the regression task is established in the classification loss.

[0063] The segmentation loss is only used to optimize the feature extraction network during the training stage of the track line detection model to obtain better network parameters; the classification loss reflects the ability to select appropriate candidates from the existing candidate tracks as the final result; the regression loss includes the loss of predicting the starting point, angle, length, and track position.

[0064] Since the classification branch and the regression branch are parallel, the former determines which pair of candidate tracks to select as the detection result, and the latter determines the position accuracy of the candidate tracks. To ensure that the candidate track with the most accurate position regression has the highest confidence level, the following method is used to calculate the classification loss:

[0065] Among them, is the confidence level prediction value, is the confidence level true value. For positive samples, y is the intersection over union (IOU) between the candidate track and the track true value. For negative samples, , As a regulatory factor, take , when , take the global minimum value. Update the model parameters through the backpropagation algorithm until the model converges.

[0066] The loss function can ensure that the losses of the two branches are balanced, so that the model can optimize the classification and regression tasks simultaneously, enabling the confidence to fully reflect the regression effect, and enabling the optimized track line detection model to output accurate track line detection results.

[0067] Optionally, in the embodiments of this specification, generating the target paired track lines from the position information of a pair of the paired track lines that meet the preset conditions may specifically include: Calculate the confidence of each pair of paired track lines; Determine the position information of the pair of paired track lines with the highest confidence; the position information is used to represent the coordinate information of several sampling points on the track line; Generate the target paired track lines according to the coordinate information of several of the sampling points.

[0068] In the embodiments of this specification, by inputting 112 pairs of paired track feature vectors, the position information of 112 pairs of candidate paired track lines can be obtained. The classification branch can calculate the confidence of each pair of paired track lines, and select the pair of paired track lines with the highest confidence according to the confidence of each pair of candidate tracks.

[0069] The position information of the pair of paired track lines with the highest confidence can be connected by interpolation, fitting a curve, or using a specific mathematical model to generate the final paired track lines.

[0070] Optionally, the method in the embodiments of this specification may further include: Set the sampling points non-uniformly in a linearly increasing manner.

[0071] In the embodiments of this specification, the sampling points are set from far to near in a manner where the spacing increases linearly. The ordinate of the th sampling point is calculated by the formula:

[0072] where is the height of the picture, .

[0073] Since the nearby images are clear, the degree of track curvature is small, and the detection difficulty is low, while the distant images are blurred, the degree of track curvature is large, and the detection difficulty is high. The non-uniform sampling points can increase the number of sampling points in the distance, focus more on the distant tracks in the regression, improve the detection accuracy of the distant track lines, and thus improve the overall detection effect. The non-uniform sampling points with more in the near and less in the far increase the attention to the regression of the distant tracks, thereby improving the detection accuracy.

[0074] Figure 3 It is a schematic diagram of an application scenario of a method for paired detection of train track lines provided in an embodiment of this specification.

[0075] As Figure 3 shown, step 301: Obtain the track line image to be detected.

[0076] Step 302: Input the track line image into the backbone convolutional neural network.

[0077] Step 303: Extract features from the track line image through the backbone convolutional neural network to obtain high-level feature maps and low-level feature maps.

[0078] Step 304: Combine 104 pairs of uniformly distributed paired anchors and 8 pairs of paired anchors generated by clustering into 112 pairs of paired anchors.

[0079] Step 305: Use 112 pairs of paired anchors to extract paired track features in the high-level feature map and the low-level feature map respectively to obtain 112 pairs of first paired track feature vectors and 112 pairs of second paired track feature vectors.

[0080] Step 306: Fuse 112 pairs of first paired track feature vectors and 112 pairs of second paired track feature vectors respectively to obtain 112 pairs of paired track local feature vectors.

[0081] Step 307: Calculate the similarity matrix between the paired track local feature vectors and the low-level feature map by using the attention mechanism to obtain 112 pairs of paired track global feature vectors.

[0082] Step 308: Input 112 pairs of paired track global feature vectors into the track line detection model to obtain 112 pairs of paired track lines.

[0083] Step 309: Select the pair of paired track lines with the highest confidence as the target paired track lines.

[0084] In the embodiments of this specification, two track lines are regarded as a whole, anchors are laid in pairs, features are extracted in pairs, and the two tracks are regressed simultaneously. When facing a serious occlusion situation, the position information of the occluded track can be estimated using the position information of the unoccluded track, improving the detection accuracy of the track lines.

[0085] Figure 4 It is a schematic diagram of another application scenario of a method for paired detection of train track lines provided by the embodiments of this specification.

[0086] As Figure 4 shown, at the bottom of the track line image, a starting point of a pair of paired anchors is selected. The starting point coordinates of the left anchor are , and the starting point coordinates of the right anchor are . The angle of the left anchor is , and the angle of the right anchor is . To are the vertical coordinates of the sampling points.

[0087] Figure 5 It is a schematic diagram of the structure of a device for paired detection of train track lines provided by the embodiments of this specification.

[0088] Corresponding to the method embodiments, this embodiment also provides a device for paired detection of train track lines, which may include: An image acquisition module 502, configured to acquire a track line image to be detected; A first feature extraction module 504, configured to extract features from the track line image using a convolutional neural network to obtain a feature map containing track features; A second feature extraction module 506, configured to extract paired track features in the feature map using multiple pairs of paired anchors to obtain multiple pairs of paired track feature vectors; the paired anchors are set according to the positional relationship between the left track and the right track in the paired tracks; A position information determination module 508, configured to input all the paired track feature vectors into a track line detection model to obtain the position information of multiple pairs of paired track lines; A paired track line generation module 510, configured to generate target paired track lines from the position information of a pair of paired track lines that meet preset conditions.

[0089] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method.

[0090] Figure 6 It is a schematic diagram of the structure of a device for paired detection of train track lines provided by the embodiments of this specification. As Figure 6As shown, a paired train track line detection device 600 provided by an embodiment of this specification includes a memory 630, a processor 610, and a computer program 620 stored on the memory. The processor 610 executes the computer program 620 to implement the paired train track line detection method described in any of the above embodiments.

[0091] A paired train track line detection device provided by an embodiment of this specification may include a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the paired train track line detection method described in any of the above embodiments.

[0092] A computer-readable storage medium provided by an embodiment of this specification has a computer program stored thereon. When the computer program is executed by a processor, it can implement the paired train track line detection method described in any of the above embodiments.

[0093] Each embodiment in this specification is described in a progressive manner. For the parts that are the same or similar among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for Figure 6 the device shown, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, reference can be made to the partial description of the method embodiment.

[0094] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software improvements (improvements to method flows). However, with the development of technology, many method flow improvements today can be regarded as direct improvements to hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement to a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user programming the device. Designers can program themselves to "integrate" a digital system onto a single PLD, without having to ask a chip manufacturer to design and fabricate a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, which is similar to the software compiler used in program development and writing. The original code before compilation also has to be written in a specific programming language, which is called a hardware description language (HDL), and there is not just one type of HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones currently are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be aware that by simply performing a little logical programming on the method flow using the above-mentioned several hardware description languages and programming it into an integrated circuit, it is easy to obtain the hardware circuit that implements the logical method flow.

[0095] The controller can be implemented in any suitable manner. For example, the controller can take the form of, for example, a microprocessor or a processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of the controller include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that, in addition to implementing the controller in the form of pure computer-readable program code, it is entirely possible to logically program the method steps to enable the controller to be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. to achieve the same function. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as either software modules for implementing the method or structures within the hardware component.

[0096] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0097] For the convenience of description, when describing the above devices, they are described separately as various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0098] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0099] The present invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each flow and / or block of the flowchart illustrations and / or block diagrams, and combinations of flows and / or blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing apparatus create means for implementing the functions specified in the flowchart Figure 1 for one or more flows and / or blocks Figure 1 of the flowchart and / or one or more blocks of the block diagram.

[0100] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement the functions specified in the flowchart Figure 1 for one or more flows and / or blocks Figure 1 of the flowchart and / or one or more blocks of the block diagram.

[0101] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart Figure 1 for one or more flows and / or blocks Figure 1 of the flowchart and / or one or more blocks of the block diagram.

[0102] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0103] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

[0104] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0105] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0106] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0107] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0108] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A train track line pair detection method, characterized in that: include: Acquire the track line image to be detected; Using a convolutional neural network to extract features from the track line image to obtain a feature map containing track features; Using multiple pairs of paired anchors to respectively extract paired track features in the feature graph, and obtain multiple pairs of paired track feature vectors; the paired anchors are set according to the positional relationship between the left track and the right track in the paired track; Inputting all the paired track feature vectors into a track line detection model to obtain position information of multiple pairs of paired track lines; The position information of a pair of paired track lines that meet the preset conditions is used to generate a target paired track line.

2. The method according to claim 1, characterized in that The feature map includes a high-level feature map and a low-level feature map, and the method of using multiple pairs of paired anchors to respectively extract paired track features in the feature map to obtain multiple pairs of paired track feature vectors specifically includes: Extracting paired track features in the high-level feature graph using multiple pairs of paired anchors to obtain multiple pairs of first paired track feature vectors; Performing convolution processing on each pair of first paired track feature vectors to obtain multiple pairs of first paired track local feature vectors; Inputting all of the first paired track feature vectors into the track line detection model to obtain position information of multiple paired track lines to be corrected; According to the position information of all the paired track lines to be corrected, paired track features are extracted from the low-level feature graphs to obtain multiple pairs of second paired track feature vectors; Convolution processing is performed on each pair of the second paired track feature vectors to obtain multiple pairs of second paired track local feature vectors.

3. The method according to claim 2, characterized in that The method further comprises: respectively calculating the similarity between each pair of the first paired track local feature vectors and the high-level feature graph, and obtaining a first similarity matrix corresponding to each of the first paired track local feature vectors; Using an attention mechanism, based on the first similarity matrix, a first pairwise track global feature vector is obtained; Respectively calculating the similarity between each pair of the second paired track local feature vectors and the low-level feature map, to obtain a second similarity matrix corresponding to each of the second paired track local feature vectors; An attention mechanism is adopted to obtain a second pairwise track global feature vector based on the second similarity matrix.

4. The method according to claim 1, characterized in that: Before using the plurality of pairs of paired anchors to respectively extract paired track features in the feature graph to obtain the plurality of pairs of paired track feature vectors, the method further includes: A preset number of starting points of the paired anchors are selected at the same horizontal interval at the bottom of the track line image to obtain a plurality of pairs of the paired anchors evenly distributed at the bottom of the track line image.

5. The method according to claim 1, characterized in that Before inputting all the paired track feature vectors into the track line detection model, the method further includes: Constructing the track line detection model according to the classification branch and the regression branch; Establishing a loss function including the association relationship between the classification branch and the regression branch; The loss function is used to optimize the track line detection model to obtain a target track line detection model.

6. The method according to claim 1, characterized in that The step of generating a target paired trajectory line from the position information of a pair of paired trajectory lines that meet the preset conditions specifically includes: Calculate the confidence of each pair of trajectory lines; Determine the position information of a pair of paired track lines with the highest confidence; the position information is used to represent the coordinate information of a number of sampling points on the track line; According to the coordinate information of the plurality of sampling points, a target paired trajectory line is generated.

7. The method according to claim 6, characterized in that The method further comprises: The sampling points are non-uniformly set in a linear increasing manner.

Images