A method, system, and edge computing device for track identification in tunnels.

Abstract

This invention provides a method, system, and edge computing device for track identification in tunnels. The track identification method includes the following steps: Step 1: Reading image data from a network camera; Step 2: Locating the track based on global feature calculations using the image data. The beneficial effects of this invention are: It solves the problem of track identification in tunnels when tracks are obscured or under poor lighting conditions, improves the processing speed of each frame, accurately describes the extension and edges of the track, and improves the real-time performance and accuracy of track identification.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, system and edge computing device for track recognition in tunnels. Background Technology

[0002] In the context of tunnel boring machine (TBM) construction for subways, intelligent transportation solutions employing unmanned driving technology require the identification of the track area to determine dangerous zones along the travel direction. When someone enters a dangerous zone or there are obstacles within the dangerous zone, the system can promptly slow down or stop and wait.

[0003] Existing technologies perform well in identifying relatively clean track areas. For example, traditional binary methods using machine learning can identify railway tracks on the ground. However, they cannot handle situations where there is a lot of water or mud in tunnels.

[0004] While existing target recognition methods can identify tracks, marking them with rectangular boxes is unreasonable and cannot accurately mark the track's extension and edge curves.

[0005] While existing deep learning technologies can segment track regions using image segmentation, they consume a lot of computing resources, resulting in low real-time performance for vehicle-mounted edge deployment schemes used in autonomous driving systems. Furthermore, they lack in-depth understanding and judgment of occluded objects. Summary of the Invention

[0006] To address the shortcomings of the prior art, this invention solves the following problems: 1. In cases where there is a lot of water and mud in the tunnel, an associative mechanism is adopted, which can largely overcome the influence of water and mud obstruction and identify the track; 2. A line description method is used to accurately describe the extension and edge of the track; 3. An anchoring is set, which greatly improves the real-time performance and accuracy of training and inference, making it more suitable for edge deployment.

[0007] This invention provides a method for track identification in tunnels, characterized by comprising the following steps:

[0008] Step 1: Read the image data from the webcam;

[0009] Step 2: Based on the calculation of global features, locate the track using the image data.

[0010] As a further improvement of the present invention, in step 2, the image is divided into a grid, dividing the image into multiple small regions of the same size. Assuming the maximum number of tracks is C, the number of anchor rows is h, the number of rows in each grid is w, X is the global feature of the image, and f... i，jFor the classifier of the j-th row of the i-th track, the mathematical formula for track prediction is written as:

[0011] P i，j = f ij (X), i ∈ [1, C], j ∈ [1, h]

[0012] Where the left term is the probability that the classification result of the j-th row of the i-th track is true, and its value ranges from 0 to 1. 0 means the probability of the result being true is 0, and 1 means the probability of the result being true is 100%; the loss function of classification is written as:

[0013]

[0014] Where, L cls represents the loss function value of track classification, P i,j represents the probability of the above track prediction, T i,j represents the true classification marked on the dataset, L CE represents the cross-entropy loss function.

[0015] As a further improvement of the present invention, in the said step 2, assume that the height of the image is H pixels, the width of the image is W pixels, the number of channels of the image is C, the grid of the divided image has h rows, and the number of grids in each row is w. Since each grid is composed of multiple pixels, so h << H and w << W. After dividing the grid, it is necessary to calculate h * w * C times of classification.

[0016] As a further improvement of the present invention, this track recognition method further includes a structural loss function calculation method. When the track in the image is not occluded, the classification results on adjacent anchor rows should have a certain similarity, that is, they should be continuously and relatively vertically extended in the longitudinal direction. Such similarity is written in a mathematical formula as:

[0017]

[0018] The above loss function calculates the L1 norm between the predicted grids between adjacent anchor rows, sums up the calculation results of all anchor rows, and uses this as the similarity loss function value of the whole image; for the probability of each grid of each anchor row, the softmax function is used to calculate its classification result, and this is used as the recognition result of this anchor row; it is written in a mathematical formula as:

[0019] Lane i，j = softmax(P i，j,1:ω )

[0020] Since the grids in the background image should not contain tracks, and at the same time the calculation of probability only calculates from 1 to ω, so the expected track coordinates are written as:

[0021]

[0022] From the above, we can see that the mathematical expression for the second derivative of the loss function can be written as:

[0023]

[0024] Therefore, the overall structural loss function can be written as the following mathematical expression:

[0025] L str =L sim +λL shp

[0026] λ is the introduced loss coefficient, which should be obtained through training.

[0027] As a further improvement to the present invention, the overall loss function is:

[0028] L total =L cls +αL str +βL seg

[0029] Where α and β are the coefficients of the loss function, which should be obtained through training; L seg This represents the loss function for segmentation.

[0030] As a further improvement of the present invention, the number of tracks contained in a single frame image is 0-4; the closer the anchored coordinate points are to the bottom of the image, the sparser the arrangement, and the closer they are to the top of the image, the denser the arrangement.

[0031] As a further improvement of the present invention, the number of anchor lines for each track is at most 8, that is, a track is described by a maximum of 8 anchor coordinate points.

[0032] As a further improvement of the present invention, the coordinate points inferred from each track are connected into a line, so that the track is described by lines instead of points.

[0033] The present invention also provides an edge computing device, comprising: a memory, a processor, and a computer program stored on the memory, the computer program being configured to implement the steps of a track identification method when invoked by the processor.

[0034] The present invention also provides a track identification system for use in tunnels, including a webcam and an edge computing device, wherein the webcam is connected to the edge computing device, enabling the edge computing device to access the webcam.

[0035] The beneficial effects of this invention are: it solves the problem of track identification when the track is obscured and the lighting is not ideal in the tunnel, improves the processing speed of each frame of image, accurately describes the extension and edge of the track, and improves the real-time performance and accuracy of track identification. Attached Figure Description

[0036] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0037] This invention relies on an in-vehicle vision hardware system and algorithm software. The in-vehicle vision hardware system is a vision hardware system installed and deployed on an unmanned vehicle inside a tunnel, including: a network camera and an edge computing device. The network camera refers to an image acquisition device that can provide visual image acquisition, encoding / decoding, and RTSP services. It is connected to the edge computing device via an Ethernet cable, allowing the edge computing device to access the video stream data acquired by the network camera. The edge computing device refers to a computer with certain graphics processing capabilities, typically an industrial control computer equipped with a GPU.

[0038] Network cameras and edge computing devices in in-vehicle vision hardware systems can also be connected wirelessly or in a hybrid wireless and wired manner. Wireless connections include Bluetooth, Wi-Fi, 4G, and 5G.

[0039] like Figure 1 As shown, the track identification method of the present invention includes the following steps:

[0040] Step 1: Read the image data from the webcam.

[0041] Step 2: The method, based on global feature calculation, locates the track on a predetermined number of rows. In the mathematical expression of the method described in this invention, the track is formed by connecting the positioning points on several predetermined rows in the vertical direction. To facilitate positioning, it is necessary to first divide the image into a grid, dividing the image into multiple small regions of the same size.

[0042] This invention defines the following mathematical variables and their types:

[0043]

[0044] Assume the maximum number of tracks is C, the number of anchor rows is h, the number of rows in each grid is w, X is the global feature, and f i，j Let j be the classifier for the j-th row of the i-th orbit. Therefore, the mathematical formula for orbit prediction can be written as:

[0045] P i，j =f ij(X), i ∈ [1, C], j ∈ [1, h]

[0046] Among them, the left term is the probability that the classification result of the j-th row of the i-th track is true, and its value ranges from 0 to 1. 0 indicates that the probability of the result being true is 0, and 1 indicates that the probability of the result being true is 100%. The loss function of the classification can be written as:

[0047]

[0048] Among them, L cls represents the loss function value of the track classification, P i，j represents the probability of the above track prediction, T i，j represents the true classification marked on the dataset, L CE represents the cross-entropy loss function.

[0049] Compared with image segmentation, the method adopts a simpler calculation method to optimize the calculation speed: Assume that the height of the image is H pixels, the width is W pixels, and the number of channels is C (if it is an RGB image, C is equal to 3). The grid of the divided image has h rows, and the number of grids in each row is w. Since each grid consists of multiple pixels, h << H and w << W. If the traditional image segmentation method is used, it is necessary to classify each pixel, and at least H * W * C classification calculations are required. After dividing the grid, only h * w * C classification calculations are required, which is much less than the calculation times of image segmentation.

[0050] Regarding the occlusion problem of the method, such as common problems of water accumulation occlusion, silt occlusion, personnel occlusion, and occlusion by other locomotives, the occluded track information can be inferred based on the aisle curvature, light change, and environmental information in the tunnel. In short, the prediction of the occluded track information requires the use of feature information of other parts of the image to judge. Therefore, the method described in the present invention proposes a structural loss function calculation method to solve the occlusion problem: When the track in the image is not occluded, the classification results on adjacent anchor rows should have a certain similarity, that is, they should be continuously and relatively vertical in the longitudinal direction. Such similarity can be written as a mathematical formula:

[0051]

[0052] The above loss function calculates the L1 norm between the predicted grids between adjacent anchor rows, and sums up the calculation results of all anchor rows to obtain the similarity loss function value of the entire image.

[0053] For the probability of each grid of each anchor row, the softmax function can be used to calculate its classification result as the recognition result of the anchor row. It is written as a mathematical formula:

[0054] Lane i，j =softmax(P i，j，1：ω )

[0055] Since the grid in the background image should not contain the orbit, and the probability calculations only range from 1 to ω, the desired orbit coordinates can be written as:

[0056]

[0057] The formula above is a summation formula, and the value of k ranges from 1 to ω.

[0058] From the above, we can see that the mathematical expression for the second derivative of the loss function can be written as:

[0059]

[0060] Therefore, the overall structural loss function can be written as the following mathematical expression:

[0061] L str =L sim +λL shp

[0062] λ is the introduced loss coefficient, which should be obtained through training.

[0063] In summary, for the overall loss function, its classification loss function, structural loss function, and segmentation loss function should be considered. This is given by the following mathematical expression:

[0064] L total =L cls +αL str +βL seg

[0065] Here, α and β are the coefficients of the loss function, which should be obtained through training.

[0066] Based on the installation method of the onboard camera on the unmanned locomotive in the tunnel and the characteristics of the track in the image, and in accordance with the principle of perspective, the method of this invention also adopts the following constraints:

[0067] A single frame of an image contains 0-4 tracks;

[0068] The maximum number of anchor lines for each track is 8, meaning a track can be described by a maximum of 8 anchor coordinate points; the number of anchor points can be designed to be other than 8.

[0069] The closer the anchored coordinate points are to the bottom of the image, the sparser the arrangement; the closer they are to the top of the image, the denser the arrangement.

[0070] Based on the inference results from the described method, the coordinate points inferred from each track in a single image are connected to form a line, thus transforming the track description from point-based to line-based. Line-based descriptions can also be replaced with point-based descriptions.

[0071] The track identification method of the present invention includes the following features:

[0072] 1. This invention provides a simple and fast-running deep learning network structure that solves the problem of track identification in tunnels under conditions of track occlusion and poor lighting. Compared to traditional image segmentation methods, the method described in this invention does not classify every pixel of the image, but selectively identifies the location of the track, thus improving the processing speed of each frame. Furthermore, the track prediction method described in this invention is based on the calculation of global features, resulting in a deeper depth of field compared to traditional image segmentation methods, thereby effectively solving the problem of track prediction under occlusion and poor lighting conditions.

[0073] 2. This invention provides a novel method for calculating the loss function, which fully utilizes the characteristics of the arrangement of tracks within the tunnel and their presentation in the image, providing an optimization method for training deep learning models.

[0074] 3. The present invention achieves state-of-the-art (SOTA) performance on the track dataset within the tunnel, exhibiting higher accuracy and inference speed compared to other similar methods. The optimized model achieves an inference speed of 150+ FPS on a GTX 1060.

[0075] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for track identification in tunnels, characterized in that, It includes the following steps: Step 1: Read the image data of the webcam; Step 2: For the image data, based on the calculation of global features, locate the tracks; In step 2, the image is divided into a grid, creating multiple small regions of equal size. Assuming the maximum number of tracks is C, the number of anchor rows is h, the number of rows in each grid is w, X represents the global features of the image, and f... i,j Let j be the classifier for the j-th row of the i-th orbit. Therefore, the mathematical formula for orbit prediction can be written as: P i,j =f ij (X),i∈[1,C],j∈[1,h] Where the left term is the probability that the classification result of the j-th row of the i-th track is true, and its value ranges from 0 to 1. 0 means the probability that the result is true is 0, and 1 means the probability that the result is true is 100%; the loss function of classification is written as: Among them, L cls P represents the loss function value for orbit classification. i，j T represents the probability of the above orbital prediction. i，j L represents the true classification of the labels on the dataset. CE This represents the cross-entropy loss function.

2. The track identification method according to claim 1, characterized in that, In the said Step 2, assume the height of the image is H pixels, the width of the image is W pixels, the number of channels of the image is C, the grid of the divided image has h rows, and the number of grids in each row is w. Since each grid is composed of multiple pixels, so h << H and w << W. After dividing the grid, it is necessary to calculate h * w * C times of classification.

3. The track identification method according to claim 1, characterized in that, This track recognition method also includes a structural loss function calculation method. When the tracks in the image are not blocked, the classification results on adjacent anchor rows are similar, that is, they should be continuously and relatively vertical extended in the longitudinal direction. Such similarity is written in a mathematical formula as: The above loss function calculates the L1 norm between the predicted grids on adjacent anchor rows, and sums up the calculation results of all anchor rows to be used as the similarity loss function value of the whole image; For the probability of each grid of each anchor row, use the softmax function to calculate its classification result, which is used as the recognition result of this anchor row; it is written in a mathematical formula as: Lane i，j =softmax(P i，j，1：ω ) Since the grids in the background image should not contain tracks, and the calculation of probability only calculates from 1 to ω, so the expected track coordinates are written as: The value range of k is from 1 to ω, and the mathematical expression of the second derivative of the loss function is written as: Therefore, the overall structural loss function is written as the following mathematical expression: THE str =L sim +λL shp λ is the introduced loss coefficient, which should be obtained through training.

4. The track identification method according to claim 3, characterized in that, The overall loss function: L total =L cls +αL str +βL seg Where α and β are the coefficients of the loss function, which should be obtained through training; L seg This represents the loss function for segmentation.

5. The track identification method according to claim 1, characterized in that, The number of tracks contained in a single-frame image is 0 - 4; the closer the anchored coordinate points are to the lower part of the image, the sparser the arrangement, and the closer they are to the upper part of the image, the denser the arrangement.

6. The track identification method according to claim 1, characterized in that, The maximum number of anchor rows for each track is 8, that is, each track is described by at most 8 anchored coordinate points.

7. The track identification method according to claim 1, characterized in that, Connect the coordinate points deduced for each track into a line, so that the track is described by points changed to be described by a line.

8. An edge computing device, characterized in that, It includes: A memory, a processor, and a computer program stored on the memory, the computer program is configured to implement the steps of the track recognition method according to any one of claims 1 - 7 when called by the processor.

9. A track identification system for use in tunnels, characterized in that, It includes a webcam and the edge computing device according to claim 8, the webcam is connected to the edge computing device, so that the edge computing device can access the webcam.

Images