A deep learning-based 2C railway catenary scene depth calculation method

Abstract

The application discloses a kind of 2C railway overhead line system scene depth calculation method based on deep learning, it is related to image analysis technical field, the present application can realize the depth estimation of high accuracy in 2C scene, by analyzing single image, the present application can deduce the distance and depth information of object, to provide accurate scene perception and distance measurement, it is very key for the application in 2C scene, such as safety monitoring, environmental perception etc., use deep learning technology to calculate the distance of entire imaging graph to camera, monocular depth estimation technology can use deep learning model, such as convolutional neural network or auto-encoder, to train and deduce, this method can more accurately calculate the distance of transverse object, especially suitable for distant object.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image analysis, and particularly relates to a 2C railway overhead line system scene depth calculation method based on deep learning. BACKGROUND

[0002] The railway overhead line system refers to an electrical device in a power supply railway system, which is used to provide power to the running train. It is composed of a series of wires and support structures, which are installed above or beside the railway track. The wires are usually made of copper or aluminum, in the form of suspension or rigidity, extending along one side or both sides of the railway track. The support structure is used to fix and support the wire and ensure the proper distance between the wire and the ground. The railway overhead line system transmits electrical energy to the wire through a power supply device, and the train obtains the required power by contacting the wire through the elastic contact device (such as the pantograph) suspended on the overhead line system. This power supply mode enables the train to obtain power during operation and provides power for other train equipment (such as lighting, braking, signaling system, etc.).

[0003] The design and specification of the railway overhead line system will vary according to different railway systems and countries, in order to adapt to different train types, speed requirements and power supply standards. In railway operation, it is essential to maintain and manage the normal operation of the overhead line system to ensure the reliability and safety of power transmission, and to reduce interference and failure to train operation.

[0004] Trees or other buildings on both sides of the railway overhead line system are prone to interfere with the overhead line system in adverse weather conditions, such as trees leaning or directly falling on the overhead line system, or trees being bent by heavy snow, which may interfere with the overhead line system. These risks need to be monitored in advance, and judging the distance of trees and other objects around the railway overhead line system is a relatively effective method. The existing technology uses the longitudinal parallax method to calculate the distance, which is relatively accurate in calculating the distance from the support to the camera. However, when calculating the distance of the dangerous trees in the horizontal direction, the acceleration of the pixel points of different distance objects in the continuous image frames captured by the vehicle-mounted camera is usually not the same. The pixel points of objects close to the camera will change faster in the image, i.e. the acceleration is greater. While the objects far away from the camera will move relatively slowly, and the acceleration of their pixel points is smaller. Conversely, when the vehicle slows down or stops, the objects close to the camera move relatively slowly, while the objects far away from the camera are stationary faster. Therefore, using equal length to calculate the longitudinal distance will produce a large error, especially for distant objects.

[0005] When measuring the distance by the longitudinal parallax method, the image information of several frames before and after the distance measuring target needs to be obtained, and the time required for the program to calculate each frame of image is about 330ms, so the total detection target time will exceed 1000ms, which cannot meet the requirement of more than 40 frames per second.

[0006] In view of the above problems, we provide a 2C railway catenary scene depth calculation method based on deep learning to solve the above-mentioned problems. SUMMARY

[0007] The purpose of the present application is to provide a 2C railway catenary scene depth calculation method based on deep learning to solve the problems mentioned in the background art.

[0008] To solve the problems of dynamic parallax and calculation timeout, the present application proposes a deep learning method for calculating depth. The present application uses deep learning technology to calculate the distance from the entire imaging graph to the camera. By analyzing the visual features, textures and geometric information in the image, we can infer the depth information of the object. The present application no longer relies on longitudinal parallax, but estimates the real distance of the object by learning and inference. Monocular depth estimation technology can use deep learning models such as convolutional neural networks or autoencoders for training and inference. The present application can more accurately calculate the distance of objects in the horizontal direction, especially for objects far away.

[0009] To solve the problem of calculation timeout and meet the real-time requirement, the present application optimizes the monocular depth estimation technology for real-time. The present application uses high-performance hardware devices such as graphics processing units (GPUs) or dedicated deep learning accelerators to speed up the inference process of the depth estimation algorithm. In addition, the present application optimizes and lightens the deep learning model to reduce the amount of calculation and inference time. Through these optimization measures, the present application can complete the entire depth estimation process in a short time and meet the real-time requirement.

[0010] To achieve the above-mentioned purposes, the present application provides the following technical solutions: a 2C railway catenary scene depth calculation method based on deep learning, specifically comprising the following steps:

[0011] Step 1: data space selection, using a depth camera to obtain a large amount of depth imaging graph data of different railway sections;

[0012] Step 2: design the network structure, which includes a sampling module, a feature extraction module and an edge algorithm;

[0013] The sampling module includes an improved encoder, which includes four sampling layers, s / 2, s / 4, s / 8 and s / 16;

[0014] The feature extraction module comprises an improved CNN convolution module;

[0015] The edge algorithm comprises the following steps:

[0016] The first step is to obtain an edge map:

[0017] After downsampling, four scale feature maps are obtained, a convolution is performed on S / 16 to obtain a new feature map S` / 16, then S / 16 and S` / 16 are subtracted to obtain an edge map L4, and the same method is used to perform the same operation on S / 8, S / 4 and S / 2 to obtain L3, L2 and L1.

[0018] The second step is to superimpose edge features:

[0019] An ASPP is performed on S / 16 to maintain a larger receptive field while retaining more detailed information, then a convolution is performed to obtain R5, and an upsampling is performed on R5 to prepare for superposition; an upsampling is performed on the feature map after the ASPP, and then a convolution is performed; L4, the ASPP upsampling convolution feature map and R5 are superimposed on a same-size map, a convolution is performed to obtain R4, and through the above method, R3, R2 and R1 are sequentially obtained; Figure Three

[0020] The third step is to superimpose results:

[0021] D represents an output result, D5 is R5, D4 is the superposition of R5 after upsampling and R4, D3 is the superposition of R3 after D4 is upsampling, D2 is the superposition of R2 after D3 is upsampling, D1 is the superposition of R1 after D2 is upsampling, through layer-by-layer superposition, the features and details are clearer and clearer, and D1 is the final input result.

[0022] As a further scheme of the present application, the improved CNN convolution module comprises a structure of 17 layers, wherein 15 are feature extraction layers, 1 is an average pooling layer and 1 is an FC layer, and the Resnet algorithm is used.

[0023] As a further scheme of the present application, the step two further comprises calculating a loss value, the loss value comprises a mean square error formula, a regularization penalty term is introduced into the mean square error formula of the loss function, the loss value is calculated, and the correlation between a predicted value and an actual value in a training process is adjusted.

[0024] As a further scheme of the present application, the mean square error formula is:

[0025] = = - ;

[0026] In the formula​ represents the conventional mean square error term, which measures the difference between the predicted value and the true value, and is the introduced regularization penalty term. This penalty term has a negative sign, when d i and d j are of the same sign, the term is positive, thus normally reducing the loss value; while when d i and d j are of opposite signs, the term is negative, thus penalizing the loss value.

[0027] Compared with the prior art, the present application has the following beneficial effects:

[0028] 1. Accurate depth estimation: The present application can achieve highly accurate depth estimation in 2C scenarios by analyzing a single image, which can infer the distance and depth information of objects, thus providing accurate scene perception and distance measurement, which is very crucial for applications in 2C scenarios such as security monitoring, environmental perception, etc.

[0029] 2. Real-time performance: The present application has real-time performance, which can quickly and efficiently perform depth estimation in 2C scenarios, and can process images in real time and generate depth estimation results in a short time, which is very important for 2C scenarios that require immediate feedback and real-time decision-making, such as intelligent driving, real-time security monitoring, etc.

[0030] 3. Robustness and adaptability: The present application exhibits robustness and adaptability in 2C scenarios, which can adapt to various complex scenarios and lighting conditions, and effectively cope with challenges in 2C scenarios such as occlusion, texture variation and lighting variation, etc., which enables the present application to maintain stable depth estimation results in different 2C environments and provide reliable scene perception.

[0031] 4. Efficient computation: The present application has efficient computing capability, which can perform depth estimation on devices with limited computing resources, and uses optimized algorithms or models to improve computing speed and reduce power consumption, which enables the present application to efficiently perform depth estimation in 2C scenarios and adapt to situations with limited computing resources. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is the overall network structure diagram of the present application.

[0033] Figure 2 is the structure diagram of the sampling part in the present application.

[0034] Figure 3 is the structure diagram of the improved CNN convolution module in the present application.

[0035] Figure 4The partial structure diagram of the convolution to obtain R5 in the application.

[0036] Figure 5 The partial structure diagram of the up-sampling of the feature map after ASPP in the application, followed by a convolution.

[0037] Figure 6 The partial structure diagram of the result superposition in the application.

[0038] Figure 7 The original diagram of the railway catenary scene in the application.

[0039] Figure 8 The input of D5. Figure 7 The depth map of D5 after input.

[0040] Figure 9 The input of D4. Figure 7 The depth map of D4 after input.

[0041] Figure 10 The input of D3. Figure 7 The depth map of D3 after input.

[0042] Figure 11 The input of D2. Figure 7 The depth map of D2 after input.

[0043] Figure 12 The input of D1. Figure 7 The depth map of D1 after input. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0045] To solve the problems of dynamic parallax and calculation timeout, the method proposes a depth learning method for calculating depth. We use deep learning technology to calculate the distance from the entire imaging graph to the camera. By analyzing the visual features, textures and geometric information in the image, we can infer the depth information of the object. This method no longer relies on longitudinal parallax, but estimates the real distance of the object through learning and inference. Monocular depth estimation technology can use deep learning models such as convolutional neural networks or autoencoders for training and inference. This method can more accurately calculate the horizontal distance of objects, especially for distant objects.

[0046] To calculate the timeout for the outcome, in order to meet the real-time requirements, we optimize the monocular depth estimation technology for real-time. We use high-performance hardware devices such as graphics processing units (GPUs) or dedicated deep learning accelerators to speed up the inference process of the depth estimation algorithm. In addition, we optimize and lightweight the deep learning model to reduce the amount of calculation and inference time. Through these optimization measures, we can complete the entire depth estimation process in a shorter time and meet the real-time requirements.

[0047] A deep learning-based 2C railway catenary scene depth calculation method, the complete technical solution is as follows:

[0048] Mainly divided into two parts.

[0049] Step one: data space selection

[0050] In order to effectively simulate real visual images in the 2C railway catenary scene, we need to train the model to adapt to various forms of real scene data of the railway, and maintain good compatibility in numerical value. For this purpose, we use a depth camera to obtain a large amount of depth imaging data of different railway scene data.

[0051] By using a depth camera, we can obtain accurate scene depth information and use it to restore the authenticity of visual images. These depth imaging data provide us with accurate shape and position information of objects in the railway catenary scene. Based on these data, we can train the model to learn the visual features of each element in the scene and generate visual images that match the real scene.

[0052] This method enables us to establish a model that can simulate real railway catenary scenes. By combining depth imaging data with other relevant data sources, we can more accurately restore the details of object shape, texture, and lighting in the railway catenary scene, thereby improving the authenticity and realism of the model.

[0053] This technology has important application value for the simulation and simulation of railway catenary scenes, and can help optimize railway system design, improve train operation planning and safety evaluation, and provide decision support and training simulation functions.

[0054] Step two: network structure design

[0055] This method uses an encoder + CNN convolution improvement + self-created edge extraction method + feature fusion method for training.

[0056] The total network structure diagram is as shown in Figure 1 :

[0057] 2.1 Encoder design

[0058] The encoder is a convolutional neural network (CNN) encoder: CNN encoders are widely used in the field of image processing. It extracts local features of images through convolution and pooling operations, and gradually reduces the size of feature maps. CNN encoders perform well in image classification, object detection, and image generation tasks. We use the encoder as the first layer of the network model.

[0059] 2.2 Sampling design

[0060] In order to maintain the diversity of features in the 2C railway scene, provide more feature maps for subsequent edge feature calculation, and realize image category recognition of images of any size, an improved encoder design is proposed. The design includes four down-sampling layers, s / 2, s / 4, s / 8 and s / 16.

[0061] s / 2 down-sampling: sampling operation is performed on the input signal, which is reduced to half the original size.

[0062] s / 4 down-sampling: sampling operation is performed on the input signal, which is reduced to one-fourth of the original size.

[0063] s / 8 down-sampling: sampling operation is performed on the input signal, which is reduced to one-eighth of the original size.

[0064] s / 16 down-sampling: sampling operation is performed on the input signal, which is reduced to one-sixteenth of the original size.

[0065] These down-sampling operations are usually performed in different levels or modules of the encoder to gradually reduce the size of the input data. Through step-by-step down-sampling, the encoder can capture feature information at different scales and extract higher-level abstract representations. This hierarchical down-sampling operation helps the encoder learn multi-scale features of the data, improving the model's expression ability and generalization ability. By adding these four down-sampling layers, the resolution of the input image can be gradually reduced, and multi-scale features can be extracted. This helps the encoder capture information at different scales in the image and provide richer feature representations. At the same time, this design also allows classification of images of any size, improving the adaptability and practicality of the model.

[0066] This improved encoder design can be effectively applied to image category recognition tasks in the 2C railway scene and provide better technical effects. It can maintain the diversity of features, provide more feature maps for subsequent edge feature calculation, and flexibly handle images of different sizes. Ensure its innovation and uniqueness.

[0067] The sampling part is as follows Figure 2 As shown in the figure, the original image has been sampled at four levels.

[0068] 2.3 Feature extraction module design

[0069] In the 2C contact network detection scenario, processing speed and accuracy are crucial requirements. To ensure fast processing speed while ensuring high accuracy results, conventional feature extraction algorithms have the following problems:

[0070] a. Overuse of CNN convolution modules: This can lead to an increase in the number of network model parameters, increasing the amount of calculation and thus prolonging the training and inference time, affecting the processing speed of 2C railway contact network. In addition, due to excessive features, overfitting phenomenon is easy to occur, which affects the accuracy of 2C railway contact network detection results.

[0071] b. Insufficient use of CNN convolution modules: Although it reduces the number of features and calculation, it speeds up the inference speed, but reduces the accuracy, making it impossible to achieve the required accuracy, and even causing underfitting phenomenon.

[0072] To solve the above problems, we propose an optimization scheme to meet the requirements in the 2C contact network detection scenario. This scheme combines the following features:

[0073] Efficient feature extraction algorithm: An optimized feature extraction algorithm is adopted to ensure sufficient speed in terms of processing speed.

[0074] Moderate application of CNN convolution modules: By carefully designing the number of CNN convolution modules, the number of features and the amount of calculation are balanced to meet the requirements of processing speed and accuracy.

[0075] Improved CNN convolution module Figure 3 : The module structure is a 17-layer structure, including 15 feature extraction layers, 1 average pooling layer and 1 FC layer, and uses the idea of Resnet.

[0076] Through this structure model optimization, we can achieve high-speed processing and high-accuracy results in the 2C contact network detection scenario, while avoiding the problems brought by conventional feature extraction algorithms. This scheme has obvious innovation advantages and broad prospects and potential in practical applications.

[0077] 2.4 Edge algorithm design

[0078] First, get the edge map:

[0079] After downsampling, we get 4 scale feature maps, we do a convolution on S / 16 to get a new feature map S` / 16, then we subtract S / 16 and S` / 16 to get the edge map L4, using the same method, we do the same operation on S / 8, S / 4, S / 2 to get L3, L2, L1.

[0080] Second step edge feature superposition:

[0081] We do an ASPP on S / 16 to maintain a larger receptive field while preserving more detailed information. Then do a convolution to get R5, R5 does an upsampling to prepare for superposition, as shown in Figure 4 .

[0082] Then we do an upsampling on the feature map after ASPP, then do a convolution, as shown in Figure 5 .

[0083] Finally we superimpose L4 (edge information map), ASPP upsampling convolution feature map, R5 downsampling Figure Three a same size map, do a convolution to get R4.

[0084] Through the above method, we get R3, R2, R1 in turn.

[0085] The third step result superposition, as shown in Figure 6 .

[0086] D represents the output result, D5 is R5, D4 is the superposition of R5 after upsampling and R4, D3 is the superposition of R3 result after D4 upsampling, D2 is the superposition of R2 after D3 upsampling, D1 is the superposition of R1 after D2 upsampling, through layer by layer superposition, the features and details are more and more clear, D1 is the final input result.

[0087] 2.5 Design of loss function

[0088] In common loss functions, the commonly used mean square error formula is:

[0089] =

[0090] However, when calculating the loss value, it is usually expected that there is a certain correlation between the predicted result and the true value. For example, for a certain pixel point in an image, if its predicted value is 70, and the true value is 80, the difference is 10. The predicted value of another point is 40, according to the correlation law, the true value should be greater than 40. Because the depth value in the real image is fixed, but in the training process, non-correlation may occur.

[0091] To address this issue, the method introduces a regularization penalty term. By introducing the regularization penalty term, we can adjust the correlation between the predicted value and the true value during the training process. The formula is improved as follows:

[0092] = = -

[0093] In this formula, represents the conventional mean squared error term, which measures the difference between the predicted value and the true value, while is the introduced regularization penalty term. This penalty term has a negative sign, and when d i and d j are of the same sign, the term is positive, thus normally reducing the loss value; while when d i and d j are of opposite signs, the term is negative, thus penalizing the loss value. This regularization penalty term can effectively constrain the prediction result to follow the same increase or decrease rule.

[0094] Through this method, we can more accurately calculate the loss value and maintain the correlation between the predicted result and the true value during the training process. This innovative method provides an effective solution for the research and application of 2C railway catenary distance measurement.

[0095] 2.6 Result Output

[0096] We analyze the final inference results, feature fusion, and the effect of adding predicted distance values as shown in Figures 7-12 , where the depth map effect is a heat map, and the depth and hue of the color are used to represent the distance of objects in the railway catenary image to the camera, and are often expressed using a color spectrum chart. Common color spectrums include a gradual change from lighter colors (such as blue or light green) to darker colors (such as red or dark red), representing a change from a distance far to a distance near.

[0097] Through detailed observation and analysis of our method, we find that during the process of superimposing feature maps of different scales, the inference effect and texture gradually improve.

[0098] Specifically, our method utilizes the strategy of multi-scale feature extraction, superimposing feature maps of different scales. This superimposition process can effectively fuse semantic information and detailed information at different scales, providing more comprehensive and rich feature representations. During the inference stage, this superimposed feature map can better capture key information in the image, thereby improving the accuracy and effect of inference.

[0099] At the same time, we notice that the texture is also gradually improved during the feature map stacking process. This is because the stacking of multi-scale features can capture the texture information of the image in a more global and local range. Through step-by-step stacking and fusion, we can better preserve and reconstruct the texture details of the image, making the inference result closer to the visual perception of the real image.

[0100] Thus, the present application provides a unique monocular depth estimation algorithm specifically for 2C railway catenary ranging scenarios. This algorithm combines the characteristics of the railway environment and the speed and accuracy requirements, and can accurately estimate the depth information of each object in the railway scene. This algorithm combines machine learning, neural networks or other computer vision techniques to achieve high precision and efficiency.

[0101] Depth and image feature association: the present application can effectively associate the features in the railway image with the real depth information. By analyzing the shape, color, texture and other features of the objects in the image, this technology can infer the depth distribution of the objects in the railway environment and generate the corresponding depth map. This association can help detect potential dangers.

[0102] Real-time performance and robustness: the technical solution of this patent has real-time performance and robustness, and can perform depth estimation in real-time 2C railway catenary ranging scenarios. It can adapt to different lighting conditions and weather conditions, and has certain fault tolerance to noise, occlusion and other factors in the image. In this way, railway workers can obtain accurate depth information of dangerous objects in a timely manner to support risk assessment and take appropriate safety measures.

[0103] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Although the present application is described in terms of embodiments, not every embodiment contains only one technical solution, and the description of the specification is for clarity only. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be appropriately combined to form other embodiments that those skilled in the art can understand.

Claims

1. A deep learning-based 2C railway catenary scene depth calculation method, characterized in that: Specifically comprising the following steps: Step one: data space selection, using a depth camera to obtain a large number of depth imaging data of different railway section scenes; Step two: design the network structure, which includes a sampling module, a feature extraction module and an edge algorithm; The sampling module includes an improved encoder, which includes four sampling layers, s / 2, s / 4, s / 8 and s / 16; The feature extraction module includes an improved CNN convolution module; The edge algorithm includes the following steps: First step: obtain the edge map: After downsampling, four scale feature maps are obtained, and S / 16 is convolved to obtain a new feature map S` / 16, then S / 16 and S` / 16 are subtracted to obtain the edge map L4, and the same method is used to obtain L3, L2 and L1 from S / 8, S / 4 and S / 2; Second step: edge feature superposition: S / 16 is subjected to ASPP to maintain a larger receptive field while retaining more detailed information, then a convolution is performed to obtain R5, and R5 is subjected to upsampling to prepare for superposition; the feature map after ASPP is subjected to upsampling, then a convolution is performed; L4, the ASPP upsampling convolution feature map and R5 down-sampling map are superimposed, and a convolution is performed to obtain R4, and through the above method, R3, R2 and R1 are obtained in turn; Third step: result superposition: D represents the output result, D5 is R5, D4 is the superposition of R5 after upsampling and R4, D3 is the superposition of R3 result after D4 is upsampling, D2 is the superposition of R2 after D3 is upsampling, and D1 is the superposition of R1 after D2 is upsampling, through layer-by-layer superposition, the features and details are clearer and clearer, and D1 is the final input result depth calculation method; the improved CNN convolution module includes a structure of 17 layers, 15 feature extraction layers, one average pooling layer and one FC layer, and the Resnet algorithm is used.

2. The deep learning-based 2C railway catenary scene depth calculation method according to claim 1, characterized in that, The step two also includes calculating the loss value, which includes the mean square error formula, and a regularization penalty term is introduced into the mean square error formula of the loss function to calculate the loss value and adjust the correlation between the predicted value and the true value in the training process.

3. The deep learning-based 2C railway catenary scene depth calculation method according to claim 2, characterized in that, The mean square error formula is: = = - ； In the formula represents the conventional mean square error term, which measures the difference between the predicted value and the true value, while is the introduced regularization penalty term; this penalty term has a negative sign, and when d i and d j are of the same sign, the term is positive, thus normally reducing the loss value; while when d i and d j are of opposite signs, the term is negative, thus penalizing the loss value.

Images