2C railway contact network scene depth algorithm based on deep learning

Abstract

The invention discloses a 2C railway contact network scene depth algorithm based on deep learning, relates to the technical field of image analysis, can realize highly accurate depth estimation in a 2C scene, and can deduce the distance and depth information of an object by analyzing a single image so as to provide accurate scene perception and distance measurement. This is very critical for applications in 2C scenes, such as security monitoring, environmental awareness and the like, a deep learning technology is used to calculate the distance from the entire image to the camera, and a monocular depth estimation technology can use a deep learning model, such as a convolutional neural network or an auto-encoder, to perform training and inference, and the monocular depth estimation technology can be used to calculate the distance from the entire image to the camera. According to the method, the transverse object distance can be calculated more accurately, and the method is particularly suitable for distant objects.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to a 2C railway contact network scene depth algorithm based on deep learning. Background Art

[0002] Railway contact network refers to an electrical device in the power supply railway system, which is used to provide power to the moving train. It consists of a series of wires and support structures, installed above or on the side of the railway track. The wires are usually made of copper or aluminum, in a suspended or rigid form, extending along one or both sides of the railway track. The support structure is used to fix and support the wires and ensure the proper spacing between them and the ground. The railway contact network transmits electrical energy to the wires through the power supply device, and the train contacts the wires through the elastic contact device (such as pantograph) suspended on the contact network to obtain the required power. This power supply method enables the train to obtain power during operation and provide power for other train equipment (such as lighting, braking, signaling systems, etc.).

[0003] The design and specifications of the railway overhead line vary according to different railway systems and countries to accommodate different train types, speed requirements and power supply standards. In railway operations, it is crucial to maintain and manage the normal operation of the overhead line to ensure the reliability and safety of power transmission while reducing interference and failures to train operations.

[0004] Trees or other buildings on both sides of the railway contact network can easily interfere with the contact network in bad weather conditions, such as trees leaning or falling directly on the contact network, or trees being bent in heavy snow and tending to interfere with the contact network. These risks need to be monitored in advance, and judging the distance of tree objects around the railway contact network has become a relatively effective method. The existing technology uses the longitudinal parallax method when calculating the distance. This method is relatively accurate in calculating the distance from the pillar to the camera. When calculating the horizontal distance of dangerous trees by longitudinal distance, the pixel acceleration of objects at different distances in the continuous image frames captured by the on-board camera is usually different. The pixel position of objects close to the camera in the image will change faster, that is, the acceleration is greater. Objects far away from the camera will move relatively slowly, and their pixel acceleration is small. On the contrary, when the vehicle slows down or stops, objects close to the camera move relatively slowly, while objects far away from the camera stop faster. Therefore, using equal distances to calculate the longitudinal distance will produce large errors, especially for distant objects.

[0005] When using the longitudinal parallax method to measure distance, it is necessary to obtain image information of several frames before and after the distance measurement target. The program takes an average of about 330ms to calculate each frame of the image. Therefore, the time required to fully calculate the entire detection target will exceed 1000ms, which cannot meet the requirement of more than 40 frames within 1s.

[0006] To address the above problems, we provide a 2C railway contact network scenario deep algorithm based on deep learning to solve the above-mentioned problems. Summary of the invention

[0007] The purpose of the present invention is to provide a 2C railway contact network scene depth algorithm based on deep learning to solve the problems raised in the above background technology.

[0008] In order to solve the problems of dynamic parallax and calculation timeout, the present invention proposes a deep learning method for calculating depth. The present invention uses deep learning technology to calculate the distance from the entire imaging image to the camera. By analyzing the visual features, texture and geometric information in the image, we can infer the depth information of the object. The present invention no longer relies on longitudinal parallax, but estimates the true distance of the object through learning and inference. Monocular depth estimation technology can use deep learning models, such as convolutional neural networks or autoencoders, for training and inference. The present invention can more accurately calculate the lateral distance of objects, especially for distant objects.

[0009] In order to solve the problem of timeout in calculation and meet the real-time requirements, the present invention optimizes the monocular depth estimation technology in real time. The present invention uses high-performance hardware devices, such as a graphics processing unit (GPU) or a dedicated deep learning accelerator, to accelerate the inference process of the depth estimation algorithm. In addition, the present invention optimizes and lightweights the deep learning model to reduce the amount of calculation and inference time. Through these optimization measures, the present invention can complete the entire depth estimation process in a shorter time and meet the real-time requirements.

[0010] To achieve the above purpose, the present invention provides the following technical solution: a 2C railway contact network scene depth algorithm based on deep learning, specifically comprising the following steps:

[0011] Step 1: Data space selection, using depth cameras to obtain a large amount of depth imaging data of different railway sections;

[0012] Step 2: Design the network structure, which includes sampling module, feature extraction module and edge algorithm;

[0013] The sampling module includes an improved encoder, and the improved encoder includes four sampling layers, namely s / 2, s / 4, s / 8 and s / 16;

[0014] The feature extraction module includes an improved CNN convolution module;

[0015] The edge algorithm includes the following steps:

[0016] Step 1: Get the edge map:

[0017] After downsampling, we get feature maps of 4 scales. We perform a convolution on S / 16 to get a new feature map S` / 16. Then we subtract S / 16 from S` / 16 to get the edge map L4. In the same way, we perform the same operation on S / 8, S / 4, and S / 2 to get L3, L2, and L1.

[0018] Step 2: Edge feature superposition:

[0019] Perform an ASPP on S / 16 to keep a larger receptive field while retaining more detail information. Then perform a convolution to get R5, and perform an upsampling on R5 to prepare for superposition. Perform an upsampling on the feature map after the ASPP, and then perform a convolution. Perform a convolution on the L4 and ASPP upsampling feature map, and downsample R5. Figure 3 Superimpose the images of the same size and perform a convolution to get R4. Through the above method, we can get R3, R2, and R1 in turn.

[0020] Step 3: Result superposition:

[0021] D represents the output result, D5 is R5, D4 is the superposition of R5 upsampling and R4, D3 is the superposition of D4 upsampling and R3, D2 is the superposition of D3 upsampling and R2, and D1 is the superposition of D2 upsampling and R1. Through layer-by-layer superposition, the features and details become clearer and clearer. D1 is the final input result.

[0022] As a further solution of the present invention: the improved CNN convolution module includes a 17-layer structure, including 15 feature extraction layers, 1 average pooling layer and 1 FC layer, and uses the Resnet algorithm.

[0023] As a further solution of the present invention: the step 2 also includes calculating the loss value, the calculation of the loss value includes the mean square error formula, a regularization penalty term is introduced into the mean square error formula of the loss function, the loss value is calculated, and the correlation between the predicted value and the true value during the training process is adjusted.

[0024] As a further solution of the present invention: the mean square error formula is:

[0025]

[0026] in the formula represents the conventional mean square error term, which is used to measure the difference between the predicted value and the true value, and is the regularization penalty term introduced. This penalty term has a negative sign. When d i With d j When d is the same sign, the term is positive, thus reducing the loss value normally; and when d i With d j When is of opposite sign, the term is negative, thus penalizing the loss value.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. Accurate depth estimation: The present invention can achieve highly accurate depth estimation in 2C scenarios. By analyzing a single image, the present invention can infer the distance and depth information of an object, thereby providing accurate scene perception and distance measurement, which is very critical for applications in 2C scenarios, such as security monitoring, environmental perception, etc.

[0029] 2. Real-time performance: The present invention has real-time performance and can quickly and efficiently perform depth estimation in 2C scenarios. The present invention can process images in real time in real-time applications and generate depth estimation results in a short time. This real-time performance is very important for 2C scenarios that require instant feedback and real-time decision-making, such as intelligent driving, real-time safety monitoring, etc.;

[0030] 3. Robustness and adaptability: The present invention demonstrates robustness and adaptability in 2C scenarios. The present invention can adapt to various complex scenarios and lighting conditions, and effectively cope with challenges in 2C scenarios, such as occlusion, texture changes, and lighting changes. This enables the present invention to maintain stable depth estimation results in different 2C environments and provide reliable scene perception.

[0031] 4. Efficient computing: The present invention has efficient computing capabilities and performs depth estimation on devices with limited computing resources. The present invention adopts optimized algorithms or models to increase computing speed and reduce power consumption, which enables the present invention to efficiently perform depth estimation in 2C scenarios and adapt to situations where computing resources are limited.

[0032] Instruction Manual

[0033] Figure 1 This is the overall network structure diagram of the present invention.

[0034] Figure 2 It is a schematic diagram of the structure of the sampling part in the present invention.

[0035] Figure 3 It is a schematic diagram of the structure of the improved CNN convolution module in the present invention.

[0036] Figure 4 It is a schematic diagram of the partial structure of R5 obtained by convolution in the present invention.

[0037] Figure 5 A schematic diagram of a partial structure of upsampling the feature map after ASPP in the present invention and then performing a convolution.

[0038] Figure 6 It is a partial structural schematic diagram of the superposition of results in the present invention.

[0039] Figure 7 This is the original picture of the railway overhead contact network scene in the embodiment of the present invention.

[0040] Figure 8 For Figure 7 It is the depth map of D5 after input.

[0041] Fig. 9 For Figure 7 It is the depth map of D4 after input.

[0042] Fig.10 For Figure 7 It is the depth map of D3 after input.

[0043] Fig.11 For Figure 7 It is the depth map of D2 after input.

[0044] Fig.12 For Figure 7 It is the depth map of D1 after input. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] To solve problems such as dynamic parallax and computational timeout, this method proposes a deep learning method for calculating depth. We use deep learning technology to calculate the distance from the entire image to the camera. By analyzing the visual features, texture, and geometric information in the image, we can infer the depth information of the object. This method no longer relies on longitudinal parallax, but estimates the true distance of the object through learning and inference. Monocular depth estimation technology can use deep learning models, such as convolutional neural networks or autoencoders, for training and inference. This method can more accurately calculate the lateral distance of objects, especially for distant objects.

[0047] To solve the problem of computation timeout and meet the real-time requirements, we optimized the monocular depth estimation technology for real-time performance. We use high-performance hardware devices, such as graphics processing units (GPUs) or dedicated deep learning accelerators, to accelerate the inference process of the depth estimation algorithm. In addition, we optimized and lightweighted the deep learning model to reduce the amount of calculation and inference time. Through these optimization measures, we can complete the entire depth estimation process in a shorter time and meet the real-time requirements.

[0048] A 2C railway contact network scenario deep algorithm based on deep learning. The complete technical solution is as follows:

[0049] It is mainly divided into 2 parts.

[0050] Step 1: Data space selection

[0051] In order to effectively simulate real visual images in the 2C railway contact network scene, we need to train the model to adapt to various forms of real railway scene data and maintain good numerical compatibility. To this end, we use a depth camera to obtain a large number of depth imaging data of different railway section scenes.

[0052] By using a depth camera, we are able to obtain accurate scene depth information and use it to restore the authenticity of visual images. These depth imaging data provide us with an understanding of the accurate shape and position of objects in the railway contact network scene. Based on this data, we are able to train the model to learn the visual features of each element in the scene and generate visual images that match the real scene.

[0053] This approach enables us to build a model that can simulate real railway overhead wire scenes. By combining the depth imaging data with other relevant data sources, we can more accurately restore the details of the railway overhead wire scene in terms of object shape, texture, and lighting, thereby improving the authenticity and realism of the model.

[0054] This technology has important application value for the simulation and emulation of railway contact network scenarios, and can help optimize railway system design, improve train operation planning and safety assessment, and provide decision support and training simulation functions.

[0055] Step 2: Network structure design

[0056] This method uses encoder + CNN convolution improvement + self-created edge extraction method + feature fusion method for training.

[0057] The overall network structure is as follows Figure 1 As shown:

[0058] 2.1 Encoder Design

[0059] The encoder is a convolutional neural network (CNN) encoder: CNN encoders are widely used in the field of image processing. They extract local features of images through convolution and pooling operations and gradually reduce the size of feature maps. CNN encoders perform well in tasks such as image classification, object detection, and image generation. We use the encoder as the first layer of the network model.

[0060] 2.2 Sampling design

[0061] In order to maintain the diversity of features in the 2C railway scene, provide more feature maps for subsequent edge feature calculation, and realize image category recognition for images of any size to be recognized, an improved encoder design is proposed. The design includes four downsampling layers, namely s / 2, s / 4, s / 8 and s / 16.

[0062] s / 2 downsampling: Sampling the input signal to reduce its scale to half of its original size.

[0063] s / 4 downsampling: Sampling the input signal to reduce its scale to one-fourth of its original size.

[0064] s / 8 downsampling: Sampling the input signal to reduce its scale to one eighth of its original size.

[0065] s / 16 downsampling: Sampling the input signal to reduce its scale to one sixteenth of its original size.

[0066] These downsampling operations are usually performed in different layers or modules of the encoder to gradually reduce the size of the input data. By gradually downsampling, the encoder can capture feature information of different scales and extract higher-level abstract representations. This hierarchical downsampling operation helps the encoder learn the multi-scale characteristics of the data and improve the expressiveness and generalization ability of the model. By adding these four downsampling layers, the resolution of the input image can be gradually reduced and multi-scale features can be extracted. This helps the encoder capture information of different scales in the image and provide richer feature representations. At the same time, this design also allows the classification of images to be identified of any size, improving the adaptability and practicality of the model.

[0067] This improved encoder design can be effectively applied to image category recognition tasks in 2C railway scenarios and provide better technical results. It can maintain feature diversity, provide more feature maps for subsequent edge feature calculations, and can flexibly process images of different sizes, ensuring its innovation and uniqueness.

[0068] The sampling part is as follows Figure 2 As shown in the figure, the original image was sampled at 4 levels.

[0069] 2.3 Feature extraction module design

[0070] In the 2C contact network detection scenario, processing speed and processing accuracy are crucial requirements. In order to ensure a sufficiently fast processing speed while ensuring high accuracy results, conventional feature extraction algorithms have the following problems:

[0071] a. Excessive use of CNN convolution modules: This will increase the number of network model parameters, which in turn increases the amount of calculation, thereby extending the training and reasoning time, affecting the processing speed of the 2C railway contact network. In addition, due to too many features, it is easy to cause overfitting, which has a certain impact on the accuracy of the 2C railway contact network detection results.

[0072] b. Insufficient use of CNN convolution modules: Although it reduces the number of features and the amount of computation, speeds up the inference, it reduces the accuracy, making it impossible to achieve the required accuracy, and even causes underfitting.

[0073] To solve the above problems, we proposed an optimization solution to meet the requirements in the 2C contact network ranging scenario. This solution combines the following features:

[0074] Efficient feature extraction algorithm: Use an optimized feature extraction algorithm to ensure sufficient processing speed.

[0075] Moderate application of CNN convolution modules: By carefully designing the number of CNN convolution modules, the number of features and the amount of computation are balanced to take into account the requirements of processing speed and accuracy.

[0076] Improve the CNN convolution module such as Figure 3 :The module structure is a 17-layer structure, including 15 feature extraction layers, 1 average pooling layer and 1 FC layer, and uses the idea of ​​Resnet.

[0077] Through this structural model optimization, we can achieve high-speed processing and high-accuracy results in 2C contact network detection scenarios, while avoiding the problems caused by conventional feature extraction algorithms. This solution has obvious innovative advantages and has broad prospects and potential in practical applications.

[0078] 2.4 Edge Algorithm Design

[0079] The first step is to obtain the edge map:

[0080] After downsampling, we get feature maps of 4 scales. We perform a convolution on S / 16 to get a new feature map S` / 16. Then we subtract S / 16 from S` / 16 to get the edge map L4. In the same way, we perform the same operation on S / 8, S / 4, and S / 2 to get L3, L2, and L1.

[0081] The second step is edge feature superposition:

[0082] We make S / 16 into an ASPP to keep a larger receptive field while retaining more detail information. Then we do a convolution to get R5, and R5 is upsampled to prepare for superposition. The structure is as follows Figure 4 shown.

[0083] After that, we will do an upsampling of the feature map after ASPP, and then do a convolution. The structure is as follows Figure 5 shown.

[0084] Finally, we use L4 (edge ​​information map), ASPP upsampling convolution special map, R5 downsampling Figure 3 Superimpose images of the same size and perform a convolution to get R4.

[0085] Through the above method, R3, R2, and R1 are obtained in sequence.

[0086] The third step results in superposition, such as Figure 6 As shown:

[0087] D represents the output result, D5 is R5, D4 is the superposition of R5 upsampling and R4, D3 is the superposition of D4 upsampling and R3, D2 is the superposition of D3 upsampling and R2, and D1 is the superposition of D2 upsampling and R1. Through layer-by-layer superposition, the features and details become clearer and clearer. D1 is the final input result.

[0088] 2.5 Design of loss function

[0089] In common loss functions, the commonly used mean square error formula is:

[0090]

[0091] However, when calculating the loss value, it is usually expected that there is a certain correlation between the predicted result and the true value. For example, for a pixel in an image, if its predicted value is 70 and the true value is 80, the difference is 10. The predicted value of another point is 40, and according to the correlation law, the true value should be greater than 40. Because the depth value in the real image is fixed, but non-correlation may occur during training.

[0092] To solve this problem, this method introduces a regularization penalty term. By introducing the regularization penalty term, we can adjust the correlation between the predicted value and the true value during the training process. The formula is improved to:

[0093]

[0094] In this formula represents the conventional mean square error term, which is used to measure the difference between the predicted value and the true value, and is the regularization penalty term introduced. This penalty term has a negative sign. When d i With d j When d is the same sign, the term is positive, thus reducing the loss value normally; and when d i With d j When is of the opposite sign, the term is negative, thus penalizing the loss value. This regularization penalty term can effectively constrain the prediction results to conform to the law of increasing or decreasing together.

[0095] This method allows us to calculate the loss value more accurately and maintain the correlation between the predicted results and the true values ​​during the training process. This innovative method provides an effective solution for the research and application of 2C railway contact network distance measurement.

[0096] 2.6 Result Output

[0097] We perform feature fusion on the final inference result, and then add the predicted distance value to achieve the following effect: Figure 7-Figure 12 As shown in the figure, the depth map effect is a heat map, and the depth and hue of the color are used to indicate the distance from the object to the camera in the railway contact network image, and are often expressed using a color spectrum. Common color spectrums include a gradient from lighter colors (such as blue or light green) to darker colors (such as red or dark red), indicating a change from far distance to close distance.

[0098] Through detailed observation and analysis of our method, we found that the inference effect and texture are gradually improved during the process of superimposing feature maps of different scales.

[0099] Specifically, our method uses a multi-scale feature extraction strategy to superimpose feature maps of different scales. This superposition process can effectively integrate semantic information and detail information at different scales, thereby providing a more comprehensive and rich feature representation. In the inference stage, this superimposed feature map can better capture the key information in the image, thereby improving the accuracy and effect of inference.

[0100] At the same time, we noticed that the texture was gradually improved during the feature map stacking process. This is because the stacking of multi-scale features can capture the texture information of the image in a more global and local range. Through gradual stacking and fusion, we can better preserve and reconstruct the texture details of the image, making the inference results closer to the visual perception of the real image.

[0101] Therefore, the present invention provides a unique monocular depth estimation algorithm specifically for 2C railway contact network ranging scenarios. The algorithm combines the characteristics of the railway environment with the requirements of speed and accuracy, and can accurately estimate the depth information of each object in the railway scene. The algorithm combines machine learning, neural networks or other computer vision technologies to achieve high accuracy and efficiency.

[0102] Correlation between depth and image features: The present invention can effectively correlate features in railway images with real depth information. By analyzing features such as shape, color, texture, etc. of objects in the image, the technology can infer the depth distribution of objects in the railway environment and generate corresponding depth maps. This correlation can help detect potential dangers.

[0103] Real-time performance and robustness: The patented technical solution has real-time performance and robustness, and can perform depth estimation in real-time 2C railway contact network ranging scenarios. It can adapt to different lighting conditions and weather conditions, and has a certain fault tolerance for factors such as noise and occlusion in the image. In this way, railway staff can obtain accurate depth information of dangerous objects in a timely manner to support risk assessment and take corresponding safety measures.

[0104] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Although this specification is described in accordance with the implementation modes, not every implementation mode includes only one technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A 2C railway contact network scene deep algorithm based on deep learning, characterized by: The specific steps include: Step 1: Data space selection, using depth cameras to obtain a large amount of depth imaging data of different railway sections; Step 2: Design the network structure, which includes sampling module, feature extraction module and edge algorithm; The sampling module includes an improved encoder, and the improved encoder includes four sampling layers, namely s / 2, s / 4, s / 8 and s / 16; The feature extraction module includes an improved CNN convolution module; The edge algorithm includes the following steps: Step 1: Get the edge map: After downsampling, we get feature maps of 4 scales. We perform a convolution on S / 16 to get a new feature map S` / 16. Then we subtract S / 16 from S` / 16 to get the edge map L4. In the same way, we perform the same operation on S / 8, S / 4, and S / 2 to get L3, L2, and L1. Step 2: Edge feature superposition: Perform an ASPP on S / 16 to keep a larger receptive field while retaining more detail information, then perform a convolution to get R5, and perform an upsampling on R5 to prepare for superposition; perform an upsampling on the feature map after ASPP, and then perform a convolution; superimpose three images of the same size, L4, ASPP upsampling convolution feature map, and R5 downsampling map, and perform a convolution to get R4. Through the above methods, we can get R3, R2, and R1 in turn; Step 3: Result superposition: D represents the output result, D5 is R5, D4 is the superposition of R5 upsampling and R4, D3 is the superposition of D4 upsampling and R3, D2 is the superposition of D3 upsampling and R2, and D1 is the superposition of D2 upsampling and R1. Through layer-by-layer superposition, the features and details become clearer and clearer. D1 is the final input result.

2. According to claim 1, a 2C railway contact network scene depth algorithm based on deep learning is characterized in that: The improved CNN convolution module includes a 17-layer structure, including 15 feature extraction layers, 1 average pooling layer and 1 FC layer, and uses the Resnet algorithm.

3. According to the 2C railway contact network scene depth algorithm based on deep learning in claim 1, it is characterized in that: The step 2 also includes calculating the loss value, which includes a mean square error formula, introducing a regularization penalty term into the mean square error formula of the loss function, calculating the loss value, and adjusting the correlation between the predicted value and the true value during the training process.

4. According to claim 3, a 2C railway contact network scene depth algorithm based on deep learning is characterized in that: The mean square error formula is: in the formula represents the conventional mean square error term, which is used to measure the difference between the predicted value and the true value, and is the regularization penalty term introduced. This penalty term has a negative sign. When d i With d j When d is the same sign, the term is positive, thus reducing the loss value normally; and when d i With d j When is of opposite sign, the term is negative, thus penalizing the loss value.

Images