A method for identifying pantograph-catenary contact points based on video surveillance cameras

Abstract

The present invention discloses a method for identifying pantograph-catenary contact points based on a video surveillance camera, comprising the following steps: S1, training a pantograph based on images captured by a video surveillance camera; S2, performing online identification of the pantograph to obtain the coordinates of the pantograph-catenary contact points. The present invention introduces multi-template matching during the pantograph positioning phase, improving the stability and accuracy of pantograph positioning. The present invention utilizes template matching in conjunction with the pantograph and prior knowledge to determine a training ROI image, significantly improving processing efficiency compared to using deep learning to determine the training ROI image. The present invention introduces a method for recording relative coordinates when marking training data, which is more adaptable to ROI scaling during training and recognition than directly recording absolute coordinates, thereby improving recognition accuracy. The present invention introduces a soft thresholding function as one of the activation functions in the original residual network's basic block, improving the network's noise resistance and robustness.

Description

Technical Field

[0001] The present invention relates to the field of rail transit pantograph-catenary detection and monitoring, and in particular to a pantograph-catenary contact point identification method based on a video surveillance camera. Background Art

[0002] The pantograph-catenary system is an important equipment in the traction power supply system of electric locomotives. The pantograph-catenary contact point is an important monitoring object during the operation of the pantograph-catenary system. It reflects the status of the pantograph-catenary system during operation, especially the detection and monitoring of the offset of the contact line relative to the pantograph center.

[0003] At present, a lot of research has been carried out both at home and abroad in the field of non-contact detection of bow nets.

[0004] Currently, there are several main detection methods:

[0005] 1) Based on the image processing method, a high-definition camera installed on the roof is used to obtain the pantograph image and calculate the relevant parameters. The detection structure is relatively simple, but the accuracy needs to be improved.

[0006] 2) An array camera and structured light are installed on the top of the inspection vehicle, and on-board dynamic measurement is performed based on line structured light vision measurement technology. This method has the advantages of high measurement accuracy, stability and reliability, and is widely used in lines with low inspection speed requirements. However, this method requires a large amount of image data for a single measurement, and has extremely high requirements for image acquisition and processing, which has certain limitations in high-speed dynamic measurement.

[0007] 3) Use canny edge detection to obtain the edge information of the pantograph and the contact line respectively, and calculate the intersection of the straight lines through Hough line detection to realize the positioning and analysis of the pantograph-catenary contact point.

[0008] 4) Contact point detection based on infrared images uses two directional enhancement operators to realize point detection of pantograph and contact network respectively, and then adopts improved RANSAC strategy to realize the positioning of contact points.

[0009] In terms of contact point detection, the above methods need to further improve their detection accuracy and real-time performance, as well as their ability to adapt to changes in the pantograph-catenary operating environment. Summary of the Invention

[0010] In response to the above-mentioned deficiencies in the prior art, the present invention provides a bow-catcher contact point identification method based on a video surveillance camera, which solves the problems of low contact point detection accuracy and real-time performance, as well as poor adaptability to changes in the bow-catcher operating environment.

[0011] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solution: a method for identifying a pantograph-catenary contact point based on a video surveillance camera, comprising the following steps:

[0012] S1. Construct a matching template set and a deep residual network, and obtain a trained deep residual network through training based on the historical data of pantograph images;

[0013] S2. Based on the trained deep residual network, the pantograph is identified online in real time to obtain the coordinates of the pantograph-net contact point.

[0014] Furthermore: the constructing of the matching template set includes:

[0015] Selecting a plurality of images from the pantograph image history data and extracting pantograph sub-images therefrom;

[0016] Extracting a contour image from a sub-image;

[0017] A matching template set is constructed according to the contour image.

[0018] Furthermore: the construction of the deep residual network includes:

[0019] On the basis of the original basic residual module, a sub-network module is set, and one or more groups of thresholds are automatically learned through the sub-network module;

[0020] By presetting the soft threshold activation function, the noise is reduced, forming the basic module of the deep residual network;

[0021] A deep residual network is formed by stacking multiple deep residual network basic modules.

[0022] Furthermore: the step S1 includes:

[0023] S11, calculating the pantograph center position of each pantograph image historical data according to the pantograph image historical data and the constructed matching template set;

[0024] S12. Selecting an area of interest based on the pantograph center position, and marking the relative position of the pantograph-catenary contact point in the area of interest;

[0025] S13. The relative positions of all marked bow-net contact points are input into the constructed deep residual network for training.

[0026] Furthermore: the step S11 includes:

[0027] S111, matching each pantograph image history data with the matching template set, and selecting the result with the highest matching score, that is, the matching template with the highest score and the matching target corresponding to the matching template;

[0028] S112, selecting the matching target with the highest score from the matching targets with the highest score as the pantograph positioning target of the current image;

[0029] S113. Calculate the pantograph center position based on the pantograph positioning target and the corresponding matching template.

[0030] Furthermore: the step S12 includes:

[0031] S121, determining a region of interest required for subsequent processing based on the pantograph center position and a preset number of up, down, left, and right offset pixels;

[0032] S122. In the image of interest, mark the relative position of the pantograph-catenary contact point in the image of interest.

[0033] Furthermore: the sub-network module includes a 1×1 convolutional layer, a fully connected layer, a 1×1 convolutional layer, a fully connected layer and a 1×1 convolutional layer connected in sequence.

[0034] Furthermore: the step S2 includes:

[0035] S21, acquiring a pantograph image in real time, and identifying the pantograph position through the matching template set;

[0036] S22. Selecting an area of interest based on the identified pantograph position;

[0037] S23: Input the region of interest into the trained deep residual network to obtain the coordinates of the contact points.

[0038] Furthermore: the step S21 includes:

[0039] S211, matching the pantograph image collected in real time with the matching template set, and selecting the result with the highest matching score, that is, the matching template with the highest score and the matching target corresponding to the matching template;

[0040] S212: Selecting the matching target with the highest score from the matching targets with the highest score as the pantograph positioning target of the pantograph image acquired in real time;

[0041] S213. Calculate the pantograph center position based on the pantograph positioning target and the corresponding matching template.

[0042] Furthermore: the step S23 includes:

[0043] Inputting the region of interest into the trained deep residual network to obtain the relative coordinates of the pantograph-catenary contact point in the real-time acquired pantograph image;

[0044] The absolute coordinates of the pantograph-catenary contact point are calculated based on the width and height of the region of interest.

[0045] The beneficial effects of the present invention are:

[0046] 1. The present invention introduces multi-template matching during the pantograph positioning stage, which improves the stability and accuracy of pantograph positioning;

[0047] 2. This invention uses template matching combined with pantograph and prior knowledge to determine the training ROI image, which greatly improves processing efficiency compared to using deep learning (such as YOLO) to determine the training ROI image;

[0048] 3. This invention introduces a method of recording relative coordinates when marking training data. Compared with directly recording absolute coordinates, it is more adaptable to the scaling of ROI during training and recognition, thereby improving recognition accuracy.

[0049] 4. This paper introduces a soft thresholding function into the original residual network as one of the activation functions in the network basic block, which improves the noise resistance and robustness of the network;

[0050] 5. The present invention introduces the attention mechanism into the original residual network, enabling the network to extract more useful feature information while suppressing redundant feature information. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 Flowchart of the present invention;

[0052] Figure 2 This is a schematic diagram of the improved network basic module in the present invention. DETAILED DESCRIPTION

[0053] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0054] like Figure 1 As shown, a method for identifying a pantograph-catenary contact point based on a video surveillance camera includes the following steps:

[0055] S1. Construct a matching template set and a deep residual network, and obtain a trained deep residual network through training based on the historical data of pantograph images;

[0056] Build a matching template set, including:

[0057] Selecting a plurality of images from the pantograph image history data and extracting pantograph sub-images therefrom;

[0058] Extracting a contour image from a sub-image;

[0059] A matching template set is constructed according to the contour image.

[0060] Assume that a total of N matching templates are created, recorded as:

[0061] S Match =[matchModel0 … matchModel n … matchModel N-1 ]

[0062] In the above formula, S Match To match the template set, matchModel n is the nth matching template.

[0063] Build a deep residual network, including:

[0064] On the basis of the original basic residual module, a sub-network module is set, and one or more groups of thresholds are automatically learned through the sub-network module;

[0065] By presetting the soft threshold activation function, the noise is reduced, forming the basic module of the deep residual network;

[0066] A deep residual network is formed by stacking multiple deep residual network basic modules.

[0067] S11, calculating the pantograph center position of each pantograph image historical data according to the pantograph image historical data and the constructed matching template set;

[0068] S111, matching each pantograph image history data with the matching template set, and selecting the result with the highest matching score, that is, the matching template with the highest score and the matching target corresponding to the matching template;

[0069] S112, selecting the matching target with the highest score from the matching targets with the highest score as the pantograph positioning target of the current image;

[0070] S113. Calculate the pantograph center position based on the pantograph positioning target and the corresponding matching template.

[0071] S12. Selecting an area of interest based on the pantograph center position, and marking the relative position of the pantograph-catenary contact point in the area of interest;

[0072] S121, determining a region of interest required for subsequent processing based on the pantograph center position and a preset number of up, down, left, and right offset pixels;

[0073] S122. In the image of interest, mark the relative position of the pantograph-catenary contact point in the image of interest.

[0074] S13. The relative positions of all marked bow-net contact points are input into the constructed deep residual network for training.

[0075] The present invention proposes an improved deep residual network for identifying bow-catenary contact points.

[0076] The present invention introduces an attention mechanism and a soft threshold activation function into the original residual network, so that the original network can discover useful information and enhance this part of information, while suppressing redundant information.

[0077] Soft thresholding can reduce noise in input data by removing features with absolute values less than a certain threshold and shrinking features with absolute values greater than the threshold towards zero. For the task you are working on, this can be achieved as follows:

[0078]

[0079] The derivative of the above soft thresholding output with respect to the input is

[0080]

[0081] This derivative form has the same properties as the ReLU activation function, which can reduce the risk of gradient diffusion and gradient explosion in deep learning algorithms.

[0082] The original residual block and the basic module after the introduction of attention mechanism and soft threshold are as follows Figure 2 As shown. By stacking multiple basic modules, an improved residual network can be formed. In this network, each basic module contains a sub-network, such as Figure 2 As shown in the dashed box, the subnetwork consists of a 1×1 convolutional layer, a fully connected layer, a 1×1 convolutional layer, a fully connected layer, and a 1×1 convolutional layer. This subnetwork automatically learns a set of thresholds for soft thresholding of feature maps. This allows each sample to have a different adaptive threshold, making the network structure most adaptable to sample variability.

[0083] S2. Based on the trained deep residual network, the pantograph is identified online in real time to obtain the coordinates of the pantograph-net contact point.

[0084] S21, acquiring a pantograph image in real time, and identifying the pantograph position through the matching template set;

[0085] S211, matching the pantograph image collected in real time with the matching template set, and selecting the result with the highest matching score, that is, the matching template with the highest score and the matching target corresponding to the matching template;

[0086]

[0087] In the above formula, matchTar n Score is the matching target with the highest score in the matching result of the nth template. n～i is the i-th matching result corresponding to the n-th template, matchTar n～i is the i-th matching target corresponding to the n-th template, and φ is the set of all matching results of the n-th template;

[0088] S212: Selecting the matching target with the highest score from the matching targets with the highest score as the pantograph positioning target of the pantograph image acquired in real time;

[0089]

[0090] In the above formula, matchTar * The pantograph positioning target of the current image, Score n matchTar n The corresponding matching score;

[0091] S213. Calculate the pantograph center position based on the pantograph positioning target and the corresponding matching template.

[0092] S22. Selecting a region of interest (ROI) based on the identified pantograph position;

[0093] According to the center position of the pantograph and combined with prior knowledge of the pantograph and camera, the number of pixels offset up, down, left, and right is manually set to determine the ROI area required for subsequent processing.

[0094]

[0095] S23: Input the region of interest into the trained deep residual network to obtain the coordinates of the contact points.

[0096] Inputting the region of interest into the trained deep residual network to obtain the relative coordinates of the pantograph-catenary contact point in the real-time acquired pantograph image;

[0097] During training, the ROI image may be scaled. In order to adapt to the change in the size of the training image, this solution proposes to record the relative positions of the markers in the ROI.

[0098] After pantograph positioning is performed on each image in the training dataset and the ROI region is determined, the relative coordinates of the pantograph-catenary contact point in the ROI image are manually marked (I i ,(x i / W,y i / H)); where I i is the i-th training image, x i is the horizontal coordinate of the bow-net contact point in the i-th training image, y i is the ordinate of the bow-net contact point in the i-th training image, W is the width, and H is the height of the image of interest.

[0099] The absolute coordinates of the pantograph-catenary contact point are calculated based on the width and height of the region of interest.

[0100] The present invention introduces multi-template matching during the pantograph positioning stage, thereby improving the stability and accuracy of pantograph positioning;

[0101] The present invention uses template matching in combination with pantograph and prior knowledge to determine the training ROI image, which greatly improves the processing efficiency compared to using deep learning (such as YOLO) to determine the training ROI image;

[0102] When marking training data, the present invention introduces a method of recording relative coordinates, which is more adaptable to the scaling of ROI during training and recognition, and improves recognition accuracy compared to directly recording absolute coordinates.

[0103] The present invention introduces a soft thresholding function into the original residual network as one of the activation functions in the network basic block, thereby improving the noise resistance and robustness of the network.

[0104] The present invention introduces an attention mechanism into the original residual network, enabling the network to extract more useful feature information while suppressing redundant feature information.

Claims

1. A method for identifying pantograph-catenary contact points based on a video surveillance camera, characterized in that: The following steps are involved: S1. Construct a matching template set and a deep residual network, and obtain a trained deep residual network through training based on the historical data of pantograph images; S2. Based on the trained deep residual network, the pantograph is identified online in real time to obtain the coordinates of the pantograph-net contact point; The constructing of the matching template set includes: Selecting a plurality of images from the pantograph image history data and extracting pantograph sub-images therefrom; Extracting a contour image from a sub-image; Constructing a matching template set according to the contour image; The constructing of a deep residual network includes: On the basis of the original basic residual module, a sub-network module is set, and one or more groups of thresholds are automatically learned through the sub-network module; By presetting the soft threshold activation function, the noise is reduced, forming the basic module of the deep residual network; Forming a deep residual network by stacking multiple deep residual network basic modules; The sub-network module includes a 1×1 convolutional layer, a fully connected layer, a 1×1 convolutional layer, a fully connected layer and a 1×1 convolutional layer connected in sequence; The step S1 comprises: S11, calculating the pantograph center position of each pantograph image historical data according to the pantograph image historical data and the constructed matching template set; S12. Selecting an area of interest based on the pantograph center position, and marking the relative position of the pantograph-catenary contact point in the area of interest; S13. The relative positions of all marked bow-net contact points are input into the constructed deep residual network for training.

2. The pantograph-catenary contact point identification method based on a video surveillance camera according to claim 1, characterized in that: The step S11 includes: S111, matching each pantograph image history data with the matching template set, and selecting the result with the highest matching score, that is, the matching template with the highest score and the matching target corresponding to the matching template; S112, selecting the matching target with the highest score from the matching targets with the highest score as the pantograph positioning target of the current image; S113. Calculate the pantograph center position based on the pantograph positioning target and the corresponding matching template.

3. The pantograph-catenary contact point identification method based on a video surveillance camera according to claim 1, characterized in that: The step S12 includes: S121, determining a region of interest required for subsequent processing based on the pantograph center position and a preset number of up, down, left, and right offset pixels; S122. In the image of interest, mark the relative position of the pantograph-catenary contact point in the image of interest.

4. The method for identifying pantograph-catenary contact points based on a video surveillance camera according to claim 1, characterized in that: The step S2 includes: S21, acquiring a pantograph image in real time, and identifying the pantograph position through the matching template set; S22. Selecting an area of interest based on the identified pantograph position; S23: Input the region of interest into the trained deep residual network to obtain the coordinates of the contact points.

5. The method for identifying pantograph-catenary contact points based on a video surveillance camera according to claim 4, characterized in that: The step S21 includes: S211, matching the pantograph image collected in real time with the matching template set, and selecting the result with the highest matching score, that is, the matching template with the highest score and the matching target corresponding to the matching template; S212: Selecting the matching target with the highest score from the matching targets with the highest score as the pantograph positioning target of the pantograph image acquired in real time; S213. Calculate the pantograph center position based on the pantograph positioning target and the corresponding matching template.

6. The method for identifying pantograph-catenary contact points based on a video surveillance camera according to claim 4, characterized in that: The step S23 includes: Inputting the region of interest into the trained deep residual network to obtain the relative coordinates of the pantograph-catenary contact point in the real-time acquired pantograph image; The absolute coordinates of the pantograph-catenary contact point are calculated based on the width and height of the region of interest.

Images