An online detection method for pantograph-catenary operation status based on key point detection

Abstract

The present invention discloses an online detection method for the operation status of pantograph-catenary based on key point detection, including the following steps: Install binocular cameras on the left and right sides of the pantograph-catenary of the train, synchronously trigger the binocular cameras, and respectively collect the left and right images of the train at the same moment. Subsequently, obtain the two-dimensional left and right corner points in the left and right images respectively, and find the midpoints of the corresponding left and right corner points in the left and right images respectively. The coordinates are recorded as (x 2d , y 2d , z 2d ). Denote the midpoint of the left and right corner points as the projection point of the contact point on the image. Perform three-dimensional reconstruction on the projection points of the contact points in the left and right images to obtain the three-dimensional contact point, and the coordinates are recorded as (x 3d , y 3d , z 3d ). Denote z 3d as the dynamic contact height. The difference between the x 3d of the three-dimensional contact point and the reconstructed pantograph center point is denoted as the dynamic pull-out value. The online detection method for the operation status of pantograph-catenary based on key point detection of the present invention can achieve high-precision pantograph-catenary contact corner point positioning under various complex line-changing modes, can measure the contact height and pull-out value in modes such as double branch lines, and can give an alarm for abnormal contact.

Description

Technical Field

[0001] The present invention belongs to the technical field of online detection of pantograph-catwalk operating conditions for intelligent operation and maintenance of railway trains, and in particular relates to an online detection method for pantograph-catwalk operating conditions based on key point detection. Background Art

[0002] Pantographs and contact lines are the main equipment for electrified railway power supply. At present, the high speed and heavy load of railway trains have put forward more stringent requirements for the power supply system. Ensuring that the pantograph-catenary system has a stable and good contact state is an important prerequisite for maintaining a good dynamic characteristic of the pantograph-catenary relationship and ensuring uninterrupted power supply and safe and stable operation of high-speed trains. In the daily operation and maintenance of railways, the common pantograph-catenary operation status detection is to measure some main static parameters of the contact line, including contact line height, pull-out value, positioning tube slope, support position, etc. Among them, the height of the contact line and the pull-out value are the most important evaluation indicators for evaluating the pantograph-catenary system. The height of the contact line is the vertical height of the contact line from the plane of the rail. If the height of the contact line is too high, the pantograph will be frequently offline to generate arcs and burn the contact line and pantograph. If it is too low, it will endanger the safety of personnel and oversized goods. The pull-out value is the distance the contact wire is pulled outward relative to the center of the rail. It is designed to make the pantograph and the contact wire rub evenly within a certain range when the electric locomotive is running, so that the pantograph slide plate will wear evenly and extend the service life. If the pull-out value is too small, the local wear of the pantograph slide plate will increase, and if it is too large, it will cause accidents such as pantograph scraping and wire breaking. Therefore, not only do these static parameters need to be tested in daily parking operation and maintenance, but online measurement of dynamic conductor height and pull-out value during operation can better ensure the safe and high-quality operation of the contact network in the electrified section. Therefore, real-time online detection of the operation status of the train pantograph network is of great significance.

[0003] The measurement of the lead height and the pull-out value are both achieved under the premise of accurately locating the contact point. The current non-contact detection method is to monitor and record the online bow-net operation status through a monocular camera installed on the roof, which cannot meet the requirements of intelligent operation and maintenance. Moreover, the above-mentioned method remains in a single-line mode for the detection of the bow-net contact point, but the actual operation will involve frequent line changes, double branch line contact and other complex on-site situations, which poses new challenges to the online detection of the bow-net operation status. From the perspective of three-dimensional reconstruction accuracy, there is currently no relevant method for evaluating the reconstruction accuracy of the bow-net contact point, so it is impossible to judge the accuracy of parameters such as the lead height or the pull-out value. Therefore, high-precision parameter measurement is also the focus of the present invention.

[0004] The image processing method based on edge detection extracts the edges of the contact wire and the pantograph, and the intersection of the edge straight lines is defined as the pantograph-catwalk contact point. A large number of background edge clutter interferences make this positioning method based on edge detection not robust in complex backgrounds.

[0005] Automatic positioning of the pantograph-net contact point is achieved based on the mode of tracking first and then positioning. This mode first uses template matching to extract the edges of the pantograph and the contact line. The intersection area is the initial tracking area. Then, the contact area of ​​each frame is continuously positioned through KCF tracking. Finally, the contact point is positioned through corner point detection. The highlight of this method is its fast speed (greater than 200FPS), but template matching involves many parameter adjustments, which makes it difficult to meet the positioning requirements of all-weather and long-term contact points. Summary of the invention

[0006] In order to solve the technical problems existing in the prior art, the purpose of the present invention is to provide an online detection method for the operation status of a bow-catenary based on key point detection, which is used to solve the measurement of high-speed guide height and pull-out value on the railway main line.

[0007] In order to achieve the above purpose and the above technical effect, the technical solution adopted by the present invention is:

[0008] An online detection method for the operation status of a pantograph and a catenary based on key point detection comprises the following steps:

[0009] The binocular cameras are installed on the left and right sides of the top of the train pantograph network, facing the monitoring area of ​​the pantograph network. Then the binocular cameras are triggered synchronously to collect the left and right images of the train at the same time. Then, the two-dimensional left and right corner points are obtained in the left and right images respectively. The midpoints of the left and right corner points corresponding to each other in the left and right images are calculated respectively, and the coordinates are marked as (x 2d ,y 2d ,z 2d ), the midpoint of the left and right corners is recorded as the projection point of the contact point on the image, and the projection points of the contact points in the left and right images are reconstructed to obtain the three-dimensional contact point, and the coordinates are marked as (x 3d ,y 3d ,z 3d ), where z 3d Denoted as dynamic height, x of the three-dimensional contact point 3d The difference with the reconstructed pantograph center point is recorded as the dynamic pull-out value.

[0010] Furthermore, obtaining two-dimensional left and right corner points in the left and right images respectively specifically includes:

[0011] 1) Input the left and right images of the train at the same time into the trained high-precision 2D key point positioning network;

[0012] The 2D key point positioning network includes a backbone network resnet18, three deconvolution layers deconv1, deconv2, deconv3, and several auxiliary feature enhancement modules that have not been pre-trained, and the auxiliary feature enhancement modules are located after layer1 of the backbone network resnet18;

[0013] 2) The features of the left and right images are extracted through the 2D key point positioning network, and the features in the backbone network resnet18 are extracted through the auxiliary feature enhancement module. The extracted features are then added to the features of layer2, layer3 and layer4 in the backbone network resnet18, and the deconvolution layer is output to obtain the parameters of the bounding box and the parameters of the key points and regress them to determine the position of the key points. Finally, after decoding, the two-dimensional left and right corner points corresponding to the left and right images are output respectively.

[0014] Furthermore, there are three auxiliary feature enhancement modules, namely ANlayer1, ANlayer2, and ANlayer3. The input of ANlayer1 is the output of layer1. The inputs of Anlayer2 and layer3 are the concatenation of the features of ANlayer1 starting from 0 and the pre-trained features of layer2, and they have both intermediate and advanced features. The inputs of Anlayer3 and layer4 are the addition of the features of Anlayer2 and the pre-trained features of layer3, and the output of layer4 is the deconvolution layers deconv1, deconv2, and deconv3.

[0015] Furthermore, in step 2), the parameters of the bounding box include: target heat map, width and height, and offset of the center of the box; the parameters of the key points include: key point heat map, key point connection heat map, and key point offset.

[0016] Furthermore, wingloss is used to regress the offset of the center of the bounding box and the offset of the key point respectively. The formula of wingloss is:

[0017]

[0018] Among them, ω=2, ε=2.

[0019] Furthermore, the improved bbox The function regresses the width and height of the bounding box, and the formula is as follows:

[0020]

[0021] Among them A p , A gt are the areas of the prediction box and the true value box respectively; the minimum convex set of the prediction box and the true value box is The area is A c ; C w , C h are the width and height of the bounding box of the minimum convex set, ρ represents the Euclidean distance between two values, ρ 2 (w,wgt ) represents the square of the Euclidean distance between the current regression width and the true value width, ρ 2 (h,h gt ) represents the square of the Euclidean distance between the current regressed height and the true height.

[0022] Furthermore, the loss function of the bounding box includes: high-precision bounding box regression and high-precision key point regression, and the overall loss function is:

[0023] loss=(l hm +l bbox +l offset )+(l hm-hp +l hp +l hp-offset )

[0024] Among them, l bbox is the regression loss function of the width and height of the bounding box, l offset is the regression loss function of the offset of the center of the bounding box; l hm-hp +l hp +l hp-offset They represent the regression of keypoint heatmap, pixel-level keypoints, and keypoint offsets respectively.

[0025] The present invention discloses an online detection system for the operation status of a pantograph and a catenary based on key point detection, comprising:

[0026] Binocular cameras are installed on the left and right sides of the top of the train pantograph network, facing the monitoring area of ​​the pantograph network. The camera located on the left side of the pantograph network collects the left image, and the camera located on the right side of the pantograph network collects the right image.

[0027] A key point detection module is configured to extract two-dimensional left and right corner points in an image acquired by a binocular camera using a 2D key point positioning network;

[0028] The 2D key point positioning network is used to extract the contact angle point where the pantograph of a high-speed train intersects with the contact line. It includes a backbone network resnet18, three deconvolution layers of deconv1, deconv2, and deconv3, and three auxiliary feature enhancement modules ANlayer1, ANlayer2, and ANlayer3 that have not been pre-trained. Features in the backbone network resnet18 are extracted through ANlayer1, ANlayer2, and ANlayer3. The extracted features are then added to the features of layer2, layer3, and layer4 in the backbone network resnet18, and the deconvolution layers deconv1, deconv2, and deconv3 are output to obtain the parameters of the bounding box and the parameters of the key points and regress them to determine the position of the key points. Finally, after decoding, the two-dimensional left and right corner points corresponding to the left and right images are output respectively.

[0029] The present invention also discloses an electronic device, comprising:

[0030] processor;

[0031] A memory, the memory being used to store instructions executable by the processor;

[0032] The processor is configured to execute the instructions to implement the above-mentioned online detection method of the pantograph-catenary operating condition based on key point detection.

[0033] The present invention also discloses a storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the above-mentioned online detection method of the operating status of the bow-catwalk based on key point detection.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The present invention discloses an online detection method for the operation status of a pantograph network based on key point detection, comprising the following steps: installing binocular cameras on the left and right sides of the top of the train pantograph network, facing the monitoring area of ​​the pantograph network, then synchronously triggering the binocular cameras, respectively collecting left and right images of the train at the same time, then respectively obtaining two-dimensional left and right corner points in the left and right images, respectively calculating the midpoints of the left and right corner points corresponding to each other in the left and right images, and the coordinates are marked as (x 2d ,y 2d ,z 2d ), the midpoint of the left and right corners is recorded as the projection point of the contact point on the image, and the projection points of the contact points in the left and right images are reconstructed to obtain the three-dimensional contact point, and the coordinates are marked as (x 3d ,y 3d ,z 3d ), where z 3dDenoted as dynamic height, x of the three-dimensional contact point 3d The difference with the reconstructed pantograph center point is recorded as the dynamic pull-out value. The present invention provides an online detection method for the operation status of the pantograph-catenary network based on key point detection, which is used to solve the measurement of high-speed conductor height pull-out value on the railway main line, can realize high-precision pantograph-catenary contact angle point positioning in various complex line-changing modes, and can realize the measurement of conductor height and pull-out value in double branch line and other modes as well as abnormal contact alarm. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A network framework diagram for positioning key points of the present invention;

[0037] Figure 2 Schematic diagram of binocular stereo vision of the three-dimensional reconstruction method of the present invention. DETAILED DESCRIPTION

[0038] The present invention is described in detail below so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0039] A brief summary of one or more aspects is given below to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceived aspects, and is neither intended to identify the key or critical elements of all aspects nor to define the scope of any or all aspects. Its only purpose is to give some concepts of one or more aspects in a simplified form as a prelude to a more detailed description that will be given later.

[0040] Example 1

[0041] like Figure 1-2 As shown, a method for online detection of the operation status of a pantograph and a catenary based on key point detection comprises the following steps:

[0042] First, install the binocular camera on the top of the train, with the binocular camera facing the monitoring area of ​​the pantograph network. The binocular camera is divided into a left camera and a right camera, and the distance between the two is as large as possible, but the public field of view must include the complete pantograph and contact area;

[0043] Then, the binocular cameras are triggered synchronously to collect the left and right images of the train at the same time. The camera on the left side of the pantograph collects the left image, and the camera on the right side of the pantograph collects the right image. Then, the two-dimensional left and right corner points are obtained in the left and right images respectively, and the midpoints of the left and right corner points corresponding to each other in the left and right images are calculated respectively, and the coordinates are marked as (x 2d ,y 2d ,z 2d); the midpoint of the left and right corners is recorded as the projection point of the contact point on the image, and the projection point of the contact point in the left and right images is reconstructed by a 3D reconstruction method (firstly, the binocular camera is calibrated using the Zhang Zhengyou calibration method to obtain the intrinsic parameters of the binocular camera and the rotation matrix R and translation matrix T of the right camera relative to the left camera) to obtain the 3D contact point, and the coordinates are marked as (x 3d ,y 3d ,z 3d ), where z 3d Denoted as dynamic height, x of the three-dimensional contact point 3d The difference with the reconstructed pantograph center point is recorded as the dynamic pull-out value.

[0044] The three-dimensional reconstruction method includes the following steps: after obtaining the coordinates of the two-dimensional corner points in the images taken by the left and right cameras, the coordinates of the three-dimensional points in the world coordinate system are reconstructed using the binocular stereo vision principle, such as Figure 2 As shown;

[0045] Assume that the left camera O-xyz is located at the origin of the world coordinate system and has no rotation, and the image coordinate system is O l -X l Y l , the effective focal length is f l ; The right camera coordinate system is o r -x r y r z r , the image coordinate system is O r -X r Y r , the camera perspective transformation model is:

[0046]

[0047]

[0048] The relationship between the left camera coordinate system and the right camera coordinate system is expressed by the rotation matrix R and the translation vector T as follows:

[0049]

[0050] The three-dimensional point coordinates are calculated as follows:

[0051]

[0052] The three-dimensional reconstruction method can adopt existing technology.

[0053] The two-dimensional left and right corner points in the left and right images are obtained by the following steps:

[0054] Design a high-precision 2D key point positioning network, namely the AN-CenterNet sub-pixel key point detection model, to extract the contact angle points where the pantograph of a high-speed train intersects with the contact line:

[0055] The AN-CenterNet sub-pixel key point detection model includes the backbone network resnet18, three deconvolution layers (deconv1, deconv2, deconv3) and three auxiliary feature enhancement modules (ANlayer1, ANlayer2, ANlayer3) that have not been pre-trained. Each of the above structures is composed of several convolutional layers and deconvolutional layers.

[0056] At present, the most advanced feature enhancement methods mostly use multi-branch dilated convolution with bottleneck design to expand the receptive field of the feature layer and reduce the number of parameters of the newly added module, or use channel attention and spatial attention to simulate the focus of human vision on the target, but such module improvements add a great burden to memory operations because they add a large number of pixel-level operations and operations of different branch re-integration connections, thereby reducing the reasoning speed of the network, even if these networks have very low parameter counts. We know that the cudnn acceleration operation is very fast for 3*3 convolution processing, so in order not to introduce redundant and complex bottleneck branches and attention mechanisms, the present invention uses three minimalist conv3*3 convolution feature extraction modules to extract features from the backbone network resnet18. These features will be simply added to the features of layer2, layer3 and layer4 in the original resnet, respectively. This feature superposition method can combine the advantages of pre-trained features and features trained from scratch.

[0057] The input of ANlayer1 is the output of layer1, and the input received by Anlayer2 is the concatenation of the features of Anlayer1 trained from scratch and the features of pre-trained layer2. Not only does the input of Anlayer2 effectively fuse the two, but the input of layer3 also changes from a single forward-transferred layer2 feature to the superposition of the pre-trained layer2 feature and ANlayer1 trained from scratch. The output of layer4 is deconvolution deconv1, deconv2, and deconv3. The design of this minimalist auxiliary network is mainly to make up for the residual module in resnet's ability to understand low-level, intermediate, and high-level semantic information. We believe that the efficient expression of high-level features comes from the fusion of multi-level features. For example, layer2 contains two residual modules, and the features of its output can be understood as strong intermediate features. ANlayer1 uses 3*3 convolution to extract low-level semantic information to a slightly weaker degree, so that the inputs of layer3 and ANlayer2 have both intermediate and low-level features, thereby improving the expressive power of the features;

[0058] Through the 2D key point positioning network output deconvolution deconv1, deconv2, deconv3, the parameters of the bounding box and the key point are obtained and regressed to determine the position of the key point. Finally, after decoding, the two-dimensional left and right corner points corresponding to the left and right images are output respectively.

[0059] Convolution layer: A layer used to extract features in a convolutional neural network. It is used to extract low-dimensional features from high-dimensional data. It is divided into three parts: convolution operation, activation operation, and pooling operation. When performing a convolution operation, a convolution kernel that has been trained and learned in advance is used to extract features. When performing an activation operation, an activation function is used to activate the feature map obtained by convolution. Common activation functions include the Rectified Linear Unit (ReLU) function, the Sigmoid function, and the Tanh function. After the convolution activation layer, the feature vector of the convolution activation output is reduced through pooling operations (i.e., the size of the feature map is reduced), while improving the overfitting problem. Commonly used pooling methods include mean-pooling, max-pooling, and stochastic-pooling.

[0060] Deconvolution layer: The function of the deconvolution layer is opposite to that of the convolution layer. The deconvolution layer is used to map low-dimensional features into high-dimensional outputs. In the present invention, the deconvolution layer is a layer used to upsample the feature vector, that is, to increase the size of the feature map;

[0061] Stitching: refers to the process of fusing two multi-channel feature maps of the same size, where the number of channels of the stitched feature map is equal to the sum of the number of channels of the two stitched feature maps;

[0062] Residual layer: A layer structure that extracts features through a residual network (Residual Network, ResNet) structure. In the present invention, we use resnet18 for backbone network feature extraction, which is used to extract features from the pantograph image and determine the key points of the pantograph-net contact area.

[0063] In order to improve the sub-pixel key point positioning accuracy, the present invention improves the loss function of key point regression, specifically including:

[0064] The tasks of this step mainly include two parts: regression of bounding box related parameters and regression of key point related parameters. Among them, the parameters of the bounding box are the three parts of the anchor-free method: target heat map, width and height, and offset of the center of the box; key points require more sophisticated regression, including: key point heat map, key point connection heat map, and key point offset. The following is a two-step introduction to the loss improvement strategy:

[0065] Bounding box (contact area) loss:

[0066] The target heat map still uses the original focal loss in CenterNet, because it solves the problem of imbalance between positive and negative samples to a certain extent. For the detection of the central key point of the bounding box, a more refined regression is required, because the offset regression of the subsequent contact key point is calculated based on the center point. In the task of facial key point regression, the regression difficulty of each key point is different. In the early stage of training, the errors of all points are large. In the middle and late stages of training, most points are basically accurate, but not accurate enough, and the errors are small. In order to regress more accurately, its loss must be amplified. L1 Loss is more sensitive to larger errors, but less sensitive to smaller errors, which results in the inability to obtain fine results for key points. Therefore, considering increasing the focus on small and medium errors, the present invention uses wingloss to regress the offset of the center of the bounding box instead of L1 Loss. The specific formula is as follows:

[0067]

[0068] Among them, ω=2, ε=2.

[0069] For the width and height of the bounding box, the original centernet uses L1 Loss to regress its values, which not only lacks position information, but also greatly introduces manual annotation errors. bbox Regress the bounding box position. Based on the width and height w and h output by the network, combined with the output heat map and offset, the prediction box is initially decoded. The information is then calculated by comparing it with the true value frame The loss function can achieve a more accurate regression bounding box. We not only use GIoU to obtain the position information between the two boxes, but also add aspect loss L to predict the width and height of the box. w&h .

[0070] Complete The function formula is as follows:

[0071]

[0072] Among them, A p , A gt are the areas of the prediction box and the true value box respectively; the minimum convex set of the prediction box and the true value box is The area is A c ; C w , C h is the width and height of the bounding box of the minimum convex set, that is ρ represents the Euclidean distance between two values; ρ 2 (w,w gt ) represents the square of the Euclidean distance between the current regression width and the true value width, ρ 2 (h,h gt ) represents the square of the Euclidean distance between the current regression height and the true value height; by adding the loss function l for width and height w&h The difference between the width and height of the target box and the anchor box is minimized, leading to faster convergence and better localization results. We designed L bbox It not only takes advantage of iouloss in position regression, but also does not discard the numerical information of w&h.

[0073] The loss function of the bounding box mainly consists of two parts: high-precision bounding box regression and high-precision key point regression. Accurate bounding box regression can provide good bounding box center point coordinates for the required key point regression. In the process of key point regression, its offset is the offset of the point from the bounding box center. The overall loss function is:

[0074] loss=(l hm +l bbox +l offset )+(l hm-hp +lhp +l hp-offset )

[0075] Among them, l bbox is the regression loss function of the width and height of the bounding box, l offset is the regression loss function of the offset of the center of the bounding box; l hm-hp +l hp +l hp-affset They represent the regression of key point heat map, pixel-level key points, and key point offset respectively. hm-hp and l hp Focal loss is used in both hp-offset Continue using wingloss.

[0076] The present invention also discloses an online detection system for the operation status of a pantograph and a catenary based on key point detection, comprising:

[0077] Binocular camera: The binocular camera is installed on the left and right sides of the top of the train pantograph network, facing the monitoring area of ​​the pantograph network. The binocular camera is divided into a left camera and a right camera, and the distance between the two is as large as possible, but the public field of view must include the complete pantograph and the contact area. The camera on the left side of the pantograph network collects the left image, and the camera on the right side of the pantograph network collects the right image;

[0078] A key point detection module is configured to extract two-dimensional left and right corner points in an image acquired by a binocular camera using a 2D key point positioning network;

[0079] The 2D key point positioning network, namely the AN-CenterNet sub-pixel key point detection model, is used to extract the contact corners where the pantograph of a high-speed train intersects with the contact line: the AN-CenterNet sub-pixel key point detection model includes a backbone network resnet18, three deconvolution layers (deconv1, deconv2, deconv3) and three auxiliary feature enhancement modules (ANlayer1, ANlayer2, ANlayer3) that have not been pre-trained. Each of the above structures is composed of several convolution layers and deconvolution layers. The three auxiliary feature enhancement modules are used to extract the features in the backbone network resnet18. The extracted features are then added to the features of layer2, layer3 and layer4 in the backbone network resnet18, and the deconvolution deconv1, deconv2 and deconv3 are output to obtain the parameters of the bounding box and the parameters of the key points and perform regression to determine the position of the key points. Finally, after decoding, the two-dimensional left and right corner points corresponding to the left and right images are output respectively.

[0080] The present invention also discloses an electronic device, comprising:

[0081] processor;

[0082] Memory, which is used to store instructions executable by the processor;

[0083] The processor is configured to execute the instructions to implement the above-mentioned online detection method of the pantograph-catenary operating condition based on key point detection.

[0084] The present invention also discloses a storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the above-mentioned online detection method of the operating status of the bow-catwalk based on key point detection.

[0085] Parts or structures not specifically described in the present invention may adopt existing technologies or existing products and will not be described in detail here.

[0086] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An online detection method for the operation status of a pantograph and a catenary based on key point detection, characterized in that: The following steps are involved: The binocular cameras are installed on the left and right sides of the top of the train pantograph network, facing the monitoring area of ​​the pantograph network. Then the binocular cameras are triggered synchronously to collect the left and right images of the train at the same time. Then, the two-dimensional left and right corner points are obtained in the left and right images respectively, and the midpoints of the corresponding left and right corner points in the left and right images are calculated respectively. The coordinates are marked as , the midpoint of the left and right corners is recorded as the projection point of the contact point on the image, and the projection points of the contact points in the left and right images are reconstructed to obtain the three-dimensional contact point, and the coordinates are marked as ,in, Recorded as dynamic height, three-dimensional contact point The difference with the reconstructed pantograph center point is recorded as the dynamic pull-out value; The two-dimensional left and right corner points are obtained in the left and right images respectively, specifically including: 1) Input the left and right images of the train at the same time into the trained high-precision 2D key point positioning network; The 2D key point positioning network includes a backbone network resnet18, three deconvolution layers deconv1, deconv2, deconv3, and several auxiliary feature enhancement modules that have not been pre-trained, and the auxiliary feature enhancement modules are located after layer1 of the backbone network resnet18; 2) The features of the left and right images are extracted through the 2D key point positioning network, and the features in the backbone network resnet18 are extracted through the auxiliary feature enhancement module. The extracted features are then added to the features of layer2, layer3 and layer4 in the backbone network resnet18, and the deconvolution layer is output to obtain the parameters of the bounding box and the parameters of the key points and regress them to determine the position of the key points. Finally, after decoding, the two-dimensional left and right corner points corresponding to the left and right images are output respectively; There are three auxiliary feature enhancement modules, namely ANlayer1, ANlayer2, and ANlayer3. The input of ANlayer1 is the output of layer1. The inputs of Anlayer2 and layer3 are the concatenation of the features of ANlayer1 starting from 0 and the features of pre-trained layer2, and they have both intermediate and advanced features. The inputs of Anlayer3 and layer4 are the addition of the features of Anlayer2 and the features of pre-trained layer3. The output of layer4 is the deconvolution layers deconv1, deconv2, and deconv3. In step 2), the parameters of the bounding box include: target heat map, width and height, and offset of the center of the box; the parameters of the key points include: key point heat map, key point connection heat map, and key point offset; Wingloss is used to regress the offset of the center of the bounding box and the offset of the key point respectively. The formula of wingloss is: ； in, ; Using improved The function regresses the width and height of the bounding box, and the formula is as follows: ； in are the areas of the prediction box and the true value box respectively; the minimum convex set of the prediction box and the true value box is , the area is are the width and height of the bounding box of the minimum convex set, ; Represents the Euclidean distance between two values, Represents the square of the Euclidean distance between the current regression width and the true value width. Represents the square of the Euclidean distance between the current regression height and the true height; The loss function of the bounding box includes: high-precision bounding box regression and high-precision key point regression, and the overall loss function is: ； in, is the regression loss function of the width and height of the bounding box, is the regression loss function of the offset of the center of the bounding box; They represent the regression of keypoint heatmap, pixel-level keypoints, and keypoint offsets respectively.

2. An online detection system for the operation status of a pantograph and a catenary based on key point detection, characterized in that: include: Binocular cameras are installed on the left and right sides of the top of the train pantograph network, facing the monitoring area of ​​the pantograph network. The camera located on the left side of the pantograph network collects the left image, and the camera located on the right side of the pantograph network collects the right image. A key point detection module is configured to extract two-dimensional left and right corner points in an image acquired by a binocular camera using a 2D key point positioning network; The 2D key point positioning network is used to extract the contact angle point where the pantograph of a high-speed train intersects with the contact line, including a backbone network resnet18, three deconvolution layers of deconv1, deconv2, and deconv3, and three auxiliary feature enhancement modules ANlayer1, ANlayer2, and ANlayer3 that have not been pre-trained. The features in the backbone network resnet18 are extracted by ANlayer1, ANlayer2, and ANlayer3, and the extracted features are respectively added to the features of layer2, layer3, and layer4 in the backbone network resnet18, and the deconvolution layers deconv1, deconv2, and deconv3 are output to obtain the parameters of the bounding box and the parameters of the key points and regress them to determine the position of the key points, and finally, after decoding, the two-dimensional left and right corner points corresponding to the left and right images are output respectively; The two-dimensional left and right corner points are obtained in the left and right images respectively, specifically including: 1) Input the left and right images of the train at the same time into the trained high-precision 2D key point positioning network; The 2D key point positioning network includes a backbone network resnet18, three deconvolution layers deconv1, deconv2, deconv3, and several auxiliary feature enhancement modules that have not been pre-trained, and the auxiliary feature enhancement modules are located after layer1 of the backbone network resnet18; 2) The features of the left and right images are extracted through the 2D key point positioning network, and the features in the backbone network resnet18 are extracted through the auxiliary feature enhancement module. The extracted features are then added to the features of layer2, layer3 and layer4 in the backbone network resnet18, and the deconvolution layer is output to obtain the parameters of the bounding box and the parameters of the key points and regress them to determine the position of the key points. Finally, after decoding, the two-dimensional left and right corner points corresponding to the left and right images are output respectively; There are three auxiliary feature enhancement modules, namely ANlayer1, ANlayer2, and ANlayer3. The input of ANlayer1 is the output of layer1. The inputs of Anlayer2 and layer3 are the concatenation of the features of ANlayer1 starting from 0 and the features of pre-trained layer2, and they have both intermediate and advanced features. The inputs of Anlayer3 and layer4 are the addition of the features of Anlayer2 and the features of pre-trained layer3. The output of layer4 is the deconvolution layers deconv1, deconv2, and deconv3. In step 2), the parameters of the bounding box include: target heat map, width and height, and offset of the center of the box; the parameters of the key points include: key point heat map, key point connection heat map, and key point offset; Wingloss is used to regress the offset of the center of the bounding box and the offset of the key point respectively. The formula of wingloss is: ； in, ; Using improved The function regresses the width and height of the bounding box, and the formula is as follows: ； in are the areas of the prediction box and the true value box respectively; the minimum convex set of the prediction box and the true value box is , the area is are the width and height of the bounding box of the minimum convex set, ; Represents the Euclidean distance between two values, Represents the square of the Euclidean distance between the current regression width and the true value width. Represents the square of the Euclidean distance between the current regression height and the true height; The loss function of the bounding box includes: high-precision bounding box regression and high-precision key point regression, and the overall loss function is: ； in, is the regression loss function of the width and height of the bounding box, is the regression loss function of the offset of the center of the bounding box; They represent the regression of keypoint heatmap, pixel-level keypoints, and keypoint offsets respectively.

3. An electronic device, characterized in that: include: processor; A memory, the memory being used to store instructions executable by the processor; The processor is configured to execute the instructions to implement the online detection method of the pantograph-catenary operating condition based on key point detection as described in claim 1.

4. A storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the online detection method of the bow-catwalk operating status based on key point detection as described in claim 1.

Images