A pantograph abnormal tilt detection method and device

Abstract

The present invention provides a method and device for detecting abnormal inclination of a pantograph. The method includes: extracting the two-dimensional rectangular coordinate frame information of the pantograph of the current pantograph image belonging to the clean pantograph category; performing pixel segmentation on the lower arm rod of the pantograph in the current pantograph image to obtain a lower arm rod mask image; extracting four boundary contour line segments of the lower arm rod of the pantograph and calculating the slope values of two boundary lines; calculating the relative inclination angle of the lower arm rod of the current pantograph, comparing the relative inclination angle of the lower arm rod of the current pantograph with a set inclination angle threshold, and judging whether there is abnormal inclination of the current pantograph according to the comparison result, and labeling the inclination label of the pantograph on the current pantograph image by using the two-dimensional rectangular coordinate frame information of the pantograph. The method of the present invention solves the problems of inaccurate and untimely pantograph detection, and realizes accurate and real-time detection of abnormal inclination of the pantograph by excluding external factor interference.

Description

Technical Field

[0001] The present invention relates to the technical field of pantographs, and in particular, to a method and device for detecting abnormal inclination of a pantograph. Background Art

[0002] A pantograph is a power supply device installed on the roof of a rail transit train to collect current from a catenary. It is a key component for the power supply of the train, and the operating state of the pantograph directly affects the safe operation of the vehicle. During the high-speed operation of the train, the pantograph experiences frequent raising and lowering actions, and is continuously subjected to vibration impacts from the catenary, as well as unevenness of the catenary and inclination of the train itself. Many factors may cause the pantograph to incline. The inclination of the pantograph head not only affects the contact stability between the pantograph and the catenary, resulting in a decrease in power transmission efficiency, but may also cause pantograph-catenary faults and even endanger the safety of train operation. Therefore, it is of great significance to detect the inclination state of the pantograph in real time.

[0003] Currently, a method for detecting abnormal inclination of a pantograph in the prior art is an image-based detection method. This detection method takes images of the pantograph through a high-speed camera and uses image processing algorithms for analysis and processing to determine whether it is inclined. The disadvantages of this detection method include being greatly affected by environmental factors such as light and weather.

[0004] Another method for detecting abnormal inclination of a pantograph in the prior art is a sensor-based detection method. This detection method directly measures the inclination angle of the pantograph head through sensors installed on the pantograph. The disadvantages of this method include high cost and complex installation and maintenance.

[0005] In summary, the existing methods for detecting abnormal inclination of a pantograph currently have problems such as low detection accuracy, poor real-time performance, and susceptibility to environmental interference, and it is difficult to meet the safety operation requirements of modern urban rail transit. Summary of the Invention

[0006] Embodiments of the present invention provide a method and device for detecting abnormal inclination of a pantograph to effectively detect whether the pantograph is abnormally inclined.

[0007] To achieve the above object, the present invention adopts the following technical solutions.

[0008] According to one aspect of the present invention, there is provided a method for detecting abnormal inclination of a pantograph, including:

[0009] Real-time collection of current pantograph video data through a pantograph image collection device, and performing a frame extraction operation on the current pantograph video data to obtain a current pantograph image;

[0010] Determine whether the current pantograph image belongs to the category of lens dirt or the category of clean pantographs. Extract the two-dimensional rectangular coordinate frame information of the pantograph in the current pantograph image belonging to the category of clean pantographs through the pantograph key area detection module;

[0011] Perform pixel segmentation on the lower arm rod of the pantograph in the current pantograph image through the lower arm rod pixel segmentation module of the pantograph,

[0012] to obtain the lower arm rod mask image;

[0013] Extract four boundary contour line segments of the lower arm rod of the pantograph through the lower arm rod boundary slope extraction module. Use parameter constraints based on the four boundary contour line segments to obtain two boundary lines of the lower arm rod of the pantograph, and calculate the slope values of the two boundary lines; Calculate the relative inclination angle of the lower arm rod of the current pantograph based on the slope values of the two boundary lines of the lower arm rod of the pantograph. Compare the relative inclination angle of the lower arm rod of the current pantograph with the set inclination angle threshold, and judge whether there is an abnormal inclination of the pantograph according to the comparison result;

[0014] In the case where the discrimination result is inclined, use the two-dimensional rectangular coordinate frame information of the pantograph to label the inclination label of the pantograph on the current pantograph image.

[0015] Preferably, the step of determining whether the current pantograph image belongs to the category of lens dirt or the category of clean pantographs through the pantograph key area detection module and extracting the two-dimensional rectangular coordinate frame information of the pantograph in the current pantograph image belonging to the category of clean pantographs includes:

[0016] Perform image-level annotation on the category information of the pantograph image. This category information includes two categories: lens dirt and clean pantographs. Use the pantograph images of all categories to construct a pantograph recognition data set;

[0017] Through the pantograph key area detection module, use the pantograph recognition data set to train and verify the real-time object detection algorithm. Use the trained object detection algorithm to judge the category information of the current pantograph image. If the category information of the current pantograph image belongs to the category of lens dirt, the algorithm terminates; If the category information of the current pantograph image belongs to the category of clean pantographs, extract and save the two-dimensional rectangular coordinate frame information of the pantograph.

[0018] Preferably, the step of performing pixel segmentation on the lower arm rod of the pantograph in the current pantograph image through the lower arm rod pixel segmentation module of the pantograph to obtain the lower arm rod mask image includes:

[0019] Perform pixel-level annotation on the content information of the pantograph image. This content information includes the lower arm rod and the background. Use the content information of all pantograph images to construct a pantograph segmentation data set;

[0020] The pantograph lower arm pixel segmentation module uses the pantograph segmentation dataset to train and validate the fast semantic segmentation algorithm, and uses the trained fast semantic segmentation algorithm to perform pixel segmentation on the lower arm in the current pantograph image to obtain an RGB color lower arm mask image; the RGB color lower arm mask image is converted into a grayscale lower arm mask image.

[0021] Preferably, the four boundary contour line segments of the lower arm of the pantograph are extracted by the lower arm boundary slope extraction module, and two boundary lines of the lower arm of the pantograph are obtained by using parameter constraints according to the four boundary contour line segments, and the slope values of the two boundary lines are calculated, including:

[0022] The four boundary contour line segments of the lower arm in the grayscale lower arm mask image are extracted by the lower arm boundary slope extraction module using the edge detection operator and the Hough line transformation algorithm, where Edges is the edge detection result, and Lines are the four boundary contour line segments, which are the left, right, upper, and lower four edge contour line segments respectively;

[0023] Edges = cv2.Canny(Image gray , 50, 150)

[0024] Lines = cv2.HoughLines(Edges, 1, Π / 180)

[0025] Using the line parameter r = 130, setting the line segment length threshold, and using the line segment length to filter out the two longer left and right boundary contour line segments, and obtaining the corresponding key fitting parameter pairs (ρ, θ) respectively, where ρ is the polar radius and θ is the polar angle;

[0026] Lines = cv2.HoughLines(Edges, 1, Π / 180, r = 130)

[0027] (ρ, θ) = Lines[0]

[0028] Using the key fitting parameters to obtain the two endpoint coordinates corresponding to the two boundary contour line segments, is the first endpoint coordinate of the i-th line segment, is the second endpoint coordinate of the i-th line segment, (ρ i , θ i ) is the parameter pair of the i-th line segment, i = 1, 2;

[0029]

[0030] Using the two endpoint coordinates of the two line segments respectively to calculate the slope k of the two boundary lines of the lower arm;

[0031]

[0032] Preferably, calculating the relative inclination angle of the lower arm rod of the current pantograph according to the slope values of the two boundary lines of the lower arm rod of the pantograph, comparing the relative inclination angle of the lower arm rod of the current pantograph with a set inclination angle threshold, and judging whether there is an abnormal inclination of the current pantograph according to the comparison result, including:

[0033] Calculating the angle α of the lower arm rod in the pantograph image by weighted calculation using the slope values of the two boundary lines of the lower arm rod;

[0034]

[0035] Setting the inclination angle threshold δ = 3, and calculating the relative inclination angle β between the angle α of the lower arm rod and 90 degrees in the vertical direction;

[0036] β = |α - 90|

[0037] If the relative inclination angle β is greater than or equal to the set inclination angle threshold δ, it is determined that there is an abnormal inclination of the pantograph; if the relative inclination angle β is less than the set threshold range δ, it is determined that there is no abnormal inclination of the pantograph.

[0038] According to another aspect of the present invention, there is provided a pantograph abnormal inclination detection device, including: a pantograph video data acquisition module, a pantograph key area detection module, a pantograph lower arm rod pixel segmentation module, a lower arm rod boundary slope extraction module, a pantograph abnormal inclination judgment module, and an inclination label annotation module;

[0039] The pantograph video data acquisition module is used to collect current pantograph video data in real time through a pantograph image acquisition device, perform a frame extraction operation on the current pantograph video data, and obtain the current pantograph image;

[0040] The pantograph key area detection module is used to judge whether the current pantograph image belongs to the lens dirt category or the clean pantograph category, and extract the pantograph two-dimensional rectangular coordinate frame information of the current pantograph image belonging to the clean pantograph category;

[0041] The pantograph lower arm rod pixel segmentation module is used to perform pixel segmentation on the lower arm rod of the pantograph in the current pantograph image to obtain a lower arm rod mask image;

[0042] The described lower arm rod boundary slope extraction module is used to extract four boundary contour line segments of the lower arm rod of the pantograph, obtain two boundary lines of the lower arm rod of the pantograph using parameter constraints based on the four boundary contour line segments, and calculate the slope values of the two boundary lines; the pantograph abnormal tilt judgment module is used to calculate the relative tilt angle of the lower arm rod of the current pantograph according to the slope values of the two boundary lines of the lower arm rod of the pantograph, compare the relative tilt angle of the lower arm rod of the current pantograph with a set tilt angle threshold, and judge whether there is an abnormal tilt of the current pantograph according to the comparison result.

[0043] The described tilt label annotation module is used to annotate the tilt label on the current pantograph image using the pantograph two-dimensional rectangular coordinate frame information in the case where the discrimination result is tilt, so as to realize the accurate detection of the abnormal tilt of the pantograph.

[0044] Preferably, the pantograph key area detection module is used to perform image-level annotation of the category information of the pantograph image. This category information includes two categories: lens dirt and clean pantograph. The pantograph recognition dataset is constructed using the pantograph images of all categories.

[0045] The real-time object detection algorithm is trained and verified using the pantograph recognition dataset. The trained object detection algorithm is used to judge the category information of the current pantograph image. If the category information of the current pantograph image belongs to the lens dirt category, the algorithm terminates; if the category information of the current pantograph image belongs to the clean pantograph category, the two-dimensional rectangular coordinate frame information of the pantograph is extracted and saved.

[0046] Preferably, the lower arm rod pixel segmentation module of the pantograph is used to perform pixel-level annotation of the content information of the pantograph image. This content information includes the lower arm rod and the background. The pantograph segmentation dataset is constructed using the content information of all pantograph images.

[0047] The fast semantic segmentation algorithm is trained and verified using the pantograph segmentation dataset. The trained fast semantic segmentation algorithm is used to perform pixel segmentation on the lower arm rod in the current pantograph image to obtain an RGB color lower arm rod mask image; the RGB color lower arm rod mask image is converted into a grayscale lower arm rod mask image.

[0048] Preferably, the lower arm rod boundary slope extraction module is used to extract four boundary contour line segments of the lower arm rod of the pantograph, obtain two boundary lines of the lower arm rod of the pantograph using parameter constraints based on the four boundary contour line segments, and calculate the slope values of the two boundary lines, including:

[0049] The lower arm lever boundary slope extraction module uses an edge detection operator and the Hough line transformation algorithm to extract four boundary contour line segments of the lower arm lever in the grayscale lower arm lever mask image, where Edges is the edge detection result, and Lines are the four boundary contour line segments, namely the left, right, upper, and lower four edge contour line segments respectively;

[0050] Edges = cv2.Canny(Image gray , 50, 150)

[0051] Lines = cv2.HoughLines(Edges, 1, Π / 180)

[0052] Using the line parameter r = 130 to set the line segment length threshold, and screening out the two boundary contour line segments on the left and right sides with longer lengths using the line segment length, and obtaining the corresponding key fitting parameter pairs (ρ, θ) respectively, where ρ is the polar radius and θ is the polar angle;

[0053] Lines = cv2.GoughLines(Edges, 1, Π / 180, r = 130)

[0054] (ρ, θ) = Lines[0]

[0055] Using the key fitting parameters to obtain the two endpoint coordinates corresponding to the two boundary contour line segments, is the first endpoint coordinate of the i-th line segment, is the second endpoint coordinate of the i-th line segment, (ρ i , θ i ) is the parameter pair of the i-th line segment, i = 1, 2;

[0056]

[0057] Using the two endpoint coordinates of each of the two line segments to calculate the slope k of the two boundary lines of the lower arm lever;

[0058]

[0059] Preferably, the pantograph abnormal tilt judgment module is used to calculate the angle α of the lower arm lever in the pantograph image by weighted calculation using the slope values of the two boundary lines of the lower arm lever;

[0060]

[0061] Set the tilt angle threshold δ = 3, and calculate the relative tilt angle β between the angle α of the lower arm lever and the vertical direction of 90 degrees;

[0062] β = |α - 90|

[0063] If the relative tilt angle β is greater than or equal to the set tilt angle threshold δ, it is determined that there is an abnormal tilt of the pantograph; if the relative tilt angle β is less than the set threshold range δ, it is determined that there is no abnormal tilt of the pantograph.

[0064] A non-transitory computer-readable storage medium for storing computer instructions, which when executed by a processor, implement the pantograph abnormal tilt detection method as described above.

[0065] A computer program product, including a computer program, which when running on one or more processors, is used to implement the rail transit tunnel intrusion detection method as described above.

[0066] An electronic device, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes to implement the pantograph abnormal tilt detection method as described above.

[0067] It can be seen from the technical solutions provided by the embodiments of the present invention above that the method of the embodiments of the present invention can effectively detect whether the pantograph has an abnormal tilt, providing strong technical support for the real-time monitoring and maintenance of the pantograph state, and meeting the safety operation requirements of modern urban rail transit.

[0068] Additional aspects and advantages of the present invention will be given in part in the following description, which will become apparent from the following description, or can be understood through the practice of the present invention. Description of the Drawings

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0070] Figure 1 It is a processing flowchart of a pantograph abnormal tilt detection method provided by an embodiment of the present invention;

[0071] Figure 2 It is a structural diagram of a pantograph provided by an embodiment of the present invention;

[0072] Figure 3 It is a mask diagram of a pantograph lower arm rod provided by an embodiment of the present invention;

[0073] Figure 4 It is a structural diagram of a pantograph abnormal tilt detection device provided by an embodiment of the present invention. Detailed implementation manners

[0074] The following is a detailed description of the implementation manners of the present invention. Examples of the implementation manners are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The implementation manners described below by referring to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be construed as a limitation to the present invention.

[0075] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the description of the present invention means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The phrase "and / or" used herein includes any unit and all combinations of one or more of the associated listed items.

[0076] Those skilled in the art of the present technology can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the art to which the present invention belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless defined as herein.

[0077] For the convenience of understanding the embodiments of the present invention, the following will further explain with several specific embodiments by referring to the accompanying drawings, and each embodiment does not constitute a limitation to the embodiments of the present invention.

[0078] Embodiment 1

[0079] The processing flow of a pantograph abnormal inclination detection method provided by an embodiment of the present invention is as Figure 1 shown, and includes the following processing steps:

[0080] Step S1, the current pantograph video data from a top-down perspective is collected in real time through a pantograph image acquisition device installed on the train roof. Most of the pixels in the picture are the pantograph. The video data is frame-extracted to obtain the current pantograph image;

[0081] Step S2: Determine whether the current pantograph image belongs to the lens dirt category or the normal pantograph category. Extract the two-dimensional rectangular coordinate frame information of the pantograph in the current pantograph image belonging to the normal pantograph category through the pantograph key area detection module, and continue the detection;

[0082] Step S3: Through the pantograph lower arm pixel segmentation module, use the PIDNet algorithm to perform pixel segmentation on the lower arm of the pantograph in the current pantograph image according to the two-dimensional rectangular coordinate frame information of the pantograph in the current pantograph image, and obtain the lower arm mask image;

[0083] Step S4: Extract the four boundary contour line segments of the lower arm of the pantograph through the lower arm boundary slope extraction module. Use parameter restrictions based on the four boundary contour line segments to obtain the two boundary lines of the lower arm of the pantograph, and calculate the slope values of the two boundary lines;

[0084] Step S5: Calculate the relative inclination angle of the lower arm of the current pantograph according to the slope values of the two boundary lines of the lower arm of the pantograph. Compare the relative inclination angle of the lower arm of the current pantograph with the set inclination angle threshold, and judge whether there is abnormal inclination of the pantograph according to the comparison result;

[0085] Step S6: In the case where the discrimination result is inclined, use the two-dimensional rectangular coordinate frame information of the pantograph to label the inclination label of the pantograph on the current pantograph image, and realize the accurate detection of the abnormal inclination of the pantograph.

[0086] Specifically, step S2 specifically includes the following steps:

[0087] Step 2.1: Use the LabelImg software to label the category information of the pantograph image at the image level. The specific categories include two categories: lens dirt and normal pantograph, and construct a pantograph recognition data set;

[0088] Step 2.2: Through the pantograph key area detection module, use the pantograph recognition data set to train and verify the YOLOV8 target detection algorithm to obtain the trained YOLOV8 target detection algorithm. Use the trained YOLOV8 target detection algorithm to judge the category information of the current pantograph image. If the category information of the current pantograph image belongs to the lens dirt category, skip the subsequent detection and the algorithm terminates; if the category information of the current pantograph image belongs to the normal pantograph category, extract and save the two-dimensional rectangular coordinate frame information of the pantograph and perform subsequent detection.

[0089] Specifically, step S3 specifically includes the following steps:

[0090] Step 3.1: Use the Labelme software to perform pixel-level annotation on the content information of the pantograph image. The content information includes the lower arm rod and the background. Construct a pantograph segmentation dataset using the content information of all pantograph images. Figure 2 This is a structural diagram of a pantograph provided by an embodiment of the present invention. As Figure 2 shown, the structure of the pantograph is complex and precise. The lower arm rod is directly connected to the carbon sliding plate, and the service state of the pantograph can be observed more obviously.

[0091] Step 3.2: Through the pantograph lower arm rod pixel segmentation module, use the pantograph segmentation dataset to train and verify the PIDNet algorithm to obtain a trained PIDNet algorithm. Use the trained PIDNet algorithm to perform pixel segmentation on the lower arm rod in the current pantograph image to obtain an RGB color lower arm rod mask image. Figure 3 This is a mask diagram of the lower arm rod of a pantograph provided by an embodiment of the present invention.

[0092] Step 3.3: Through the pantograph lower arm rod pixel segmentation module, convert the RGB color lower arm rod mask image into a grayscale lower arm rod mask image. Image RGB is the color lower arm rod mask image, and Image gray is the gray lower arm rod mask image;

[0093] Image gray = cv2.cvtColor(Image RGB , cv2.COLOR_RGB2GRAY)

[0094] Specifically, step S4 specifically includes the following steps:

[0095] Step 4.1: Through the lower arm rod boundary slope extraction module, use the Canny edge detection operator and the Hough line transformation algorithm to extract four boundary contour line segments of the lower arm rod in the grayscale lower arm rod mask image, where Edges is the edge detection result and Lines is the four boundary contour line segments; they are the left, right, upper, and lower four edge contour line segments respectively;

[0096] Edges = cv2.Canny(Image gray , 50, 150)

[0097] Lines = cv2.HoughLines(Edges, 1, Π / 180)

[0098] Step 4.2: Using the lower arm bar boundary slope extraction module, set the line segment length threshold using the line parameter r, and filter out the two longer boundary contour line segments on the left and right sides using the line segment length to obtain the corresponding key fitting parameter pairs (ρ, θ) respectively, where ρ is the polar radius and θ is the polar angle;

[0099] Lines = cv2.HoughLines(Edges, 1, Π / 180, r = 130)

[0100] (ρ, θ) = Lines[0]

[0101] Using the line parameter r = 130, filter out the two relatively long and relatively vertical boundary contour line segments to obtain the corresponding key fitting parameter pairs (ρ, θ) respectively;

[0102] Lines = cv2.HoughLines(Edges, 1, Π / 180, r = 130)

[0103] (ρ, θ) = Lines[0]

[0104] Step 4.3: Use the key fitting parameters to obtain the coordinates of the two endpoints corresponding to the two boundary contour line segments. is the coordinate of the first endpoint of the i-th line segment. is the coordinate of the second endpoint of the i-th line segment, (ρ i , θ i ) is the parameter pair of the i-th line segment, i = 1, 2;

[0105]

[0106] Step 4.4: Calculate the slopes k of the two boundary lines of the lower arm bar using the coordinates of the two endpoints of each of the two line segments.

[0107]

[0108] Specifically, step S5 specifically includes the following steps:

[0109] Step 5.1: Calculate the angle α of the lower arm bar in the pantograph image by weighted calculation based on the slope values of the two boundary lines of the lower arm bar.

[0110]

[0111] Step 5.2: Set the tilt angle threshold δ = 3, and calculate the relative tilt angle β between the angle of the lower arm bar and 90 degrees in the vertical direction.

[0112] β = |α - 90|

[0113] Step 5.3: If the relative tilt angle β is greater than or equal to the set tilt angle threshold δ, it is determined that there is an abnormal tilt of the pantograph; if the relative tilt angle β is less than the set threshold range δ, it is determined that there is no abnormal tilt of the pantograph.

[0114] Step 6: In the case where the discrimination result is tilt, using the above coordinate frame information, finally label the tilt label of the pantograph on the image, achieve precise detection of the abnormal tilt of the pantograph, and perform alarm processing.

[0115] The pantograph abnormal tilt detection algorithm proposed by the present invention can achieve a detection accuracy of 100% in the pantograph dataset, and the detection speed is 66 FPS, meeting the requirements of real-time precise detection.

[0116] Embodiment 2

[0117] The structure of a pantograph abnormal tilt detection device provided by this embodiment is as Figure 4 shown, including the following modules: Pantograph video data acquisition module 1, Pantograph key area detection module 2, Pantograph lower arm rod pixel segmentation module 3, Lower arm rod boundary slope extraction module 4, Pantograph abnormal tilt judgment module 5, and Tilt label annotation module 6.

[0118] The described pantograph video data acquisition module is used to collect the current pantograph video data in real time through the pantograph image acquisition device, perform a frame extraction operation on the current pantograph video data, and obtain the current pantograph image;

[0119] The described pantograph key area detection module is used to determine whether the current pantograph image belongs to the lens dirt category or the clean pantograph category, and extract the pantograph two-dimensional rectangular coordinate frame information of the current pantograph image belonging to the clean pantograph category;

[0120] The described pantograph lower arm rod pixel segmentation module is used to perform pixel segmentation on the lower arm rod of the pantograph in the current pantograph image to obtain a lower arm rod mask image;

[0121] The described lower arm rod boundary slope extraction module is used to extract four boundary contour line segments of the lower arm rod of the pantograph, obtain two boundary lines of the lower arm rod of the pantograph by using parameter restrictions according to the four boundary contour line segments, and calculate the slope values of the two boundary lines; the described pantograph abnormal tilt judgment module is used to calculate the relative tilt angle of the current pantograph's lower arm rod according to the slope values of the two boundary lines of the pantograph's lower arm rod, compare the relative tilt angle of the current pantograph's lower arm rod with the set tilt angle threshold, and judge whether there is an abnormal tilt of the current pantograph according to the comparison result;

[0122] The described inclined label annotation module is used to, when the discrimination result is inclined, annotate the pantograph with an inclined label on the current pantograph image by using the two-dimensional rectangular coordinate frame information of the pantograph, so as to achieve precise detection of abnormal inclination of the pantograph.

[0123] Preferably, the pantograph key area detection module is used to annotate the category information of the pantograph image at the image level. This category information includes two categories: lens dirt and clean pantograph. The pantograph recognition dataset is constructed by using the pantograph images of all categories;

[0124] The real-time object detection algorithm is trained and verified by using the pantograph recognition dataset. The trained object detection algorithm is used to judge the category information of the current pantograph image. If the category information of the current pantograph image belongs to the lens dirt category, the algorithm terminates; if the category information of the current pantograph image belongs to the clean pantograph category, the two-dimensional rectangular coordinate frame information of the pantograph is extracted and saved.

[0125] Preferably, the pantograph lower arm pixel segmentation module is used to annotate the content information of the pantograph image at the pixel level. This content information includes the lower arm and the background. The pantograph segmentation dataset is constructed by using the content information of all pantograph images;

[0126] The fast semantic segmentation algorithm is trained and verified by using the pantograph segmentation dataset. The trained fast semantic segmentation algorithm is used to perform pixel segmentation on the lower arm of the current pantograph image to obtain an RGB color lower arm mask image; the RGB color lower arm mask image is converted into a grayscale lower arm mask image.

[0127] Preferably, the lower arm boundary slope extraction module is used to extract four boundary contour line segments of the lower arm of the pantograph, and two boundary lines of the lower arm of the pantograph are obtained by using parameter constraints according to the four boundary contour line segments, and the slope values of the two boundary lines are calculated, including:

[0128] The four boundary contour line segments of the lower arm in the grayscale lower arm mask image are extracted by the lower arm boundary slope extraction module by using an edge detection operator and a Hough line transform algorithm, where Edges is the edge detection result and Lines are the four boundary contour line segments, which are the left, right, upper and lower four edge contour line segments respectively;

[0129] Edges = cv2.Canny(Image gray , 50, 150)

[0130] Lines = cv2.HoughLines(Edges, 1, Π / 180)

[0131] Using the straight-line parameter r = 130, set the line segment length threshold, and use the line segment length to filter out two boundary contour line segments on the left and right sides with longer lengths, and obtain the corresponding key fitting parameter pairs (ρ, θ) respectively, where ρ is the polar radius and θ is the polar angle;

[0132] Lines = cv2.GoughLines(Edges, 1, Π / 180, r = 130)

[0133] (ρ, θ) = Lines[0]

[0134] Obtain the coordinates of the two end points corresponding to the two boundary contour line segments using the key fitting parameters, is the coordinate of the first end point of the i-th line segment, is the coordinate of the second end point of the i-th line segment, (ρ i , θ i ) is the parameter pair of the i-th line segment, i = 1, 2;

[0135]

[0136] Calculate the slopes k of the two boundary lines of the lower arm using the coordinates of the two end points of each of the two line segments;

[0137]

[0138] Preferably, the pantograph abnormal tilt judgment module is used to calculate the angle α of the lower arm in the pantograph image by weighted calculation using the slope values of the two boundary lines of the lower arm;

[0139]

[0140] Set the tilt angle threshold δ = 3, and calculate the relative tilt angle β between the angle α of the lower arm and 90 degrees in the vertical direction;

[0141] β = |α - 90|

[0142] If the relative tilt angle β is greater than or equal to the set tilt angle threshold δ, it is determined that there is an abnormal tilt of the pantograph; if the relative tilt angle β is less than the set threshold range δ, it is determined that there is no abnormal tilt of the pantograph.

[0143] The processing process of using the device of the present invention for pantograph abnormal tilt detection is the same as that of the above method embodiment. Similar to the foregoing method embodiment, it will not be repeated here.

[0144] Embodiment 3

[0145] A non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the pantograph abnormal tilt detection method as described in Embodiment 1.

[0146] A computer program product includes a computer program that, when running on one or more processors, is used to implement the rail transit tunnel intrusion detection method as described in Embodiment 1.

[0147] An electronic device includes: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory to enable the electronic device to execute the pantograph abnormal tilt detection method as described in Embodiment 1.

[0148] In summary, the method of the embodiment of the present invention extracts the two-dimensional rectangular coordinate frame information of the pantograph through the YOLOV8 target detection algorithm, obtains the RGB color mask image of the lower arm through the PIDNet algorithm, and uses the Canny edge detection operator and the Hough line transformation algorithm to extract the four boundary contour line segments of the lower arm in the grayscale mask image of the lower arm. By closely combining the methods based on deep learning and traditional image processing, it reduces the interference of environmental factors on detection, significantly improves the accuracy and speed of pantograph abnormal tilt detection, solves the problems of low accuracy, slow detection speed, and low detection efficiency commonly existing in traditional methods, and provides strong technical support for the real-time monitoring and maintenance of the pantograph state.

[0149] Those of ordinary skill in the art can understand that the drawings are only schematic diagrams of one embodiment, and the modules or processes in the drawings are not necessarily essential for implementing the present invention.

[0150] From the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0151] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative work.

[0152] As described above, the above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A pantograph abnormal tilt detection method, characterized in that, Including: Real-time collect the current pantograph video data through the pantograph image acquisition device, perform a frame extraction operation on the current pantograph video data to obtain the current pantograph image; Judge whether the current pantograph image belongs to the lens dirt category or the clean pantograph category, and extract the two-dimensional rectangular coordinate frame information of the pantograph of the current pantograph image belonging to the clean pantograph category through the pantograph key area detection module; Perform pixel segmentation on the lower arm rod of the pantograph in the current pantograph image through the lower arm rod pixel segmentation module of the pantograph to obtain a lower arm rod mask image; Extract four boundary contour line segments of the lower arm rod of the pantograph through the lower arm rod boundary slope extraction module, obtain two boundary lines of the lower arm rod of the pantograph using parameter constraints, and calculate the slope values of the two boundary lines; Calculate the relative inclination angle of the lower arm rod of the current pantograph based on the slope values of the two boundary lines of the lower arm rod of the pantograph, compare the relative inclination angle of the lower arm rod of the current pantograph with the set inclination angle threshold, and judge whether there is an abnormal inclination of the pantograph according to the comparison result; In the case where the discrimination result is inclined, label the inclination label of the pantograph on the current pantograph image using the two-dimensional rectangular coordinate frame information of the pantograph; The extracting four boundary contour line segments of the lower arm rod of the pantograph through the lower arm rod boundary slope extraction module, obtaining two boundary lines of the lower arm rod of the pantograph using parameter constraints, and calculating the slope values of the two boundary lines includes: Extract four boundary contour line segments of the lower arm rod in the grayscale lower arm rod mask image through the lower arm rod boundary slope extraction module using an edge detection operator and the Hough line transformation algorithm, where Edges is the edge detection result, and Lines are the four boundary contour line segments, which are the left, right, upper, and lower four edge contour line segments respectively; Edges = cv2.Canny(Image gray , 50, 150) Lines = cv2.HoughLines(Edges, 1, Π / 180) Use the line parameter r = 130, set the line segment length threshold, and filter out the two longer left and right boundary contour line segments using the line segment length to obtain the corresponding key fitting parameter pairs (ρ, θ) respectively, where ρ is the polar radius and θ is the polar angle; Lines = cv2.HoughLines(Edges, 1, Π / 180, r = 130) (ρ, θ) = Lines[0] Obtain the coordinates of the two endpoints corresponding to the two boundary contour line segments by using the key fitting parameters. is the coordinate of the first endpoint of the i-th line segment. is the coordinate of the second endpoint of the i-th line segment, (ρ i , θ i ) is the parameter pair of the i-th line segment, i = 1, 2; Calculate the slope k of the two boundary lines of the lower arm rod using the respective two endpoint coordinates of the two line segments; The calculating the relative inclination angle of the lower arm rod of the current pantograph based on the slope values of the two boundary lines of the lower arm rod of the pantograph, comparing the relative inclination angle of the lower arm rod of the current pantograph with the set inclination angle threshold, and judging whether there is an abnormal inclination of the pantograph according to the comparison result includes: Calculate the angle α of the lower arm rod in the pantograph image by weighted calculation using the slope values of the two boundary lines of the lower arm rod; Set the inclination angle threshold δ = 3, and calculate the relative inclination angle β between the angle α of the lower arm rod and 90 degrees in the vertical direction; β＝|α-90| If the relative tilt angle β is greater than or equal to the set tilt angle threshold δ, it is determined that there is an abnormal tilt of the pantograph; if the relative tilt angle β is less than the set threshold range δ, it is determined that there is no abnormal tilt of the pantograph.

2. The method according to claim 1, wherein The pantograph key area detection module is used to determine whether the current pantograph image belongs to the lens dirt category or the clean pantograph category, and extract the two-dimensional rectangular coordinate frame information of the pantograph of the current pantograph image belonging to the clean pantograph category, including: Image-level annotation of the category information of the pantograph image. This category information includes two categories: lens dirt and clean pantograph. Use the pantograph images of all categories to construct a pantograph recognition data set; The pantograph key area detection module uses the pantograph recognition data set to train and verify the real-time object detection algorithm. Use the trained object detection algorithm to judge the category information of the current pantograph image. If the category information of the current pantograph image belongs to the lens dirt category, the algorithm terminates; if the category information of the current pantograph image belongs to the clean pantograph category, extract and save the two-dimensional rectangular coordinate frame information of the pantograph.

3. The method according to claim 2, wherein The pantograph lower arm pixel segmentation module is used to perform pixel segmentation on the lower arm of the pantograph in the current pantograph image to obtain a lower arm mask image, including: Pixel-level annotation of the content information of the pantograph image. This content information includes the lower arm and the background. Use the content information of all pantograph images to construct a pantograph segmentation data set; The pantograph lower arm pixel segmentation module uses the pantograph segmentation data set to train and verify the semantic segmentation algorithm, and uses the trained semantic segmentation algorithm to perform pixel segmentation on the lower arm of the current pantograph image to obtain an RGB color lower arm mask image; convert the RGB color lower arm mask image to a grayscale lower arm mask image.

4. A pantograph abnormal inclination detection device, characterized in that, Including: Pantograph video data acquisition module, pantograph key area detection module, pantograph lower arm pixel segmentation module, lower arm boundary slope extraction module, pantograph abnormal tilt judgment module and tilt label annotation module; The pantograph video data acquisition module is used to collect the current pantograph video data in real time through the pantograph image acquisition device, and perform a frame extraction operation on the current pantograph video data to obtain the current pantograph image; The pantograph key area detection module is used to determine whether the current pantograph image belongs to the lens dirt category or the clean pantograph category, and extract the two-dimensional rectangular coordinate frame information of the pantograph of the current pantograph image belonging to the clean pantograph category; The pantograph lower arm pixel segmentation module is used to perform pixel segmentation on the lower arm of the pantograph in the current pantograph image to obtain a lower arm mask image; The lower arm rod boundary slope extraction module is used to extract four boundary contour line segments of the lower arm rod of the pantograph, obtain two boundary lines of the lower arm rod of the pantograph according to the four boundary contour line segments, and calculate the slope values of the two boundary lines; the pantograph abnormal tilt judgment module is used to calculate the relative tilt angle of the current lower arm rod of the pantograph according to the slope values of the two boundary lines of the lower arm rod of the pantograph, compare the relative tilt angle of the current lower arm rod of the pantograph with the set tilt angle threshold, and judge whether there is an abnormal tilt of the current pantograph according to the comparison result. The tilt label annotation module is used to annotate the tilt label of the pantograph on the current pantograph image by using the pantograph two-dimensional rectangular coordinate frame information when the discrimination result is tilt. The lower arm rod boundary slope extraction module is specifically used to extract four boundary contour line segments of the lower arm rod in the grayscale lower arm rod mask image by using the edge detection operator and the Hough line transformation algorithm in the lower arm rod boundary slope extraction module, where Edges is the edge detection result, and Lines are the four boundary contour line segments, which are the left, right, upper and lower four edge contour line segments respectively. Edges = cv2.Canny(Image gray , 50, 150) Lines = cv2.HoughLines(Edges, 1, Π / 180) Using the straight line parameter r = 130, set the line segment length threshold, and use the line segment length to screen out the two longer left and right boundary contour line segments, and obtain the corresponding key fitting parameter pairs (ρ, θ) respectively, where ρ is the polar radius and θ is the polar angle. Lines = cv2.HoughLines(Edges, 1, Π / 180, r = 130) (ρ, θ) = Lines[0] Obtain the two endpoint coordinates corresponding to the two boundary contour line segments by using the key fitting parameters. is the first endpoint coordinate of the i-th line segment. is the second endpoint coordinate of the i-th line segment, (ρ i , θ i ) is the parameter pair of the i-th line segment, i = 1, 2; Calculate the slope k of the two boundary lines of the lower arm rod by using the respective two endpoint coordinates of the two line segments. Calculate the angle α of the lower arm rod in the pantograph image by weighted calculation of the slope values of the two boundary lines of the lower arm rod. Set the tilt angle threshold δ = 3, and calculate the relative tilt angle β between the angle α of the lower arm rod and 90 degrees in the vertical direction. β＝|α-90| If the relative tilt angle β is greater than or equal to the set tilt angle threshold δ, it is determined that there is an abnormal tilt of the pantograph; if the relative tilt angle β is less than the set threshold range δ, it is determined that there is no abnormal tilt of the pantograph.

5. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the pantograph abnormal tilt detection method described in any one of claims 1-3 is implemented.

6. A computer program product, characterized in that, It includes a computer program, and when the computer program runs on one or more processors, it is used to implement the pantograph abnormal tilt detection method described in any one of claims 1-3.

7. An electronic device, characterized in that, It includes: A processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes the pantograph abnormal tilt detection method described in any one of claims 1-3.

Images