A method and system for collecting pantograph status information
Through deep learning technology, the size and posture information of pantographs are extracted from 3D images using background and object detection algorithms, which solves the problem that the state of the arch cannot be detected in the prior art, and achieves efficient and reliable pantograph state detection .
Patent Information
- Application Number
- CN202010694168.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-17
- Publication Date
- 2024-12-31
- Estimated Expiration
- 2040-07-17
AI Technical Summary
The prior art cannot effectively detect pantographs in the lowered bow state, resulting in low detection efficiency and quality, and artificial control of the lifting bow is required to ensure detection coverage.
Deep learning is performed using background detection algorithm and object detection algorithm. By obtaining the 3D image of the target area, segmenting it into several image frames, removing redundant image frames, annotating key frames containing pantographs, and obtaining pantograph size and posture information to achieve Detection is performed without pantograph triggering.
It improves the stability and reliability of pantograph detection, ensures that both the raised bow and the lower bow state can be effectively detected, avoids the shortcomings of artificial control, and improves the detection efficiency.
Smart Images
Figure CN111754512B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of pantograph detection on top of a train, and in particular to a method and system for collecting pantograph status information. Background Art
[0002] Pantograph refers to the key component that contacts the power supply wire on the top of the train (including EMU, locomotive, vehicle, subway, tram, etc.) to obtain electricity to drive the train. Its working state directly affects the current-carrying capacity of the electric locomotive. Generally, pantograph accidents easily lead to damage to the pantograph slider or burn damage to the contact network. In severe cases, it may lead to current interruption, locomotive failure or even burn the electric drive part of the electric locomotive. Therefore, the state of the pantograph directly affects the safety of train operation.
[0003] In daily maintenance, the main measurements are on the thickness of the carbon slide plate, the pantograph posture, etc. The pantograph status detection system is a system that detects the working status of the pantograph by dynamically capturing images of the pantograph passing through the detection interval.
[0004] However, in the actual detection process, the image acquisition device is generally triggered by a sensor, such as the technical solution disclosed in CN104567729B. The sensor detects the distance of the object in front. When it is less than the set value, the image acquisition device is triggered to detect the pantograph image. It can be seen that the prior art can only trigger the image acquisition device in the pantograph raising state. The carbon slide board in the pantograph lowering state cannot be effectively detected due to the complex background and the inability to accurately trigger the image acquisition device. In order to overcome this problem, it is generally solved by the driver raising the pantograph cyclically to ensure that all pantographs have the opportunity to be detected when passing the detection equipment. However, this method requires manual control and cannot always guarantee the efficiency and quality of maintenance. Summary of the invention
[0005] In view of this, the present application provides a method for collecting pantograph status information, which performs deep learning through background detection algorithm and target detection algorithm to extract images containing pantograph size information, which is used for slide plate wear measurement, slide plate block falling, pantograph posture detection, etc., overcoming the defect that the prior art cannot collect pantograph lowering images, improves detection efficiency, and improves the stability and reliability of image acquisition.
[0006] In order to solve the above technical problems, the technical solution provided by the present invention is based on this. The present application provides a method for collecting pantograph status information, comprising the following steps:
[0007] Acquire a 3D image of the target area;
[0008] Segmenting the 3D image into a number of image frames;
[0009] The redundant image frames are removed by using the background detection algorithm, and the image frames are annotated by using the target detection algorithm to obtain the key frame containing the pantograph image;
[0010] The key frame contains pantograph size and posture information, which is used for pantograph status detection.
[0011] Preferably, the step of acquiring a 3D image of the target area specifically includes:
[0012] The image acquisition device is triggered by receiving a start instruction and continuously acquires structured light images of the target area at a certain frequency until receiving an end instruction.
[0013] Preferably, when the image acquisition device is triggered, the laser corresponding to the image acquisition device receives a synchronous trigger signal, so that the laser always keeps working during the period when the image acquisition device acquires images and projects structured light to the target area.
[0014] Preferably, the light emitting path of the laser is arranged perpendicular to the running track of the pantograph, and the field of view of the image acquisition device has an overlapping part with the light emitting path of the laser.
[0015] Preferably, the step of dividing the 3D image into a plurality of image frames specifically includes:
[0016] Smoothing 3D images;
[0017] The 3D image is divided into a plurality of image frames, and two adjacent image frames have overlapping image parts.
[0018] Preferably, the step of removing redundant images from the image specifically includes:
[0019] Through the ViBe algorithm, the first frame of the continuously acquired images is set as the initial frame and background modeling is performed;
[0020] Perform foreground segmentation on the subsequent image sequence to determine whether the subsequent image contains a foreground image. If not, mark it as a redundant image. If so, mark it as a plurality of image frames including vehicle images in sequence.
[0021] Preferably, the step of obtaining a key frame including a pantograph specifically comprises:
[0022] The YOLO algorithm is used to annotate the continuous image frames containing the pantograph area;
[0023] Select the middle frame of continuous image frames as the key frame.
[0024] Preferably, when the number of consecutive frames is an even number, any one of the two middle frames is selected as the key frame;
[0025] When the number of consecutive frames is an odd number, the middle frame is selected as the key frame.
[0026] A pantograph status acquisition system, comprising:
[0027] An image acquisition device, which continuously acquires 3D images of the target area;
[0028] An image processing module is used to divide the 3D image into a plurality of image frames, and remove redundant images in the image by a background detection algorithm to obtain a plurality of image frames including images of vehicles;
[0029] A key frame extraction module is used to annotate the image frame through the target detection algorithm to obtain the key frame containing the pantograph;
[0030] The key frame contains pantograph size and attitude information for pantograph status detection
[0031] Compared with the prior art, the present application is described in detail as follows:
[0032] The present application discloses a method for collecting pantograph status information. The method is not triggered by the pantograph itself, nor does it perform preliminary identification and judgment of target objects. Instead, the method directly collects images of the target area, removes images that do not contain the foreground through a background detection algorithm, and obtains continuous image frames that only contain vehicles. Then, the image frames are annotated through a target detection algorithm to filter out key frames containing complete pantograph images. The key frames collected by this method will not be affected by the pantograph's raised or lowered state. Since the image acquisition device indiscriminately collects the image below to form a 3D image, the image acquisition device can capture the pantograph in either the raised or lowered state, thereby avoiding the problem in the prior art that the image acquisition device needs to be repeatedly raised and lowered to trigger the image acquisition device.
[0033] Since a relatively complete 3D image is directly acquired, when the 3D image is processed, the image inevitably contains parts that interfere with the detection targets such as the pantograph. The present application sequentially uses the background detection algorithm to identify the background image, then extracts the frame containing the foreground image of the vehicle, and then uses the target detection algorithm to find the key frame containing the complete pantograph area. This allows the method of the present application to complete the acquisition of the pantograph image without the need for pantograph triggering or accurate identification of the pantograph in the early stage, thereby improving the stability and reliability of pantograph detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 is a flow chart of the method of the present invention;
[0035] Figure 2 A schematic diagram of the positions of the image acquisition device and the laser in the system shown in the present invention;
[0036] Figure 3 It is a system schematic diagram of the present invention. DETAILED DESCRIPTION
[0037] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0038] like Figure 1 As shown, the present application discloses a method for collecting pantograph status information, which collects a 3D image of a target area below by an image acquisition device 1 arranged above a vehicle's running route; the 3D image is processed and segmented into a plurality of image frames, and then redundant images in the image frames are removed by a background detection algorithm to obtain a plurality of image frames including vehicle images; the image frames are annotated by a target detection algorithm to obtain key frames including a pantograph, the key frames including pantograph size and posture information, and the extracted key frames can be used for pantograph status detection.
[0039] The method specifically includes:
[0040] S1: The image acquisition device 1 receives a start instruction, and after being triggered, continuously acquires images of the target area at a certain frequency until receiving an end instruction.
[0041] The start instruction and the end instruction can be remotely provided by a computer or a control center, such as when a train is about to pass through the detection area, the program gives an instruction or a manual instruction, or it can be triggered by a sensor near the detection area, for example, when the train approaches the detection area, the sensor triggers the start instruction, and when the train leaves the detection area, the sensor triggers the end instruction. The image acquisition device 1 always keeps working from receiving the start instruction to receiving the end instruction, and continuously acquires images of the target area at a preset frequency.
[0042] Generally, the image acquisition device 1 is arranged above the track of the pantograph. After being triggered, the image acquisition device 1 can capture the image of the target area below. If the vehicle partially or completely runs to the target area, a 3D image including the upper surface of the vehicle is captured through continuous acquisition. If the vehicle has not yet run to the target area, it is also possible that the captured image is a ground image. It should be noted that the image of the target area continuously acquired in the present application may include all of the above situations, that is, the image of the target area continuously acquired may contain images that do not include the vehicle or only include images of partially incomplete vehicles. These situations have no negative impact on the method shown in the present application. The image acquisition device 1 continuously acquires images at a certain frequency, and its frequency may be restricted by the structure of the optical acquisition device itself, or it may be an artificially set frequency to ensure the continuity and effectiveness of image acquisition, which is not limited here. It can be expected that at this frequency, at least the image acquisition device 1 is guaranteed to acquire and generate a 3D image, and the 3D image can form a number of image frames after processing.
[0043] A laser 2 is also matched next to the image acquisition device 1. Generally, the light path of the laser 2 is set perpendicular to the pantograph running track, and the field of view of the image acquisition device 1 overlaps with the light path of the laser 2. Since the installation angle and position of the image acquisition device 1 are adjustable, the overlap between the field of view of the image acquisition device 1 and the light path of the laser 2 can be adjusted; the pantograph running track can pass through the overlapped part, so that the image acquisition device 1 can obtain a clearer and more accurate image.
[0044] When the image acquisition device 1 is triggered by a start instruction, the laser 2 receives a synchronization trigger signal, so that the laser 2 always keeps working during the period when the image acquisition device 1 continuously acquires images. Correspondingly, when the image acquisition device 1 is triggered to stop working by an end instruction, the laser 2 can also receive a synchronization signal to stop working. The signal received by the laser 2 can come from the image acquisition device 1. At this time, there is a direct signal transceiver link between the laser 2 and the image acquisition device 1; the signal received by the laser 2 can also come from the sending end of the start instruction.
[0045] Since the image acquisition device 1 continuously acquires images containing laser structured light, if the image acquired by the device includes the pantograph area, the three-dimensional contour of the pantograph containing size information can be reconstructed through an algorithm to be applied to pantograph status detection. After the image acquisition is completed, it can be stored and then processed.
[0046] S2: Smoothing the 3D image; dividing the 3D image into a plurality of image frames, where two adjacent image frames have overlapping image parts.
[0047] When different parameters are adjusted in the smoothing process, the sizes of the segmented images may be different. Since the smoothing algorithm is well known to those skilled in the art, no limitation is made to the specific smoothing algorithm.
[0048] In the segmented image frames, two adjacent image frames have at least partially overlapping image portions. Since two adjacent images overlap when the image is segmented, by setting the width of the overlapping area to be greater than or equal to the width of the pantograph in the image, it can be ensured that at least one image frame contains a complete pantograph image.
[0049] S31: Using the ViBe algorithm, the first frame of the continuously acquired images is set as the initial frame and background modeling is performed.
[0050] The ViBe algorithm uses neighborhood pixels to create a background model, and compares the pixel values of other frames with the background model to distinguish the background from the foreground. In this application, the so-called background includes the unchanging background information such as the ground and track in the target area, and also includes the image of the top surface of the vehicle collected in the target area. The foreground is the image of the pantograph located on the top surface of the vehicle, including the state of lowering and raising the pantograph.
[0051] The first frame image is set as the initial frame, and the background model constructed by the pixels in the initial frame is:
[0052]
[0053] Among them, N G (x, y) represents the adjacent pixel values in the spatial domain, f 0 (x, y) represents the pixel value of the current point.
[0054] S32: Perform foreground segmentation on the subsequent image sequence of the first frame, assuming that the subsequent image sequence is t,
[0055] When t=k, the background model of the pixel (x, y) is The pixel value is
[0056] f k (x, y). Determine whether the pixel value is foreground as follows:
[0057]
[0058] When formula (a) is satisfied, the pixel is a pixel of the foreground image, and when formula (b) is satisfied, the pixel is a pixel of the background image.
[0059] When the proportion of foreground image pixels in any frame to all pixels is less than 20%, the image is judged to contain no foreground and is marked as a redundant image. Otherwise, the image is judged to contain foreground and is marked in sequence as several image frames including vehicle images.
[0060] S33: Extract the pantograph contour area through the YOLO network, and mark the continuous frames containing the complete pantograph area image in the image frame.
[0061] Specifically, assume that the images of the entire acquisition process are marked in sequence as #1, #2, ..., #10, #11, #12, #13, ..., #n-1, #n, remove the redundant images marked in step S32, and set them as #1, #2, ..., #k-1, #k, and #nm, #n-m+1, ..., #n-1, #n, and extract the frames containing the complete pantograph area image through the YOLO network to obtain several consecutive frames, which are set as #k+i, #k+i+1...#k+i+j.
[0062] Among them, n, k, i, j, and m are all natural numbers, and k+i+j≤nm.
[0063] The YOLO network is used to perform multiple image magnification processing on consecutive frames, and no less than 2 feature maps are output respectively. The one with the highest resolution is selected for key frame screening, thereby improving the detection capability of small-sized pantographs.
[0064] S34: Selecting the middle frame of the continuous frames as a key frame, where:
[0065] When the number of consecutive frames is even, any of the two middle frames is selected as the key frame;
[0066] When the number of consecutive frames is an odd number, the middle frame is selected as the key frame.
[0067] Suppose the consecutive frames are #k+i, #k+i+1, ... #k+i+j. If j+1 is an even number, any one of the two middle frames is selected as the key frame. If j+1 is an odd number, the middle frame is selected as the key frame.
[0068] The key frame is a three-dimensional image containing a complete pantograph image, including structured light data or three-dimensional size data, and can be applied to pantograph state detection. Since the use of pantograph three-dimensional images for state detection is well known to those skilled in the art, the method for pantograph state detection is not further elaborated in this application document.
[0069] like Figure 2 and Figure 3 As shown, the present application also discloses a pantograph status acquisition system, which can apply the method disclosed in the present application, including:
[0070] An image acquisition device 1 is used to continuously acquire 3D images of a target area;
[0071] The image processing module 3 is used to process the 3D image and divide it into a plurality of image frames, and remove redundant images in the image by a background detection algorithm to obtain a plurality of image frames including the image of the vehicle;
[0072] A key frame extraction module 4 is used to annotate the image frame through a target detection algorithm to obtain a key frame containing the pantograph;
[0073] The key frame contains pantograph size and attitude information for pantograph status detection
[0074] The light emitting path of the laser 2 is arranged perpendicular to the pantograph running track, and the field of view of the image acquisition device 1 overlaps with the light emitting path of the laser 2. Since the installation angle and position of the image acquisition device 1 are adjustable, the pantograph running track can pass through the overlapped part of the field of view of the image acquisition device 1 and the light emitting path of the laser 2, ensuring that the image acquisition device 1 can obtain a valid image that meets the needs of subsequent steps.
[0075] The above are only preferred embodiments of the present invention. It should be noted that the above preferred embodiments should not be regarded as limiting the present invention, and the protection scope of the present invention should be based on the scope defined by the claims. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the spirit and scope of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention.
Claims
1. A method for collecting pantograph status information, characterized in that: The following steps are involved: Acquire a 3D image of the target area; The 3D image is divided into a plurality of image frames, specifically comprising: performing smoothing processing on the 3D image, dividing the 3D image into a plurality of image frames, wherein two adjacent image frames have overlapping image portions; Removing redundant image frames from image frames by using a background detection algorithm, including: using a ViBe algorithm, setting the first frame of the continuously acquired images as an initial frame and performing background modeling; performing foreground segmentation on a subsequent image sequence, determining whether the subsequent image contains a foreground image, and if not, marking it as a redundant image, and if so, sequentially marking it as a number of image frames including vehicle images; The image frames are annotated by the target detection algorithm to obtain the key frame containing the pantograph image; The key frame contains pantograph size and posture information, which is used for pantograph status detection.
2. A method for collecting pantograph status information according to claim 1, characterized in that: The step of obtaining a 3D image of the target area specifically includes: The image acquisition device is triggered by receiving a start instruction and continuously acquires structured light images of the target area at a certain frequency until receiving an end instruction.
3. A method for collecting pantograph status information according to claim 2, characterized in that: When the image acquisition device is triggered, the laser corresponding to the image acquisition device receives a synchronous trigger signal, so that the laser always keeps working during the period when the image acquisition device acquires images, and projects structured light to the target area.
4. A method for collecting pantograph status information according to claim 3, characterized in that: The light emitting path of the laser is arranged perpendicular to the running track of the pantograph, and the field of view of the image acquisition device has an overlapping part with the light emitting path of the laser.
5. The method for collecting pantograph status information according to claim 1, characterized in that: The step of obtaining the key frame including the pantograph specifically includes: The YOLO algorithm is used to annotate the continuous image frames containing the pantograph area; Select the middle frame of continuous image frames as the key frame.
6. A method for collecting pantograph status information according to claim 5, characterized in that: When the number of consecutive frames is even, any of the two middle frames is selected as the key frame; When the number of consecutive frames is an odd number, the middle frame is selected as the key frame.
7. A pantograph status acquisition system, characterized in that: include: An image acquisition device, which continuously acquires 3D images of the target area; The image processing module is used to divide the 3D image into a plurality of image frames, specifically including: performing smoothing processing on the 3D image, dividing the 3D image into a plurality of image frames, wherein two adjacent image frames have overlapping image parts, Removing redundant image frames from image frames by using a background detection algorithm, including: using a ViBe algorithm, setting the first frame of the continuously acquired images as an initial frame and performing background modeling; performing foreground segmentation on a subsequent image sequence, determining whether the subsequent image contains a foreground image, and if not, marking it as a redundant image, and if so, sequentially marking it as a number of image frames including vehicle images; A key frame extraction module is used to annotate the image frame through the target detection algorithm to obtain the key frame containing the pantograph; The key frame contains pantograph size and posture information, which is used for pantograph status detection.