Locomotive pantograph lowering detection method, equipment, medium and product
By obtaining train track pictures and using detection models to identify locomotive and rising bow state pantographs, the problem of locomotive pantograph status detection errors caused by visual fatigue in the prior art is solved, and automatic detection is achieved, improving efficiency and accuracy.
Patent Information
- Application Number
- CN202411796212.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-09
- Publication Date
- 2025-04-04
AI Technical Summary
In the prior art, confirmation of whether the locomotive pantograph is in the lowered state depends on the naked eye observation of the on-site personnel, which is prone to errors due to visual fatigue, resulting in safety hazards and inefficient detection.
By obtaining the train track picture at the entrance of the overturner, using the detection model to identify the locomotive and rising bow status pantographs, determine whether the locomotive is located in the predetermined stop area, and determine whether the rising bow status pantograph is the locomotive pantograph in this area, and realize automatic detection.
Automatic detection of whether the locomotive pantograph is in the lowered bow state is achieved, which improves detection efficiency, reduces the safety risks of on-site prosecutors, and improves the accuracy of detection, avoids errors caused by visual fatigue.
Smart Images
Figure CN119763036A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of visual inspection, and in particular to a method, equipment, medium and product for detecting the lowering of a locomotive pantograph. Background Art
[0002] In order to efficiently unload materials from the train at the unloading platform or unloading facility, a car tipper system is used, and a positioning vehicle is used to tow the train into the car tipper. However, before the locomotive is towed into the car tipper by the positioning vehicle, on-site personnel are required to confirm whether the locomotive pantograph is in the lowered state. If it is not in the lowered state, the positioning vehicle is not allowed to tow the locomotive into the car tipper, and the train driver is reminded to lower the pantograph. Only after the locomotive pantograph is in the lowered state can the positioning vehicle tow the locomotive into the car tipper to prevent the pantograph and the car tipper from colliding.
[0003] At present, most car tipping machine systems use the method of on-site personnel visually observing to confirm whether the locomotive pantograph is in the lowered state. Since the car tipping machine system operates 24 hours a day, the method of confirming whether the locomotive pantograph is in the lowered state by on-site personnel visually observing may lead to errors due to visual fatigue of the personnel. Summary of the invention
[0004] The purpose of the present application is to provide a method, device, medium and product for detecting the lowering of a locomotive pantograph, so as to realize automatic detection of whether the locomotive pantograph is in a lowered state.
[0005] To achieve the above objectives, this application provides the following solutions:
[0006] In a first aspect, the present application provides a method for detecting a pantograph lowering condition of a locomotive, comprising:
[0007] Get a picture of the train tracks at the entrance to the dumper;
[0008] Detecting whether there is a locomotive and a pantograph in a raised state in the train track picture;
[0009] If there is a locomotive in the train track image, detecting whether the locomotive is located in a predetermined locomotive parking zone LMZ;
[0010] If the locomotive is located in the LMZ and there is a pantograph in the raised state in the train track picture, detect whether the pantograph in the raised state is the pantograph of the locomotive located in the LMZ; if the locomotive is located in the LMZ and there is no pantograph in the raised state in the train track picture, or if the pantograph in the raised state is not the pantograph of the locomotive located in the LMZ, it means that the pantograph of the locomotive located in the LMZ is in the lowered state.
[0011] Optionally, the locomotive pantograph lowering detection method further includes:
[0012] If the pantograph in the raised state is the pantograph of a locomotive located in the LMZ, the control performs a pantograph lowering operation on the pantograph in the raised state, or the control issues a second alarm to remind the user to perform a pantograph lowering operation on the pantograph in the raised state.
[0013] Optionally, the detecting whether there is a locomotive and a pantograph in a raised state in the train track picture includes:
[0014] A detection model is constructed, and the train track image is input into the detection model, and the detection model is used to detect whether there is a locomotive and a pantograph in a raised state in the train track image.
[0015] Optionally, the detection model includes a locomotive detection model and a pantograph detection model in a pantograph-raised state;
[0016] The constructing of the detection model, inputting the train track picture into the detection model, and detecting whether there is a locomotive and a pantograph in a raised state in the train track picture by the detection model, comprises:
[0017] Constructing the locomotive detection model and the pantograph detection model in the pantograph raising state;
[0018] The train track picture is respectively input into the locomotive detection model and the pantograph detection model in the raised state, and the locomotive detection model is used to detect whether there is a locomotive in the train track picture, and the pantograph detection model in the raised state is used to detect whether there is a pantograph in the raised state in the train track picture.
[0019] Optionally, the detecting whether there is a locomotive and a pantograph in a raised state in the train track picture further includes:
[0020] If a locomotive is detected in the train track picture, marking the locomotive in the train track picture by using a first detection frame;
[0021] If it is detected that there is a pantograph in the raised state in the train track image, the pantograph in the raised state is marked in the train track image through a second detection frame.
[0022] Optionally, the detecting whether the locomotive is located in a predetermined locomotive parking zone LMZ comprises:
[0023] Detect whether the first detection frame and the LMZ have overlapping parts;
[0024] If the first detection frame overlaps with the LMZ, the locomotive is located in the LMZ.
[0025] Optionally, the detecting whether the pantograph in the raised state is a pantograph of a locomotive located in a LMZ includes:
[0026] Detect whether the second detection frame and the first detection frame marked on the locomotive located at the LMZ have overlapping parts;
[0027] If the second detection frame and the second detection frame marked on the locomotive located in the LMZ have an overlapping portion, the pantograph in the raised state is the pantograph of the locomotive located in the LMZ.
[0028] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-mentioned methods for detecting pantograph lowering of a locomotive.
[0029] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-mentioned methods for detecting pantograph lowering of a locomotive.
[0030] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for detecting pantograph lowering of a locomotive.
[0031] According to the specific embodiments provided in this application, this application discloses the following technical effects:
[0032] The present application provides a method, device, medium and product for detecting a lowered pantograph of a locomotive. The method comprises the steps of obtaining a train track picture at the entrance of a car dumper, detecting whether there is a locomotive and a pantograph in a raised state in the train track picture. If the locomotive is located in the LMZ and there is a pantograph in a raised state in the train track picture, detecting whether the pantograph in the raised state is the pantograph of the locomotive located in the LMZ. If the locomotive is located in the LMZ and there is no pantograph in a raised state in the train track picture, or if the pantograph in a raised state is not the pantograph of the locomotive located in the LMZ, it indicates that the pantograph of the locomotive located in the LMZ is in a lowered state. Automatic detection of whether the locomotive pantograph is in a lowered state is achieved, solving the problem that errors may occur due to visual fatigue of personnel when confirming whether the locomotive pantograph is in a lowered state through naked eye observation by on-site personnel. The inspection efficiency is improved, while the safety risk of on-site inspectors themselves is reduced. In addition, by detecting whether the locomotive is located in the predetermined locomotive parking area, the locomotives on adjacent train tracks can be prevented from interfering with the detection results when multiple tipping machines are adjacent. By detecting whether the pantograph in the raised state is the pantograph of the locomotive located in the LMZ, the interference with the detection results of some pantographs that are misidentified at a distance can be effectively avoided, and the detection accuracy of whether the locomotive pantograph is in the lowered state can be comprehensively improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0034] Figure 1 A schematic diagram of a flow chart of a method for detecting pantograph lowering of a locomotive provided in one embodiment of the present application;
[0035] Figure 2 A schematic diagram of functional modules of a locomotive pantograph lowering detection device provided in one embodiment of the present application;
[0036] Figure 3 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0038] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0039] In an exemplary embodiment, Figure 1 As shown, a method for detecting pantograph lowering of a locomotive is provided, comprising the following steps 101 to 104. Among them:
[0040] Step 101, obtaining a train track image at the entrance of the dumper.
[0041] In the embodiment of the present application, a picture of the train track at the entrance of the car dumper is obtained by taking a picture of the train track at the entrance of the car dumper room through a camera installed at the entrance of the car dumper room. The camera is preferably installed at the top of the entrance of the car dumper room in the middle of the train track, and the camera is tilted downward to ensure that the train track picture can be taken. There is no specific limitation on the downward tilt angle of the camera (the angle between the direction of the camera and the horizontal plane), which can be set according to actual needs. For example, the camera is set to tilt downward by 30°.
[0042] Step 102, detecting whether there is a locomotive and a pantograph in the raised state in the train track image.
[0043] In the embodiment of the present application, the pantograph in the raised state is a pantograph in the raised state.
[0044] Step 103 : if there is a locomotive in the train track image, it is detected whether the locomotive is located in a predetermined locomotive zone LMZ (LMZ).
[0045] In an embodiment of the present application, by demarcating the locomotive parking area for the acquired train track image (i.e., the surveillance video image) and detecting whether the locomotive is located in the predetermined locomotive parking area, it is possible to prevent the locomotives on adjacent train tracks from interfering with the detection results when multiple tipping machines are adjacent.
[0046] Step 104, if the locomotive is located in the LMZ and there is a pantograph in the raised state in the train track picture, detect whether the pantograph in the raised state is the pantograph of the locomotive located in the LMZ; if the locomotive is located in the LMZ and there is no pantograph in the raised state in the train track picture, or if the pantograph in the raised state is not the pantograph of the locomotive located in the LMZ, it means that the pantograph of the locomotive located in the LMZ is in the lowered state.
[0047] In the embodiment of the present application, on the basis of detecting whether the locomotive is located in the predetermined locomotive parking area, it is detected whether the pantograph in the raised state in the train track picture is the pantograph of the locomotive located in the LMZ, effectively avoiding interference of the detection result by the pantograph in the raised state that is misidentified in some distance, and improving the detection accuracy. If the pantograph of the locomotive located in the LMZ is in the lowered state, a first alarm is issued to remind the staff to tow the locomotive into the tipping machine by the tractor, or directly control the tractor to perform the operation of towing the locomotive into the tipping machine, so as to match the fully automatic tipping machine system.
[0048] Implementing the above steps 101 to 104 realizes the automatic detection of the pantograph lowering of the locomotive based on vision, and solves the problem that the method of visually observing and confirming whether the locomotive pantograph is in the lowered state may cause errors due to visual fatigue of personnel. In addition, by detecting whether the locomotive is located in the predetermined locomotive parking area, the locomotives on adjacent train tracks when multiple car tippers are adjacent are prevented from interfering with the detection results. By detecting whether the pantograph in the raised state is the pantograph of the locomotive located in the LMZ, the interference detection results of some pantographs that are misidentified at a distance are effectively avoided, and the detection accuracy of whether the locomotive pantograph is in the lowered state is comprehensively improved. At the same time, the traditional method of confirming whether the locomotive pantograph is in the lowered state by visual observation of on-site personnel cannot meet the requirements of the fully automatic car tipping system for rapid car tipping operations. The locomotive pantograph lowering detection method proposed in the embodiment of the present application can match the fully automatic car tipping system to meet the requirements of the fully automatic car tipping system for rapid car tipping operations, and improve the car tipping operation efficiency of the fully automatic car tipping system.
[0049] In another exemplary embodiment of the present application, in order to ensure that the pantograph on the locomotive towed by the tractor is in a lowered state and to avoid the pantograph on the locomotive towed by the tractor being in a raised state and colliding with the tipping machine, the locomotive pantograph lowering detection method further includes:
[0050] Step 105, if the pantograph in the raised state is the pantograph of the locomotive located in the LMZ, control the lowering operation of the pantograph in the raised state, or control the issuance of a second alarm to remind the staff to lower the pantograph in the raised state.
[0051] As an optional implementation, in order to ensure that the pantograph of the locomotive located at the entrance of the dumper can be photographed in the raised state, the camera needs to be able to photograph the contact network end sign on the high-voltage line.
[0052] As an optional implementation, in order to ensure that the locomotive located at the entrance of the tipping machine is located in the middle of the train track picture, in the train track picture taken by the camera, the distance between the end sign of the contact network under the high-voltage line and the bottom edge of the train track picture is one-fourth of the height of a pixel.
[0053] In another exemplary embodiment of the present application, in order to achieve rapid and accurate detection of the locomotive and the pantograph in the raised state in the train track picture, the above step 102 includes:
[0054] Step 201, construct a detection model, and input the train track image into the detection model, and use the detection model to detect whether there is a locomotive and a pantograph in the raised state in the train track image.
[0055] In an embodiment of the present application, a detection model is used to simultaneously realize rapid detection of the locomotive and the pantograph in the raised state in the train track image, so as to reduce the computational workload of the processor and the utilization rate of storage resources.
[0056] In another exemplary embodiment of the present application, the detection model includes a locomotive detection model and a pantograph detection model in a pantograph-raised state. Accordingly, the above step 201 includes:
[0057] Step 301, constructing a locomotive detection model and a pantograph detection model in a pantograph-raised state;
[0058] Step 302, input the train track picture into the locomotive detection model and the pantograph detection model in the raised state respectively, detect whether there is a locomotive in the train track picture through the locomotive detection model, and detect whether there is a pantograph in the raised state in the train track picture through the pantograph detection model in the raised state.
[0059] In another exemplary embodiment of the present application, the above step 102 further includes:
[0060] Step 401, if a locomotive is detected in the train track image, the locomotive is marked by a first detection frame;
[0061] Step 402: If a pantograph in the raised state is detected in the train track image, the pantograph in the raised state is marked by a second detection frame.
[0062] In another exemplary embodiment of the present application, in the above step 201, building a detection model includes:
[0063] The above detection model is obtained by using training samples and training the neural network through supervised learning.
[0064] In the training samples, for the train track image samples with a locomotive, the locomotive is marked in the train track image samples through the first detection frame. For the train track image samples with a pantograph in the raised state, the pantograph in the raised state is marked in the train track image samples through the second detection frame.
[0065] In the embodiment of the present application, there is no specific limitation on the neural network, and it can be selected according to actual needs. For example, the yolov7 network is used.
[0066] In another exemplary embodiment of the present application, in the above step 201, building a detection model further includes:
[0067] Before training the neural network through supervised learning, negative samples are added to the training samples. The negative samples include train track picture samples without locomotives and train track picture samples with locomotives without pantographs in the raised state, so as to increase the recognition robustness of the detection model and reduce misidentification.
[0068] In another exemplary embodiment of the present application, the adding of negative samples to the training samples includes:
[0069] For the same negative sample (i.e. the same train track image sample without a locomotive or the same train track image sample with a locomotive without a pantograph in the raised state), Gaussian convolution kernels with different standard deviations are used for smoothing, and additional negative samples are expanded. That is, a new smoothed image is obtained by convolving the Gaussian convolution kernel with each pixel position and its neighborhood in each negative sample to increase generalization, reduce the interference of complex environment and light problems on recognition, and reduce misrecognition.
[0070] In another exemplary embodiment of the present application, each negative sample is smoothed according to the Gaussian convolution kernel shown in the following formula:
[0071]
[0072] Among them, σ is the standard deviation, (x, y) is the coordinate point in two-dimensional space, and is the offset relative to the center of the convolution kernel (x0, y0).
[0073] In the embodiment of the present application, by selecting a suitable σ value, different additional negative samples are expanded according to each negative sample. There is no specific limitation on the number and value of σ, which can be selected according to actual needs. For example, σ is 0.2, 0.5, 0.8, 1.0, and 1.6 respectively, so that n additional negative samples can be expanded for n negative samples.
[0074] In another exemplary embodiment of the present application, the above step 102 further includes:
[0075] Step 403, if a locomotive is detected in the train track image, control the detection model to output: first detection frame parameters and the train track image marked by the first detection frame; wherein the first detection frame parameters include any one of the coordinates of the upper left corner, lower left corner, upper right corner and lower right corner of the first detection frame, the first detection frame width and the first detection frame height;
[0076] Step 404, if a pantograph in a raised state is detected in the train track image, control the detection model to output: second detection frame parameters and the train track image marked by the second detection frame; wherein the second detection frame parameters include any one of the coordinates of the upper left corner, lower left corner, upper right corner and lower right corner of the second detection frame, the second detection frame width and the height of the second detection frame;
[0077] Step 405, if a locomotive and a pantograph in the raised state are detected in the train track image, the detection model is controlled to output: detection frame parameters and the train track image marked by the detection frame, wherein the detection frame is a first detection frame and a second detection frame.
[0078] In another exemplary embodiment of the present application, the above step 102 further includes:
[0079] Step 501, detecting whether there is a first detection frame and a second detection frame in the train track image output by the above detection model;
[0080] Step 502: If the first detection frame exists in the train track image output by the detection model, it means that there is a locomotive in the train track image; otherwise, it means that there is no locomotive in the train track image.
[0081] Step 503, if the second detection frame exists in the train track image output by the detection model, it means that there is a pantograph in the raised state in the train track image, otherwise, it means that there is no pantograph in the raised state in the train track image.
[0082] In another exemplary embodiment of the present application, in the above step 201, building the detection model further includes:
[0083] The train track image samples in the training samples are labeled with a locomotive or without a locomotive label;
[0084] The train track image samples in the training samples are annotated with labels of pantographs in the raised state or pantographs in the non-raised state.
[0085] Correspondingly, in the above step 403, if a locomotive is detected in the train track image, the detection model output is controlled to further include a locomotive label or a locomotive-free label.
[0086] In the above step 404, if it is detected that there is a pantograph in the raised state in the train track image, the control detection model output also includes a pantograph in the raised state or a pantograph in the non-raised state.
[0087] In the embodiment of the present application, by limiting the detection model to directly output the label of having a locomotive or not having a locomotive and the pantograph in the raised state or not having the pantograph in the raised state, it is avoided to detect whether there is a first detection box and a second detection box in the train track image output by the above-mentioned detection model, thereby simplifying the detection process.
[0088] In another exemplary embodiment of the present application, in the above step 103, detecting whether the locomotive is located in the predetermined locomotive parking zone LMZ includes:
[0089] Step 601, detecting whether the first detection frame and the LMZ have overlapping parts;
[0090] Step 602: If the first detection frame overlaps the LMZ, the locomotive is located in the LMZ.
[0091] In another exemplary embodiment of the present application, the above step 601 includes:
[0092] According to the coordinates (x1, y1), width w1, height h1 of the first corner of the first detection frame and the coordinates (x2, y2), width w2, height h2 of the second corner of the rectangular structure, determine whether the first detection frame and the rectangular structure have overlapping parts. The rectangular structure includes LMZ, the first corner is any one of the upper left corner, lower left corner, upper right corner and lower right corner of the first detection frame, and the second corner is any one of the upper left corner, lower left corner, upper right corner and lower right corner of the rectangular structure.
[0093] In another exemplary embodiment of the present application, in the above step 104, detecting whether the pantograph in the raised state is the pantograph of the locomotive located in the LMZ includes:
[0094] Step 701, detecting whether the second detection frame and the first detection frame marked on the locomotive located at the LMZ have overlapping parts;
[0095] Step 702: If the second detection frame and the second detection frame marked on the locomotive located in the LMZ have overlapping parts, the pantograph in the raised state is the pantograph of the locomotive located in the LMZ.
[0096] In another exemplary embodiment of the present application, the above step 701 includes:
[0097] According to the coordinates (x1, y1), width w1, height h1 of the first corner of the first detection frame and the coordinates (x2, y2), width w2, height h2 of the second corner of the rectangular structure, determine whether the first detection frame and the rectangular structure have overlapping parts. The rectangular structure includes the second detection frame, the first corner is any one of the upper left corner, lower left corner, upper right corner and lower right corner of the first detection frame, and the second corner is any one of the upper left corner, lower left corner, upper right corner and lower right corner of the rectangular structure.
[0098] In another exemplary embodiment of the present application, the above-mentioned determining whether the first detection frame and the rectangular structure have an overlapping portion according to the coordinates (x1, y1), width w1, height h1 of the first corner of the first detection frame and the coordinates (x2, y2), width w2, height h2 of the second corner of the rectangular structure includes the following steps 1 to 3, wherein:
[0099] Step 1: Determine the intersection of the first detection frame and the rectangular structure according to the coordinates (x1, y1), width w1, height h1 of the first detection frame and the coordinates (x2, y2), width w2, height h2 of the rectangular structure.
[0100] Step 2: If the intersection of the first detection frame and the rectangular structure is zero, it means that the first detection frame has no overlapping part with the rectangular structure.
[0101] Step 3: If the intersection of the first detection frame and the rectangular structure is greater than zero, it means that the first detection frame and the rectangular structure have overlapping parts.
[0102] In the embodiment of the present application, it is assumed that (x1, y1) is the coordinate of the upper left corner of the first detection frame, and (x2, y2) is the coordinate of the upper left corner of the rectangular structure. Accordingly, the above step 1 includes:
[0103] If the first detection box and the rectangular structure meet at least one of the following conditions, it means that the intersection of the first detection box and the rectangular structure is equal to zero. Otherwise (that is, the first detection box and the rectangular structure do not meet all of the following conditions), it means that the intersection of the first detection box and the rectangular structure is greater than zero:
[0104] min(x1+w1,x2+w2)≤max(x1,x2);
[0105] min(y1,y2)≤max(y1-h1,y2-h2).
[0106] Assuming that (x1, y1) is the coordinate of the upper right corner of the first detection frame, and (x2, y2) is the coordinate of the upper right corner of the rectangular structure, accordingly, the above step 1 includes:
[0107] If the first detection frame and the rectangular structure meet at least one of the following conditions, it means that the intersection of the first detection frame and the rectangular structure is equal to zero; otherwise, it means that the intersection of the first detection frame and the rectangular structure is greater than zero:
[0108] min(x1,x2)≤max(x1-w1,x2-w2);
[0109] min(y1,y2)≤max(y1-h1,y2-h2).
[0110] Assuming that (x1, y1) is the coordinate of the lower left corner of the first detection frame, and (x2, y2) is the coordinate of the lower left corner of the rectangular structure, accordingly, the above step 1 includes:
[0111] If the first detection frame and the rectangular structure meet at least one of the following conditions, it means that the intersection of the first detection frame and the rectangular structure is equal to zero; otherwise, it means that the intersection of the first detection frame and the rectangular structure is greater than zero:
[0112] min(x1+w1,x2+w2)≤max(x1,x2);
[0113] min(y1+h1,y2+h2)≤max(y1,y2).
[0114] Assuming that (x1, y1) is the coordinate of the lower right corner of the first detection frame, and (x2, y2) is the coordinate of the lower right corner of the rectangular structure, accordingly, the above step 1 includes:
[0115] If the first detection frame and the rectangular structure meet at least one of the following conditions, it means that the intersection of the first detection frame and the rectangular structure is equal to zero; otherwise, it means that the intersection of the first detection frame and the rectangular structure is greater than zero:
[0116] min(x1,x2)≤max(x1-w1,x2-w2);
[0117] min(y1+h1,y2+h2)≤max(y1,y2).
[0118] In another exemplary embodiment of the present application, the above-mentioned determining whether the first detection frame and the rectangular structure have an overlapping portion according to the coordinates (x1, y1), width w1, and height h1 of the first corner of the first detection frame and the coordinates (x2, y2), width w2, and height h2 of the second corner of the rectangular structure includes the following steps (1) to (3), wherein:
[0119] Step (1), determining the intersection-over-union ratio of the first detection frame and the rectangular structure according to the coordinates (x1, y1), width w1, height h1 of the first detection frame and the coordinates (x2, y2), width w2, height h2 of the rectangular structure.
[0120] Step (2): if the intersection-and-union ratio of the first detection frame and the rectangular structure is zero, it means that there is no overlapping portion between the first detection frame and the rectangular structure.
[0121] Step (3): if the intersection-and-union ratio of the first detection frame and the rectangular structure is greater than zero, it means that the first detection frame and the rectangular structure have overlapping parts.
[0122] In another exemplary embodiment of the present application, the above-mentioned determination of whether the first detection frame and the rectangular structure have an overlapping portion based on the coordinates (x1, y1), width w1, and height h1 of the first corner of the first detection frame and the coordinates (x2, y2), width w2, and height h2 of the second corner of the rectangular structure includes the following steps a to b. Among them:
[0123] Step a, determining the positional relationship between the first detection frame and the rectangular structure according to the coordinates (x1, y1), width w1, height h1 of the first detection frame and the coordinates (x2, y2), width w2, height h2 of the rectangular structure.
[0124] Step b: if the first detection frame is completely located on the left side, right side, above or below the rectangular structure, the first detection frame has no overlapping portion with the rectangular structure; otherwise, the first detection frame has overlapping portion with the rectangular structure.
[0125] In the embodiment of the present application, it is assumed that (x1, y1) is the coordinate of the upper left corner of the first detection frame, and (x2, y2) is the coordinate of the upper left corner of the rectangular structure. Accordingly, the above step a includes:
[0126] If the first detection box and the rectangular structure meet at least one of the following conditions, it means that the first detection box is completely located on the left, right, above, or below the rectangular structure:
[0127] x1+w1≤x2;
[0128] x2+w2≤x1;
[0129] y1-h1>y2;
[0130] y2-h2>y1;
[0131] Among them, x1+w1≤x2 means that the first detection box is completely located on the left side of the rectangular structure, x2+w2≤x1 means that the first detection box is completely located on the right side of the rectangular structure, y1-h1>y2 means that the first detection box is completely located above the rectangular structure, and y2-h2>y1 means that the first detection box is completely located below the rectangular structure.
[0132] Assuming that (x1, y1) is the coordinate of the upper right corner of the first detection frame, and (x2, y2) is the coordinate of the upper right corner of the rectangular structure, accordingly, the above step a includes:
[0133] If the first detection box and the rectangular structure meet at least one of the following conditions, it means that the first detection box is completely located on the left, right, above, or below the rectangular structure:
[0134] x1-w1>x2;
[0135] x2-w2>x1;
[0136] y1-h1>y2;
[0137] y2-h2>y1;
[0138] Among them, x1-w1>x2 means that the first detection box is completely located on the right side of the rectangular structure, and x2-w2>x1 means that the first detection box is completely located on the left side of the rectangular structure.
[0139] Assuming that (x1, y1) is the coordinate of the lower left corner of the first detection frame, and (x2, y2) is the coordinate of the lower left corner of the rectangular structure, accordingly, the above step a includes:
[0140] If the first detection box and the rectangular structure meet at least one of the following conditions, it means that the first detection box is completely located on the left, right, above, or below the rectangular structure:
[0141] x1+w1≤x2;
[0142] x2+w2≤x1;
[0143] y1+h1≤y2;
[0144] y2+h2≤y1;
[0145] Among them, y1+h1≤y2 means that the first detection frame is completely located below the rectangular structure, and y2+h2≤y1 means that the first detection frame is completely located above the rectangular structure.
[0146] Assuming that (x1, y1) is the coordinate of the lower right corner of the first detection frame, and (x2, y2) is the coordinate of the lower right corner of the rectangular structure, accordingly, the above step a includes:
[0147] If the first detection box and the rectangular structure meet at least one of the following conditions, it means that the first detection box is completely located on the left, right, above, or below the rectangular structure:
[0148] x1-w1>x2;
[0149] x2-w2>x1;
[0150] y1+h1≤y2;
[0151] y2+h2≤y1.
[0152] Based on the same inventive concept, the embodiment of the present application also provides a locomotive pantograph lowering detection device for implementing the locomotive pantograph lowering detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more locomotive pantograph lowering detection device embodiments provided below can refer to the limitations of the locomotive pantograph lowering detection method above, and will not be repeated here.
[0153] In an exemplary embodiment, Figure 2 As shown, a locomotive pantograph lowering detection device 80 is provided, comprising:
[0154] The picture acquisition module 801 is used to acquire the train track picture at the entrance of the dumper;
[0155] The first detection module 802 is used to detect whether there is a locomotive and a pantograph in a raised state in the train track picture;
[0156] The second detection module 803 is used to detect whether the locomotive is located in a predetermined locomotive parking zone LMZ if there is a locomotive in the train track image;
[0157] The third detection module 804 is used to detect whether the pantograph in the raised state is the pantograph of the locomotive located in the LMZ if the locomotive is located in the LMZ and there is a pantograph in the raised state in the train track picture. If the locomotive is located in the LMZ and there is no pantograph in the raised state in the train track picture, or if the pantograph in the raised state is not the pantograph of the locomotive located in the LMZ, it means that the pantograph of the locomotive located in the LMZ is in the lowered state.
[0158] In another exemplary embodiment of the present application, the locomotive pantograph lowering detection device 80 further includes:
[0159] The pantograph lowering operation control module 805 is used to control the pantograph lowering operation of the pantograph in the raised state if the pantograph in the raised state is the pantograph of the locomotive located in the LMZ, or to control the issuance of a second alarm to remind the pantograph lowering operation of the pantograph in the raised state.
[0160] In another exemplary embodiment of the present application, the first detection module 802 is further used to:
[0161] Build a detection model and input the train track image into the detection model. Use the detection model to detect whether there is a locomotive and a pantograph in the raised state in the train track image.
[0162] In another exemplary embodiment of the present application, the detection model includes a locomotive detection model and a pantograph detection model in a raised state. Accordingly, the first detection module 802 is further used to:
[0163] Construct locomotive detection model and pantograph detection model in pantograph raising state;
[0164] The train track picture is input into the locomotive detection model and the pantograph detection model in the raised state respectively. The locomotive detection model is used to detect whether there is a locomotive in the train track picture, and the pantograph detection model in the raised state is used to detect whether there is a pantograph in the raised state in the train track picture.
[0165] In another exemplary embodiment of the present application, the first detection module 802 is further used to:
[0166] If a locomotive is detected in the train track image, the locomotive is marked by a first detection frame;
[0167] If a pantograph in the raised state is detected in the train track image, the pantograph in the raised state is marked by a second detection frame.
[0168] In another exemplary embodiment of the present application, the first detection module 802 is further used to:
[0169] Build a detection model, including:
[0170] The above detection model is obtained by using training samples and training the neural network through supervised learning.
[0171] In another exemplary embodiment of the present application, the first detection module 802 is further used to:
[0172] Before training the neural network through supervised learning, negative samples are added to the training samples, and the negative samples include train track picture samples without a locomotive and train track picture samples with a locomotive without a pantograph in a raised state. The introduction to adding negative samples to the training samples is detailed in the above method embodiment, and will not be repeated here.
[0173] In another exemplary embodiment of the present application, the first detection module 802 is further used to:
[0174] If a locomotive is detected in the train track image, control the detection model to output: first detection frame parameters and the train track image marked by the first detection frame; wherein the first detection frame parameters include any one of the coordinates of the upper left corner, the lower left corner, the upper right corner and the lower right corner of the first detection frame, the width of the first detection frame and the height of the first detection frame;
[0175] If a pantograph in a raised state is detected in the train track image, the detection model is controlled to output: a second detection frame parameter and a train track image marked by the second detection frame; wherein the second detection frame parameter includes any one of the coordinates of the upper left corner, the lower left corner, the upper right corner and the lower right corner of the second detection frame, the second detection frame width and the second detection frame height;
[0176] If a locomotive and a pantograph in the raised state are detected in the train track image, the detection model is controlled to output: detection frame parameters and the train track image marked by the detection frame, wherein the detection frames are the first detection frame and the second detection frame.
[0177] In another exemplary embodiment of the present application, the first detection module 802 is further used to:
[0178] Detect whether the first detection frame and the second detection frame exist in the train track image output by the above detection model;
[0179] If the first detection frame exists in the train track image output by the detection model, it means that there is a locomotive in the train track image; otherwise, it means that there is no locomotive in the train track image;
[0180] If the second detection frame exists in the train track image output by the detection model, it means that there is a pantograph in the raised state in the train track image; otherwise, it means that there is no pantograph in the raised state in the train track image.
[0181] In another exemplary embodiment of the present application, the first detection module 802 is further used to:
[0182] The train track image samples in the training samples are labeled with a locomotive or without a locomotive label;
[0183] The train track image samples in the training samples are labeled with a label indicating a pantograph in a raised state or a pantograph in a non-raised state;
[0184] If a locomotive is detected in the train track image, the control detection model output also includes a locomotive or no locomotive label;
[0185] If a pantograph in a raised state is detected in the train track image, the control detection model output also includes a pantograph in a raised state or a pantograph in a non-raised state.
[0186] In another exemplary embodiment of the present application, the second detection module 803 is further used to:
[0187] Check whether the first detection frame and LMZ have overlapping parts;
[0188] If the first detection frame overlaps with the LMZ, the locomotive is located in the LMZ.
[0189] In another exemplary embodiment of the present application, the second detection module 803 is further used to:
[0190] According to the coordinates (x1, y1), width w1, height h1 of the first corner of the first detection frame and the coordinates (x2, y2), width w2, height h2 of the second corner of the rectangular structure, determine whether the first detection frame and the rectangular structure have overlapping parts, wherein the rectangular structure includes LMZ, the first corner is any one of the upper left corner, lower left corner, upper right corner and lower right corner of the first detection frame, and the second corner is any one of the upper left corner, lower left corner, upper right corner and lower right corner of the rectangular structure. For detailed introduction, please refer to the records of the above method embodiment, which will not be repeated here.
[0191] In another exemplary embodiment of the present application, the third detection module 804 is further used to:
[0192] Detect whether the second detection frame and the first detection frame marked on the locomotive located at the LMZ have overlapping parts;
[0193] If the second detection frame and the second detection frame marked on the locomotive located in the LMZ have overlapping parts, the pantograph in the raised state is the pantograph of the locomotive located in the LMZ.
[0194] In another exemplary embodiment of the present application, the third detection module 804 is further used to:
[0195] According to the coordinates (x1, y1), width w1, height h1 of the first corner of the first detection frame and the coordinates (x2, y2), width w2, height h2 of the second corner of the rectangular structure, determine whether the first detection frame and the rectangular structure have overlapping parts, wherein the rectangular structure includes the second detection frame, the first corner is any one of the upper left corner, lower left corner, upper right corner and lower right corner of the first detection frame, and the second corner is any one of the upper left corner, lower left corner, upper right corner and lower right corner of the rectangular structure. For detailed introduction, please refer to the description of the above method embodiment, which will not be repeated here.
[0196] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 3As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store locomotive pantograph lowering detection data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a locomotive pantograph lowering detection method is implemented.
[0197] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0198] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0199] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0200] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0201] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0202] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0203] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.
[0204] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0205] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. A method for detecting pantograph lowering of a locomotive, characterized in that: include: Get a picture of the train tracks at the entrance to the dumper; Detecting whether there is a locomotive and a pantograph in a raised state in the train track picture; If there is a locomotive in the train track image, detecting whether the locomotive is located in a predetermined locomotive parking zone LMZ; If the locomotive is located in the LMZ and there is a pantograph in the raised state in the train track picture, detect whether the pantograph in the raised state is the pantograph of the locomotive located in the LMZ; if the locomotive is located in the LMZ and there is no pantograph in the raised state in the train track picture, or if the pantograph in the raised state is not the pantograph of the locomotive located in the LMZ, it means that the pantograph of the locomotive located in the LMZ is in the lowered state.
2. The method for detecting pantograph lowering of a locomotive according to claim 1, characterized in that: Also includes: If the pantograph in the raised state is the pantograph of a locomotive located in the LMZ, the control performs a pantograph lowering operation on the pantograph in the raised state, or the control issues a second alarm to remind the user to perform a pantograph lowering operation on the pantograph in the raised state.
3. The pantograph lowering detection method according to claim 1 is characterized in that: The detecting whether there is a locomotive and a pantograph in a raised state in the train track picture includes: A detection model is constructed, and the train track image is input into the detection model, and the detection model is used to detect whether there is a locomotive and a pantograph in a raised state in the train track image.
4. The method for detecting pantograph lowering of a locomotive according to claim 3, characterized in that: The detection model includes a locomotive detection model and a pantograph detection model in a raised state; The constructing of the detection model, inputting the train track picture into the detection model, and detecting whether there is a locomotive and a pantograph in a raised state in the train track picture by the detection model, comprises: Constructing the locomotive detection model and the pantograph detection model in the pantograph raising state; The train track picture is respectively input into the locomotive detection model and the pantograph detection model in the raised state, and the locomotive detection model is used to detect whether there is a locomotive in the train track picture, and the pantograph detection model in the raised state is used to detect whether there is a pantograph in the raised state in the train track picture.
5. The method for detecting pantograph lowering of a locomotive according to claim 1, characterized in that: The detecting whether there is a locomotive and a pantograph in a raised state in the train track picture also includes: If a locomotive is detected in the train track picture, marking the locomotive in the train track picture by using a first detection frame; If it is detected that there is a pantograph in the raised state in the train track image, the pantograph in the raised state is marked in the train track image through a second detection frame.
6. The method for detecting pantograph lowering of a locomotive according to claim 5, characterized in that: The detecting whether the locomotive is located in a predetermined locomotive parking zone LMZ comprises: Detect whether the first detection frame and the LMZ have overlapping parts; If the first detection frame overlaps with the LMZ, the locomotive is located in the LMZ.
7. The method for detecting pantograph lowering of a locomotive according to claim 5, characterized in that: The detecting whether the pantograph in the raised state is the pantograph of the locomotive located in the LMZ comprises: Detect whether the second detection frame and the first detection frame marked on the locomotive located at the LMZ have overlapping parts; If the second detection frame and the second detection frame marked on the locomotive located in the LMZ have an overlapping portion, the pantograph in the raised state is the pantograph of the locomotive located in the LMZ.
8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the pantograph lowering detection method for a locomotive according to any one of claims 1 to 7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the locomotive pantograph lowering detection method described in any one of claims 1 to 7 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the locomotive pantograph lowering detection method described in any one of claims 1 to 7 are implemented.