A method for anti-collision inspection of electric locomotives for mines

Abstract

The invention discloses a method for anti-collision inspection of mining electric locomotives in the field of anti-collision inspection technology, comprising an explosion-proof camera, a Mini server, a storage module, a power supply module, a communication controller and a flameproof shell; step one: the explosion-proof camera collects real-time video data and transmits it to the Mini server, the user can set the duration of a single backup video and the total number of backup videos, the Mini server automatically backs up in the storage module according to the user's settings, when the total number of backed up videos exceeds the total number of videos set by the user, the Mini server can automatically clean up old videos, only one monocular camera is needed to realize the function, while ensuring accuracy, reducing costs; the system structure adopts a modular design, with strong reconfigurability, good expansibility and compatibility; the use of the Mini server can provide better computing performance and development support, and more functional frameworks can be built on the platform later.

Description

Technical Field

[0001] The invention relates to the technical field of anti-collision inspection, and in particular to an anti-collision inspection method for a mining electric locomotive. Background Art

[0002] Unmanned driving of electric locomotives is an important part of realizing unmanned mines. Unmanned driving technology frees drivers from heavy work underground. Combined with scientific decision-making vehicle dispatching technology, it can reduce the idle rate of locomotives, and has obvious advantages and practical significance in transportation efficiency and safety.

[0003] There may be underground workers, fallen coal piles or other electric locomotives occupying the tracks on the locomotive's route. The most basic function of unmanned driving technology is to identify obstacles ahead and take braking measures to prevent injuries, rollovers and rear-end collisions.

[0004] Compared with traditional anti-collision detection, the anti-collision detection environment of underground electric locomotives is more stringent. The light underground is very weak, the brightness is low, there is a lot of dust, the humidity is high, and it is often accompanied by water and corrosion. The operation of the electric locomotive is also accompanied by vibration, extrusion, etc. These harsh working environments make the actual working conditions of the anti-collision detection platform very complicated, and all equipment must meet the intrinsic safety and explosion-proof requirements. These have become technical difficulties that need to be urgently solved in underground anti-collision detection. Based on this, the present invention designs a method for anti-collision inspection of mining electric locomotives to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a method for anti-collision inspection of a mining electric locomotive to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions: a method for anti-collision inspection of a mining electric locomotive, comprising an explosion-proof camera, a Mini server, a storage module, a power supply module, a communication controller and a flameproof housing;

[0007] Step 1: The explosion-proof camera collects real-time video data and transmits it to the Mini server. The user can set the duration of a single backup video and the total number of backup videos. The Mini server automatically backs up in the storage module according to the user's settings. When the total number of backed up videos exceeds the total number of videos set by the user, the Mini server can automatically clean up old videos.

[0008] Step 2: The Mini server captures real-time video data and forwards it to the image processing program, which includes the functions of target recognition, target ranging, overlap detection, and lane line detection;

[0009] Step 3: After the image processing program processes a frame of video data, it can pass the result data packet to the logic processing program. The program can read user-defined parameters: distance threshold, overlap threshold. After the image processing completes each frame of the picture, the server will give a binary data packet in the following format [[category 1, distance 1, overlap 1], [category 2, distance 2, overlap 2], ... [category n, distance n, overlap n]].

[0010] The communication program (C++ / Rust) in the server will parse and process the data packets, based on the position of the track, combined with the threshold Tl of the anti-collision operation distance of the locomotive, assuming that the overlap threshold is [Tc_min, Tc_max], and after completing the processing through the logic algorithm, the alarm signal will be transmitted to the communication controller through the TCP / CAN protocol;

[0011] Step 4: When the result data packet parsed by the logic processing program meets different threshold intervals, the logic processing program will obtain different alarm signals, which are green for safety signals, blue for potential danger signals, and red for danger signals according to the levels;

[0012] Step 5: Finally, the communication program sends the alarm signal to the back-end communication controller through the TCP protocol, which controls the electric locomotive to take corresponding actions.

[0013] Preferably, the image backup strategy in step 2 adopts a multi-threaded program, thread T1 is responsible for forwarding the image acquisition data directly to the image processing module, thread T2 is responsible for copying the image data to the local disk and saving the file with a timestamp name, and the system can set the video duration Ts for a single storage and the maximum number Nmax of videos that can be stored locally.

[0014] Preferably, the image cleaning strategy is to detect the number of data N in the storage location each time the system is started, and determine whether N is greater than the set maximum storage number Nmax. If it is greater, traverse and delete the earliest video in timestamp order, N minus one, and then determine the number of N again. If it is less than Nmax at this time, store a new video, N plus one, otherwise repeat the video deletion operation again; after storing the video N plus one, the next video still needs to determine whether the number N is greater than Nmax, and when a system interruption occurs, exit the system and shut down the device.

[0015] Preferably, the AI ​​processing program inside the Mini server can automatically identify the attributes, distance and other parameter information of the obstacle in front of the locomotive, and make an intelligent judgment based on the safety threshold parameters set by the user, and transmit the alarm signal to the communication controller, which controls the locomotive to take corresponding measures.

[0016] Preferably, a single convolutional neural network (CNN) is applied to the entire image for object detection, dividing the image into grids and predicting the class probability and bounding box for each grid.

[0017] Preferably, for each grid cell, the network predicts a bounding box and the probability corresponding to each class.

[0018] Each bounding box can be described using four descriptors:

[0019] 1. The center of the bounding box;

[0020] 2. Height;

[0021] 3. Width;

[0022] 4. The value is mapped to the class to which the object belongs;

[0023] In addition, the algorithm can also predict the probability of an object existing in the bounding box; if the center of an object falls in a grid cell, then the grid cell is responsible for detecting the object; there will be multiple bounding boxes in each grid; when training, we want to have only one bounding box per object; therefore, we assign a box to be responsible for predicting the object based on which box has the highest overlap with the ground truth box;

[0024] Finally, we apply a method called “Non Max Suppression” to objects of each class to filter out bounding boxes whose “confidence” is less than a threshold;

[0025] Preferably, the target distance measurement is to calculate the distance from the object or target to the camera using similar triangles. Similar triangles are used to calculate the distance from the camera to a known object or target. Assume that there is a target or object with a width W; then the target is placed at a distance D from the camera; the camera is used to take a picture of the object and the pixel width P of the object is measured;

[0026] This gives us the formula for the focal length of the camera: F = (PxD) / W

[0027] When the camera is continuously moved closer to or farther away from an object or target, similar triangles can be used to calculate the distance between the object and the camera: D'=(WxF) / P.

[0028] Preferably, firstly, according to the camera calibration principle, an intrinsic parameter matrix of the camera is obtained;

[0029] Then, a single frame image containing the target is collected, and the target area in the image is identified. The maximum value of the vertical coordinate of the pixel coordinates in the area and the corresponding average value of the horizontal coordinate are calculated, and the obtained coordinates are used as observation points;

[0030] Finally, the three-dimensional information of the observation point in the world coordinate system is obtained based on the image pixels, and the distance of the observation point is calculated based on the three-dimensional information of the observation point in the body coordinate system.

[0031] The following steps are involved:

[0032] Step 1: Calibrate the monocular camera and obtain the camera's intrinsic parameter matrix [fx,0,u0; 0,fy,v0; 0,0,1];

[0033] Step 2: Use a calibrated monocular camera to capture a single frame image containing the target, process the obtained image, extract the target area, calculate the maximum value of the ordinate of the pixel coordinates in the area, and the average value of the corresponding abscissa, and use the combined coordinates as the observation point P;

[0034] Step 3: According to the camera calibration principle, the three-dimensional coordinate value Pb (xb, yb, zb) of the observation point in the body coordinate system can be obtained from the pixel coordinates Pp (u, v) of the observation point, and then the distance of the observation point can be calculated. The specific method is as follows:

[0035] 1) The intrinsic parameter matrix of the camera can be obtained by step 1, and the pixel point of the observation point can be obtained by step 2, where λ represents the angle between the light at the observation point and the optical axis of the camera, y represents the ordinate of the observation point in the image coordinate system, y0 represents the ordinate of the optical center of the camera in the image coordinate system, and f represents the focal length of the camera.

[0036] Further simplified to the pixel coordinate system, the relationship is as follows:

[0037] Where v represents the ordinate of the observation point in the pixel coordinate system, v0 represents the ordinate of the optical center of the camera in the pixel coordinate system, dy represents the physical size of each pixel in the y direction, and fy represents the normalized focal length on the y axis;

[0038] 2) In triangular geometry, the O1P2 distance is expressed as follows:

[0039] O1O2 is the height of the camera from the ranging plane, the projection point of P1 on the camera optical axis is P2, and O1P2 represents the depth information, that is, the value of zc in the observation point Pc (xc, yc, zc) in the camera coordinate system.

[0040] 3) Convert pixel coordinates to camera coordinates, and the conversion relationship between pixel coordinate system and camera coordinate system.

[0041] Combine the pixel coordinates of the observation point with the depth information zc of the observation point in the camera coordinate system obtained in the previous step to obtain the coordinates Pc (xc, yc, zc) of the observation point in the camera coordinate system;

[0042] Calculate the conversion formula from the camera coordinate system to the body coordinate system, where Rcb is the rotation matrix from the camera coordinate system to the body coordinate system, and Tcb is the translation matrix from the camera coordinate system to the body coordinate system.

[0043] 4) Calculate the distance D of the observation point;

[0044] 5) where xOy is the image coordinate system, Zc represents the Z axis and optical axis of the camera coordinate system, XbO2Yb represents the Z=0 plane (i.e., the ranging plane) in the body coordinate system, O1 represents the camera lens, the two dotted lines a and b represent the visual field range, θ represents the camera pitch angle, H represents the height from the camera to the ranging plane, point P1 is the observation point, the imaging point of observation point P1 on the image plane is point P, the projection point on the optical axis is point P2, the projection point on the X-axis is point P3, and the imaging point of point P3 on the image plane is point P0.

[0045] 9. A method for anti-collision inspection of a mining electric locomotive according to claim 8, characterized in that the lane line detection includes the following steps:

[0046] 1) Obtain a dataset;

[0047] First, a sliding window of fixed width (width w and height h) is selected, and training samples are selected from the video set, with the label a (0: no track, 1: dual tracks, 2: multiple tracks). Secondly, a large-scale data set is a prerequisite for the successful application of deep neural networks. In order to ensure the complexity of the sample set, a series of random changes are made to the image through image augmentation technology to generate a sufficient amount of training data set;

[0048] 2) Define and initialize the model;

[0049] Create a neural network as the target model so that the final output is equal to the category of the target dataset. Let the weight and bias parameters of softmax regression be W and b respectively, and the i-th image sample be x (i) , the output layer is o (i) , the predicted probability distribution is The calculation expression of the classification vector of the sample by softmax regression is:

[0050] o (i) =x (i) W+b

[0051]

[0052] Constructing a suitable loss function, such as cross entropy, is a common method.

[0053]

[0054] 3) Training model;

[0055] In the training model, we need to generate multi-scale aiming frames during the forward calculation process of the model, set the prediction category for each aiming frame, and select the target that is closest to the drone and centered as the current track.

[0056] 4) Track line detection based on edge algorithm;

[0057] By choosing a certain step size and sliding window, we can easily find double or triple track areas. Failure or no track is considered a fault. Here we only discuss the confirmation of further track lines under normal circumstances.

[0058] Based on the edge recognition algorithm, all straight lines in the track area are generated.

[0059] y=k(α)x+b

[0060] In the formula, α is the angle between the line and the horizontal viewing angle. It is easy to remove the cross lines and noise lines in the track by angle classification. By setting the line density, the distance from the origin to the line, etc. to group the tracks, the left and right tracks can be identified.

[0061] Preferably, the target overlap determination is to calculate the pixel overlap between the target and the track based on the result of the target detection. Assuming that the overlapping pixels are PC and the pixels selected by the target frame are PA, the overlap degree is: S=PC / PA.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] Image processing is completed based on deep learning methods, and only one monocular camera is needed to achieve the function, which reduces costs while ensuring accuracy;

[0064] The system structure adopts modular design, with strong reconfigurability, good expandability and compatibility.

[0065] Using Mini servers can provide better computing performance and development support, and more functional frameworks can be built on this platform later.

[0066] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0068] Figure 1This is the overall system control block diagram of the present invention;

[0069] Figure 2 It is a flow chart of the image cleaning strategy of the present invention;

[0070] Figure 3 Schematic diagram of similar triangle algorithm of the present invention;

[0071] Figure 4 A coordinate diagram for calculating the distance D of the observation points in the present invention. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] See also Figures 1 to 4 , the present invention provides a technical solution for a method for anti-collision inspection of electric locomotives for mining: an explosion-proof camera collects real-time video data and transmits it to a Mini server. The user can set the duration of a single backup video and the total number of backup videos. The server automatically backs up in the storage module according to the user's settings. When the total number of backed-up videos exceeds the total number of videos set by the user, the server can automatically clean up old videos. After capturing real-time video data, the Mini server forwards it to an image processing program, which includes the functions of target recognition, target ranging, overlap detection, and lane line detection. After processing a frame of video data, the image processing program can pass the result data packet to the logic processing program, which can read user-defined parameters: distance threshold and overlap threshold. When the result data packet parsed by the logic processing program meets different threshold intervals, the logic processing program will obtain different alarm signals, which are green (safe), blue (there are hidden dangers), and red (dangerous) according to the levels. Finally, the communication program sends the alarm signal to the back-end communication controller through the TCP protocol, which controls the electric locomotive to take corresponding actions,

[0074] The number of explosion-proof cameras can be set at multiple placement points according to the actual use location. Each placement point is equipped with a detection power circuit to detect whether the camera is faulty. If a fault occurs, an alarm is used for alarm processing. The alarm is a photoelectric alarm, which is used for people around to find and maintain it. The detection power circuit is a detection photoelectric alarm set in series with the explosion-proof camera and the main power circuit. If there is a power outage, the photoelectric alarm goes out, which means that the explosion-proof camera is in a power-off and non-working state, so that the staff can find it immediately to avoid affecting the subsequent anti-collision behavior of the electric locomotive, and ensure that the device has stable preliminary preparations.

[0075] Image acquisition and backup

[0076] Image backup strategy:

[0077] A multi-threaded program is used, thread T1 is responsible for forwarding the image acquisition data directly to the image processing module, thread T2 is responsible for copying the image data to the local disk and saving the file with a timestamp name. The system can set the video storage duration Ts for a single time and the maximum number of videos that can be stored locally Nmax.

[0078] Image Cleanup Strategy:

[0079] Each time the system starts, it detects the number of data N in the storage location and determines whether N is greater than the set maximum storage number Nmax. If it is greater, it traverses and deletes the earliest video in timestamp order, N minus one, and then determines the number of N again. If it is less than Nmax at this time, it stores a new video and N plus one. Otherwise, it repeats the video deletion operation again. After storing the video N plus one, the next video still needs to determine whether the number of N is greater than Nmax. When a system interruption occurs, the system is exited and the device is shut down.

[0080] Object Detection

[0081] During the driving process, the drone must identify obstacles or targets ahead based on the operation of the track line. This case is based on deep learning technology to achieve track, pedestrian or obstacle detection.

[0082] Redmon et al. proposed the YOLO (You Only Look Once) target detection algorithm in a 2016 research paper. YOLO redefined object detection as a regression problem. It applies a single convolutional neural network (CNN) to the entire image, divides the image into grids, and predicts the class probability and bounding box for each grid. For example, take a 100x100 image. We divide it into grids, such as 7x7.

[0083] Then, for each grid, the network predicts a bounding box and the probability corresponding to each class (car, pedestrian, traffic light, etc.).

[0084] Each bounding box can be described using four descriptors:

[0085] 1. The center of the bounding box

[0086] 2. Height

[0087] 3. Width

[0088] 4. The value is mapped to the class to which the object belongs

[0089] In addition, the algorithm can also predict the probability of an object being present in a bounding box. If the center of an object falls in a grid cell, then that grid cell is responsible for detecting the object. There will be multiple bounding boxes in each grid. When training, we want to have only one bounding box per object. Therefore, we assign a box to be responsible for predicting an object based on which box has the highest overlap with the ground truth box.

[0090] Finally, we apply a method called Non Max Suppression to each class of objects to filter out the bounding boxes whose Confidence is less than a threshold. This gives us the image prediction.

[0091] Target distance measurement

[0092] Similar triangles method

[0093] Rationale

[0094] Use similar triangles to calculate the distance from the object or target to the camera. Similar triangles will be used to calculate the distance from the camera to a known object or target.

[0095] Assume there is a target or object with a width of W. Then place the target at a distance D from the camera. Take a picture of the object with the camera and measure the pixel width P of the object.

[0096] This gives us the formula for the focal length of the camera: F = (PxD) / W

[0097] When the camera is moved closer to or farther away from an object or target, similar triangles can be used to calculate the distance between the object and the camera: D' = (WxF) / P

[0098] Specific operations

[0099] First, according to the camera calibration principle, the camera's intrinsic parameter matrix is ​​obtained;

[0100] Then, a single frame image containing the target is collected, and the target area in the image is identified. The maximum value of the vertical coordinate of the pixel coordinates in the area and the corresponding average value of the horizontal coordinate are calculated, and the obtained coordinates are used as observation points;

[0101] Finally, the three-dimensional information of the observation point in the world coordinate system is obtained based on the image pixels, and the distance of the observation point is calculated based on the three-dimensional information of the observation point in the body coordinate system.

[0102] The following steps are involved:

[0103] Step 1: Calibrate the monocular camera and obtain the camera's intrinsic parameter matrix [fx,0,u0; 0,fy,v0; 0,0,1];

[0104] Step 2: Use a calibrated monocular camera to capture a single frame image containing the target, process the obtained image, extract the target area, calculate the maximum value of the ordinate of the pixel coordinates in the area, and the average value of the corresponding abscissa, and use the combined coordinates as the observation point P;

[0105] Step 3: According to the camera calibration principle, the three-dimensional coordinate value Pb (xb, yb, zb) of the observation point in the body coordinate system can be obtained from the pixel coordinates Pp (u, v) of the observation point, and then the distance of the observation point can be calculated. The specific method is as follows:

[0106] 1) The intrinsic parameter matrix of the camera can be obtained by step 1, and the pixel point of the observation point can be obtained by step 2, where λ represents the angle between the light at the observation point and the optical axis of the camera, y represents the ordinate of the observation point in the image coordinate system, y0 represents the ordinate of the optical center of the camera in the image coordinate system, and f represents the focal length of the camera.

[0107] Further simplified to the pixel coordinate system, the relationship is as follows:

[0108] Where v represents the ordinate of the observation point in the pixel coordinate system, v0 represents the ordinate of the optical center of the camera in the pixel coordinate system, dy represents the physical size of each pixel in the y direction, and fy represents the normalized focal length on the y axis;

[0109] 2) In triangular geometry, the O1P2 distance is expressed as follows:

[0110] O1O2 is the height of the camera from the ranging plane, the projection point of P1 on the camera optical axis is P2, and O1P2 represents the depth information, that is, the value of zc in the observation point Pc (xc, yc, zc) in the camera coordinate system.

[0111] 3) Convert pixel coordinates to camera coordinates, and the conversion relationship between pixel coordinate system and camera coordinate system.

[0112] Combine the pixel coordinates of the observation point with the depth information zc of the observation point in the camera coordinate system obtained in the previous step to obtain the coordinates Pc (xc, yc, zc) of the observation point in the camera coordinate system;

[0113] Calculate the conversion formula from the camera coordinate system to the body coordinate system, where Rcb is the rotation matrix from the camera coordinate system to the body coordinate system, and Tcb is the translation matrix from the camera coordinate system to the body coordinate system.

[0114] 4) Calculate the distance D of the observation point;

[0115] xOy is the image coordinate system, Zc represents the Z axis and optical axis of the camera coordinate system, XbO2Yb represents the Z=0 plane (i.e., the ranging plane) in the body coordinate system, O1 represents the camera lens, the two dotted lines a and b represent the visual field range, θ represents the camera pitch angle, H represents the height from the camera to the ranging plane, point P1 is the observation point, the imaging point of observation point P1 on the image plane is P, the projection point on the optical axis is P2, the projection point on the X-axis is P3, and the imaging point of point P3 on the image plane is P0.

[0116] Image scale distance measurement

[0117] Theoretically, when the camera's viewing angle is fixed, the actual distance between different pixels on the image should be fixed, that is, the mapping relationship is satisfied. Therefore, we can estimate the actual distance by calculating the position of the pixel on the image. When the function mapping deviation is large, we can use the hash table method to create a hash for the pixel position and the actual distance one by one, and directly look up the table to obtain it during calculation.

[0118] Lane Detection

[0119] The study detects all the tracks during the drone driving process and identifies the track lines during the driving process. Based on deep learning related technologies, this study first identifies the tracks in the video, uses edge technology to extract track lines in the area with the highest recognition probability, and finally uses classification technology to identify the track lines.

[0120] 1) Get the dataset

[0121] First, a sliding window of fixed width (width w and height h) is selected, and training samples are selected from the video set, with the label a (0: no track, 1: dual tracks, 2: multiple tracks). Secondly, a large-scale data set is a prerequisite for the successful application of deep neural networks. In order to ensure the complexity of the sample set, a series of random changes are made to the image through image augmentation technology to generate a sufficient amount of training data set.

[0122] 2) Define and initialize the model

[0123] Create a neural network as the target model so that the final output is equal to the category of the target dataset. Let the weight and bias parameters of softmax regression be W and b respectively, and the i-th image sample be x (i) , the output layer is o (i) , the predicted probability distribution is The calculation expression of the classification vector of the sample by softmax regression is:

[0124] o (i) =x (i) W+b

[0125]

[0126] Constructing a suitable loss function, such as cross entropy, is a common method.

[0127]

[0128] 3) Train the model.

[0129] In the training model, we need to generate multi-scale aiming frames during the forward calculation process of the model, set the prediction category for each aiming frame, and select the target that is closest to the drone and centered as the current track.

[0130] 4) Track line detection based on edge algorithm

[0131] By choosing a certain step size and sliding window, we can easily find double or triple track areas. Failure or no track is considered a fault. Here we only discuss the confirmation of further track lines under normal circumstances.

[0132] Based on the edge recognition algorithm, all straight lines in the track area are generated.

[0133] y=k(α)x+b

[0134] In the formula, α is the angle between the line and the horizontal viewing angle. It is easy to remove the cross lines and noise lines in the track by angle classification. By setting the line density, the distance from the origin to the line, etc. to group the tracks, the left and right tracks can be identified.

[0135] Target overlap determination

[0136] Based on the result of target detection, the pixel overlap between the target and the track is calculated. Assuming that the overlapping pixels are PC and the pixels selected by the target box are PA, the overlap degree is: S = PC / PA.

[0137] Data Transfer

[0138] After completing the image processing for each frame, the server will give a binary data packet in the following format (python description): [[category 1, distance 1, overlap 1], [category 2, distance 2, overlap 2], … [category n, distance n, overlap n]].

[0139] The communication program (C++ / Rust) in the server will parse and process the data packets, based on the position of the track, combined with the threshold Tl of the anti-collision operation distance of the locomotive, assuming that the overlap threshold is [Tc_min, Tc_max]. The algorithm logic is as follows:

[0140]

[0141] After processing is completed, the alarm signal is transmitted to the communication controller via TCP / CAN protocol.

[0142] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0143] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for anti-collision inspection of a mining electric locomotive, characterized in that: It includes explosion-proof camera, Mini server, storage module, power supply module, communication controller and explosion-proof housing; Step 1: The explosion-proof camera collects real-time video data and transmits it to the Mini server. The user sets the duration of a single backup video and the total number of backup videos. The Mini server automatically backs up in the storage module according to the user's settings. When the total number of backed-up videos exceeds the total number of videos set by the user, the Mini server automatically cleans up old videos. The number of explosion-proof cameras is set at multiple placement points according to the actual placement location. Each placement point is equipped with a detection power-on circuit to detect whether the camera is faulty. If a fault occurs, an alarm is used for alarm processing. The alarm is a photoelectric alarm. The detection power-on circuit is a detection photoelectric alarm that is set in series with the explosion-proof camera and the main power circuit. If there is no power, the photoelectric alarm goes out; Step 2: The Mini server captures real-time video data and forwards it to the image processing program, which includes the functions of target recognition, target ranging, overlap detection, and lane line detection. In the overlap detection, the target overlap is determined by calculating the pixel overlap between the target and the track based on the result of target detection. The overlapping pixels are PC, and the pixels selected by the target are PA. The overlap is: S=PC / PA. Step 3: After the image processing program processes a frame of video data, it can pass the result data packet to the logic processing program. The program can read the user-defined parameters: distance threshold, overlap threshold. After the image processing completes each frame of the picture, the server will give a binary data packet in the following format [[category 1, distance 1, overlap 1], [category 2, distance 2, overlap 2], ... [category n, distance n, overlap n]]; The communication program in the server will parse and process the data packet, based on the position of the track, combined with the threshold Tl of the anti-collision operation distance of the locomotive, and the overlap threshold [Tc_min, Tc_max]. After completing the processing, the alarm signal is transmitted to the communication controller through the TCP / CAN protocol; wherein, the algorithm logic is to obtain different alarm signals according to the size relationship between the distance L and the threshold Tl and the size relationship between the overlap S and the overlap threshold; Step 4: When the result data packet parsed by the logic processing program meets different threshold intervals, the logic processing program will obtain different alarm signals, which are green for safety signals, blue for potential danger signals, and red for danger signals according to the levels; Step 5: Finally, the communication program sends the alarm signal to the back-end communication controller through the TCP protocol, which controls the electric locomotive to take corresponding actions; In step 2, the image backup strategy uses a multi-threaded program. Thread T1 is responsible for forwarding the image acquisition data directly to the image processing module. Thread T2 is responsible for copying the image data to the local disk and saving the file with a timestamp. The system sets the video duration Ts for a single storage and the maximum number of videos that can be stored locally Nmax. The image cleaning strategy is to detect the number of data N in the storage location every time the system starts, and determine whether N is greater than the set maximum storage number Nmax. If it is greater, traverse and delete the earliest video in timestamp order, N minus one, and then determine the number of N again. If it is less than Nmax at this time, store a new video, N plus one, otherwise repeat the video deletion operation again; after storing the video N plus one, the next video still needs to determine whether the number N is greater than Nmax. When a system interruption occurs, exit the system and shut down the device.

2. A method for anti-collision inspection of a mining electric locomotive according to claim 1, characterized in that: In object detection, a single convolutional neural network (CNN) is applied to the entire image, dividing the image into grids and predicting the class probability and bounding box for each grid.

3. A method for anti-collision inspection of a mining electric locomotive according to claim 2, characterized in that: For each grid, the network predicts a bounding box and the probability corresponding to each class.

4. A method for anti-collision inspection of a mining electric locomotive according to claim 1, characterized in that: The target distance measurement is to calculate the distance from the object or target to the camera using similar triangles.

5. A method for anti-collision inspection of a mining electric locomotive according to claim 4, characterized in that: First, according to the camera calibration principle, the camera's intrinsic parameter matrix is ​​obtained; Then, a single frame image containing the target is collected, and the target area in the image is identified. The maximum value of the vertical coordinate of the pixel coordinates in the area and the average value of the corresponding horizontal coordinate are calculated, and the obtained coordinates are used as observation points; Finally, the three-dimensional information of the observation point in the world coordinate system is obtained based on the image pixels, and the distance of the observation point is calculated according to the three-dimensional information of the observation point in the body coordinate system.

6. A method for anti-collision inspection of a mining electric locomotive according to claim 5, characterized in that: Lane detection includes the following steps: 1) Get the dataset; First, a sliding window with a fixed width is selected to select training samples from the video set. Second, a series of random changes are made to the image through image augmentation technology to generate a sufficient amount of training data set. 2) Define and initialize the model; Create a neural network as the target model so that the final output is equal to the category of the target dataset; 3) Training the model; In the training model, multi-scale aiming frames are generated during the forward calculation process of the model, and a prediction category is set for each aiming frame. The target that is currently closest to the drone and centered is selected as the current track. 4) Track line detection based on edge algorithm; Select a certain step size and sliding window to find the double-track or triple-track area; consider the failure or no track as a fault; Based on the edge recognition algorithm, all straight lines in the track area are generated.

Images