Autonomous anti-collision control device and method for coal mining machine

Abstract

The invention discloses an automatic anti-collision control device and method for a coal mining machine. The automatic anti-collision control device of the coal mining machine comprises sensor assemblies, an intrinsic safety type sensor acquisition unit and a coal mining machine main controller, the sensor assemblies are arranged for a left rocker arm and a right rocker arm of the coal mining machine respectively, and each sensor assembly comprises a mining intrinsic safety type image processing camera and an intrinsic safety type millimeter wave radar module. The extending side protection plate is identified based on image information, whether the extending side protection plate is in an operation space in the advancing direction of the coal mining machine or not is judged in combination with the fused point cloud, and the coal mining machine is automatically prevented from colliding with a front hydraulic support according to the traction speed, the rocker arm angle and obstacle identification information of the coal mining machine; therefore, the traditional passive anti-collision mode of dispersedly collecting the spatial logic relation between the coal mining machine and the hydraulic support is converted into the autonomous anti-collision mode.

Description

Technical Field

[0001] The present invention relates to an automatic anti-collision control device and method for a shearer. Background Art

[0002] Intelligent fully mechanized mining is based on the mutual relationship between the coal seam mining space, the coal seam to be mined, and the mining space and mining equipment as the decision-making basis to realize that intelligent shearers, hydraulic supports, and scraper conveyors automatically adapt to coal cutting, support, and propulsion operations following the changes of the coal seam. Intelligent shearers, scraper conveyors, and hydraulic supports are an integrated mechanized equipment that cooperates with each other, has strong coupling, and collaborates with each other. They have a mutually dependent operating relationship with high coupling and mutual cooperation. The hydraulic support provides a support space for the shearer, and its rib protection plate is opened and attached to the coal wall. During coal mining operations, the shearer and the scraper conveyor are connected by a rigid gear to drive the shearer to reciprocate and operate on the working face. The rib protection plates of the hydraulic supports 2 to 5 frames before the drum in the advancing direction of the shearer will automatically retract. After the shearer cuts the coalfield, the hydraulic supports behind its walking support the top and bottom coal seams, push the scraper conveyor towards the coal wall direction, and then the hydraulic supports lower the support height and complete the follow-up of the supports through the direction contraction cylinder to form a complete operation cycle process. During the process of less personnel and unmanned operation, due to some failures in the mechanical, hydraulic, and sensing monitoring and control of the hydraulic supports, it is easy to cause accidents such as the rib protection plate not being retracted, resulting in collisions between the shearer and the hydraulic supports, causing significant losses.

[0003] The current anti-collision system of the shearer mainly relies on the electro-hydraulic control system of the hydraulic support. As Figure 1 shown, the hydraulic support is equipped with sensors such as proximity sensors, rib protection plate stroke sensors, height measurement sensors, and high-definition cameras. By using multi-sensor means to perceive the spatial position relationship between the shearer and the support, information such as the distance between the shearer drum and the front end of the support can be identified to a certain extent. Combining the shearer mining height information collected by the gateway centralized control center, the gateway centralized control center comprehensively judges the spatial logical relationship between the shearer drum and the support, and can realize the control of the shearer to automatically stop when colliding and lower the drum height to avoid collision with the hydraulic support to a certain extent.

[0004] The existing anti-collision technologies for shearers and hydraulic supports mainly adopt passive detection means. There are harsh conditions such as high humidity, strong vibration, strong electromagnetic interference, and weak light in the coal mining face of coal mines. The failure rate of sensors on the hydraulic support is relatively high, and after the comprehensive decision-making collision information of sensors and videos is processed by the gateway centralized control center and then transmitted to the shearer control system, there is often a large delay, and the shearer cannot be controlled to react in time. The anti-collision effect of the shearer needs to be further improved. Summary of the Invention

[0005] The object of the present invention is to provide a self - collision - prevention control device and method for a shearer. Based on an intrinsically safe millimeter - wave radar device and a mine - use flameproof image - processing camera fixedly connected thereto, they are installed at the left and right rocker arms of the shearer to collect multi - source sensing data of radar point cloud and images. Based on image information recognition technology, the extended rib protection plate is detected, and combined with the fused point cloud, it is determined whether it is within the working space of 2 - 5 hydraulic supports in the advancing direction of the shearer. According to the traction speed, rocker arm angle and obstacle recognition information of the shearer, the shearer can autonomously prevent collision with the front - end hydraulic support, thus transforming the traditional passive anti - collision that separately collects the spatial logical relationship between the shearer and the hydraulic support into autonomous anti - collision.

[0006] To achieve the above - mentioned technical object, the present invention will adopt the following technical solutions: A self - collision - prevention control device for a shearer, comprising a sensor assembly, an intrinsically safe sensor acquisition unit and a shearer main controller. For each of the left and right rocker arms of the shearer, a set of sensor assemblies is provided. Each set of sensor assemblies includes a mine - use intrinsically safe image - processing camera and an intrinsically safe millimeter - wave radar module; The mine - use intrinsically safe image - processing camera is used to detect the image information in front of the advancing direction of the shearer, and transmit the detected image information to the shearer main controller through the intrinsically safe sensor acquisition unit; The intrinsically safe millimeter - wave radar module is used to detect the point cloud data in front of the advancing direction of the shearer, and transmit the detected point cloud data to the shearer main controller through the intrinsically safe sensor acquisition unit; The shearer main controller identifies the extended rib protection plate in the advancing direction of the shearer through the received image information and obtains the pixel coordinates of the center of the extended rib protection plate; by fusing the received image information and point cloud data, the spatial coordinates corresponding to the pixel coordinates of the center of the extended rib protection plate are obtained, and then the distance between the spatial coordinates of the center of the rib protection plate and the shearer is calculated; it is judged whether the extended rib protection plate is within the working space of the shearer through the received image information, and according to this judgment result, it is determined whether it is necessary to further judge whether the distance between the spatial coordinates of the center of the rib protection plate calculated and the shearer exceeds the limit, and then whether the shearer has a collision risk is identified.

[0007] Preferably, the mine - use intrinsically safe image - processing camera and the intrinsically safe millimeter - wave radar module are fixedly connected as a whole and installed on the rocker arm.

[0008] Preferably, the intrinsically safe millimeter - wave radar module includes an explosion - proof housing, a power distribution circuit board, a signal - processing and main CPU board, a radio - frequency front - end and antenna circuit, an explosion - proof wave - transmitting lens and a cable connector; among them: The explosion - proof wave - transmitting lens and the cable connector are assembled on the explosion - proof housing; The power distribution circuit board, the signal processing and main CPU board, and the RF front-end and antenna circuit are respectively installed in the explosion-proof enclosure; The power distribution circuit board is electrically connected to the external power supply through a cable connector.

[0009] Preferably, the main controller of the shearer detects the image information based on the loaded deep learning algorithm model to identify the rib protection plate extending in the advancing direction of the shearer; The deep learning algorithm model is built based on the deep learning architecture of the fusion of radar point cloud and RGB image, and includes a radar point cloud input layer, an image input layer, a dual-branch feature extraction module, a cross-modal fusion layer, a fully connected layer 1, a Dropout layer, a fully connected layer 2, and an output layer, where: The input of the radar point cloud input layer is the 3D point cloud data collected by the intrinsically safe millimeter-wave radar module; The input of the image input layer is the RGB image captured by the intrinsically safe mine image processing camera; The dual-branch feature extraction module includes a radar feature extraction branch and an image feature extraction branch; The radar feature extraction branch includes a point cloud convolutional layer 1, a batch normalization layer 1, and a pooling layer 1, where: the point cloud convolutional layer 1 extracts the spatial geometric features of the 3D point cloud data through sparse convolution; the batch normalization layer 1 normalizes the point cloud spatial geometric features output by the point cloud convolutional layer 1; the pooling layer 1 performs max pooling dimensionality reduction on the output of the batch normalization layer 1; The image feature extraction branch includes an image convolutional layer 1, a batch normalization layer 2, and a pooling layer 2, where: the image convolutional layer 1 extracts the image texture features of the RGB image through a convolutional kernel; the batch normalization layer 2 normalizes the image texture features output by the image convolutional layer 1; the pooling layer 2 performs spatial pyramid pooling on the output of the batch normalization layer 2; The cross-modal fusion layer uses the attention mechanism to align the resolutions of the radar feature map output by the radar feature extraction branch and the image feature map output by the image feature extraction branch, and fuses the geometric features and texture features through channel concatenation; The fully connected layer 1 maps the fusion features output by the cross-modal fusion layer to a high-dimensional space; The Dropout layer randomly masks some neurons to prevent overfitting, and its input is the output of the fully connected layer 1; The input of the fully connected layer 2 is the output of the Dropout layer, and is used to perform another full connection on the output of the Dropout layer; The output layer generates a collision probability prediction and an obstacle distance regression value based on the fully connected layer 2, as the basis for the anti-collision control instruction.

[0010] Another technical object of the present invention is to provide a shearer autonomous anti-collision control method, including: Obtain the image information and point cloud data in front of the shearer's traveling direction respectively; Identify the rib protection plate protruding in the advancing direction of the shearer based on the image information and obtain the pixel coordinates of the center point of the protruding rib protection plate; Map the point cloud data onto the image, and during the projection process, judge whether the pixel coordinates corresponding to the three-dimensional coordinate points of any point cloud data are within the pixel coordinate range of the image. If it is abnormal, discard the point cloud data, otherwise retain it, so as to realize the fusion of the point cloud data and the image information, and further obtain the spatial coordinates corresponding to the pixel coordinates of the center point of the protruding rib protection plate; the spatial coordinates of the center point of the protruding rib protection plate are the average of the spatial coordinates of the pixel points within the neighborhood range around the center point of the protruding rib protection plate; Calculate the Euclidean distance D between the spatial coordinates of the center point of the protruding rib protection plate and the shearer, and judge whether the protruding rib protection plate is within the working space of the shearer based on the image information; When the judgment result shows that the protruding rib protection plate is not within the working space of the shearer, the shearer is in a safe working state at this time; When the judgment result shows that the protruding rib protection plate is within the working space of the shearer, further judge whether the distance D between the spatial coordinates of the center point of the protruding rib protection plate and the shearer is less than the distance threshold limit D min : When the judgment result is D≥ D min At this time, adjust the traction speed and boom angle of the shearer based on the distance D until the protruding rib protection plate is not within the working space of the shearer, realizing the anti-collision autonomous adjustment of the shearer; when the judgment result is D< D min At this time, brake the shearer emergently.

[0011] Preferably, the image information in front of the shearer's traveling direction is detected by a mine intrinsically safe image processing camera installed on the boom; The point cloud data in front of the shearer's traveling direction is detected by an intrinsically safe millimeter wave radar module installed on the boom.

[0012] Preferably, the mine intrinsically safe image processing camera and the intrinsically safe millimeter wave radar module are connected into one body.

[0013] Preferably, a mine intrinsically safe image processing camera and an intrinsically safe millimeter wave radar module are respectively provided at the left and right booms of the shearer; When the shearer travels left, read the image information in front of the shearer's traveling direction through the mine intrinsically safe image processing camera installed on the left boom, and read the point cloud data in front of the shearer's traveling direction through the intrinsically safe millimeter wave radar module installed on the left boom.

[0014] Preferably, for the rib protection plate extending in the advancing direction of the shearer, based on the collected images of the extended rib protection plate and radar point cloud information, through the constructed deep learning algorithm model, the extension and retraction states of the rib protection plate are classified into two categories for target detection and recognition.

[0015] Based on the above technical objectives, compared with the prior art, the present invention has the following advantages: The present invention uses the fusion of multi-source sensing data of point cloud and image to achieve autonomous anti-collision of the shearer, timely controls the shearer to make reaction control, and effectively improves the anti-collision effect of the shearer. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is the schematic diagram of the anti-collision principle between the current shearer and the hydraulic support; Figure 2 is the electrical principle block diagram of the shearer autonomous anti-collision control device of the present invention; Figure 3 is the installation position diagram of the detection device on the shearer in the shearer autonomous anti-collision control device of the present invention; In the figure: (a) is the schematic structural diagram in one direction when the shearer is running; (b) is the schematic structural diagram in another direction when the shearer is running; Figure 4 is the schematic structural diagram of the millimeter wave radar device of the present invention; In the figure: (a) is the front view of the millimeter wave radar device; (b) is the cross-sectional view of the millimeter wave radar device; Figure 5 is the control flow chart of the shearer autonomous anti-collision control device of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. The description of at least one exemplary embodiment is actually only illustrative and in no way restrictive of the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention. Unless otherwise specifically stated, the relative arrangements, expressions and numerical values of the components and steps set forth in these embodiments do not limit the scope of the present invention. Technologies, methods and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods and devices should be regarded as part of the specification. In all the examples shown and discussed here, any specific value should be construed as merely exemplary and not as a limitation. Therefore, other examples of the exemplary embodiments may have different values.

[0018] For ease of description, spatial relative terms such as "above", "over", "on the upper surface", "upper" etc. may be used herein to describe the spatial positional relationship of one device or feature to other devices or features as shown in the figures. It should be understood that the spatial relative terms are intended to encompass different orientations in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figure is inverted, the device described as "above" or "over" other devices or structures will then be positioned "below" or "under" the other devices or structures. Thus, the exemplary term "above" can include both the orientations of "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations).

[0019] As Figure 2 - 5 shown, the self-collision prevention control device of the shearer according to the present invention is disposed on the shearer 1 and includes a sensor assembly, an intrinsically safe sensor acquisition unit, and a shearer main controller, wherein: The sensor assembly includes two groups, corresponding to the left and right sensor assemblies; the left sensor assembly includes a left mining intrinsically safe image processing camera 3, a left intrinsically safe millimeter wave radar module 2, a left rocker arm inclination sensor, a left rocker arm lifting cylinder stroke sensor, and a left traveling rotary encoder; the right sensor assembly includes a right mining intrinsically safe image processing camera 5, a right intrinsically safe millimeter wave radar module 4, a right rocker arm inclination sensor, a right rocker arm lifting cylinder stroke sensor, and a right traveling rotary encoder. After the left mining intrinsically safe image processing camera and the left intrinsically safe millimeter wave radar module are fixedly connected into one body, they are installed at the left rocker arm of the shearer, and after the right mining intrinsically safe image processing camera and the right intrinsically safe millimeter wave radar module are fixedly connected into one body, they are installed at the right rocker arm of the shearer. In the figure, the distance from the installation site A of the left intrinsically safe millimeter wave radar module on the left rocker arm to the center of the left drum is d, the angle between the tangent line of the installation site A relative to the left drum and the extension direction of the rocker arm length is denoted as α, and the diameter of the left drum is denoted as R. The installation position of the right intrinsically safe millimeter wave radar module on the right rocker arm is similar to that of the left intrinsically safe millimeter wave radar module on the left rocker arm and will not be described herein again. In Figure 3 (a) thereof, the shearer 1 tunnels the coal wall 7 through the drum, and there is a hydraulic support 6 in front of the running direction of the shearer 1.

[0020] The intrinsically safe sensor acquisition unit is mainly used to collect information of various sensors such as the attitude, angle, speed, video images, etc. installed on the shearer. It uses the form of industrial Ethernet to interact and transmit data with the shearer, and can access 2 Ethernet channels and 8 CAN bus sensors. In the present invention, the intrinsically safe sensor acquisition unit includes two groups, corresponding to the left and right intrinsically safe sensor acquisition units. The left intrinsically safe sensor acquisition unit is used to collect the information detected by the left sensor assembly, and the right intrinsically safe sensor acquisition unit is used to collect the information detected by the right sensor assembly.

[0021] The left / right mine intrinsically safe image processing camera mainly monitors the video information in front of the advancing shearer, and transmits the image information back to the shearer main controller through the intrinsically safe sensor acquisition unit on the same side. The shearer main controller fuses the received video information with the point cloud data fed back by the intrinsically safe millimeter-wave radar module on the same side, and then comprehensively judges the obstacle information of the shearer.

[0022] The left / right intrinsically safe millimeter-wave radar module is mainly used to monitor the obstacle information in front of the advancing shearer, and feeds back the point cloud data of the obstacles to the shearer through the intrinsically safe sensor acquisition unit on the same side.

[0023] The left boom inclination sensor mainly monitors the angle of the shearer boom. According to the mathematical and geometric relationship between the shearer boom inclination and the shearer height adjustment structure, the cutting height of the drum is calculated; the boom cylinder stroke sensor is mainly used to monitor the stroke of the shearer boom lifting cylinder. According to the mathematical and geometric relationship between the boom height adjustment cylinder stroke and the shearer height adjustment structure, the cutting height of the drum is calculated. These two sensors are redundant and mutually verified, using magnetostrictive displacement sensors and outputting CAN signals.

[0024] The traveling rotary encoder is used to monitor the relative position of the shearer on the working face. The left and right rotary encoders are mutually verified, using absolute magnetic heavy-duty type and outputting CAN signals.

[0025] The shearer's autonomous anti-collision control system is mainly based on Ethernet and CAN bus. The main controller is the core of data processing, and connects the left and right intrinsically safe sensor acquisition units through Ethernet. The intrinsically safe sensor acquisition unit can access 2 Ethernet channels and 8 CAN bus sensors. The communication rate of the Ethernet CAN bus is 250 kbps, and the CAN OPEN protocol is used. The intrinsically safe sensor acquisition unit collects information such as the explosion-proof millimeter-wave radar module, the shearer height adjustment cylinder stroke, the shearer boom inclination, the shearer traction speed, the real-time position, etc. The shearer controller completes data acquisition, analysis and calculation, and combines the distance information, angle information, etc. of the obstacles detected by the millimeter-wave radar module to comprehensively adjust the height and speed of the shearer drum.

[0026] In the present invention, the intrinsically safe millimeter-wave radar device, such as Figure 3As shown in the figure, the millimeter-wave radar device for shearer mainly consists of a power management circuit 21 (i.e., a power distribution circuit board), a signal processing and main CPU board 22, a radio frequency front-end and antenna circuit 23, an explosion-proof housing 24, an explosion-proof wave-transparent lens 25, and a cable connector 26.

[0027] The power management circuit mainly completes the functions of external power conversion and power supply distribution. It mainly adopts four buck DC / DC converter cores, provides 4-way single-phase output, has an I2C serial interface and supports enable signal control, and has output short-circuit, overheat, and overload protection. The input voltage supports a wide voltage input of 3.5V to 36V, and the output voltages include 5V, 3.3V, 1.8V, 1.7V, 1.2V, and 1.0V.

[0028] The signal processing and main CPU board mainly consists of a microcontroller and a digital signal processing unit. The microcontroller, as the core control part of the radar chip, on the one hand, receives the distance, speed, and angle information transmitted by the DSP, combines the spatial logical relationships such as the shearer rocker arm angle for conversion, radar signal calibration, etc., and on the other hand, sets relevant parameters for the radar radio frequency front-end. The DSP mainly transfers the intermediate-frequency signal data in the analog-to-digital memory of the received radar echo signal to the buffer area, performs relevant algorithm processing on the intermediate-frequency signal, calculates the distance, speed, and angle information of the detected target, and simultaneously transmits it to the MCU for processing.

[0029] Radio frequency front-end and antenna circuit: The radio frequency front-end mainly integrates a phase-locked loop (PLL), a voltage-controlled oscillator (VCO), and a mixer, which are used for the generation and mixed reception of high-frequency millimeter-wave radar signals. The antenna circuit mainly adopts a microstrip comb-shaped antenna, with a 4-transmit 4-receive system, and can obtain a detection width of horizontal and vertical FOV ±30°.

[0030] The intrinsically safe special housing is a mechanical housing for installing the radar module in the dangerous environment of the coal mine underground.

[0031] The explosion-proof wave-transparent lens is mainly used to enclose and protect the millimeter-wave radar circuit and antenna, and uses PBT-GF30 (polybutylene terephthalate) as the wave-transparent protective cover for the electromagnetic transparency of the millimeter-wave radar.

[0032] Shearer, scraper conveyor, hydraulic support and other coal mining face mechanical equipment are a set of closely matched and strongly coupled systems. Especially in the less-manned and unmanned working faces, the anti-collision technology of the shearer is very indispensable. The explosion-proof millimeter-wave radar device of the shearer is installed at the goaf side end of the shearer cutting motor (as Figure 2 shown), the radar module uses the 77GHz frequency band, the maximum detection distance ≥10m, the detection accuracy is ±0.01m, and the obstacle recognition feedback time ≤100ms, which is used to identify obstacles such as hydraulic supports above the drum.

[0033] Based on the above-mentioned coal mining machine autonomous anti-collision control device, the present invention provides a coal mining machine autonomous anti-collision control method, the overall process of which is as follows: Figure 5 As shown. The process mainly includes the following main control processes: (1) System initialization and sensor calibration The main task is to initialize the parameter thresholds and operating parameters of the coal mining machine main controller, initialize the parameter configuration, working mode, communication parameters and other information of the intrinsically safe millimeter-wave radar device, mining explosion-proof image processing camera and intrinsically safe sensor acquisition unit, and judge and establish the system communication status.

[0034] Jointly calibrate the peripheral monitoring sensors, send a unified clock source inside the coal mining machine controller and provide the same reference time, synchronize the millimeter wave radar module, the mine flameproof image processing camera and the clocks of other attitude monitoring. Calibrate the internal and external parameters of the intrinsically safe millimeter wave radar device and the mine flameproof image processing camera for subsequent point cloud and image fusion.

[0035] (2) Data collection Verify the communication status between the main controller of the coal mining machine and the intrinsically safe millimeter-wave radar, and read the point cloud data of the intrinsically safe millimeter-wave radar when the communication is normal. In addition, collect relevant information such as temperature and current.

[0036] (3) Data preprocessing The coal mining machine anti-collision control system reads data from multiple sensors such as intrinsically safe millimeter-wave radars. The collected point cloud is limited to the region of interest through preprocessing such as direct filtering to reduce the number of irrelevant points, and the point cloud is filtered to remove abnormal noise points. Finally, the point cloud is mapped to the image, that is, the mapping of the two-dimensional pixel coordinates of the image and the three-dimensional spatial coordinates of the point cloud is realized, thereby completing the data preprocessing.

[0037] (4) Data fusion Based on the deep learning algorithm model installed, the extended guard plate in the image field of view is detected, and the target detection is performed on the RGB image. According to the extended and retracted state of the guard plate, the detection target is divided into two categories: extended guard plate and retracted guard plate. The pixel horizontal coordinates of all the detection frames of the extended guard plate in the RGB image are selected. The pixel coordinates of the center point of the detection result are obtained. Based on the calibrated intrinsically safe millimeter wave radar device and the internal and external parameters of the mine flameproof image processing camera, the point cloud is mapped to the image to obtain the spatial coordinates corresponding to the pixel coordinates of the center point of the detection result. In order to improve the positioning accuracy of the extended guard plate closest to the coal mining machine, the average value of the spatial coordinates corresponding to the pixel points within a certain neighborhood of the center point of the detection frame is taken as its positioning result.

[0038] Images of the rib protection plate extending in the advancing direction of the shearer and radar point cloud information are collected, and shallow feature channels are spliced. Through the attention weighting mechanism, the radar point cloud and the RGB feature map of the image are cascaded by element-wise addition. Finally, the depth features of the point cloud and the global features such as the color and texture of the RGB image are aggregated and trained to obtain a deep learning algorithm model for target monitoring. Redundant feature layers are cropped, and knowledge distillation is used to perform low-precision quantization on the weight parameters, reducing the computational complexity while maintaining the detection accuracy.

[0039] Through the constructed cropped lightweight deep learning algorithm model based on the fused features of RGB image information and depth information, the state behavior targets of the rib protection plate are identified and output to the main controller of the shearer as the basis for the next decision on the target state, realizing the autonomous anti-collision technology of the shearer to meet intelligent operation according to multi-information recognition and processing.

[0040] The cropped lightweight deep learning algorithm model is built based on the deep learning architecture of the fusion of radar point cloud and RGB image, including a radar point cloud input layer, an image input layer, a dual-branch feature extraction module, a cross-modal fusion layer, a fully connected layer 1, a Dropout layer, a fully connected layer 2, and an output layer, where: The input of the radar point cloud input layer is the 3D point cloud data collected by the shearer radar; The input of the image input layer is the RGB image captured by the shearer camera; The dual-branch feature extraction module includes a radar feature extraction branch and an image feature extraction branch; Radar feature extraction branch: 1. The point cloud convolutional layer 1 extracts the spatial geometric features of the point cloud through sparse convolution; 2. The batch normalization layer 1 normalizes the point cloud features; 3. The pooling layer 1 performs max pooling dimensionality reduction on the features; Image feature extraction branch:  1. The image convolutional layer 1 extracts the image texture features through a convolutional kernel;  2. The batch normalization layer 2 normalizes the image features;  3. The pooling layer 2 performs spatial pyramid pooling on the features; Cross-modal fusion layer:  1. The attention mechanism is used to align the resolutions of the radar feature map and the image feature map;  2. The geometric features and texture features are fused through channel concatenation; The fully connected layer 1 maps the fused features output by the cross-modal fusion layer to a high-dimensional space; The Dropout layer randomly masks some neurons to prevent overfitting, and its input is the output of the fully connected layer 1; The input of the fully connected layer 2 is the output of the Dropout layer, which is used to perform another fully connected operation on the output of the Dropout layer.

[0041] The output layer generates a collision probability prediction (a value in the range of 0 - 1) and an obstacle distance regression value based on the fully connected layer 2, which serves as the basis for the anti-collision control command.

[0042] Residual connections are used between layers to ensure the efficiency of gradient propagation. During training, a synchronous data augmentation strategy is adopted to ensure the spatial consistency of multi-modal data.

[0043] (5)Obstacle Judgment Calculate the distance between its detection center and the shearer, and determine whether the extended rib protection plate in the image field of view is within the working space, that is, within the working space of the rib protection plates of 2 - 5 hydraulic supports in front of the drum in the advancing direction of the shearer, so as to identify whether there is a collision risk.

[0044] (6)Decision-making and Response When the shearer operates according to the working conditions, set the distance threshold limit between the equipment that may collide D min . Based on the distance between the shearer and the extended rib protection plate of the hydraulic support in its advancing direction D , it can be divided into three situations in total: a. When the extended rib protection plate in the advancing direction of the shearer is not within the working space, the shearer is in a safe working state at this time, and there will be no collision between the shearer and the hydraulic support. The shearer can continue to maintain its current operating speed, boom angle and other states; b. When the extended rib protection plate is within the working space of the shearer, and the distance between it and the shearer D is not less than D min , the shearer should adjust the traction speed and boom angle of the shearer based on the distance D and the data such as the current traction speed, boom angle, cutting current of the shearer fed back, to achieve the autonomous anti-collision of the shearer; c. When the extended rib protection plate is within the working space of the shearer, and the distance between it and the shearer D is less than D min , the shearer takes emergency braking.

[0045] (7)Data Storage and Playback Transmit the multi-source sensor data such as the collected point cloud and image back to the gateway gathering control center to achieve data storage and playback, which is used for subsequent fault analysis and diagnosis, etc.

Claims

1. An autonomous anti-collision control device for a coal mining machine, comprising a sensor assembly, an intrinsically safe sensor acquisition unit and a main controller for a coal mining machine, characterized in that: The sensor components are provided in groups for the left and right rocker arms of the coal mining machine, and each group of sensor components includes a mining intrinsically safe image processing camera and an intrinsically safe millimeter wave radar module; The intrinsically safe image processing camera for mining is used to detect image information in front of the coal mining machine in the direction of travel, and transmit the detected image information to the main controller of the coal mining machine through the intrinsically safe sensor acquisition unit; The intrinsically safe millimeter wave radar module is used to detect point cloud data in front of the coal mining machine in the walking direction, and transmit the detected point cloud data to the main controller of the coal mining machine through the intrinsically safe sensor acquisition unit; The main controller of the coal mining machine identifies the guard plate extended in the forward direction of the coal mining machine through the received image information and obtains the pixel coordinates of the center of the extended guard plate; obtains the spatial coordinates corresponding to the pixel coordinates of the center of the extended guard plate by fusing the received image information and the point cloud data, and then calculates the distance between the spatial coordinates of the center of the guard plate and the coal mining machine; determines whether the extended guard plate is in the working space of the coal mining machine through the received image information, and determines whether it is necessary to further determine whether the distance between the calculated spatial coordinates of the center of the guard plate and the coal mining machine exceeds the limit based on this judgment result, and then identifies whether the coal mining machine has a collision risk.

2. The autonomous anti-collision control device for coal mining machine according to claim 1, characterized in that: The intrinsically safe image processing camera for mining is fixedly connected with the intrinsically safe millimeter wave radar module to form an integral whole and then installed on the rocker arm.

3. The autonomous anti-collision control device for coal mining machine according to claim 1, characterized in that: The intrinsically safe millimeter wave radar module includes an explosion-proof housing, a power distribution circuit board, a signal processing and main CPU board, a radio frequency front end and antenna circuit, an explosion-proof wave-transmitting mirror and a cable connector; wherein: The explosion-proof housing is equipped with an explosion-proof wave-transmitting lens and a cable connector; The power distribution circuit board, signal processing and main CPU board, RF front end and antenna circuit are installed in explosion-proof enclosures respectively; The power distribution circuit board is electrically connected to the external power supply through a cable connector.

4. The autonomous anti-collision control device for coal mining machine according to claim 1, characterized in that: The main controller of the coal mining machine detects image information based on the deep learning algorithm model on board to identify the guard plates extending in the direction of the coal mining machine's advance; The deep learning algorithm model is built based on the deep learning architecture of radar point cloud and RGB image fusion, including radar point cloud input layer, image input layer, dual-branch feature extraction module, cross-modal fusion layer, fully connected layer 1, Dropout layer, fully connected layer 2 and output layer, among which: The input of the radar point cloud input layer is the 3D point cloud data collected by the intrinsically safe millimeter-wave radar module; The input of the image input layer is the RGB image captured by the mine intrinsically safe image processing camera; The dual-branch feature extraction module includes a radar feature extraction branch and an image feature extraction branch; The radar feature extraction branch includes point cloud convolution layer 1, batch normalization layer 1 and pooling layer 1, wherein: point cloud convolution layer 1 extracts the spatial geometric features of 3D point cloud data through sparse convolution; batch normalization layer 1 standardizes the spatial geometric features of the point cloud output by point cloud convolution layer 1; pooling layer 1 performs maximum pooling dimensionality reduction on the output of batch normalization layer 1; The image feature extraction branch includes image convolution layer 1, batch normalization layer 2 and pooling layer 2, wherein: image convolution layer 1 extracts image texture features of RGB images through convolution kernels; batch normalization layer 2 standardizes the image texture features output by image convolution layer 1; pooling layer 2 performs spatial pyramid pooling on the output of batch normalization layer 2; The cross-modal fusion layer uses the attention mechanism to align the resolution of the radar feature map output by the radar feature extraction branch and the image feature map output by the image feature extraction branch, and fuses the geometric features and texture features through channel cascading; The fully connected layer 1 maps the fusion features output by the cross-modal fusion layer to a high-dimensional space; The Dropout layer randomly blocks some neurons to prevent overfitting, and its input is the output of the fully connected layer 1; The input of the fully connected layer 2 is the output of the Dropout layer, which is used to fully connect the output of the Dropout layer again; The output layer generates collision probability prediction and obstacle distance regression value based on the fully connected layer 2 as the basis for anti-collision control instructions.

5. A method for autonomous anti-collision control of a coal mining machine, characterized in that: include: Respectively obtain image information and point cloud data in front of the coal mining machine's travel direction; Based on the image information, the guard plate extending in the forward direction of the coal mining machine is identified and the pixel coordinates of the center point of the extended guard plate are obtained; The point cloud data is mapped to the image, and during the projection process, it is determined whether the pixel coordinates corresponding to the three-dimensional coordinate point of any point cloud data are within the pixel coordinate range of the image. If abnormal, the point cloud data is discarded, otherwise it is retained, thereby realizing the fusion of point cloud data and image information, and then obtaining the spatial coordinates corresponding to the pixel coordinates of the center point of the extended guard plate; the spatial coordinates of the center point of the extended guard plate are the average spatial coordinates of the pixel points in the neighborhood range around the center point of the extended guard plate; Calculate the Euclidean distance D between the spatial coordinates of the center point of the extended side guard plate and the coal mining machine, and determine whether the extended side guard plate is within the working space of the coal mining machine based on the image information; When the judgment result shows that the extended side guard plate is not in the working space of the coal mining machine, the coal mining machine is in a safe working state; When the judgment result shows that the extended side guard plate is in the working space of the coal mining machine, it is further judged whether the spatial coordinate of the center point of the extended side guard plate and the Euclidean distance D of the coal mining machine is less than the distance threshold limit. D min :When the judgment result is D≥ D min When the distance D is less than 0, the traction speed and the rocker arm angle of the coal mining machine are adjusted until the extended side guard plate is not in the working space of the coal mining machine, so as to realize the anti-collision autonomous adjustment of the coal mining machine; when the judgment result is D< D min When the coal mining machine is emergency braked.

6. The method for autonomous anti-collision control of a coal mining machine according to claim 5, characterized in that: The image information in front of the coal mining machine in its travel direction is detected by a mining intrinsically safe image processing camera installed on the rocker arm; The point cloud data in front of the coal mining machine in its travel direction is detected by the intrinsically safe millimeter-wave radar module installed on the rocker arm.

7. The method for autonomous anti-collision control of a coal mining machine according to claim 6, characterized in that: The mine-used intrinsically safe image processing camera is integrated with the intrinsically safe millimeter-wave radar module.

8. The method for autonomous anti-collision control of a coal mining machine according to claim 7, characterized in that: The left and right rocker arms of the coal mining machine are each equipped with a mining intrinsically safe image processing camera and an intrinsically safe millimeter wave radar module; When the coal mining machine moves to the left, the mine-used intrinsically safe image processing camera installed on the left rocker arm reads the image information in front of the coal mining machine's travel direction, and the intrinsically safe millimeter-wave radar module installed on the left rocker arm reads the point cloud data in front of the coal mining machine's travel direction.

9. The method for autonomous anti-collision control of a coal mining machine according to claim 5, characterized in that: The guard plates extended in the forward direction of the coal mining machine are classified into two categories of extended and retracted states for target detection and identification based on the collected images of the extended guard plates and radar point cloud information through the constructed deep learning algorithm model.

Images