Coal mine cage personnel detection method and system

By adopting the lightweight target detection model YOLO-Speed ​​and dual network card structure in the coal mine tank cage, many challenges in personnel detection in the tank cage are solved, fast and accurate pedestrian detection and locking signal transmission are achieved, and the safety of coal mine transportation is improved.

CN119007238APending Publication Date: 2024-11-22ANHUI UNIV OF SCI & TECH
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202410956267.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

There are many challenges in personnel detection in coal mine tank cages, including large depth of the tank cage, weak internal lighting, poor robustness of traditional algorithms, easy interference to infrared sensors and lack of computing resources, resulting in high false detection and missed detection rates, and the inability to quickly and accurately identify the tank cages. Insiders are unable to send out locking signals in time, which poses serious safety hazards.

Method used

Using the lightweight object detection model YOLO-Speed, combined with ELAN-Speed ​​and MP-Speed ​​modules, the dual network card structure is designed to isolate video and control signals, and improve transmission speed and model Real-time, realize fast and accurate pedestrian detection and locking signal transmission.

Benefits of technology

It realizes rapid and accurate detection of pedestrians in the can cage under the complex environment of coal mine tank cages, and reduces the transmission delay of locking signals, improves detection accuracy and speed, and ensures the safety of tank cage transportation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007238A_ABST
    Figure CN119007238A_ABST
Patent Text Reader

Abstract

The invention discloses a coal mine cage personnel detection method and system, and relates to the technical field of coal mine processing, and the method comprises the steps: employing a dual-network-port structure in hardware design, avoiding the problem of control signal transmission delay caused by network congestion of a coal mine video network, and improving the transmission speed of a control signal; an ELAN-Speed module and an ELAN-Sped2 module are provided, the model feature extraction capacity is effectively improved, and meanwhile the real-time performance of the model is guaranteed by controlling the calculation amount, the parameter amount and the memory access cost of the model; an SPPCSPC-Speed module is provided, the number of branches of traditional pyramid pooling is reduced by adopting a series pyramid pooling mode, and the memory access cost of model reasoning is reduced; on the basis of the designed lightweight neural network module, a YOLO-Speed target detection model is provided in combination with depth separable convolution, and real-time accurate detection of pedestrians in the auxiliary shaft cage is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of coal mine processing, and in particular to a method and system for detecting personnel in a coal mine cage. Background Art

[0002] The auxiliary shaft cage is a key equipment of the auxiliary transportation system of coal mines, responsible for the transportation of personnel and materials up and down the wellhead. In order to prevent injuries such as squeezing of personnel due to the shaking of materials in the cage during the transportation of materials, coal mine enterprises have stipulated that it is strictly forbidden to mix people and materials. The working modes of the cage are usually divided into two modes: lifting people and lifting objects. If the cage finds pedestrians entering in the lifting mode, it should immediately enter the locked state to prevent the mixing of people and objects from causing safety hazards. However, during the operation of the auxiliary shaft, due to various factors such as the negligence of the winch driver and the uneven safety awareness of the personnel, the failure to find people in the tank in the lifting mode, the misoperation to enter the lifting mode when transporting personnel, and the mixing of people and materials occasionally occur, which brings serious hidden dangers to the personal safety of front-line workers. Therefore, there is an urgent need for a method that can accurately identify people in the cage and give a locking signal immediately when people enter the tank, so as to quickly eliminate the safety hazards of cage transportation.

[0003] However, there are many challenges in accurately detecting people in coal mine cages. First, the cage is deep, but low in height, and the lighting inside is very weak. In dark conditions, the headlights of workers can easily cause uneven lighting. Therefore, even if an ultra-wide-angle spherical camera is installed inside the cage, its field of view can only cover a small part of the cage, and its imaging effect is extremely poor. At the same time, since the camera inside the cage needs to use wireless network transmission, the transmission delay and image quality will be affected by multiple interferences from various equipment and environments in the coal mine. Second, there are many types of materials that need to be transported in the cage, and many of them are similar in shape to people. Therefore, traditional algorithms are prone to false detection during detection. Third, the shapes of people are changeable. People in the cage may be in various postures such as standing, squatting, and bending over for construction. Due to the poor robustness of traditional algorithms, it is easy to miss detection during detection.

[0004] Infrared sensors detect people by sensing infrared radiation emitted by the human body. However, the sensing range of infrared sensors is usually fan-shaped or circular, and it is difficult to control its range within a specific shape or space. Therefore, when it is used for detecting people in a cage, its detection results are easily interfered by people outside the cage. In addition, infrared sensors are easily affected by temperature, and climate change will affect its detection accuracy.

[0005] Some companies have deployed algorithms in the front-end microprocessors of cameras to process and transmit videos immediately after collecting image information, thus achieving rapid response to on-site security incidents. However, the front-end microprocessors of cameras have limited computing resources and can only implement real-time operations of some traditional personnel recognition algorithms. Currently, they cannot support more accurate and robust video intelligent processing algorithms, resulting in high false detection and missed detection rates.

[0006] Through the deep convolutional neural network (DCNN), the target detection algorithm is constructed for personnel detection and has been applied in many security fields. However, the number of parameters and the amount of calculation of the high-precision target detection model built based on DCNN are very large. It is difficult for the network edge close to the data source to provide abundant storage and computing resources. Therefore, it is usually deployed in a high-performance server to implement video intelligent processing services. When implementing video intelligent processing based on this solution, it is necessary to first transmit the video to the server, wait for the server to complete the processing, and then feed back the video intelligent processing results to the control device for control. In this process, serious computing delays are generated, which cannot effectively protect the safety of pedestrians in the cage. Summary of the invention

[0007] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a coal mine cage personnel detection method and system.

[0008] In the first aspect, the purpose of the present invention can be achieved by the following technical solution: a method for detecting personnel in a coal mine cage, the method comprising the following steps:

[0009] Acquire coal mine related images, perform resize processing on the coal mine related images, and obtain processed coal mine related images;

[0010] The processed coal mine-related images are input into a pre-established lightweight target detection model, and the position coordinates of the personnel in the coal mine-related images are output;

[0011] Set the coordinate range of the coal mine cage area, and compare the position coordinates of the personnel in the coal mine related image with the coordinate range of the coal mine cage area. If the personnel are within the coordinate range of the coal mine cage area, the personnel are inside the cage and a locking signal is sent to lock the cage. Otherwise, no locking signal is sent.

[0012] In combination with the first aspect, in some implementations of the first aspect, the method further includes: a processing process before the coal mine related image is generated:

[0013] Pull the RTSP video stream of the corresponding camera and decode it;

[0014] The decoded image information is stored in the message queue. When the length of the message queue is greater than 1, the image information first sent to the message queue is popped out, the images accumulated in the message queue are deleted, and the video processing thread takes out the latest frame of the image from the message queue for processing.

[0015] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: the pre-established lightweight target detection model is established based on the target detection model YOLO-Speed, including a backbone network, a fusion network and an output.

[0016] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: the backbone network is used to extract deep feature information in the image, the fusion network is used to fuse different scale feature information output by the backbone network, and the output part outputs the detected personnel location information based on the multi-scale features as the location coordinates of the personnel in the coal mine-related image.

[0017] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the backbone network includes an ELAN-Speed ​​module and an MP-Speed ​​module, and is combined with a CBH convolution structure, where the CBH is composed of a convolution layer + a BN layer + a Hard-Swish activation function;

[0018] In combination with the first aspect, in some implementations of the first aspect, the method further includes: an expression of CBH:

[0019] f cbh (x) = Hard_Swish(BN(f conv (x)))

[0020] Among them, Hard_Swish(·) is the Hard_Swish activation function, BN(·) is the batch normalization operation, and f conv (·) is the convolution operation.

[0021] In combination with the first aspect, in some implementations of the first aspect, the method also includes: the DBH in the target detection model YOLO-Speed ​​is composed of a depth-separable convolution + a BN layer + a Hard-Swish activation function, and the CBM is composed of a convolution layer + a BN layer + a Sigmoid activation function.

[0022] In a second aspect, in order to achieve the above-mentioned purpose, the present invention discloses a coal mine cage personnel detection system, comprising:

[0023] An image processing module is used to obtain coal mine related images, resize the coal mine related images, and obtain processed coal mine related images;

[0024] A coordinate acquisition module is used to input the processed coal mine related images into a pre-established lightweight target detection model, and output the position coordinates of the personnel in the coal mine related images;

[0025] The position comparison module is used to set the coordinate range of the coal mine cage area and compare the position coordinates of the personnel in the coal mine related image with the coordinate range of the coal mine cage area. If the personnel are within the coordinate range of the coal mine cage area, the personnel are inside the cage and a locking signal is sent to lock the cage. Otherwise, no locking signal is sent.

[0026] In conjunction with the second aspect, in some implementations of the second aspect, the system further includes: a processing process before the generation of the coal mine related image in the image processing module:

[0027] Pull the RTSP video stream of the corresponding camera and decode it;

[0028] The decoded image information is stored in the message queue. When the length of the message queue is greater than 1, the image information first sent to the message queue is popped out, the images accumulated in the message queue are deleted, and the video processing thread takes out the latest frame of the image from the message queue for processing.

[0029] Among them, the lightweight target detection model pre-established in the coordinate acquisition module is established based on the target detection model YOLO-Speed, including the backbone network, fusion network and output;

[0030] The backbone network in the coordinate acquisition module is used to extract the deep feature information in the image, and the fusion network is used to fuse the different scale feature information output by the backbone network. The output part outputs the detected personnel position information according to the multi-scale features as the position coordinates of the personnel in the coal mine related images;

[0031] The backbone network in the coordinate acquisition module includes the ELAN-Speed ​​module and the MP-Speed ​​module, and is combined with the CBH convolutional structure. The CBH is composed of a convolutional layer + a BN layer + a Hard-Swish activation function.

[0032] The expression of CBH is:

[0033] f cbh (x) = Hard_Swish(BN(f conv (x)))

[0034] Among them, Hard_Swish(·) is the Hard_Swish activation function, BN(·) is the batch normalization operation, and f conv (·) is the convolution operation;

[0035] The DBH in the target detection model YOLO-Speed ​​in the coordinate acquisition module is composed of depthwise separable convolution + BN layer + Hard-Swish activation function, and the CBM is composed of convolution layer + BN layer + Sigmoid activation function.

[0036] Beneficial effects of the present invention:

[0037] The hardware design of the present invention adopts a dual network card structure. Since the coal mine video network needs to transmit a large amount of video information, although the device is deployed in a network location close to the data source, the transmission of the control signal in the video network will inevitably cause delays. The dual network port structure of the device separates the coal mine video network from the control network signal, improves the transmission speed of the control signal, and avoids delays in the control signal caused by network congestion. After testing, by setting up a separate network card to send control instructions to the cage control PLC, the transmission delay of the control signal is reduced by about 1.5 seconds compared with the method of directly transmitting from the video network. The ELAN-Speed ​​and ELAN-Speed2 modules are proposed to effectively improve the model feature extraction capability while controlling the model's calculation amount, parameter amount and memory access cost, thereby ensuring the real-time performance of the model. The SPPCSPC-Speed ​​module is proposed, and the number of branches of the traditional pyramid pooling is reduced by the serial pyramid pooling method, which reduces the memory access cost of model reasoning; by increasing the convolutional neural network branches and using deep separable convolution, the multi-scale feature fusion capability of the model is improved, and the YOLO-Speed ​​target detection model is designed. Through a variety of feature extraction modules, the effective feature extraction capability of the cage in complex environments is improved, ensuring the accuracy of cage pedestrian detection. Through the lightweight network structure design, the speed of the entire system to respond to pedestrians in the cage is guaranteed, and a coal mine auxiliary shaft cage safety locking device is constructed, which can quickly and accurately detect pedestrians in the cage and immediately send a locking signal. Compared with other systems, this device has the advantages of fast response speed and high precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 It is a schematic flow chart of the method of the present invention;

[0040] Figure 2 It is a schematic diagram of a safety locking signal generating device for a cage in a coal mine auxiliary shaft of the present invention;

[0041] Figure 3 It is a schematic diagram of the program flow of the present invention;

[0042] Figure 4 Schematic diagram of the YOLO-Speed ​​model architecture of the present invention;

[0043] Figure 5 is a schematic diagram of the ELAN-Speed ​​module of the present invention;

[0044] Figure 6 is a schematic diagram of the MP-Speed ​​module of the present invention;

[0045] Figure 7 It is a schematic diagram of the SPPCSPC-Speed ​​module of the present invention;

[0046] Figure 8 is a schematic diagram of the ELAN-Speed2 module of the present invention;

[0047] Fig. 9 It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Embodiment 1:

[0050] The following is an introduction to the relevant terms involved in the embodiments of the present application:

[0051] Coal mine cage is a kind of mine hoisting container used to transport personnel, ores, equipment, materials, etc. It can generally carry several tons and its use is similar to that of an elevator.

[0052] like Figure 1 As shown, a method for detecting personnel in a coal mine cage comprises the following steps:

[0053] Acquire coal mine related images, perform resize processing on the coal mine related images, and obtain processed coal mine related images;

[0054] Specifically, the present invention is further described below by way of embodiments:

[0055] In order to detect personnel in the cage and provide a locking signal to the control system in time, the present invention provides a coal mine auxiliary shaft cage safety locking signal generating device based on a lightweight video intelligent processing module. The structure of the device is as follows Figure 2As shown in the figure, the device is based on the NVIDIA Jetson TX2 embedded microprocessor, and an Ethernet interface is additionally extended through the USB interface, and a 12V power supply is provided for it through a 220V power supply. One of the Ethernet ports is connected to the coal mine video network to obtain the video information of the cage, and the other Ethernet port is connected to the control network to transmit the video signal using the Modbus-TCP protocol. The device needs to be deployed in a network location close to the data source (i.e., the camera) to reduce the delay caused by video transmission.

[0056] Processing process before generating coal mine related images:

[0057] The software is divided into two threads: decoding and processing. The decoding thread continuously pulls the RTSP video stream of the corresponding camera and decodes it to prevent the delay of directly processing the video from interfering with the decoding of the video stream (there must be no delay when decoding the RTSP video stream, otherwise errors will occur), and finally stores the decoded image information in the message queue. To prevent processing delays caused by backlogs in the message queue, when the length of the message queue is greater than 1, the image information first sent to the message queue will be popped up.

[0058] The processed coal mine-related images are input into a pre-established lightweight target detection model, and the position coordinates of the personnel in the coal mine-related images are output;

[0059] The key to the present invention lies in the design of a lightweight target detection model, which first needs to achieve accurate detection of pedestrians under complex conditions in the auxiliary shaft cage. In addition, high real-time detection is also required to promptly respond to safety hazards occurring on site.

[0060] The pre-established lightweight target detection model is based on the target detection model YOLO-Speed, including the backbone network, fusion network and output. Figure 4 As shown in the figure, the backbone network is used to extract the deep feature information in the image, the fusion network is used to fuse the feature information of different scales output by the backbone network, and the output part outputs the detected personnel location information based on the multi-scale features. Figure 4 In the figure, the input image size of the model is (416, 416), and there are three output heads, Y1, Y2, and Y3, to adapt to different human body scales in the image.

[0061] In the backbone network, the present invention designs an ELAN-Speed ​​module, the structure of which is as follows: Figure 5As shown. Where c is the number of feature channels, s is the step size, and k is the convolution kernel size. The module first splits the input features into two parts, so that the number of input and output feature channels of the left and right convolution modules remain consistent, reducing the memory access cost caused by model inference, reducing the number of input feature channels of the convolution layer, and further reducing the amount of calculation and the number of parameters; secondly, the model uses a depth-wise separable convolution (DWC) with a convolution kernel size of 3×3, reducing the amount of calculation and the number of parameters; finally, the convolution output features are batch normalized based on the BN layer, and the linear activation function is used to directly output the features. After all the features are connected by the Concate operation, the module obtains nonlinear performance based on the Hard-Swish activation function, improves the fitting ability of the model, greatly reduces the element-level operations in the model, and reduces the memory access cost of the module. At the same time, the Hard-Swish activation function is used to avoid exponential operations and improve the hardware friendliness of the model. The ELAN-Speed ​​module uses a dual-branch structure to improve the model's ability to perceive different depth features. It also introduces an attention mechanism to improve the model's anti-interference ability in complex environments, enabling the backbone network to effectively extract the key features of pedestrians under complex conditions in the cage.

[0062] In the backbone network, the present invention also designs an MP-Speed ​​module, whose structure is as follows: Figure 6 As shown in the figure, this module is used to implement downsampling of image features. The module first uses Split to divide the input features into two parts, and then one branch uses a depth-wise separable convolution with a step size of 2 to implement feature downsampling, and the other branch uses maximum pooling and convolution feature extraction to implement feature downsampling, effectively avoiding the loss of feature information caused by a single downsampling method.

[0063] The backbone network uses the ELAN-Speed ​​and MP-Speed ​​modules combined with the CBH convolution structure to achieve efficient extraction of image features. Among them, CBH is composed of convolution layer + BN layer + Hard-Swish activation function, CBH-1 means the convolution step size is 1, and CBH-1 means the convolution step size is 2.

[0064] The expression of CBH is:

[0065] f cbh (x) = Hard_Swish(BN(f conv (x)))

[0066] Among them, Hard_Swish(·) is the Hard_Swish activation function, BN(·) is the batch normalization operation, and f conv (·) is the convolution operation;

[0067] In the fusion network, the present invention designs the SPPCSPC-Speed ​​module, whose structure is as follows Figure 7 As shown in the figure, this module strengthens the fusion capability of different depth features and the multi-scale feature extraction capability of the model, and applies lightweight network design to ensure the real-time performance of the model. First, the SPPCSPC-Speed ​​module uses the Split operation to divide the input features into two parts, and inputs them into the left and right branches respectively, so that the number of input and output feature channels of the left convolution layer and the first convolution layer on the right are consistent, reducing the amount of calculation and parameters of the convolution layer, and reducing the memory access cost; then, the left branch directly uses the convolution layer for feature integration, and the right branch uses the serial pyramid pooling to realize multi-scale feature extraction after two convolution layers. At the same time, the serial structure plays a role in reducing the number of branches and reducing the memory access cost; finally, the feature information extracted by the serial pyramid pooling is connected, and after feature extraction and integration through two convolution layers, it is fused with the features of the left branch, and feature integration is performed through the CBH-1 module.

[0068] In the converged network, the present invention designs an ELAN-Speed2 module, whose structure is as follows Figure 8 As shown, this module is used in the fusion network to enhance the model's ability to extract important information from deep feature information.

[0069] In addition to the above structures, the DBH mentioned in the model of the present invention is composed of depth-separable convolution + BN layer + Hard-Swish activation function, and the CBM is composed of convolution layer + BN layer + Sigmoid activation function.

[0070] Set the coordinate range of the coal mine cage area, and compare the position coordinates of the personnel in the coal mine related image with the coordinate range of the coal mine cage area. If the personnel are within the coordinate range of the coal mine cage area, the personnel are inside the cage and a locking signal is sent to lock the cage. Otherwise, no locking signal is sent.

[0071] The YOLO-Speed ​​model proposed in the present invention only requires 22.45GnFLOPs of computation and 18.54M parameters, which are 49.77% and 50.72% lower than the YOLOv7 target detection model, respectively. When detecting people walking down a coal mine shaft, the accuracy reaches 92.72%. Table 1 shows the performance comparison of YOLO-Speed ​​and several other models on the coal mine shaft pedestrian data set. The model achieves the best balance between detection accuracy and speed. The detection speed on the NVIDIArtx3090 platform can reach 98fps, and the detection speed on the embedded microprocessor NVIDIAJetson TX2 can reach 11.2fps. According to the test, the time delay from the personnel entering the cage to the system sending the locking signal can be controlled within 100ms.

[0072] Table 1: Performance comparison of YOLO-Speed ​​and other centralized models on the coal mine pedestrian dataset

[0073]

[0074] Embodiment 2: In a second aspect, in order to achieve the above-mentioned purpose, the present invention discloses a coal mine cage personnel detection system, comprising:

[0075] An image processing module is used to obtain coal mine related images, resize the coal mine related images, and obtain processed coal mine related images;

[0076] A coordinate acquisition module is used to input the processed coal mine related images into a pre-established lightweight target detection model, and output the position coordinates of the personnel in the coal mine related images;

[0077] The position comparison module is used to set the coordinate range of the coal mine cage area and compare the position coordinates of the personnel in the coal mine related image with the coordinate range of the coal mine cage area. If the personnel are within the coordinate range of the coal mine cage area, the personnel are inside the cage and a locking signal is sent to lock the cage. Otherwise, no locking signal is sent.

[0078] In conjunction with the second aspect, in some implementations of the second aspect, the system further includes: a processing process before the generation of the coal mine related image in the image processing module:

[0079] Pull the RTSP video stream of the corresponding camera and decode it;

[0080] The decoded image information is stored in the message queue. When the length of the message queue is greater than 1, the image information first sent into the message queue is popped out as the coal mine related image;

[0081] Among them, the lightweight target detection model pre-established in the coordinate acquisition module is established based on the target detection model YOLO-Speed, including the backbone network, fusion network and output;

[0082] The backbone network in the coordinate acquisition module is used to extract the deep feature information in the image, and the fusion network is used to fuse the different scale feature information output by the backbone network. The output part outputs the detected personnel position information according to the multi-scale features as the position coordinates of the personnel in the coal mine related images;

[0083] The backbone network in the coordinate acquisition module includes the ELAN-Speed ​​module and the MP-Speed ​​module, and is combined with the CBH convolutional structure. The CBH is composed of a convolutional layer + a BN layer + a Hard-Swish activation function.

[0084] The expression of CBH is:

[0085] f cbh (x) = Hard_Swish(BN(f conv (x)))

[0086] Among them, Hard_Swish(·) is the Hard_Swish activation function, BN(·) is the batch normalization operation, and f conv (·) is the convolution operation;

[0087] The DBH in the target detection model YOLO-Speed ​​in the coordinate acquisition module is composed of depthwise separable convolution + BN layer + Hard-Swish activation function, and the CBM is composed of convolution layer + BN layer + Sigmoid activation function.

[0088] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0089] It needs to be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to execute the above method. The storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0090] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0091] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure may have various changes and improvements, and these changes and improvements fall within the scope of the present disclosure to be protected.

Claims

1. A method for detecting personnel in a coal mine cage, characterized in that: The method comprises the following steps: Acquire coal mine related images, perform resize processing on the coal mine related images, and obtain processed coal mine related images; The processed coal mine-related images are input into a pre-established lightweight target detection model, and the position coordinates of the personnel in the coal mine-related images are output; Set the coordinate range of the coal mine cage area, and compare the position coordinates of the personnel in the coal mine related image with the coordinate range of the coal mine cage area. If the personnel are within the coordinate range of the coal mine cage area, the personnel are inside the cage and a locking signal is sent to lock the cage. Otherwise, no locking signal is sent.

2. A coal mine cage personnel detection method according to claim 1, characterized in that: The processing process before the coal mine related image is generated: Pull the RTSP video stream of the corresponding camera and decode it; The decoded image information is stored in the message queue. When the length of the message queue is greater than 1, the image information first sent into the message queue is popped out.

3. A coal mine cage personnel detection method according to claim 1, characterized in that: The pre-established lightweight target detection model is the target detection model YOLO-Speed, which includes a backbone network, a fusion network and an output.

4. A coal mine cage personnel detection method according to claim 3, characterized in that: The backbone network is used to extract deep feature information from the image, the fusion network is used to fuse the different scale feature information output by the backbone network, and the output part outputs the detected personnel position information based on the multi-scale features as the position coordinates of the personnel in the coal mine related images.

5. A coal mine cage personnel detection method according to claim 4, characterized in that: The backbone network includes an ELAN-Speed ​​module and an MP-Speed ​​module, and is combined with a CBH convolutional structure, where the CBH is composed of a convolutional layer + a BN layer + a Hard-Swish activation function.

6. A coal mine cage personnel detection method according to claim 5, characterized in that: The expression of the CBH is: f cbh (x)=Hard_Swish(BN(f conv (x))) Among them, Hard_Swish(·) is the Hard_Swish activation function, BN(·) is the batch normalization operation, and f conv (·) is the convolution operation.

7. A coal mine cage personnel detection method according to claim 3, characterized in that: The DBH in the target detection model YOLO-Speed ​​is composed of depth-separable convolution + BN layer + Hard-Swish activation function, and the CBM is composed of convolution layer + BN layer + Sigmoid activation function.

8. A coal mine cage personnel detection system, characterized in that: include: An image processing module is used to obtain coal mine related images, resize the coal mine related images, and obtain processed coal mine related images; A coordinate acquisition module is used to input the processed coal mine related images into a pre-established lightweight target detection model, and output the position coordinates of the personnel in the coal mine related images; The position comparison module is used to set the coordinate range of the coal mine cage area and compare the position coordinates of the personnel in the coal mine related image with the coordinate range of the coal mine cage area. If the personnel are within the coordinate range of the coal mine cage area, the personnel are inside the cage and a locking signal is sent to lock the cage. Otherwise, no locking signal is sent.

9. A coal mine cage personnel detection system according to claim 8, characterized in that: The processing process before the generation of coal mine related images in the image processing module: Pull the RTSP video stream of the corresponding camera and decode it; The decoded image information is stored in the message queue. When the length of the message queue is greater than 1, the image information first sent into the message queue is popped out, and the image processing thread takes out the latest frame of the image from the message queue for processing; Among them, the lightweight target detection model pre-established in the coordinate acquisition module is established based on the target detection model YOLO-Speed, including the backbone network, fusion network and output; The backbone network in the coordinate acquisition module is used to extract the deep feature information in the image, and the fusion network is used to fuse the different scale feature information output by the backbone network. The output part outputs the detected personnel position information according to the multi-scale features as the position coordinates of the personnel in the coal mine related images; The backbone network in the coordinate acquisition module includes the ELAN-Speed ​​module and the MP-Speed ​​module, and is combined with the CBH convolutional structure. The CBH is composed of a convolutional layer + a BN layer + a Hard-Swish activation function. The expression of CBH is: f cbn (x)=Hard_Swish(BN(f conv (x))) Among them, Hard_Swish(·) is the Hard_Swish activation function, BN(·) is the batch normalization operation, and f conv (·) is the convolution operation; The DBH in the target detection model YOLO-Speed ​​in the coordinate acquisition module is composed of depthwise separable convolution + BN layer + Hard-Swish activation function, and the CBM is composed of convolution layer + BN layer + Sigmoid activation function.

Citation Information

Patent Citations

  • Video monitoring method and device based on deep learning

    CN113723361A

  • Method and device for detecting unsafe behaviors and states of miners in cage of coal mine elevator

    CN115761306A

  • Coal mine underground target identification system based on improved YOLOv7

    CN117197750A