Method and device for detecting large coal under mine, electronic equipment and computer program

Abstract

The invention provides an under-mine large coal detection method which comprises the following steps: presetting a polygonal detection area in a shot image according to the shooting position of a current camera; detecting a coal briquette target based on an image shot by the camera for the transfer belt to obtain a center point coordinate, a width w in the horizontal direction and a longitudinal height h of the target coal briquette; for a target coal briquette falling into the polygonal detection area, calculating a horizontal distance d of the polygonal detection area at the central point of the target coal briquette; the larger value of the width w and the height h of the target coal briquette and the distance d are proportionally calculated, and the coal briquettes exceeding a threshold value are judged as large coal briquettes. According to the method disclosed by the invention, a pure computer vision processing scheme is used for carrying out automatic detection on the large coal, and self-adaptive proportion judgment is carried out on the coal briquettes falling into the detection area, so that the influence of a near-far effect caused by image shooting is overcome, the detection accuracy is improved, and false alarms outside a preset area can be filtered out.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition technology, and in particular to a method, device, electronic device and computer program for detecting bulk coal in a mine. Background Art

[0002] In the coal transfer system, the transfer belt is an important transportation equipment. If abnormally large coal lumps appear on the belt, it is easy to cause the belt to get stuck, tear, and even lead to equipment failure and other accidents.

[0003] Currently, the detection of large lumps of coal can be achieved through image processing technology. A monitoring camera or intelligent shooting device is used to capture real-time images of the transport belt. Large lumps of coal are then identified through image recognition technology. The identified lumps of coal are then screened based on preset width, height, or area thresholds. Those exceeding the threshold are identified as abnormally large lumps of coal. In the prior art, whether the current coal lump is a large lump of coal can be determined based on the ratio of the width of the coal lump detection frame to the width of the preset area. This technical solution has the following shortcomings: when the coal lump is running along the belt, the image captured by the camera fixed on the camera position has a near-far effect (i.e., the same object is larger when closer and smaller when farther away). The size of the coal lump detection frame identified at different positions will also be larger when closer and smaller when farther away. The preset area has a fixed width. The comparison and judgment result is accurate only when the coal lump is exactly at a specific position, which limits the detection range of the technology and increases the difficulty of detection and the risk of false alarms.

[0004] Therefore, how to accurately determine whether the coal blocks on the belt are large blocks of coal under the current camera environment is a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The present invention provides a method, device, electronic equipment and computer program for detecting large coal lumps in mines, which are used to solve the problems of limited detection range and high risk of false alarms in the prior art.

[0006] The present invention solves the above technical problems through the following aspects:

[0007] In one aspect, the present invention provides a method for detecting bulk coal in a mine, comprising:

[0008] According to the current camera shooting position, a polygon detection area is preset in the captured image;

[0009] Detecting a coal block target based on the image captured by the camera on the transfer belt, and obtaining the center coordinates, horizontal width w, and vertical height h of the target coal block;

[0010] For a target coal block falling within the polygonal detection area, calculating a horizontal distance d between the polygonal detection area and the center point of the target coal block;

[0011] The larger value of the target coal block width w and height h is taken to be proportional to the distance d, and the coal exceeding the threshold is determined to be a large lump of coal.

[0012] In another aspect, the present invention provides a device for detecting bulk coal in a mine, comprising:

[0013] The preset area module is used to preset a polygon detection area in the captured image according to the current camera shooting position;

[0014] A target detection module is used to detect a coal block target based on the image captured by the camera on the transfer belt, and obtain the center point coordinates, horizontal width w, and vertical height h of the target coal block;

[0015] A distance calculation module, for a target coal block falling within the polygonal detection area, calculates a horizontal distance d between the polygonal detection area and the center point of the target coal block;

[0016] The comparison and determination module calculates the ratio of the larger value of the width w and height h of the target coal block to the distance d, and determines that the coal block exceeding the threshold is a large lump of coal.

[0017] On the other hand, the present invention also provides an electronic device for detecting large lumps of coal in mines, comprising: a microprocessor, and a memory communicatively connected to the microprocessor; the memory stores instructions that can be executed by the microprocessor, and when the instructions are executed by the microprocessor, the aforementioned method for detecting large lumps of coal can be implemented.

[0018] The present invention also provides a computer program, which, when executed by a processor, enables the processor to implement the aforementioned large coal detection method.

[0019] The technical solution of the present invention uses a pure computer vision processing solution for automatic detection. It is simple and easy to implement, and does not require human intervention. It makes up for the deficiency of the original deep learning target detection that can only identify lump coal and cannot filter small coal lumps based on size. Through the preset polygonal detection area, the coal lumps falling into the detection area are adaptively proportionally determined. This not only overcomes the influence of the near-far effect brought by the captured image, improves the detection accuracy, but also filters out false alarms outside the preset area. The technical solution of the present invention can flexibly adjust the scale threshold according to the definition of the large coal size in different coal mines, effectively eliminate small coal lumps, and the detection process is flexible and efficient, and the detection results are highly accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 An example of an image captured by a camera above the transfer belt;

[0022] Figure 2 A flow chart of a method for detecting bulk coal in a mine provided by an embodiment of the present disclosure;

[0023] Figure 3 Schematic diagram of coal blocks and detection areas in the embodiment;

[0024] Figure 4 This is a structural block diagram of the device for detecting large coal lumps in a mine provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure. In addition, for the sake of clarity, parts that are not related to the description of the exemplary embodiments are omitted in the drawings.

[0026] In this specification, it should be understood that terms such as "including" or "having" are intended to indicate the presence of the disclosed features, numbers, steps, actions, components, parts, or combinations thereof, and are not intended to exclude the possibility of one or more other features, numbers, steps, actions, components, parts, or combinations thereof being present or added. It should also be noted that, unless there is a conflict, the embodiments of this disclosure and the features therein may be combined with each other.

[0027] Figure 1 Here is an example of an image captured by a camera above the transfer belt.

[0028] Industrial intelligent cameras are installed along the coal and mineral material transportation production line. The cameras are placed at the inspection station to collect running images of the conveyor belts. The collected images are processed online in real time to identify large pieces of coal in the images. Figure 1 As shown, the blue frame is the identified coal mine detection frame, and the red frame is the belt area manually marked at the camera's shooting point.

[0029] After detecting a large lump of coal, the disclosed technical solution extracts its center point, width, and height data. Based on the lump's landing location, the belt width at that point is calculated, and then a ratio is calculated. In theory, even with perspective effects in the captured image, the ratio of the lump's width to the belt width at that point remains essentially unchanged, enabling accurate determination of lump size.

[0030] The following describes the method through specific examples.

[0031] Figure 2 This is a flow chart of a method for detecting large coal lumps in a mine provided by an embodiment of the present disclosure.

[0032] like Figure 2 As shown, the method of this embodiment includes steps S210 to S240.

[0033] S210: Preset a polygonal detection area in the captured image according to the current camera shooting position.

[0034] The polygonal detection area has two sides that adapt to the boundary of the transfer belt in the image. The polygon is generally a trapezoid, such as Figure 3 The polygon detection frame shown in the figure, where (x1, y1), (x2, y2), (x3, y3), and (x4, y4) are the coordinates of the four vertices of the polygon detection area. Figure 1 It corresponds to the belt border in the actual image. It can be seen that the two oblique sides of the polygon are aligned with the boundary of the transfer belt, which is consistent with the characteristics of larger objects near and smaller objects far away in the actual captured image. During implementation, since the shooting position of the camera is fixed, the position of the belt in the image is also fixed. The belt boundary in the image can be manually marked to obtain the four vertex coordinates of a suitable polygonal detection area. Subsequent coal block detection can be based on this detection area, and usually no changes are required. Since the shooting angles of different cameras are different, the detection area can be set for the images taken by different cameras. During actual calculations, the vertex coordinates of the polygonal detection area obtained through the front-end web page frame do not correspond one-to-one to the actual belt boundary. At this time, the upper left, lower left, upper right and lower right coordinate points can be located according to the centroid of the coordinates. The line connecting the upper left and lower left vertices is used as one boundary of the belt, and the line connecting the upper right and lower right vertices is used as the other boundary of the belt.

[0035] S220: Detect the coal block target based on the image captured by the camera on the transfer belt, and obtain the center point coordinates, horizontal width w, and vertical height h of the target coal block.

[0036] When the camera is working normally, it can continuously collect images within the shooting range and obtain live image streams. After necessary preprocessing, such as filtering and denoising, and size normalization, the deep learning detection model YOLOV5 is used to detect coal block targets and record the center point coordinates, width and height of the detected coal block targets. Figure 3 In the 2D image, the center coordinates (x, y), horizontal width w, and vertical height h of the coal block are obtained. Other deep learning object detection algorithms, such as SSD, YOLOv3, and YOLOv7, can also be used to detect coal blocks. The disclosed embodiment uses YOLOv5, which balances computational efficiency and detection performance, making it more suitable for object recognition in images captured underground.

[0037] Before detecting coal lumps, the belt's motion state can also be checked. If the belt is in motion, coal lumps are detected; otherwise, no detection is performed. Belt motion can be detected using the OpenCV optical flow method. Large coal lumps are detected only when the belt is in motion, reducing false alarms and improving detection efficiency.

[0038] S230: For the target coal block falling within the polygonal detection area, calculate the horizontal distance d between the polygonal detection area and the center point of the target coal block.

[0039] The center point of the target coal block is compared to a pre-set polygonal detection area for point-in-point detection. When the center point of the target coal block falls within the polygonal detection area, the coal block is considered to be within the detection area. Coal blocks falling outside the area are not further detected, and only those falling within the detection area are processed. This reduces false positives and improves detection accuracy. Because the detection area is larger than the coal block area, multiple images in the captured image stream can be used for large coal block detection, thereby improving detection robustness.

[0040] By using the coordinates of the four vertices in the preset detection area, calculate the intersection of the horizontal line of the coal block target center point with the line connecting the upper left and lower left vertices and the intersection of the line connecting the upper right and lower right vertices. The distance d between the two intersections is also the belt width at that location. Figure 3 As shown in the figure, the center point of the coal target is (x, y). The intersection of the horizontal line passing through this point and the two oblique sides of the detection area are points a and b, respectively, and the distance between them is d. Based on the center point of the coal target, which is also the landing point of the coal, the belt width automatically adapts to the image characteristics of larger near objects and smaller far objects. Compared with a fixed preset area width, it better matches the actual image, thereby improving the accuracy of the comparison.

[0041] S240: Take the larger value of the target coal block width w and height h and perform proportional calculation with the distance d, and determine that the coal block that exceeds the threshold is a large lump of coal.

[0042] During the comparison, the larger value of the large lump coal h and w is taken and the ratio is calculated with the belt width d. The ratio is then compared with the set threshold to determine whether the coal is large. The threshold can be set by each coal production enterprise, for example, 25%.

[0043] After obtaining the judgment results, the images containing large coal lumps can be annotated based on the judgment results and alarm data can be generated and uploaded to the host system. The alarm data can include information such as timestamp, camera number, and large coal type.

[0044] According to the method of the embodiment of the present disclosure, a pure computer vision processing solution is used to automatically detect large lumps of coal without the need for human intervention, which makes up for the deficiency of the original deep learning target detection that can only identify lumpy coal and cannot filter small lumps of coal based on size. Through the preset polygonal detection area, the coal lumps falling into the detection area are adaptively proportionally determined. This not only overcomes the influence of the near-far effect brought by the captured image and improves the detection accuracy, but also filters out false alarms outside the preset area. This technical solution can flexibly adjust the scale threshold according to the definition of the large coal size in different coal mines, effectively eliminate small coal lumps, and the detection process is flexible and efficient, and the detection results are highly accurate.

[0045] Correspondingly, the present disclosure also provides a device for detecting large coal masses in mines. Figure 4 This is a structural block diagram of a device 400 for detecting large coal lumps in a mine according to an embodiment of the present disclosure. The device 400 can be implemented as part or all of an electronic device through software, hardware, or a combination of both.

[0046] like Figure 4 As shown, the device 400 includes a preset area module 410 , a target detection module 420 , a distance calculation module 430 and a comparison and determination module 440 .

[0047] The preset area module 410 is used to preset a polygon detection area in the captured image according to the current camera shooting position.

[0048] The target detection module 420 is used to detect the coal block target based on the image captured by the camera on the transfer belt, and obtain the center point coordinates, horizontal width w and vertical height h of the target coal block.

[0049] The distance calculation module 430 calculates the horizontal distance d between the target coal block and the center point of the polygonal detection area for the target coal block that falls within the polygonal detection area.

[0050] The comparison and determination module 440 calculates the ratio of the larger value of the width w and height h of the target coal block to the distance d, and determines that the coal block exceeding the threshold is a large lump of coal.

[0051] The embodiment of the present disclosure also provides an electronic device for detecting large coal lumps in mines, including a microprocessor and a memory connected to the microprocessor for communication; the memory stores instructions that can be executed by the microprocessor, and when the instructions are executed by the microprocessor, the various methods described above can be implemented.

[0052] The embodiment of the present disclosure also provides a computer program, which, when executed by a processor, enables the processor to implement the aforementioned method for detecting large coal lumps in a mine.

[0053] The apparatus, electronic device, computer program and method provided in the embodiments of the present disclosure correspond to each other, and therefore also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, they will not be repeated here.

[0054] Each embodiment of this disclosure is described in a progressive manner. Similar portions between embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments. In particular, the device, electronic device, and computer program embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant details, refer to the description of the method embodiments on the label side.

[0055] The units or modules involved in the embodiments described in this disclosure may be implemented by software or programmable hardware. The units or modules described may also be provided in a processor, and the names of these units or modules do not, in certain circumstances, constitute limitations on the units or modules themselves.

[0056] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0057] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0058] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0059] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0060] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0061] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data optimization device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data optimization device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0062] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data optimization device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions may also be loaded onto a computer or other programmable data optimization device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide for implementing the process described in the flow. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0064] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0065] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0066] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0067] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0068] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0069] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0070] The foregoing is merely an embodiment of the present invention and is not intended to limit the present application. For those skilled in the art, various modifications and variations may be made to the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for detecting bulk coal in a mine, characterized in that: include: According to the current camera shooting position, a polygon detection area is preset in the captured image; Detecting a coal block target based on the image captured by the camera on the transfer belt, and obtaining the center coordinates, horizontal width w, and vertical height h of the target coal block; For a target coal block falling within the polygonal detection area, calculating a horizontal distance d between the polygonal detection area and the center point of the target coal block; The larger value of the target coal block width w and height h is taken to be proportional to the distance d, and the coal exceeding the threshold is determined to be a large lump of coal.

2. The method according to claim 1, characterized in that The polygonal detection area has two sides adapted to the transfer belt boundary in the image.

3. The method according to claim 1, characterized in that The threshold is 25%.

4. The method according to claim 1, wherein The detecting of the coal block target based on the image captured by the camera on the transport belt comprises: Based on the image stream captured by the camera on the transfer belt, the coal block target is detected by the deep learning detection model YOLOV5.

5. The method according to claim 1, wherein Before detecting the coal block target, the method further includes: detecting the motion state of the belt; if the belt is in motion, performing coal block target detection.

6. The method according to claim 5, characterized in that The detecting the motion state of the belt includes: detecting the motion state of the belt by using an opencv optical flow method.

7. The method according to claim 1, characterized in that Also includes: Based on the judgment results, the images containing large coal pieces are marked and alarm data is generated and uploaded to the host system.

8. A device for detecting large coal in a mine, characterized in that: include: The preset area module is used to preset a polygon detection area in the captured image according to the current camera shooting position; A target detection module is used to detect a coal block target based on the image captured by the camera on the transfer belt, and obtain the center point coordinates, horizontal width w, and vertical height h of the target coal block; A distance calculation module, for a target coal block falling within the polygonal detection area, calculates a horizontal distance d between the polygonal detection area and the center point of the target coal block; The comparison and determination module calculates the ratio of the larger value of the width w and height h of the target coal block to the distance d, and determines that the coal block exceeding the threshold is a large lump of coal.

9. An electronic device for detecting large coal lumps in mines, characterized in that: include: a microprocessor, and a memory communicatively connected to the microprocessor; The memory stores instructions that can be executed by the microprocessor, and when the instructions are executed by the microprocessor, the method for detecting large coal according to any one of claims 1 to 8 can be implemented.

10. A computer program, which, when executed by a processor, enables the processor to implement the bulk coal detection method according to any one of claims 1 to 7.

Images