Underground coal mine target detection method and device based on deep learning

Abstract

The invention provides a coal mine underground target detection method and device based on deep learning. The underground coal mine target detection method based on deep learning comprises the following steps: acquiring an underground coal mine scene image, and preprocessing the scene image; inputting the preprocessed scene image into the trained deep learning model, carrying out target detection, and outputting target information; according to the target information, performing association matching on a detected target through a data association method, and determining a motion track of the target; and outputting the target information and the motion trail. According to the process, the underground coal mine image can be accurately collected and preprocessed, the target is efficiently detected by using the deep learning model, the target is accurately matched through data association, the motion trail is determined, finally detailed target information and the motion trail are output, and powerful support is provided for coal mine safety production.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a method and device for detecting underground targets in coal mines based on deep learning. Background Art

[0002] Target detection in coal mines is an important part of coal mine safety production. Due to the complex environment of coal mines, there are problems such as insufficient lighting, low illumination, and large target overlap areas. Traditional target detection methods often fail to meet actual needs. With the rapid development of deep learning technology, its application in the field of image processing is becoming more and more extensive, providing a new solution for target detection in coal mines.

[0003] Deep learning technology can automatically learn feature information in images and accurately identify and locate targets by building deep neural network models. In underground coal mine target detection, deep learning technology can overcome the limitations of traditional methods and improve the accuracy and robustness of detection. The core of the target detection algorithm based on deep learning is to input the image into the model, return the target's bounding anchor box, position, and category information, and realize end-to-end detection. However, the algorithm still has problems such as too many parameters, high computational cost, and low efficiency. Summary of the invention

[0004] The embodiments of the present application provide a method and device for underground target detection in coal mines based on deep learning, which can at least to a certain extent solve the problems of excessive number of parameters, high computational cost, and low efficiency.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0006] According to one aspect of the present application, a deep learning-based target detection method for underground coal mines is provided, including: collecting scene images underground in coal mines, and preprocessing the scene images; inputting the preprocessed scene images into a trained deep learning model, performing target detection, and outputting target information; according to the target information, associating and matching the detected targets through a data association method to determine the motion trajectory of the target; and outputting the target information and the motion trajectory.

[0007] In the present application, based on the aforementioned scheme, the scene images underground in a coal mine are collected and the scene images are preprocessed, including: shooting videos of multiple scenes through underground monitoring cameras; editing the videos into scene images frame by frame; and preprocessing the scene images.

[0008] In the present application, based on the above-mentioned solution, the preprocessing of the scene image includes: obtaining the brightness value corresponding to each pixel coordinate in the scene image, and performing nonlinear enhancement on the scene image according to the brightness value.

[0009] In the present application, based on the aforementioned scheme, the preprocessed scene image is input into a trained deep learning model to perform target detection and output target information, including: inputting the preprocessed scene image into a trained deep learning model, adaptively generating a prediction frame according to the size of the scene image and the size distribution of the target; comparing the prediction frame with the real frame and calculating the loss function; judging the detection result according to the loss function and outputting the target information.

[0010] In the present application, based on the aforementioned scheme, the detected target is associated and matched according to the target information through a data association method to determine the motion trajectory of the target, including: according to the target information, the targets of the previous and next frames are matched through a data association method to determine the associated frame; the motion information and appearance information of the target in the associated frame are obtained; the similarity is determined according to the motion information and appearance information, and the motion trajectory of the target is determined according to the similarity.

[0011] In the present application, based on the aforementioned solution, the outputting of the target information and the motion trajectory includes: acquiring a scene map where the target is located; and generating the motion trajectory in the scene map according to the target information.

[0012] In the present application, based on the above-mentioned solution, it also includes: uploading the motion trajectory to a cloud platform.

[0013] According to one aspect of the present application, a coal mine underground target detection device based on deep learning is provided, comprising:

[0014] A collection unit, used for collecting scene images in the coal mine and preprocessing the scene images;

[0015] The detection unit is used to input the preprocessed scene image into the trained deep learning model, perform target detection, and output target information;

[0016] A trajectory unit, used to perform association matching on the detected target through a data association method according to the target information, so as to determine the motion trajectory of the target;

[0017] An output unit is used to output the target information and the motion trajectory.

[0018] In the present application, based on the aforementioned scheme, the scene images underground in a coal mine are collected and the scene images are preprocessed, including: shooting videos of multiple scenes through underground monitoring cameras; editing the videos into scene images frame by frame; and preprocessing the scene images.

[0019] In the present application, based on the above-mentioned solution, the preprocessing of the scene image includes: obtaining the brightness value corresponding to each pixel coordinate in the scene image, and performing nonlinear enhancement on the scene image according to the brightness value.

[0020] In the present application, based on the aforementioned scheme, the preprocessed scene image is input into a trained deep learning model to perform target detection and output target information, including: inputting the preprocessed scene image into a trained deep learning model, adaptively generating a prediction frame according to the size of the scene image and the size distribution of the target; comparing the prediction frame with the real frame and calculating the loss function; judging the detection result according to the loss function and outputting the target information.

[0021] In the present application, based on the aforementioned scheme, the detected target is associated and matched according to the target information through a data association method to determine the motion trajectory of the target, including: according to the target information, the targets of the previous and next frames are matched through a data association method to determine the associated frame; the motion information and appearance information of the target in the associated frame are obtained; the similarity is determined according to the motion information and appearance information, and the motion trajectory of the target is determined according to the similarity.

[0022] In the present application, based on the aforementioned solution, the outputting of the target information and the motion trajectory includes: acquiring a scene map where the target is located; and generating the motion trajectory in the scene map according to the target information.

[0023] In the present application, based on the above-mentioned solution, it also includes: uploading the motion trajectory to a cloud platform.

[0024] According to one aspect of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the deep learning-based coal mine underground target detection method as described in the above embodiment is implemented.

[0025] According to one aspect of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the coal mine underground target detection method based on deep learning as described in the above-mentioned embodiments.

[0026] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the coal mine underground target detection method based on deep learning provided in the above-mentioned various optional implementations.

[0027] In the technical solution of the present application, scene images of underground coal mines are collected, and the scene images are preprocessed; the preprocessed scene images are input into a trained deep learning model to perform target detection and output target information; according to the target information, the detected targets are associated and matched by a data association method to determine the motion trajectory of the target; the target information and the motion trajectory are output. The above process can accurately collect and preprocess underground coal mine images, use deep learning models to efficiently detect targets, accurately match targets and determine motion trajectories through data association, and finally output detailed target information and motion trajectories, providing strong support for safe production in coal mines.

[0028] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 A flowchart of a method for detecting underground coal mine targets based on deep learning in one embodiment of the present application is schematically shown.

[0031] Figure 2 The flowchart of detecting target information in one embodiment of the present application is schematically shown.

[0032] Figure 3 A schematic diagram of a coal mine underground target detection device based on deep learning in one embodiment of the present application is schematically shown.

[0033] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0035] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0036] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0037] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0038] The implementation details of the technical solution of this application are described in detail below:

[0039] Figure 1 A flow chart of a method for detecting underground coal mine targets based on deep learning according to an embodiment of the present application is shown. Figure 1 As shown, the coal mine underground target detection method based on deep learning includes at least steps S110 to S140, which are described in detail as follows:

[0040] In step S110, scene images of the coal mine are collected and preprocessed.

[0041] In one embodiment of the present application, a scene image of a coal mine is collected and preprocessed, including:

[0042] Shoot videos of multiple scenes through underground monitoring cameras;

[0043] Editing the video into scene images frame by frame;

[0044] The scene image is preprocessed.

[0045] In one embodiment of the present application, the underground monitoring camera is first started through a software or hardware interface to ensure that the camera is working properly and in a monitoring state. The camera starts recording multiple scene videos of the coal mine, and the video data is captured in the form of a continuous frame sequence and stored in a storage device of the computer.

[0046] Using a video processing library or tool, the computer cuts the video into separate scene images frame by frame. By decoding the video file, the video data is accessed frame by frame and saved as an image file. The extracted scene image is loaded and preprocessed to improve the image quality or meet the needs of subsequent processing, including image resizing, noise removal, contrast enhancement, color correction, etc.

[0047] The entire process is controlled by a computer to record the video of the downhole monitoring camera, which is then edited into individual frame images and pre-processed to prepare for subsequent target detection or analysis tasks.

[0048] In one embodiment of the present application, preprocessing the scene image includes: obtaining a brightness value corresponding to each pixel coordinate in the scene image, and performing nonlinear enhancement on the scene image according to the brightness value.

[0049] In one embodiment of the present application, the brightness value Bi(x) corresponding to each pixel coordinate x in the scene image is obtained, and the scene image is nonlinearly enhanced according to the brightness value to generate a preprocessed scene image Bo(x) as follows:

[0050] Bo(x)=Bi(x)+A(x)·sigmoid{α·[Bi(x)-μ]}

[0051] Where x represents the pixel coordinate, Bo(x) represents the brightness value of the pixel, A(x) is a parameter map with the same size as the given image, which is used to adjust the degree of brightness enhancement; α is the control parameter, μ is the mean brightness, and the sigmoid function is used to achieve nonlinear brightness enhancement.

[0052] Furthermore, in order to maintain the spatial consistency of the enhanced image, the loss function L is calculated spa for:

[0053]

[0054] Where K is the number of local regions in the scene image, Ω(i) is the set of adjacent regions centered on region i, and Y i and Y jRespectively represent the average brightness values ​​of regions i and j in the enhanced image. This loss function is used to constrain the brightness difference between adjacent regions. When the calculated loss function is greater than or equal to the set threshold, the original scene image is reprocessed until the loss function is less than the set threshold.

[0055] The above enhancement processing can be used in the dark environment of coal mines to avoid incomplete information due to the low brightness of the collected scene images, and can significantly improve the image quality, making the targets in the image clearer and more identifiable. This helps the subsequent target detection algorithm to more accurately identify and analyze the key information in the image, improve the display of image information and detection effect,

[0056] In step S120, the preprocessed scene image is input into the trained deep learning model to perform target detection and output target information.

[0057] like Figure 2 As shown, in one embodiment of the present application, the preprocessed scene image is input into a trained deep learning model to perform target detection and output target information, including:

[0058] S210, inputting the preprocessed scene image into the trained deep learning model, and adaptively generating a prediction box according to the size of the scene image and the size distribution of the target;

[0059] S220, compare the predicted box and the real box, and calculate the loss function;

[0060] S230, judging the detection result according to the loss function, and outputting the target information.

[0061] In one embodiment of the present application, the preprocessed video frame is input into the improved YOLOv5 model. The deep learning model in this embodiment uses the CSPDarknet53 network structure for feature extraction, and introduces the attention mechanism to enhance the feature representation capability. In order to increase data diversity and improve the generalization ability of the model, a prediction box is adaptively generated according to the size of the feature map and the size distribution of the target for subsequent target detection.

[0062] In this embodiment, an improved YOLOv5 model is used for target detection, including bounding box regression, category confidence calculation and category prediction. In order to more accurately measure the positioning error and classification error, a comparison is made between the predicted box and the true box, and the loss function is calculated as:

[0063]

[0064] Among them, IoU is the intersection over union ratio, ρ is the Euclidean distance between the center points of the predicted box and the real box, c is the diagonal length of the minimum circumscribed rectangle of the predicted box and the real box, α is the weight coefficient, and f is the shape similarity measure.

[0065] In the above process, the quality of the detection results is judged according to the value of the loss function, and the target information (such as the location and category of the target) is output in real time, providing strong support for subsequent decision-making and analysis. The output target information can be used as input data for subsequent processing tasks (such as target tracking, behavior analysis, etc.), further mining and utilizing the information value in the image, and realizing accurate detection and analysis of targets in coal mine underground scene images.

[0066] In step S130, the detected target is associated and matched according to the target information by a data association method to determine the motion trajectory of the target.

[0067] In one embodiment of the present application, according to the target information, the detected target is associated and matched by a data association method to determine the motion trajectory of the target, including:

[0068] According to the target information, matching the targets of the previous and next frames by a data association method to determine an associated frame;

[0069] Acquire motion information and appearance information of the target in the associated frame;

[0070] The similarity is determined according to the motion information and the appearance information, and the motion trajectory of the target is determined according to the similarity.

[0071] In one embodiment of the present application, a target detection algorithm is used to perform target detection on the front and back frames of the video to obtain information such as the position (such as the prediction box), category, and confidence of the target. The target detection algorithm can accurately identify the target in the video frame, providing a reliable basis for subsequent data association. According to the characteristics of the target such as the position, size, and category, the selected data association method is used to match the targets of the front and back frames. The matching process considers the motion consistency and possible appearance changes between the targets to ensure the accuracy of the match. Through the matching results, it is determined which frames have targets belonging to the same moving entity, that is, the associated frames are determined. The determination of the associated frames provides a basis for the subsequent acquisition of motion information and appearance information.

[0072] For the target in the associated frame, calculate its motion information, such as position change (displacement), velocity, acceleration, etc. This information can reflect the motion state and motion trend of the target and provide key data for subsequent motion trajectory generation. Kalman filtering and other methods can be used to smooth and predict motion information to improve the accuracy and reliability of the information.

[0073] The appearance features of the target, such as color histogram, are extracted to reflect the appearance attributes of the target and are used to measure the similarity of the target between different frames. The extraction of appearance features can be performed using a deep learning model to improve the expressiveness and robustness of the features.

[0074] Combine motion information and appearance information to calculate the similarity of the target between different frames. Similarity measurement can be based on distance measurement (such as Euclidean distance, Mahalanobis distance) or similarity measurement (such as cosine similarity). The calculation of similarity can reflect the degree of association between targets in different frames, providing an important basis for subsequent motion trajectory generation.

[0075] According to the similarity calculation results, the targets in the associated frames are connected to form the target's motion trajectory. The trajectory contains the target's identity information (such as ID), location information (such as coordinate sequence), time information, etc. The generation of the trajectory can intuitively display the target's motion path and motion state, providing strong support for subsequent analysis and application.

[0076] Smooth and optimize the generated trajectory to remove noise and jitter. The trajectory can be optimized using methods such as smooth curve fitting to improve the accuracy and smoothness of the trajectory. Manage and store the trajectory for subsequent analysis and application. A database or file system can be used to store trajectory data and provide query and retrieval functions.

[0077] This embodiment can significantly improve the accuracy of target tracking by combining motion information and appearance information for target matching and trajectory generation. Even when the target is occluded or deformed, it can still be matched and tracked through appearance features. It can handle target tracking problems in complex scenes, such as occlusion, intersection, and rapid movement between targets, enhance the robustness and adaptability of the algorithm, and enable it to perform well in various practical application scenarios. This method can significantly improve the accuracy and robustness of target tracking by combining motion information and appearance information for target matching and trajectory generation.

[0078] In step S140, the target information and the motion trajectory are output.

[0079] In one embodiment of the present application, outputting the target information and the motion trajectory includes:

[0080] Get the scene map where the target is located;

[0081] The motion trajectory is generated in the scene map according to the target information.

[0082] In one embodiment of the present application, obtaining a scene map where the target is located may be obtaining a map of an underground coal mine. In addition, for simple path planning, a vector map may be used; for three-dimensional scenes that need to display complex terrain and buildings, a three-dimensional model or point cloud data may need to be used.

[0083] The motion trajectory is generated in the scene map according to the target information, and the position coordinates in the target information are converted from the image coordinate system, the sensor coordinate system or other coordinate systems to the map coordinate system. According to the time series in the target information, the position points of the target are sequentially drawn in the scene map to form a motion trajectory.

[0084] Optionally, methods such as straight-line interpolation and curve fitting can be selected to smooth the trajectory.

[0085] The generated trajectory is optimized to reduce noise and jitter. This can be achieved through filtering. The optimized trajectory can be displayed through a graphical interface and interactive functions can be added for users to view and analyze. Through the above process, we can combine the target information with the scene map, generate the target's motion trajectory, and display and analyze it on the graphical interface.

[0086] In one embodiment of the present application, the method further includes: uploading the motion trajectory to a cloud platform.

[0087] In one embodiment of the present application, the acquired data is converted into a format accepted by the cloud platform. The formatted data is packaged into a request body, usually in JSON, XML or form data format.

[0088] The computer establishes a connection with the cloud platform through the network protocol. This includes parsing the URL, setting the request header, selecting the transport layer protocol, etc. The packaged data is sent to the cloud platform as the request body. After the cloud platform processes the request, the computer receives and parses the response.

[0089] In the technical solution of the present application, scene images of underground coal mines are collected, and the scene images are preprocessed; the preprocessed scene images are input into a trained deep learning model to perform target detection and output target information; according to the target information, the detected targets are associated and matched by a data association method to determine the motion trajectory of the target; the target information and the motion trajectory are output. The above process can accurately collect and preprocess underground coal mine images, use deep learning models to efficiently detect targets, accurately match targets and determine motion trajectories through data association, and finally output detailed target information and motion trajectories, providing strong support for safe production in coal mines.

[0090] The following introduces an embodiment of the device of the present application, which can be used to execute the coal mine underground target detection method based on deep learning in the above-mentioned embodiment of the present application. It can be understood that the device can be a computer program (including program code) running in a computer device, for example, the device is an application software; the device can be used to execute the corresponding steps in the method provided in the embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the coal mine underground target detection method based on deep learning mentioned above in the present application.

[0091] Figure 3 A block diagram of a coal mine underground target detection device based on deep learning according to an embodiment of the present application is shown.

[0092] Reference Figure 3 As shown, according to an embodiment of the present application, a coal mine underground target detection device based on deep learning includes:

[0093] The acquisition unit 310 is used to acquire scene images in the coal mine and pre-process the scene images;

[0094] The detection unit 320 is used to input the preprocessed scene image into the trained deep learning model, perform target detection, and output target information;

[0095] The trajectory unit 330 is used to perform association matching on the detected target through a data association method according to the target information to determine the motion trajectory of the target;

[0096] The output unit 340 is used to output the target information and the motion trajectory.

[0097] In the present application, based on the aforementioned scheme, the scene images underground in a coal mine are collected and the scene images are preprocessed, including: shooting videos of multiple scenes through underground monitoring cameras; editing the videos into scene images frame by frame; and preprocessing the scene images.

[0098] In the present application, based on the above-mentioned solution, the preprocessing of the scene image includes: obtaining the brightness value corresponding to each pixel coordinate in the scene image, and performing nonlinear enhancement on the scene image according to the brightness value.

[0099] In the present application, based on the aforementioned scheme, the preprocessed scene image is input into a trained deep learning model to perform target detection and output target information, including: inputting the preprocessed scene image into a trained deep learning model, adaptively generating a prediction frame according to the size of the scene image and the size distribution of the target; comparing the prediction frame with the real frame and calculating the loss function; judging the detection result according to the loss function and outputting the target information.

[0100] In the present application, based on the aforementioned scheme, the detected target is associated and matched according to the target information through a data association method to determine the motion trajectory of the target, including: according to the target information, the targets of the previous and next frames are matched through a data association method to determine the associated frame; the motion information and appearance information of the target in the associated frame are obtained; the similarity is determined according to the motion information and appearance information, and the motion trajectory of the target is determined according to the similarity.

[0101] In the present application, based on the aforementioned solution, the outputting of the target information and the motion trajectory includes: acquiring a scene map where the target is located; and generating the motion trajectory in the scene map according to the target information.

[0102] In the present application, based on the above-mentioned solution, it also includes: uploading the motion trajectory to a cloud platform.

[0103] In the technical solution of the present application, scene images of underground coal mines are collected, and the scene images are preprocessed; the preprocessed scene images are input into a trained deep learning model to perform target detection and output target information; according to the target information, the detected targets are associated and matched by a data association method to determine the motion trajectory of the target; the target information and the motion trajectory are output. The above process can accurately collect and preprocess underground coal mine images, use deep learning models to efficiently detect targets, accurately match targets and determine motion trajectories through data association, and finally output detailed target information and motion trajectories, providing strong support for safe production in coal mines.

[0104] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0105] It should be noted that the computer system 400 of the electronic device shown in the figure is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0106] The computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 to the random access memory (RAM) 403, such as executing the method described in the above embodiment. In the RAM 403, various programs and data required for system operation are also stored. The CPU 401, the ROM 402 and the RAM 403 are connected to each other through the bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0107] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed so that a computer program read therefrom is installed into the storage section 408 as needed.

[0108] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 409, and / or installed from a removable medium 411. When the computer program is executed by a central processing unit (CPU) 401, various functions defined in the system of the present application are executed.

[0109] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0110] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0111] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0112] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0113] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0114] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0115] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0116] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0117] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A coal mine underground target detection method based on deep learning, characterized in that: include: Collecting scene images underground in a coal mine and preprocessing the scene images; Input the preprocessed scene image into the trained deep learning model to perform target detection and output target information; According to the target information, the detected target is associated and matched by a data association method to determine the motion trajectory of the target; The target information and the motion trajectory are output.

2. The method according to claim 1, characterized in that Collecting scene images in a coal mine and preprocessing the scene images include: Shoot videos of multiple scenes through underground monitoring cameras; Editing the video into scene images frame by frame; The scene image is preprocessed.

3. The method according to claim 2, characterized in that Preprocessing the scene image includes: The brightness value corresponding to each pixel coordinate in the scene image is obtained, and the scene image is nonlinearly enhanced according to the brightness value.

4. The method according to claim 1, characterized in that: The preprocessed scene image is input into the trained deep learning model to perform target detection and output target information, including: The preprocessed scene image is input into the trained deep learning model, and the prediction box is adaptively generated according to the size of the scene image and the size distribution of the target; Compare the predicted box and the true box and calculate the loss function; The detection result is judged according to the loss function, and the target information is output.

5. The method according to claim 1, characterized in that According to the target information, the detected target is associated and matched by a data association method to determine the motion trajectory of the target, including: According to the target information, matching the targets of the previous and next frames by a data association method to determine an associated frame; Acquire motion information and appearance information of the target in the associated frame; The similarity is determined according to the motion information and the appearance information, and the motion trajectory of the target is determined according to the similarity.

6. The method according to claim 1, characterized in that Outputting the target information and the motion trajectory includes: Get the scene map where the target is located; The motion trajectory is generated in the scene map according to the target information.

7. The method according to claim 1, characterized in that Also includes: The motion trajectory is uploaded to the cloud platform.

8. A coal mine underground target detection device based on deep learning, characterized in that: include: A collection unit, used for collecting scene images in the coal mine and preprocessing the scene images; The detection unit is used to input the preprocessed scene image into the trained deep learning model, perform target detection, and output target information; A trajectory unit, used to perform association matching on the detected target through a data association method according to the target information, so as to determine the motion trajectory of the target; An output unit is used to output the target information and the motion trajectory.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the deep learning-based underground coal mine target detection method as described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the coal mine underground target detection method based on deep learning as described in any one of claims 1 to 7.

Images