A method and system for intelligent perception of thermal images of coal gangue transport situation

Abstract

The present invention discloses a method and system for intelligent perception of thermal images based on coal gangue migration situation, belonging to the technical field of intelligent mining of coal mines. The method solves the problems of poor image contrast and low clarity by performing data enhancement processing on the image; trains the coal gangue detection model through data in a laboratory simulation environment, verifies the effect of the model through a laboratory simulation environment, and puts the verified model into practical use to improve the accuracy of the model; uses the verified model to detect the target, obtains the detection frame corresponding to the target, obtains the average temperature of coal and gangue according to the detection frame, and then obtains the temperature data points of coal and gangue according to the average temperature, thereby calculating the mixed gangue rate; due to the high accuracy of the coal gangue target detection model, the detection frame corresponding to the target can be accurately obtained, and the average temperature of the target can be accurately obtained according to the detection frame; at the same time, the mixed gangue rate is accurately calculated by counting the number of temperature data points.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent mining and sorting of coal mines, and specifically relates to a method and system for intelligent perception of thermal images based on coal gangue migration status. Background Art

[0002] Automatic identification of gangue is a key core technology for realizing intelligence. At present, the main problems of gangue identification are low identification accuracy and efficiency. Because gangue is black-gray solid waste mixed in coal, it is difficult to distinguish the two substances with the naked eye. However, there are differences between the infrared thermal images of coal and gangue, so the intelligent identification technology of gangue based on infrared images has attracted much attention. This method takes advantage of the infrared thermal imager's ability to accurately read target temperature, detect at ultra-long distances, and identify hidden targets under harsh conditions.

[0003] This technology generally obtains visible light images and infrared images of coal gangue first; after preprocessing the visible light images and infrared images of coal gangue, the mixed gangue rate of coal gangue is calculated. The mixed gangue rate is calculated by the area corresponding to the coal on the infrared image and the area of ​​the coal gangue on the visible light image. However, in this method of calculating the mixed gangue rate, the area of ​​coal is obtained by infrared, and the total area of ​​coal gangue is obtained by visible light image. The two data sources are different, and there are differences in the performance of the area of ​​coal on the two images. The calculated mixed gangue rate result has errors, and making decisions based on the biased mixed gangue rate will have more adverse effects.

[0004] In summary, the existing coal and gangue identification methods have the problem of large errors in the calculation of mixed gangue ratio. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method and system for intelligent perception of coal gangue migration status based on thermal sensitive imaging in view of the deficiencies in the above-mentioned prior art, which has a novel and reasonable design, precise calculation structure, strong practicality and is easy to promote and use.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A method for intelligent perception of coal gangue transport situation based on thermal imaging, the method comprising the following steps:

[0008] Perform data enhancement processing on the frame images of the infrared thermal image data of coal gangue; construct a coal gangue detection model based on the deep learning model, and use the data after data enhancement to train the coal gangue detection model; input the infrared thermal image data of coal gangue collected in the laboratory simulation environment into the trained coal gangue detection model, and output the corresponding coal and gangue detection results. When the detection results of coal and gangue are consistent with the known coordinate positions of coal and gangue, the verification of the coal gangue target detection model is completed;

[0009] The infrared thermal image data of coal gangue collected under actual working conditions is input into the verified coal gangue target detection model, the corresponding coal and gangue detection results are output, and the detection frames corresponding to coal and gangue are extracted from them;

[0010] According to the temperature data corresponding to the detection frame on the frame image, the average temperature in the coal and gangue detection frames is calculated respectively, and it is used as the average temperature of coal and the average temperature of gangue. According to the average temperature of coal and the average temperature of gangue, the number of temperature data points of coal and the number of temperature data points of gangue on the frame image are determined, and the corresponding mixed gangue rate is calculated. The calculation formula of the mixed gangue rate is as follows:

[0011]

[0012] Among them, R infrared is the mixed waste ratio, S r is the number of temperature data points of gangue, S c is the number of coal temperature data points.

[0013] Furthermore, the process of determining the number of temperature data points of coal and the number of temperature data points of gangue on the frame image is as follows: obtaining the temperature data of all points on the frame image, counting the number of temperature data points whose temperatures are within the temperature range of coal, and using them as the number of temperature data points of coal; counting the number of temperature data points whose temperatures are within the temperature range of gangue, and using them as the number of temperature data points of gangue; the temperature range of the coal is ±T℃ of the average temperature of the coal, and the temperature range of the gangue is ±T℃ of the average temperature of the gangue, T∈[2,5].

[0014] Furthermore, the deep learning model is constructed using the DeepSort model. The training process of the above-mentioned coal gangue target detection model is as follows:

[0015] 1) The gangue infrared thermal image data is used as the input of the gangue target detection model. The YOLOv7 detector performs target detection on the frame image of the input image and creates corresponding trajectories based on the target detection results;

[0016] 2) The Kalman filter predicts the target information of the current frame based on the target information of the previous frame. When the same target is successfully matched in three consecutive frames, it is marked as a confirmed state, otherwise it is marked as a non-confirmed state;

[0017] 3) When the trajectory is judged to be in the confirmed state, the gangue trajectory and the new detection are cascade matched, and the trackers are matched in sequence according to the difference in parameter size. The matching priority of the tracker with smaller parameters is higher, and the matching priority of the tracker with larger parameters is lower. The tracker that has not been matched for several consecutive frames will gradually reduce its priority; when the trajectory is judged to be in the unconfirmed state, the gangue trajectory and the detection are matched by IOU, and the matching output has three results: matching completed, detection not matched, and trajectory not matched;

[0018] 4) Repeat steps 2) and 3) until the image is finished.

[0019] Furthermore, the above DeepSort model includes three parts: a detector, a feature extraction network and a tracker; the detector is used to detect the target object in the frame image, the feature extraction network is used to extract the features of the target, and the tracker is used to determine the position data and target type of the target on the frame image based on the features, and the feature extraction network adopts the Resnet18 network.

[0020] Furthermore, the Pconv-ELAN network is used in the convolution layer of the YOLOv7 detector backbone network. In the module, the convolution kernel of the convolution layer is 3×3Pconv. The SA module is inserted after the convolution layer of the YOLOv7 detector backbone network. The SA module groups the feature maps output by the convolution layer. The feature information of the grouped sub-features is extracted separately in the two dimensions of channel attention and spatial attention and then merged. Finally, the Channel Shuffle operation is used to rearrange the sub-features and output the final feature map after the fusion of different sub-features.

[0021] Furthermore, infrared thermal imaging data of coal gangue are acquired by a camera in a simulated underground working condition, and the simulated underground working condition includes a simulated dust working condition, a simulated humidity working condition and a simulated coal gangue transport state working condition.

[0022] Furthermore, the data enhancement processing includes contrast enhancement processing and dark channel dehazing processing.

[0023] Furthermore, the contrast enhancement process is as follows: each frame of the image data is segmented according to the n×m specification to obtain n×m sub-regions, and the grayscale value h(x) of the grayscale histogram of the sub-region is calculated. When the grayscale value h(x) ≥ the upper threshold, some pixels are cropped according to the cropping amplitude, and the cropped pixels are averaged and added to each grayscale level of the grayscale histogram.

[0024] Furthermore, the process of dark channel dehazing is as follows: obtaining a frame of image in the current coal gangue infrared thermal imaging data, estimating the atmospheric light intensity of the image, estimating the transmittance based on the image and the atmospheric light intensity, and performing image dehazing and restoration on the image based on the atmospheric light intensity and the transmittance to obtain the image after dark channel dehazing processing.

[0025] The present invention discloses a control system based on thermal imaging intelligent perception of coal gangue transport status. The system comprises a processor and a controller. The processor obtains a corresponding gangue mixing rate according to coal gangue infrared thermal imaging data using the above-mentioned method for intelligent perception of thermal imaging of coal gangue transport status, and sends the gangue mixing rate to the controller; the controller performs closing control on a coal discharge port according to the gangue mixing rate.

[0026] Compared with the prior art, the present invention has the following advantages:

[0027] Based on the intelligent perception method of thermal images of coal gangue migration situation, data enhancement processing is performed on the image to solve the problems of poor image contrast and low clarity; the coal gangue detection model is trained with data in the laboratory simulation environment to reduce the difficulty of data acquisition. A large amount of data can be obtained at a low cost through the laboratory simulation environment, and the effect of the model is actually tested in the laboratory simulation environment. The tested model is put into practical use to improve the accuracy of the model; the verified model is used for target detection, and the detection frame corresponding to the target is obtained from it. The average temperature of coal and gangue is obtained according to the detection frame, and the temperature data points of coal and gangue are obtained according to the average temperature, so as to calculate the mixed gangue rate; due to the high accuracy of the coal gangue target detection model, the detection frame corresponding to the target can be accurately obtained, and the average temperature of the target can be obtained according to the detection frame; at the same time, the mixed gangue rate is accurately calculated by counting the number of temperature data points. Mixed gangue rate Mixed gangue rate

[0028] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a structural schematic diagram of an embodiment of the thermal-sensitive imaging intelligent perception system for coal gangue transport situation of the present invention;

[0030] Figure 2 This is a structural diagram of an improved YOLOv7 model according to an embodiment of the method for intelligently sensing thermal images of coal gangue transport situation of the present invention;

[0031] Figure 3 It is a schematic diagram of collecting images under simulated working conditions based on the thermal-sensitive image intelligent perception method of coal gangue transport situation of the present invention;

[0032] Figure 4 It is a structural diagram of a coal gangue target detection model according to an embodiment of the method for intelligently sensing coal gangue transport situation based on thermal-sensitive imaging of the present invention;

[0033] Figure 5 It is a schematic diagram of the SA module structure of an embodiment of the method for intelligent perception of coal gangue transport situation based on thermal-sensitive imaging of the present invention;

[0034] Figure 6 It is a schematic diagram of a 3×3 CBS structure used in an ELAN module of an embodiment of a method for intelligently sensing coal gangue transport situation based on thermal images of the present invention;

[0035] Figure 7 It is a schematic diagram of a 3×3Pconv structure used in an ELAN module of an embodiment of a method for intelligently sensing thermal images of coal gangue transport situation according to the present invention;

[0036] Figure 8 It is a schematic diagram of the Resnet18 network structure of an embodiment of the method for intelligent perception of thermal images of coal gangue transport situation of the present invention;

[0037] Fig. 9 It is a schematic diagram of a process for determining the number of data points of coal and gangue temperature based on the embodiment of the method for intelligent perception of coal gangue transport situation based on thermal sensitive images of the present invention;

[0038] Description of Figure Numbers:

[0039] 1. Camera; 2. Dust generation device; 3. Humidity control device; 4. Belt conveyor;

[0040] 5. Coal gangue. DETAILED DESCRIPTION

[0041] Embodiment of the intelligent perception method based on thermal image of coal gangue transport situation:

[0042] like Figure 2 As shown, based on the thermal image intelligent perception method of coal gangue migration situation, the method includes the following steps:

[0043] Data enhancement processing is performed on the frame images of coal gangue infrared thermal imaging data. Figure 3 As shown, the infrared thermal image data of coal gangue is obtained by camera 1 in the simulated underground working condition, and the simulated underground working condition includes simulated dust working condition, simulated humidity working condition and simulated coal gangue transport state working condition. The camera 1 adopts a high frame rate infrared camera, and transports coal gangue 5 through a belt conveyor 4. A dust manufacturing device 2 is set above the belt conveyor 4 to simulate the actual dust working condition, and a humidity control device 3 is set above the belt conveyor 4 to simulate the actual humidity working condition. An infrared camera 1 is set above the transmission.

[0044] Among them, data enhancement processing includes contrast enhancement processing and dark channel dehazing processing.

[0045] The process of contrast enhancement is as follows: each frame of the image data is segmented according to the n×m specification to obtain n×m sub-regions, and the gray value h(x) of the gray histogram of the sub-region is calculated. When the gray value h(x) ≥ the upper threshold, some pixels are cropped according to the cropping amplitude, and the cropped pixels are averaged and added to each gray level of the gray histogram. The cropping amplitude calculation formula is as follows:

[0046]

[0047] Assuming that N pixels are cropped, each gray level will be assigned to A pixels, then the following relationship is satisfied:

[0048]

[0049] Among them, L is the cropping amplitude, N and M represent the number of pixels in the horizontal and vertical directions respectively, and T clip is the cropping coefficient, k represents the grayscale level of the sub-region, x is the sub-region number, and max[h(x)-L] is the maximum value of the difference between h(x)-L.

[0050] The dark channel defogging process is as follows: obtain a frame of the current coal gangue infrared thermal image data, estimate the atmospheric light intensity of the image, estimate the transmittance based on the image and the atmospheric light intensity, perform image defogging and restoration on the image based on the atmospheric light intensity and transmittance, and obtain the image after dark channel defogging. Specifically, the dark channel prior defogging principle is based on the prior assumption that most of the foggy images we obtain have similar brightness values ​​in the dark channel.

[0051] The dark channel refers to the part of the pixels in the non-sky area of ​​the foggy image that have a lower pixel value in at least one of the R, G, and B color channels, or even tend to 0. Its mathematical definition is:

[0052]

[0053] Among them, Ω(e) represents the local area centered at the coordinate e in the foggy image, J c (y) represents the gray value of the fog-free image in the c channel, y is the value of any pixel in the region, and c belongs to one of the three color channels R, G, and B. The smaller the color value of the pixel in the foggy image, the closer it is to the fog-free original color. From this, the transmittance can be estimated, and the relationship between the image transmittance and the dark channel can be obtained as follows:

[0054]

[0055] Where, t(e) is the transmittance, I c (y) represents the gray value in channel c of the input foggy image, A c represents the atmospheric light value, and ω is the defogging weight coefficient. Since this paper mainly considers the impact of coal mine dust, the concentration of coal dust is much higher than that of ordinary atmospheric haze, so the value range of ω is set to 0.75-0.90 to meet the needs of experiments and actual working conditions. When the transmittance is too small, it will cause the overall white field of the image to be excessive, so the lower limit of the transmittance is set to t0, and its formula is expressed as:

[0056]

[0057] Among them, J is the fog-free image, I is the foggy image, e is the pixel space coordinate, and A is a constant.

[0058] A coal gangue detection model is constructed based on the deep learning model, and the coal gangue detection model is trained using the data after data enhancement. The infrared thermal image data of coal gangue collected in the laboratory simulation environment is input into the trained coal gangue detection model, and the corresponding coal and gangue detection results are output. When the detection results of coal and gangue are consistent with the known coordinate positions of coal and gangue, the verification of the coal gangue target detection model is completed.

[0059] Specifically, Figure 4 As shown in the figure, the deep learning model is constructed using the DeepSort model, and the training process of the coal gangue target detection model is:

[0060] 1) The gangue infrared thermal image data is used as the input of the gangue target detection model. The YOLOv7 detector performs target detection on the frame image of the input image and creates corresponding trajectories based on the target detection results;

[0061] 2) The Kalman filter predicts the target information of the current frame based on the target information of the previous frame. When the same target is successfully matched in three consecutive frames, it is marked as a confirmed state, otherwise it is marked as a non-confirmed state;

[0062] 3) When the trajectory is judged to be in the confirmed state, the gangue trajectory and the new detection are cascade matched, and the trackers are matched in sequence according to the difference in parameter size. The matching priority of the tracker with smaller parameters is higher, and the matching priority of the tracker with larger parameters is lower. The tracker that has not been matched for several consecutive frames will gradually reduce its priority; when the trajectory is judged to be in the unconfirmed state, the gangue trajectory and the detection are matched by IOU, and the matching output has three results: matching completed, detection not matched, and trajectory not matched;

[0063] 4) Repeat steps 2) and 3) until the image is finished.

[0064] like Figure 5 As shown in the figure, the SA module is inserted after the convolution layer of the YOLOv7 detector backbone network. The SA module groups the feature maps output by the convolution layer, extracts feature information from the grouped sub-features in the two dimensions of channel attention and spatial attention, and then merges them. Finally, the Channel Shuffle operation is used to rearrange the sub-features and output the final feature map after the fusion of different sub-features. This effectively solves the problem of small target detection in the coal gangue flow during the transportation process and improves the detection accuracy of the model.

[0065] like Figure 6 , 7As shown in the figure, the Pconv-ELAN network is used in the convolution layer of the YOLOv7 detector backbone network. In the module, the conventional 3×3CBS is replaced by 3×3Pconv. The partial convolution Pconv only performs convolution operations in the area containing valid information, thereby reducing the calculation of invalid redundant information, and dynamically adjusts the weight of the convolution kernel during the convolution operation, so that the network can capture key features more flexibly and effectively, improving the reasoning speed and feature extraction capabilities. Therefore, using Pconv in the ELAN module can effectively reduce redundant calculations, improve the ability to extract key features, and effectively improve the training ability of the coal gangue thermal sensitivity perception model.

[0066] like Figure 8 As shown in the figure, the above DeepSort model includes three parts: detector, feature extraction network and tracker; the detector is used to detect the target object in the frame image, the feature extraction network is used to extract the features of the target, and the tracker is used to determine the position data and target type of the target on the frame image according to the features, that is, the tracker determines the detection results of coal and gangue according to the features, and the feature extraction network adopts Resnet18 network. The original feature extraction network in DeepSort has poor feature extraction effect on the target, so the original feature extraction network is replaced by Resnet18 network. The residual block in Resnet18 network overcomes the gradient problem caused by too deep network training. The scales of the coal gangue targets that need to be detected in the experiment of thermal image acquisition of coal gangue transportation and the actual underground production process are quite different, so the image of (3, 256, 256) is input and the feature vector of 256 dimensions is output.

[0067] like Fig. 9 As shown in the figure, the infrared thermal image data of coal gangue collected under actual working conditions is input into the verified coal gangue target detection model, and the corresponding detection results of coal and gangue are output, and the detection frames corresponding to coal and gangue are extracted from them; according to the temperature data corresponding to the detection frames on the frame image, the average temperatures in the coal and gangue detection frames are calculated respectively, and they are used as the average temperature of coal and the average temperature of gangue. According to the average temperature of coal and the average temperature of gangue, the number of temperature data points of coal and the number of temperature data points of gangue on the frame image are determined, and the corresponding mixed gangue rate is calculated. The calculation formula of the mixed gangue rate is as follows:

[0068]

[0069] Among them, R infrared is the mixed waste ratio, S r is the number of temperature data points of gangue, S c is the number of coal temperature data points.

[0070] The process of determining the number of temperature data points of coal and gangue on the frame image is as follows: obtain the temperature data of all points on the frame image, count the number of temperature data points with temperature within the temperature range of coal, and use it as the temperature data points of coal; count the number of temperature data points with temperature within the temperature range of gangue, and use it as the temperature data points of gangue; the temperature range of coal is ±T℃ of the average temperature of coal, and the temperature range of gangue is ±T℃ of the average temperature of gangue, T∈[2,5]. The process of counting the number of temperature data points is to count the temperature matrix in the csv file of the image.

[0071] Embodiment of thermal image intelligent perception control system based on coal gangue transport situation:

[0072] like Figure 1 As shown, the control system based on thermal imaging intelligent perception of coal gangue transport status includes a processor and a controller. The processor obtains the corresponding gangue mixing rate based on the infrared thermal imaging data of coal gangue using the above-mentioned embodiment of the method based on thermal imaging intelligent perception of coal gangue transport status, and sends the gangue mixing rate to the controller. The controller controls the closure of the coal discharge port according to the gangue mixing rate. Specifically, the controller makes real-time decisions based on the gangue mixing rate and other relevant data to determine whether the coal discharge port needs to be closed. This embodiment is implemented with reference to the above-mentioned embodiment of the method based on thermal imaging intelligent perception of coal gangue transport status, and will not be repeated here.

[0073] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A method for intelligent perception of coal gangue transport situation based on thermal imaging, characterized by: The method comprises the following steps: Perform data enhancement processing on the frame images of the infrared thermal image data of coal gangue; construct a coal gangue detection model based on the deep learning model, and use the data after data enhancement to train the coal gangue detection model; input the infrared thermal image data of coal gangue collected in the laboratory simulation environment into the trained coal gangue detection model, and output the corresponding coal and gangue detection results. When the detection results of coal and gangue are consistent with the known coordinate positions of coal and gangue, the verification of the coal gangue target detection model is completed; The infrared thermal image data of coal gangue collected under actual working conditions is input into the verified coal gangue target detection model, the corresponding coal and gangue detection results are output, and the detection frames corresponding to coal and gangue are extracted from them; According to the temperature data corresponding to the detection frame on the frame image, the average temperature in the coal and gangue detection frames is calculated respectively, and it is used as the average temperature of coal and the average temperature of gangue. According to the average temperature of coal and the average temperature of gangue, the number of temperature data points of coal and the number of temperature data points of gangue on the frame image are determined, and the corresponding mixed gangue rate is calculated. The calculation formula of the mixed gangue rate is as follows: Among them, R infrared is the mixed waste ratio, S r is the number of temperature data points of gangue, S c is the number of data points of coal temperature; The deep learning model is constructed using the DeepSort model, and the training process of the coal gangue target detection model is as follows: 1) The gangue infrared thermal image data is used as the input of the gangue target detection model. The YOLOv7 detector performs target detection on the frame image of the input image and creates corresponding trajectories based on the target detection results; 2) The Kalman filter predicts the target information of the current frame based on the target information of the previous frame. When the same target is successfully matched in three consecutive frames, it is marked as a confirmed state, otherwise it is marked as a non-confirmed state; 3) When the trajectory is judged to be in the confirmed state, the gangue trajectory and the new detection are cascade matched, and the trackers are matched in sequence according to the difference in parameter size. The matching priority of the tracker with smaller parameters is higher, and the matching priority of the tracker with larger parameters is lower. The tracker that has not been matched for several consecutive frames will gradually reduce its priority; when the trajectory is judged to be in the unconfirmed state, the gangue trajectory and the detection are matched by IOU, and the matching output has three results: matching completed, detection not matched, and trajectory not matched; 4) Repeat steps 2) and 3) until the image is finished.

2. The method for intelligently sensing coal gangue transport situation based on thermal imaging according to claim 1, characterized in that: The process of determining the number of temperature data points of coal and the number of temperature data points of gangue on the frame image is as follows: obtaining the temperature data of all points on the frame image, counting the number of temperature data points whose temperature is within the temperature range of coal, and using them as the number of temperature data points of coal; counting the number of temperature data points whose temperature is within the temperature range of gangue, and using them as the number of temperature data points of gangue; the temperature range of coal is ±T℃ of the average temperature of coal, and the temperature range of gangue is ±T℃ of the average temperature of gangue, T∈[2,5].

3. The method for intelligently sensing coal gangue transport situation based on thermal imaging according to claim 1, characterized in that: The DeepSort model includes three parts: a detector, a feature extraction network and a tracker; the detector is used to detect the target object in the frame image, the feature extraction network is used to extract the features of the target, and the tracker is used to determine the position data and target type of the target on the frame image based on the features. The feature extraction network adopts the Resnet18 network.

4. The method for intelligently sensing coal gangue transport situation based on thermal imaging according to claim 1, characterized in that: The Pconv-ELAN network is used in the convolution layer of the YOLOv7 detector backbone network. In the module, the convolution kernel of this convolution layer uses 3×3Pconv. The SA module is inserted after the convolution layer of the YOLOv7 detector backbone network. The SA module groups the feature maps output by the convolution layer, extracts feature information from the grouped sub-features in two dimensions: channel attention and spatial attention, and then merges them. Finally, the ChannelShuffle operation is used to rearrange the sub-features and output the final feature map after the fusion of different sub-features.

5. The method for intelligently sensing coal gangue transport situation based on thermal imaging according to claim 1, characterized in that: The gangue infrared thermal image data is acquired by a camera in a simulated underground working condition, and the simulated underground working condition includes a simulated dust working condition, a simulated humidity working condition and a simulated gangue transport state working condition.

6. The method for intelligently sensing coal gangue transport situation based on thermal imaging according to claim 1, characterized in that: The data enhancement processing includes contrast enhancement processing and dark channel defogging processing.

7. The method for intelligently sensing coal gangue transport situation based on thermal imaging according to claim 6, characterized in that: The process of contrast enhancement is as follows: each frame of the image data is segmented according to the n×m specification to obtain n×m sub-regions, and the grayscale value h(x) of the grayscale histogram of the sub-region is calculated. When the grayscale value h(x) ≥ the upper threshold, some pixels are cropped according to the cropping amplitude, and the cropped pixels are averaged and added to each grayscale level of the grayscale histogram.

8. The method for intelligently sensing coal gangue transport situation based on thermal imaging according to claim 6 is characterized in that: The dark channel defogging process is as follows: obtaining a frame of image in the current coal gangue infrared thermal image data, estimating the atmospheric light intensity of the image, estimating the transmittance based on the image and the atmospheric light intensity, and performing image defogging and restoration on the image based on the atmospheric light intensity and the transmittance to obtain an image after dark channel defogging processing.

9. An intelligent perception control system based on thermal imaging of coal gangue transport situation, characterized in that: The system includes a processor and a controller. The processor obtains a corresponding gangue mixing ratio according to the gangue infrared thermal image data using the intelligent perception method based on thermal image of coal gangue migration situation described in any one of claims 1 to 8, and sends the gangue mixing ratio to the controller; the controller controls the closing of the coal discharge port according to the gangue mixing ratio.

Images