Key target detection method for coal mine driving face

Abstract

The invention relates to a coal mine driving face key target detection method, and belongs to the technical field of target detection. Comprising the steps of performing foreground and background segmentation on an original image to obtain a foreground image and a background image; performing image enhancement on the background image through a CLAHE algorithm; denoising processing is carried out on the foreground image, image enhancement is carried out on the denoised foreground image through an improved CLAHE algorithm, and the improved CLAHE algorithm adaptively adjusts the value of an enhancement parameter according to an image quality index; fusing the enhanced background image and the enhanced foreground image to obtain a target image; and inputting the target image into a pre-trained improved YOLOv8n model for key target detection. The image enhanced by the method is high in contrast, low in noise and clear in edge; the improved YOLOv8n model has a larger receptive field, context information can be better extracted, the detection precision of multi-target shielding can be enhanced, and the convergence speed and the regression precision are increased.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a method for detecting key targets on a coal mine excavation working face. Background Art

[0002] With the increasing complexity of coal mining, traditional mechanized and automated mining technologies can no longer meet the requirements for further improvement of mining efficiency and safety level. Therefore, automated and intelligent mining has become an inevitable development trend. Key target detection of operators and equipment based on machine vision can provide technical support and guarantee for understanding personnel behavior and troubleshooting, which is of great significance to the realization of intelligent safety in coal mines.

[0003] Under low light conditions, the tunneling face is affected by backlight, non-uniform illumination and weak light, and the video images have problems such as low contrast, high noise, information loss, and blurred details, which affect the subsequent key target detection tasks of the tunneling face. Currently, many algorithms have been studied and applied in operating scenarios such as underground conveyor belt monitoring and motor vehicle transportation in coal mines. However, because the tunneling work scene environment is more extreme, there is still little research on target detection algorithms for tunneling work scenes, and there is a lack of algorithm research that can simultaneously address problems such as high dust and fog, low illumination, and multiple targets occluding each other.

[0004] In order to improve the image quality, the image can be enhanced by the CLAHE algorithm. The CLAHE algorithm is currently mature in medical image processing and remote sensing image processing. At present, most of the use of the original CLAHE algorithm in underground excavation working faces is to directly use the original CLAHE algorithm. When the original CLAHE algorithm processes different images, its enhancement parameters are fixed values. It cannot automatically change the enhancement parameters according to the image changes to improve the enhancement effect, and it is easy to over-enhance or under-enhance. However, over-enhancement will amplify the noise in the image, making the noise more obvious, affecting the clarity and details of the image, and will introduce artifacts or stripe effects in the image, especially at the edge or uniform area of ​​the image, and will cause local contrast imbalance, making the brightness of some areas too high, while the details of other areas are lost; insufficient enhancement will lead to limited image contrast improvement, and the details and structural information in the image are not fully displayed. Therefore, it is very important to be able to adaptively select appropriate enhancement parameters to improve the image enhancement effect. In addition, the coal mine excavation working face has a narrow space, a large number of equipment and personnel, and a more serious occlusion problem, which seriously affects the accuracy of key target detection. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method for detecting key targets in a coal mine excavation working face. The technical solution of the present invention is as follows:

[0006] A method for detecting key targets in a coal mine excavation working face, comprising:

[0007] S1, after obtaining the video collected from the coal mine excavation working face, the video is processed into frames to obtain original images frame by frame;

[0008] S2, using the color-based K-means clustering algorithm to segment the original image into foreground and background, and obtain the foreground image and background image of the original image;

[0009] S3, performing image enhancement on the background image by using a CLAHE algorithm, wherein the value of an enhancement parameter of the CLAHE algorithm is a fixed value when performing image enhancement on background images of different frames;

[0010] S4, denoising the foreground image, and performing image enhancement on the denoised foreground image using an improved CLAHE algorithm, wherein the improved CLAHE algorithm adaptively adjusts the value of the enhancement parameter according to the image quality index when performing image enhancement on the denoised foreground images of different frames;

[0011] S5, fusing the enhanced background image and the enhanced foreground image to obtain a target image corresponding to the original image;

[0012] S6, input the target image into the pre-trained improved YOLOv8n model for key target detection.

[0013] Optionally, when S2 is specifically implemented, it includes:

[0014] S21, converting the original image from RGB color space to Lab color space;

[0015] S22, randomly select two points C1 and C2 as initial centroids, where the centroid is the point representing the center of two clusters;

[0016] S23, for pixel x in the original image i , calculate its distance from the two centroids respectively, and assign it to the cluster to which the nearest centroid belongs according to its distance from the two centroids;

[0017] S24, calculate the average value of all data points of each cluster and update the centroid to this average value;

[0018] S25, repeating steps S23 and S24 until the distance between the current centroid and the previous centroid is less than a preset centroid convergence standard value, and determining the current centroid as the target centroid;

[0019] S26, according to the positions and color distributions of the two target centroids, the two clusters are distinguished into a foreground cluster and a background cluster, and the pixels in the foreground cluster and the background cluster are converted back to the RGB color space to obtain the foreground image and the background image of the original image.

[0020] Optionally, S23 calculates the pixel xi and one of the centroids C j The distance d(x i ,C j ), the Euclidean distance is used, which is expressed as:

[0021]

[0022] In formula (1), x i =[x i1 , x i2 , x i3 ], x i1 、x i2 and x i3 x i The values ​​of L, a, and b in the Lab color space; C j =[C j1 , C j2 , C j3 ], indicating the j-th centroid.

[0023] Optionally, the enhancement parameters of the CLAHE algorithm include a grid size and a contrast limiting factor, and S3, when implemented specifically, includes:

[0024] S31, converting the background image from the RGB color space to the Lab color space, wherein the Lab color space of each pixel includes a brightness component and two color components;

[0025] S32, configuring fixed values ​​for the grid size and contrast limiting factor in the CLAHE algorithm, and performing contrast enhancement on the brightness component of each pixel in the background image through the configured CLAHE algorithm while keeping the color component unchanged;

[0026] S33, converting the contrast-enhanced background image back into the RGB color space to obtain an enhanced background image.

[0027] Optionally, when performing denoising on the foreground image, the step S4 includes:

[0028] S41, performing discrete wavelet transform in row and column directions to decompose the foreground image into sub-bands of different scales to obtain a low-frequency sub-band and three high-frequency sub-bands;

[0029] S42, performing threshold processing on detail coefficients of the three high frequency sub-bands to remove noise therein;

[0030] S43, performing inverse discrete wavelet transform on the three high-frequency sub-bands and the low-frequency sub-band after the threshold processing, and reconstructing the denoised foreground image.

[0031] Optionally, the image quality index includes at least one of grayscale variance, Brenner function value and gradient energy;

[0032] The grayscale variance σ of the denoised foreground image 2 Calculated by formula (2):

[0033]

[0034] In formula (2), I(i, j) represents the gray value of the denoised foreground image at position (i, j); μ is the average gray value of the denoised foreground image, and the calculation formula is: H and W are the height and width of the denoised foreground image, respectively;

[0035] The Brenner function value B of the denoised foreground image is calculated by formula (3):

[0036]

[0037] The gradient energy E of the denoised foreground image is calculated by formula (4):

[0038]

[0039] In formula (4), G x (i,j) and G y (i, j) represent the horizontal gradient value and vertical gradient value of the denoised foreground image at position (i, j), respectively.

[0040] Optionally, it is characterized in that, when the image quality index includes grayscale variance, Brenner function value and gradient energy, the image quality index of the denoised foreground image is calculated by the following steps:

[0041] S4-1, normalize the grayscale variance, Brenner function value and gradient energy;

[0042] S4-2: Multiply the normalized grayscale variance, Brenner function value and gradient energy by their corresponding weights to obtain the image quality index of the denoised foreground image.

[0043] Optionally, in a specific implementation, S5 is performed by fusing the enhanced background image and the enhanced foreground image through formula (5) to obtain a target image corresponding to the original image:

[0044] I final (i,j)=w(i,j)·I f (i,j)+(1-w(i,j))·I b (i,j)(5);

[0045] In formula (5), I final (i, j) represents the grayscale value of the target image at position (i, j), I f (i, j) represents the gray value of the enhanced foreground image at position (i, j), I b (i, j) represents the grayscale value of the enhanced background image at position (i, j); w(i, j) represents the weight at position (i, j), and its value range is between 0 and 1.

[0046] Optionally, the improved YOLOv8n model includes a backbone network, a neck network and a SEAM improved detection head connected in sequence;

[0047] The neck network includes a DWR-modified C2f module, wherein the DWR-modified C2f module replaces the Bottleneck residual module in the C2f module with the DWR module;

[0048] The SEAM-improved detection head uses the SEAM module to replace the second Conv convolution module of the detection head;

[0049] The loss function of the improved YOLOv8n model is MPDIoU, where:

[0050]

[0051] MPDIoU loss = 1 - MPDIoU (8);

[0052] In formula (6) to formula (8), B pred and B gt Represent the predicted box and the true annotation box respectively, w and h are the width and height of the image respectively, d1 and d2 are the distances between the upper left corner and lower right corner of the predicted box and the true annotation box respectively.

[0053] Optionally, a particle swarm optimization algorithm is used to determine the values ​​of enhancement parameters of the foreground images after denoising of different frames.

[0054] All the above optional technical solutions can be combined arbitrarily, and the present invention does not provide detailed descriptions of the structures after the combinations.

[0055] By means of the above scheme, the beneficial effects of the present invention are as follows:

[0056] By using the improved CLAHE algorithm to enhance the foreground images after denoising different frames, the value of the enhancement parameter is adaptively adjusted according to the image quality index, which can improve the image enhancement effect of each original image, avoid the problems caused by over-enhancement and under-enhancement, and can adaptively enhance the foreground image according to the image quality index of the foreground image in the original image. The enhanced foreground image has high contrast, low noise, and clear edges, which can be better suitable for the target detection task of the underground excavation working face. The improved YOLOv8n model has a larger receptive field, can better extract contextual information, and can enhance the detection accuracy of multi-target occlusion, while also accelerating the convergence speed and regression accuracy.

[0057] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flow chart of a method for detecting key targets in a coal mine excavation working face provided by an embodiment of the present invention.

[0059] Figure 2 It is a schematic diagram of the composition structure of the C2f module improved by DWR in an embodiment of the present invention.

[0060] Figure 3 is a structural diagram of a DWR module in an embodiment of the present invention.

[0061] Figure 4 1 is a structural diagram of the detection head improved by SEAM in an embodiment of the present invention.

[0062] Figure 5 is a structural diagram of a SEAM module in an embodiment of the present invention.

[0063] Figure 6 It is a comparison diagram of the image enhancement effect of the key target detection method for the coal mine excavation working face provided by an embodiment of the present invention.

[0064] Figure 7 3 is a comparison chart of target detection effects of different target detection models in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0066] The key target detection method for a coal mine excavation face provided by the embodiment of the present invention can be implemented by any electronic device with computing function, such as a PC, a mobile terminal or a server. Figure 1 As shown, the key target detection method for a coal mine excavation working face provided by an embodiment of the present invention includes the following steps S1 to S6:

[0067] S1, after acquiring the video collected from the coal mine excavation working face, the video is processed into frames to obtain original images frame by frame.

[0068] Specifically, the industrial cameras installed on the coal mine excavation working face will collect video in real time, and the electronic equipment can obtain the video from the industrial cameras.

[0069] S2, using the color-based K-means clustering algorithm to segment the original image into foreground and background, and obtain the foreground image and background image of the original image.

[0070] Among them, the background image mainly includes the lane, and the foreground image mainly includes pedestrians, equipment, etc. The goal of the color-based K-means clustering algorithm is to divide the pixels of the original image into two clusters (categories), namely the foreground cluster and the background cluster.

[0071] Specifically, the color-based K-means clustering algorithm is implemented by minimizing the distance from the sample to its nearest cluster center. In a specific embodiment, the S2, when implemented, includes:

[0072] S21, converting the original image from RGB color space to Lab color space.

[0073] The Lab color space is more in line with human perception characteristics. The Lab color space consists of three components: L (brightness component), a (green-red component) and b (blue-yellow component). The design of this color space separates brightness and chromaticity, which is convenient for color adjustment and contrast enhancement in image processing.

[0074] S22, randomly select two points C1 and C2 as initial centroids, where the centroids are points representing the centers of two clusters.

[0075] S23, for pixel x in the original image i , calculate its distance from the two centroids respectively, and assign it to the cluster to which the nearest centroid belongs according to its distance from the two centroids.

[0076] In a specific embodiment, the S23 calculates the pixel x i With one of the centroids C j The distance d(x i ,C j ), the Euclidean distance is used, which is expressed as:

[0077]

[0078] In formula (1), x i =[x i1 , x i2 , x i3 ], x i1 、x i2 and x i3 x i The values ​​of L, a, and b in the Lab color space; C j =[C j1 , C j2 , C j3 ], indicating the j-th centroid.

[0079] S24, calculate the average value of all data points of each cluster and update the centroid to this average value.

[0080] Specifically, this step can be implemented by the following formula:

[0081]

[0082] Among them, Cluster j is the set of all pixels assigned to the jth cluster, |Cluster j | is the number of pixels in cluster j, c j is the average of all data points in cluster j.

[0083] S25, repeating steps S23 and S24 until the distance between the current centroid and the previous centroid is less than a preset centroid convergence standard value, and determining the current centroid as the target centroid.

[0084] This step can be expressed by the formula:

[0085] in, and are the current centroid and the previous centroid of the jth cluster respectively; ε is the preset centroid convergence standard value, which is a very small threshold.

[0086] S26, according to the positions and color distributions of the two target centroids, the two clusters are distinguished into a foreground cluster and a background cluster, and the pixels in the foreground cluster and the background cluster are converted back to the RGB color space to obtain the foreground image and the background image of the original image.

[0087] Through the above clustering, two clusters and their centroids are finally output, and each cluster contains the pixel points closest to its centroid. Since the centroids and color distributions of different clusters are different, the embodiment of the present invention divides the original image into a foreground image and a background image according to the positions and color distributions of the two target centroids.

[0088] Furthermore, for the key target detection task, the background image contains little useful information, and the key target mainly appears in the foreground image. Therefore, the embodiment of the present invention performs image enhancement on the background image and the foreground image respectively through different image enhancement methods, and the specific contents are as follows:

[0089] S3, performing image enhancement on the background image by using a CLAHE algorithm, wherein the value of an enhancement parameter of the CLAHE algorithm is a fixed value when performing image enhancement on background images of different frames.

[0090] Since the background image contains little useful information, the embodiment of the present invention directly applies the existing CLAHE algorithm with a fixed enhancement parameter to enhance the background image of the original image of different frames, so as to save computing resources and reduce processing time. The embodiment of the present invention does not specifically limit the specific type of the enhancement parameter. The enhancement parameters selected in the embodiment of the present invention include the tile grid size (tileGridSize) and the contrast limit factor (clipLimit).

[0091] Based on the above content, the S3, when implemented specifically, includes:

[0092] S31, converting the background image from the RGB color space to the Lab color space, wherein the Lab color space of each pixel includes a brightness component and two color components.

[0093] S32, configuring fixed values ​​for the division grid size and the contrast limiting factor in the CLAHE algorithm, and performing contrast enhancement on the brightness component of each pixel in the background image through the configured CLAHE algorithm (i.e., the CLAHE algorithm configured with the division grid size and the contrast limiting factor), while keeping the color component unchanged.

[0094] The background image processed in this way can both enhance the contrast and avoid color distortion.

[0095] S33, converting the contrast-enhanced background image back into the RGB color space to obtain an enhanced background image.

[0096] S4, denoising the foreground image, and performing image enhancement on the denoised foreground image using an improved CLAHE algorithm, wherein the improved CLAHE algorithm adaptively adjusts the value of the enhancement parameter according to the image quality index when performing image enhancement on the denoised foreground images of different frames.

[0097] The foreground image is the object of the main target detection. In order to prevent the subsequent CLAHE algorithm from amplifying the noise, the embodiment of the present invention first performs denoising on the foreground image. When denoising, the embodiment of the present invention can select the Mallat denoising method, which can be implemented by the following steps:

[0098] S41, performing discrete wavelet transform (DWT) in row and column directions to decompose the foreground image into sub-bands of different scales to obtain a low-frequency sub-band and three high-frequency sub-bands.

[0099] Among them, the low-frequency subband contains the main information of the foreground image, while the high-frequency subband mainly contains the details and noise of the foreground image. For the foreground image f, the discrete wavelet transform can be expressed as: f→(cA, cH, cV, cD). cA (Approximation coefficients) is the approximation coefficient of the low-frequency subband, cH (Horizontal detail coefficients), cV (Vertical detail coefficients) and cD (Diagonal detail coefficients) are the detail coefficients of the high-frequency subband in the horizontal, vertical and diagonal directions respectively.

[0100] This step is implemented through the following steps:

[0101] S411, performing a one-dimensional wavelet transform on the foreground image rows, specifically: firstly represent the foreground image as a two-dimensional matrix f(x, y), the elements in the two-dimensional matrix are grayscale, x represents the rows of the foreground image, y represents the columns of the foreground image. Then, performing a one-dimensional discrete wavelet transform on each row of the foreground image to decompose it into a low-frequency component and a high-frequency component.

[0102] For the low-frequency component (row direction), the pixels of each row are filtered and downsampled by a wavelet low-pass filter, and the result is represented as Lr. For the high-frequency component (row direction), the pixels of each row are filtered and downsampled by a wavelet high-pass filter, and the result is represented as Hr. After this operation, each row of f(x, y) is decomposed into two parts: the low-frequency component Lr(x, y) and the high-frequency component Hr(x, y).

[0103] S412, performing one-dimensional wavelet transform on the foreground image column, specifically: performing one-dimensional wavelet transform on the low-frequency component and high-frequency component of the foreground image after the one-dimensional wavelet transform in the row direction respectively in the column direction.

[0104] Among them, the specific method of one-dimensional wavelet transform in the column direction of low-frequency components is:

[0105] Perform a column-wise discrete wavelet transform on Lr(x, y) to obtain two new sub-bands Lc(x, y) and Hc(x, y). Lc(x, y) is the low-frequency part of the low-frequency component in the column direction, that is, the approximate coefficient cA of the final low-frequency sub-band. Hc(x, y) is the high-frequency part of the low-frequency component in the column direction, that is, the detail coefficient cH in the horizontal direction.

[0106] The specific method of one-dimensional wavelet transform in the column direction of high-frequency components is:

[0107] Perform discrete wavelet transform on Hr(x, y) in the column direction to obtain two new sub-bands Lc′(x, y) and Hc′(x, y). Lc′(x, y) is the low-frequency part of the high-frequency component in the column direction, that is, the detail coefficient cV in the vertical direction. Hc′(x, y) is the high-frequency part of the high-frequency component in the column direction, that is, the detail coefficient cD in the diagonal direction.

[0108] Through the above steps, f(x, y) is decomposed into the following four sub-bands. The low-frequency sub-band is obtained by L(x, y), which represents the approximate information of the foreground image and contains the main structure and overall shape of the foreground image. The horizontal detail sub-band is obtained by Hc(x, y), which contains the horizontal detail information of the foreground image and reflects the changes in the foreground image in the horizontal direction. The vertical detail sub-band is obtained by Lc′(x, y), which contains the vertical detail information of the foreground image and reflects the changes in the foreground image in the vertical direction. The diagonal detail sub-band is obtained by Hc′(x, y), which contains the diagonal detail information of the foreground image and reflects the changes in the foreground image in the diagonal direction.

[0109] S42, performing threshold processing on detail coefficients of the three high frequency sub-bands to remove noise therein.

[0110] Specifically, the threshold processing method uses a hard threshold to retain larger wavelet coefficients, has no effect on the main features of the signal, is simple to calculate, and has a fast execution speed.

[0111] This step is expressed by the formula:

[0112]

[0113] Among them, d is the detail coefficient, which are cH, cV, cD respectively. is the detail coefficient after threshold processing, λ is the threshold; α is the standard deviation of the noise, which can be estimated by cD, the formula is Median is the median; N is the length of the signal, which is equal to the number of pixels in the foreground image, that is, the width of the foreground image multiplied by the height.

[0114] S43, performing inverse discrete wavelet transform (IDWT) on the three high-frequency sub-bands and the low-frequency sub-band after the threshold processing, and reconstructing the denoised foreground image.

[0115] Furthermore, after denoising, S4 performs image enhancement on the denoised foreground images of different frames by using the improved CLAHE algorithm. When performing image enhancement on the denoised foreground images of different frames, the improved CLAHE algorithm adaptively adjusts the value of the enhancement parameter according to the image quality index. The image quality index is an important objective evaluation index that reflects the difficulty of the target detection task.

[0116] In a specific embodiment, the image quality index includes at least one of grayscale variance, Brenner function value and gradient energy. Grayscale variance is an index to measure the discrete degree of grayscale value distribution of the denoised foreground image, and can reflect the contrast of the denoised foreground image. Brenner function is mainly used to measure the clarity of the image, and reflects the clarity of the image by calculating the sum of squares of grayscale value differences between adjacent pixels. Gradient energy measures the strength of edges and textures in the denoised foreground image, and is the sum of squares of gradient values ​​of each pixel in the denoised foreground image.

[0117] Specifically, the grayscale variance σ of the denoised foreground image is 2 Calculated by the following formula (2):

[0118]

[0119] In formula (2), I(i, j) represents the gray value of the denoised foreground image at position (i, j); μ is the average gray value of the denoised foreground image, and the calculation formula is: H and W are the height and width of the denoised foreground image, respectively.

[0120] The Brenner function value B of the denoised foreground image is calculated by the following formula (3):

[0121]

[0122] The gradient energy E of the denoised foreground image is calculated by the following formula (4):

[0123]

[0124] In formula (4), G x (i,j) and G y (i, j) represent the horizontal gradient value and vertical gradient value of the denoised foreground image at position (i, j), respectively.

[0125] In a specific embodiment, when the image quality index includes grayscale variance, Brenner function value and gradient energy at the same time, the image quality index of the denoised foreground image is calculated by the following steps:

[0126] S4-1, normalize the grayscale variance, Brenner function value and gradient energy.

[0127] Specifically, during normalization, the minimum and maximum values ​​of each image quality index are found for normalization (the grayscale variance, Brenner function value and gradient energy of the foreground image after all original images are denoised are calculated, and the minimum and maximum values ​​of each image quality index are determined from all grayscale variances, Brenner function values ​​and gradient energies respectively), and then normalized using the following formula:

[0128] normalize=(value-min_value) / (max_value-min_value);

[0129] Among them, normalize represents the normalization result of a certain image quality indicator, value represents the value of the image quality indicator, min_value represents the minimum value of the image quality indicator, and max_value represents the maximum value of the image quality indicator.

[0130] S4-2: Multiply the normalized grayscale variance, Brenner function value and gradient energy by their corresponding weights to obtain the image quality index of the denoised foreground image.

[0131] Specifically, this step is expressed by the formula:

[0132] score=(σ 2 _weight×normalize_σ 2 )+(B_weight×normalize_B)+

[0133] (E_weight×normalize_E);

[0134] Among them, σ 2 _weight represents the weight of grayscale variance, normalize_σ 2 It represents the normalized result of grayscale variance, B_weight represents the weight of Brenner function value, normalize_B represents the normalized result of Brenner function value, E_weight represents the weight of gradient energy, and normalize_E represents the normalized result of gradient energy.

[0135] In a specific embodiment, a particle swarm optimization algorithm is used to determine the values ​​of enhancement parameters of the foreground images after denoising of different frames.

[0136] Specifically, because the original images of adjacent frames in the coal mine excavation working face video have high similarity, the optimal enhancement parameters in the improved CLAHE algorithm are also similar. When setting the initial range of the particle swarm optimization algorithm, the result of the optimization of the previous frame of the original image can be used as a reference to narrow the search range and accelerate convergence.

[0137] For example, prev_optimal_params is the optimal enhancement parameter of the previous frame. If prev_optimal_params does not exist (i.e., for the first frame of the original image), the search range is initialized to the global range. If this parameter exists, a smaller search range is defined with it as the center and a fixed value is added or subtracted to achieve a narrowing of the search range.

[0138] According to the determined search range, the particle swarm optimization algorithm updates prev_optimal_params to the optimal enhancement parameters of the original image of this frame after finding the optimal enhancement parameters. The improved CLAHE algorithm performs image enhancement on the foreground image after denoising the original image of this frame according to the optimal enhancement parameters.

[0139] In summary, the embodiment of the present invention can adaptively enhance the foreground image according to the image quality index of the foreground image through the improved CLAHE algorithm. The enhanced foreground image has high contrast, low noise, and clear edges, and can be better suitable for target detection tasks in underground excavation working faces.

[0140] It should be noted that, in order to increase the processing speed, the embodiment of the present invention can utilize multi-threading in parallel to perform image enhancement on the foreground image and the background image, and use GPU acceleration.

[0141] S5, fusing the enhanced background image and the enhanced foreground image to obtain a target image corresponding to the original image.

[0142] In a specific embodiment, when S5 is implemented, the enhanced background image and the enhanced foreground image are fused by the following formula (5) to obtain a target image corresponding to the original image:

[0143] I final (i,j)=w(i,j)·I f (i,j)+(1-w(i,j))·I b (i,j)(5);

[0144] In formula (5), I final (i, j) represents the grayscale value of the target image at position (i, j), I f (i, j) represents the gray value of the enhanced foreground image at position (i, j), I b(i, j) represents the grayscale value of the enhanced background image at position (i, j); w(i, j) represents the weight at position (i, j), and its value range is between 0 and 1.

[0145] By fusing the enhanced background image and the enhanced foreground image in this way, the transition area between the foreground image and the background image can be smoothly fused. The weight of the transition area is generated using a Gaussian function to ensure smooth fusion.

[0146] S6, input the target image into the pre-trained improved YOLOv8n model for key target detection.

[0147] The improved YOLOv8n model described in the embodiment of the present invention uses the DWR module to construct the DWR-improved C2f module, and uses the DWR-improved C2f module to replace the original C2f module of the neck network in the YOLOv8n model. The SEAM module is used to replace the second Conv convolution module in the detection head of the YOLOv8n model to build a SEAM-improved detection head, and MPDIoU is used to replace the original CIoU of the YOLOv8n model as the loss function. That is to say, the improved YOLOv8n model described in the embodiment of the present invention includes a backbone network, a neck network and a SEAM (Separated and Enhancement Attention Module) improved detection head connected in sequence. The neck network includes a DWR-improved C2f module, and the DWR-improved C2f module uses the DWR module to replace the Bottleneck residual module in the C2f module; the SEAM-improved detection head uses the SEAM module to replace the second Conv convolution module of the detection head; the loss function of the improved YOLOv8n model is MPDIoU, where:

[0148]

[0149] MPDIoU loss = 1 - MPDIoU (8);

[0150] In formula (6) to formula (8), B pred and B gt Represent the predicted box and the true annotation box respectively, w and h are the width and height of the image respectively, d1 and d2 are the distances between the upper left corner and lower right corner of the predicted box and the true annotation box respectively.

[0151] The three improvements of the improved YOLOv8n model are described in detail below:

[0152] Improvement 1: The C2f module contains many Bottleneck residual modules. These Bottleneck residual modules can be connected in series to achieve the extraction and fusion of features of different scales. However, the Bottleneck residual module causes the network to superimpose a large amount of information at the same position, that is, the high-frequency position, and generates many redundant features in the process of obtaining key features, which may cause the loss of key information. Multi-target detection of excavation working faces is often carried out under harsh working conditions, and the loss of information will seriously reduce the accuracy of target detection. Therefore, the embodiment of the present invention uses a DWR (Dilation-wise Residual) module to replace the original Bottleneck residual module to obtain a DWR-improved C2f module, such as Figure 2 As shown in Figure 2. The DWR module is designed in a residual way. Inside the residual, a two-step method is used to effectively extract multi-scale context information, and then the feature map generated by the multi-scale receptive field is fused. The DWR module is applied to the deep layer of the network, and the multi-branch structure is used to adapt to the requirements of receptive fields of different sizes in a layer. For each branch, as Figure 3 As shown in the figure, the DWR module first uses a normal 3x3 convolution combined with a batch normalization (BN) layer and a ReLU layer to complete feature extraction, and obtains regional residual features RR with different region sizes; then the RR features are divided into 3 groups, and the dilated deep convolutions with multiple dilation rates are used to perform morphological filtering on regional features of different sizes, which is called semantic residual SR. By increasing the flexibility of the dilation rate, the network is more adaptable to targets of different scales in terms of receptive field size and density; then BN is performed on the feature map, and all feature maps are merged using point-by-point convolution to generate the final residual corresponding to the input feature map; finally, the final residual is added to the input feature map to construct a stronger and more comprehensive feature representation. In the target detection task, especially for multi-scale targets, the DWR module can effectively improve the receptive field range of the network, better capture the details and contextual information of the target, and thus improve the accuracy and robustness of detection.

[0153] Improvement point 2: Due to the small space of the excavation scene, the large number of personnel, the large size of the excavator, and the fixed camera angle, there will be serious occlusion when detecting the personnel and tools in the excavation scene, which will greatly increase the difficulty of detection and reduce the accuracy of detection. Therefore, the embodiment of the present invention introduces the SEAM module into the detection head to construct a SEAM-improved detection head with occlusion perception, such as Figure 4 As shown in Figure 1. The SEAM improved detection head uses the SEAM module to replace the second Conv (convolution module) of the original detection head of the YOLOv8n model. Figure 5As shown in the figure, the first part of the SEAM module is a depthwise separable convolution with residual connection. Depthwise separable convolution operates by depth, that is, the convolution separates channels by channel. Although depthwise separable convolution can learn the importance of different channels and reduce the amount of parameters, it ignores the information relationship between channels. To make up for this loss, the outputs of convolutions of different depths are then combined through 1x1 point-by-point convolution. Then a two-layer fully connected network (i.e., two fully connected layers) is used to fuse the information of each channel, so that the network can strengthen the connection between all channels, hoping that the model can make up for the above loss in the occluded scene through the relationship between the occluded target and the unoccluded target learned in the previous step. Then the output learned by the fully connected layer is processed by an exponential function to expand the value range from [0, 1] to [1, e]. This exponential normalization provides a monotonic mapping relationship, making the result more tolerant to position errors. Finally, the output of the SEAM module is multiplied by the original feature as attention, so that the model can more effectively deal with problems such as alignment errors, local aliasing, and feature missing caused by occlusion. The CSMM module in the SEAM module uses different patches to obtain multi-scale features and uses deep separable convolution to learn the correlation between spatial dimensions and channels.

[0154] Improvement 3: Use MPDIoU instead of the original CIoU of the YOLOv8n model. Most existing bounding box regression loss functions cannot be optimized when the predicted box and the actual annotated box have the same aspect ratio but completely different width and height values. MPDIoU includes all relevant factors considered in existing loss functions, such as overlapping or non-overlapping area, center point distance, and width and height deviation, while simplifying the calculation process. It directly minimizes the distance between the upper left and lower right corners of the predicted bounding box and the actual annotated bounding box, which can improve the training effect of bounding box regression, and improve the convergence speed and regression accuracy.

[0155] In summary, the improved YOLOv8n model proposed in the embodiment of the present invention has a larger receptive field, can better extract context information, and can enhance the detection accuracy of multi-target occlusion, while also accelerating the convergence speed and regression accuracy.

[0156] It should be noted that before executing step S6, the improved YOLOv8n model needs to be trained first. For the specific implementation method of training the improved YOLOv8n model, please refer to the existing model training method, which is not elaborated in detail in the embodiment of the present invention.

[0157] In order to verify the effectiveness of the improved CLAHE algorithm and the improved YOLOv8n model proposed in the embodiment of the present invention, some images are selected for comparative experiments, such as Figure 6As shown, (a) and (b) are schematic diagrams of the original image and the result of adaptively adjusting the enhancement parameters through the improved CLAHE algorithm. The comparison of the enhanced image and the original image indicators is shown in Table 1:

[0158] Table 1

[0159]

[0160] It can be seen from Table 1 that the three function values ​​of the enhanced image are much higher than those of the original image, indicating that the improved CLAHE algorithm proposed in the embodiment of the present invention can effectively solve the problems of unclear images, low contrast, blurred edge information, etc. caused by weak light and dusty fog environment in coal mines, achieve the effect of image enhancement, and make the reasoning result of the model more accurate.

[0161] like Figure 7 As shown in , it is a comparison chart of target detection effects of different target detection models. Figure 7 (a) and Figure 7 From the second and third rows of images in (b), we can see that the improved YOLOv8n model can detect occluded targets without missing any and with a higher confidence level, but it can also detect targets in the first row with extreme dust and fog. Figure 7 After adding the adaptive image enhancement parameters through the improved CLAHE algorithm, the image (c) not only becomes clearer and detects the targets in the first row of images, but also further improves the confidence of the targets in the second and third rows of images. This proves the effectiveness of each part of the method proposed in the embodiment of the present invention.

[0162] In order to verify the superiority of the method proposed in the embodiment of the present invention, the accuracy of the method is compared with that of the YOLOv5s, YOLOv7, YOLOv8n, and YOLOv8s models. The mAP is shown in Table 2 below:

[0163] Table 2

[0164]

[0165] mAP (MeanAverage Precision) can comprehensively reflect the detection performance of the model in different categories and is one of the important indicators for measuring the performance of the target detection model. As can be seen from Table 2, the method proposed in the embodiment of the present invention is higher than other popular target detection algorithms, proving the overall effectiveness of the method proposed in the embodiment of the present invention.

[0166] In summary, the method provided by the embodiment of the present invention has the following characteristics:

[0167] 1. The particle swarm optimization algorithm is used to adaptively determine the enhancement parameters of the improved CLAHE algorithm, which can improve the enhancement effect of each original image and avoid the problems caused by over-enhancement and under-enhancement.

[0168] 2. Image quality index is an important objective evaluation index that reflects the difficulty of target detection tasks. Using image quality index as an indicator for finding the optimal enhancement parameters of the improved CLAHE algorithm can obtain images that are comprehensively optimized in terms of brightness distribution, edge clarity, image texture, etc., which are more suitable for subsequent key target detection tasks.

[0169] 3. Because the original images of adjacent frames usually do not change much, the best enhancement parameters of the previous frame are referenced during the parameter optimization process, which narrows the search range of the current frame and improves the optimization efficiency and stability.

[0170] 4. The improved YOLOv8n model proposed in the embodiment of the present invention uses the DWR module to construct the DWR-improved C2f module to replace the original C2f module of the neck network. In key target detection tasks, especially for multi-scale targets, the DWR-improved C2f module can effectively improve the receptive field range of the network and better capture the details and contextual information of the target, thereby improving the accuracy and robustness of detection.

[0171] 5. The SEAM module is used to replace the second Conv convolution module in the YOLOv8n model detection head to build a SEAM-improved detection head, so that the improved YOLOv8n model can more effectively handle problems such as alignment errors, local aliasing, and feature missing caused by occlusion.

[0172] 6. Using MPDIoU instead of CIoU in the original YOLOv8n model as the loss function directly minimizes the distance between the upper left and lower right corners of the predicted bounding box and the actual annotated bounding box, which can improve the training effect of bounding box regression and increase the convergence speed and regression accuracy.

[0173] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the technical principles of the present invention, and these improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for detecting key targets in a coal mine excavation working face, characterized in that: include: S1, after obtaining the video collected from the coal mine excavation working face, the video is processed into frames to obtain original images frame by frame; S2, using the color-based K-means clustering algorithm to segment the original image into foreground and background, and obtain the foreground image and background image of the original image; S3, performing image enhancement on the background image by using a CLAHE algorithm, wherein the value of an enhancement parameter of the CLAHE algorithm is a fixed value when performing image enhancement on background images of different frames; S4, denoising the foreground image, and performing image enhancement on the denoised foreground image using an improved CLAHE algorithm, wherein the improved CLAHE algorithm adaptively adjusts the value of the enhancement parameter according to the image quality index when performing image enhancement on the denoised foreground images of different frames; S5, fusing the enhanced background image and the enhanced foreground image to obtain a target image corresponding to the original image; S6, input the target image into the pre-trained improved YOLOv8n model for key target detection.

2. The key target detection method for coal mine excavation working face according to claim 1 is characterized in that: When S2 is specifically implemented, it includes: S21, converting the original image from RGB color space to Lab color space; S22, randomly select two points C1 and C2 as initial centroids, where the centroid is the point representing the center of two clusters; S23, for pixel x in the original image i , calculate its distance from the two centroids respectively, and assign it to the cluster to which the nearest centroid belongs according to its distance from the two centroids; S24, calculate the average value of all data points of each cluster and update the centroid to this average value; S25, repeating steps S23 and S24 until the distance between the current centroid and the previous centroid is less than a preset centroid convergence standard value, and determining the current centroid as the target centroid; S26, according to the positions and color distributions of the two target centroids, the two clusters are distinguished into a foreground cluster and a background cluster, and the pixels in the foreground cluster and the background cluster are converted back to the RGB color space to obtain the foreground image and the background image of the original image.

3. The key target detection method for coal mine excavation working face according to claim 2 is characterized in that: S23 calculates the pixel xi and one of the centroids C j The distance d(x i ,C j ), the Euclidean distance is used, which is expressed as: In formula (1), x i =[x i1 , x i2 , x i3 ], x i1 、x i2 and x i3 x i The values ​​of L, a, and b in the Lab color space; C j =[C j1 , C j2 , C j3 ], indicating the j-th centroid.

4. The key target detection method for coal mine excavation working face according to claim 1 is characterized in that: The enhancement parameters of the CLAHE algorithm include the grid size and the contrast limiting factor. When S3 is implemented specifically, it includes: S31, converting the background image from the RGB color space to the Lab color space, wherein the Lab color space of each pixel includes a brightness component and two color components; S32, configuring fixed values ​​for the grid size and contrast limiting factor in the CLAHE algorithm, and performing contrast enhancement on the brightness component of each pixel in the background image through the configured CLAHE algorithm while keeping the color component unchanged; S33, converting the contrast-enhanced background image back into the RGB color space to obtain an enhanced background image.

5. The key target detection method for coal mine excavation working face according to claim 1 is characterized in that: When performing denoising on the foreground image, S4 includes: S41, performing discrete wavelet transform in row and column directions to decompose the foreground image into sub-bands of different scales to obtain a low-frequency sub-band and three high-frequency sub-bands; S42, performing threshold processing on detail coefficients of the three high frequency sub-bands to remove noise therein; S43, performing inverse discrete wavelet transform on the three high-frequency sub-bands and the low-frequency sub-band after the threshold processing, and reconstructing the denoised foreground image.

6. The key target detection method for coal mine excavation working face according to claim 1 is characterized in that: The image quality index includes at least one of grayscale variance, Brenner function value and gradient energy; The grayscale variance σ of the denoised foreground image 2 Calculated by formula (2): In formula (2), I(i, j) represents the grayscale value of the denoised foreground image at position (i, j); μ is the average grayscale value of the denoised foreground image, and the calculation formula is: H and W are the height and width of the denoised foreground image, respectively; The Brenner function value B of the denoised foreground image is calculated by formula (3): The gradient energy E of the denoised foreground image is calculated by formula (4): In formula (4), G x (i,j) and G y (i, j) represent the horizontal gradient value and vertical gradient value of the denoised foreground image at position (i, j), respectively.

7. The key target detection method for coal mine excavation working face according to claim 1 or 6, characterized in that: When the image quality index includes grayscale variance, Brenner function value and gradient energy, the image quality index of the denoised foreground image is calculated by the following steps: S4-1, normalize the grayscale variance, Brenner function value and gradient energy; S4-2: Multiply the normalized grayscale variance, Brenner function value and gradient energy by their corresponding weights to obtain the image quality index of the denoised foreground image.

8. The key target detection method for coal mine excavation working face according to claim 1 is characterized in that: In the specific implementation of S5, the enhanced background image and the enhanced foreground image are fused by formula (5) to obtain the target image corresponding to the original image: I final (i,j)=w(i,j)·I f (i,j)+(1-w(i,j))·I b (i,j)(5); In formula (5), I final (i, j) represents the grayscale value of the target image at position (i, j), I f (i, j) represents the gray value of the enhanced foreground image at position (i, j), I b (i, j) represents the grayscale value of the enhanced background image at position (i, j); w(i, j) represents the weight at position (i, j), and its value range is between 0 and 1.

9. The key target detection method for coal mine excavation working face according to claim 1 is characterized in that: The improved YOLOv8n model includes a backbone network, a neck network and a SEAM improved detection head connected in sequence; The neck network includes a DWR-modified C2f module, wherein the DWR-modified C2f module replaces the Bottleneck residual module in the C2f module with the DWR module; The SEAM-improved detection head uses the SEAM module to replace the second Conv convolution module of the detection head; The loss function of the improved YOLOv8n model is MPDIoU, where: MPDIoU loss = 1 - MPDIoU (8); In formula (6) to formula (8), B pred and B gt Represent the predicted box and the true annotation box respectively, w and h are the width and height of the image respectively, d1 and d2 are the distances between the upper left corner and lower right corner of the predicted box and the true annotation box respectively.

10. The key target detection method for coal mine excavation working face according to claim 1, characterized in that: The particle swarm optimization algorithm is used to determine the values ​​of enhancement parameters of the foreground image after denoising in different frames.

Images