Key target detection method for coal mine heading working face

Abstract

The present invention relates to a method for detecting key targets in a coal mine excavation working face, and belongs to the technical field of target detection. The method comprises: performing foreground and background segmentation on an original image to obtain a foreground image and a background image; performing image enhancement on the background image using a CLAHE algorithm; performing denoising on the foreground image, and performing image enhancement on the denoised foreground image using an improved CLAHE algorithm, wherein the improved CLAHE algorithm adaptively adjusts the value of the enhancement parameter according to an image quality index; fusing the enhanced background image and the enhanced foreground image to obtain a target image; and inputting the target image into a pre-trained improved YOLOv8n model for key target detection. The enhanced image of the present invention has high contrast, low noise, and clear edges; the improved YOLOv8n model has a larger receptive field, can better extract contextual information, and can enhance the detection accuracy of multi-target occlusions, and also accelerates the convergence speed and regression accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a method for detecting key targets on a coal mine excavation working face. Background Art

[0002] As the complexity of coal mining continues to increase, traditional mechanized and automated mining technologies are no longer able to meet the demands for further improvements in mining efficiency and safety. Therefore, automated and intelligent mining has become an inevitable development trend. Machine vision-based key target detection for operators and equipment provides technical support and assurance for understanding human behavior and identifying potential safety hazards, playing a crucial role in achieving intelligent and safe coal mining.

[0003] Under low-light conditions, tunneling faces are affected by backlight, uneven illumination, and weak light. Video images suffer from low contrast, high noise, information loss, and blurred details, hindering subsequent tasks such as key target detection. Numerous algorithms have been extensively researched and applied in scenarios such as underground coal mine conveyor belt monitoring and motor vehicle transportation. However, due to the more extreme environmental conditions of tunneling, research on object detection algorithms specifically for these scenarios is limited. There is a lack of algorithms that can simultaneously address issues such as high dust and fog, low illumination, and multiple objects occluding each other.

[0004] To improve image quality, the CLAHE algorithm can be used for image enhancement. While the CLAHE algorithm is currently well-established in medical and remote sensing image processing, its original CLAHE algorithm is often used directly in underground tunneling work faces. The original CLAHE algorithm uses fixed enhancement parameters for different images and cannot automatically adjust these parameters to improve the effect based on image changes. This can lead to over-enhancement or under-enhancement. However, over-enhancement can amplify image noise, making it more noticeable and affecting image clarity and detail. It can also introduce artifacts or streaking, especially at edges or in uniform areas. It can also lead to local contrast imbalance, causing some areas to be too bright while others lose detail. Under-enhancement can also lead to limited contrast improvement, preventing the full display of detail and structural information. Therefore, being able to adaptively select appropriate enhancement parameters is crucial for improving image enhancement. Furthermore, the confined space and the large number of equipment and personnel in coal mine tunneling work faces exacerbate occlusion issues, which significantly impacts the accuracy of key target detection. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a method for detecting key targets in a coal mine excavation working face. The technical solution of the present invention is as follows:

[0006] A method for detecting key targets in a coal mine excavation working face, comprising:

[0007] S1, after obtaining the video collected from the coal mine excavation working face, the video is frame-processed to obtain original images frame by frame;

[0008] S2, using the color-based K-means clustering algorithm to segment the original image into foreground and background, and obtain the foreground image and background image of the original image;

[0009] S3, performing image enhancement on the background image using a CLAHE algorithm, wherein the value of an enhancement parameter of the CLAHE algorithm is a fixed value when performing image enhancement on background images of different frames;

[0010] S4, performing denoising on the foreground image and performing image enhancement on the denoised foreground image using an improved CLAHE algorithm, wherein the improved CLAHE algorithm adaptively adjusts the value of the enhancement parameter according to the image quality index when performing image enhancement on the denoised foreground images of different frames;

[0011] S5, fusing the enhanced background image and the enhanced foreground image to obtain a target image corresponding to the original image;

[0012] S6, input the target image into the pre-trained improved YOLOv8n model for key target detection.

[0013] Optionally, when S2 is specifically implemented, it includes:

[0014] S21, converting the original image from RGB color space to Lab color space;

[0015] S22, randomly select two points C1 and C2 as the initial centroids, where the centroids are the points representing the centers of the two clusters;

[0016] S23, for pixel x in the original image i , calculate its distance from the two centroids respectively, and assign it to the cluster to which the nearest centroid belongs according to its distance from the two centroids;

[0017] S24, calculate the average value of all data points of each cluster and update the centroid to this average value;

[0018] S25, repeating steps S23 and S24 until the distance between the current centroid and the previous centroid is less than a preset centroid convergence standard value, and determining the current centroid as the target centroid;

[0019] S26, according to the positions and color distributions of the two target centroids, the two clusters are divided into a foreground cluster and a background cluster, and the pixels in the foreground cluster and the background cluster are converted back to the RGB color space to obtain the foreground image and the background image of the original image.

[0020] Optionally, the S23 calculates the pixel xi and one of the centroids C j The distance d(x i ,C j ), the Euclidean distance is used, which is expressed as:

[0021]

[0022] In formula (1), x i =[x i1 , x i2 , x i3 ], x i1 、x i2 and x i3 x i The values ​​of L, a, and b in the Lab color space; C j =[C j1 , C j2 , C j3 ], representing the j-th centroid.

[0023] Optionally, the enhancement parameters of the CLAHE algorithm include a grid size and a contrast limiting factor. When S3 is specifically implemented, it includes:

[0024] S31, converting the background image from the RGB color space to the Lab color space, wherein the Lab color space of each pixel includes a brightness component and two color components;

[0025] S32, configuring fixed values ​​for the grid size and contrast limiting factor in the CLAHE algorithm, and performing contrast enhancement on the brightness component of each pixel in the background image using the configured CLAHE algorithm while keeping the color component unchanged;

[0026] S33, converting the contrast-enhanced background image back into RGB color space to obtain an enhanced background image.

[0027] Optionally, when performing denoising on the foreground image, the step S4 includes:

[0028] S41, performing discrete wavelet transform in the row and column directions to decompose the foreground image into sub-bands of different scales, obtaining a low-frequency sub-band and three high-frequency sub-bands;

[0029] S42, performing threshold processing on the detail coefficients of the three high-frequency sub-bands to remove noise therein;

[0030] S43, performing inverse discrete wavelet transform on the three high-frequency sub-bands and the low-frequency sub-band after the threshold processing, and reconstructing the denoised foreground image.

[0031] Optionally, the image quality indicator includes at least one of grayscale variance, Brenner function value, and gradient energy;

[0032] The grayscale variance σ of the denoised foreground image 2 Calculated by formula (2):

[0033]

[0034] In formula (2), I(i, j) represents the grayscale value of the denoised foreground image at position (i, j); μ is the average grayscale value of the denoised foreground image, and the calculation formula is: H and W are the height and width of the denoised foreground image, respectively;

[0035] The Brenner function value B of the denoised foreground image is calculated using formula (3):

[0036]

[0037] The gradient energy E of the denoised foreground image is calculated using formula (4):

[0038]

[0039] In formula (4), G x (i,j) and G y (i, j) represents the horizontal gradient value and vertical gradient value of the denoised foreground image at position (i, j), respectively.

[0040] Optionally, it is characterized in that, when the image quality index includes grayscale variance, Brenner function value and gradient energy, the image quality index of the denoised foreground image is calculated by the following steps:

[0041] S4-1, normalize the grayscale variance, Brenner function value and gradient energy;

[0042] S4-2: Multiply the normalized grayscale variance, Brenner function value, and gradient energy by their corresponding weights to obtain the image quality index of the denoised foreground image.

[0043] Optionally, in a specific implementation of S5, the enhanced background image and the enhanced foreground image are fused by formula (5) to obtain a target image corresponding to the original image:

[0044] I final (i,j)=w(i,j)·I f (i,j)+(1-w(i,j))·I b (i,j)(5);

[0045] In formula (5), I final (i, j) represents the grayscale value of the target image at position (i, j), I f (i, j) represents the grayscale value of the enhanced foreground image at position (i, j), I b (i, j) represents the grayscale value of the enhanced background image at position (i, j); w(i, j) represents the weight at position (i, j), and its value range is between 0 and 1.

[0046] Optionally, the improved YOLOv8n model includes a backbone network, a neck network, and a SEAM-improved detection head connected in sequence;

[0047] The neck network includes a DWR-modified C2f module, wherein the DWR-modified C2f module replaces the Bottleneck residual module in the C2f module with the DWR module;

[0048] The SEAM-improved detection head uses the SEAM module to replace the second Conv convolution module of the detection head;

[0049] The loss function of the improved YOLOv8n model is MPDIoU, where:

[0050]

[0051] MPDIoU loss = 1 - MPDIoU (8);

[0052] In formula (6) to formula (8), B pred and B gt Denote the predicted box and the true annotation box respectively, w and h are the width and height of the image respectively, d1 and d2 are the distances between the upper left corner and the lower right corner of the predicted box and the true annotation box respectively.

[0053] Optionally, a particle swarm optimization algorithm is used to determine the values ​​of enhancement parameters of the foreground images after denoising of different frames.

[0054] All the above optional technical solutions can be combined arbitrarily, and the present invention does not provide detailed descriptions of the structures after each combination.

[0055] By means of the above solution, the beneficial effects of the present invention are as follows:

[0056] The improved CLAHE algorithm enhances the denoised foreground images of different frames by adaptively adjusting the enhancement parameters based on image quality indicators. This improves the image enhancement effect of each original image, avoiding problems caused by over-enhancement and under-enhancement. It adaptively enhances the foreground image based on the image quality indicators of the original image. The enhanced foreground image has high contrast, low noise, and clear edges, making it more suitable for target detection tasks in underground tunneling working faces. The improved YOLOv8n model has a larger receptive field, can better extract contextual information, and can enhance the detection accuracy of multiple occluded targets, while also accelerating convergence speed and regression accuracy.

[0057] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of a method for detecting key targets in a coal mine excavation working face provided by an embodiment of the present invention.

[0059] Figure 2 Schematic diagram of the structure of the C2f module improved by DWR in an embodiment of the present invention.

[0060] Figure 3 4 is a structural diagram of the DWR module in an embodiment of the present invention.

[0061] Figure 4 2 is a structural diagram of the detection head improved by SEAM in an embodiment of the present invention.

[0062] Figure 5 4 is a structural diagram of the SEAM module in an embodiment of the present invention.

[0063] Figure 6 This is a comparison diagram of the image enhancement effects of the key target detection method for a coal mine heading working face provided by an embodiment of the present invention.

[0064] Figure 7 3 is a comparison chart of target detection effects of different target detection models in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0066] The method for detecting key targets in a coal mine excavation working face provided by the embodiment of the present invention can be implemented by any electronic device with computing capabilities, such as a PC, a mobile terminal, or a server. Figure 1 As shown, the method for detecting key targets in a coal mine excavation working face provided by an embodiment of the present invention includes the following steps S1 to S6:

[0067] S1, after obtaining the video collected from the coal mine excavation working face, the video is frame-processed to obtain original images frame by frame.

[0068] Specifically, the industrial cameras installed on the coal mine excavation working face will collect videos in real time, and the electronic equipment can obtain the videos from the industrial cameras.

[0069] S2, using the color-based K-means clustering algorithm to segment the original image into foreground and background, and obtain the foreground image and background image of the original image.

[0070] The background image mainly includes lanes, and the foreground image mainly includes pedestrians, equipment, etc. The goal of the color-based K-means clustering algorithm is to divide the pixels of the original image into two clusters (categories), namely the foreground cluster and the background cluster.

[0071] Specifically, the color-based K-means clustering algorithm is implemented by minimizing the distance between the sample and its nearest cluster center. In a specific embodiment, the S2, when implemented, includes:

[0072] S21, converting the original image from RGB color space to Lab color space.

[0073] The Lab color space better aligns with human perception. It consists of three components: L (lightness), a (green-red component), and b (blue-yellow component). This color space design separates lightness and chromaticity, facilitating color adjustment and contrast enhancement in image processing.

[0074] S22, randomly select two points C1 and C2 as initial centroids, where the centroid is a point representing the center of two clusters.

[0075] S23, for pixel x in the original image i , calculate its distance from the two centroids respectively, and assign it to the cluster to which the nearest centroid belongs according to its distance from the two centroids.

[0076] In a specific embodiment, the S23 calculates the pixel x i With one of the centroids C j The distance d(x i ,C j ), the Euclidean distance is used, which is expressed as:

[0077]

[0078] In formula (1), x i =[x i1 , x i2 , x i3 ], x i1 、x i2 and x i3 x i The values ​​of L, a, and b in the Lab color space; C j =[C j1 , C j2 , C j3 ], representing the j-th centroid.

[0079] S24, calculate the average value of all data points of each cluster and update the centroid to this average value.

[0080] Specifically, this step can be achieved by the following formula:

[0081]

[0082] Among them, Cluster j is the set of all pixels assigned to the jth cluster, |Cluster j | is the number of pixels in cluster j, c j is the mean of all data points in cluster j.

[0083] S25, repeating steps S23 and S24 until the distance between the current centroid and the previous centroid is less than a preset centroid convergence standard value, and determining the current centroid as the target centroid.

[0084] This step can be expressed as:

[0085] in, and are the current centroid and the previous centroid of the j-th cluster respectively; ε is the preset centroid convergence standard value, which is a very small threshold.

[0086] S26, according to the positions and color distributions of the two target centroids, the two clusters are divided into a foreground cluster and a background cluster, and the pixels in the foreground cluster and the background cluster are converted back to the RGB color space to obtain the foreground image and the background image of the original image.

[0087] The above clustering ultimately outputs two clusters and their centroids, where each cluster contains the pixels closest to its centroid. Because different clusters have different centroids and color distributions, this embodiment of the present invention segments the original image into a foreground image and a background image based on the positions and color distributions of the two target centroids.

[0088] Furthermore, for the key target detection task, the background image contains little useful information, and the key targets mainly appear in the foreground image. Therefore, the embodiment of the present invention performs image enhancement on the background image and the foreground image using different image enhancement methods, as follows:

[0089] S3, performing image enhancement on the background image using a CLAHE algorithm, wherein the enhancement parameter value of the CLAHE algorithm is a fixed value when performing image enhancement on background images of different frames.

[0090] Because the background image contains little useful information, the present embodiment directly applies the existing CLAHE algorithm with fixed enhancement parameters to enhance the background images of different original frames, thereby conserving computing resources and reducing processing time. The present embodiment does not specifically limit the specific type of enhancement parameters; however, the selected enhancement parameters in the present embodiment include the tileGridSize and the contrast limiting factor (clipLimit).

[0091] Based on the above content, the specific implementation of S3 includes:

[0092] S31 , converting the background image from the RGB color space to the Lab color space, wherein the Lab color space of each pixel includes a brightness component and two color components.

[0093] S32, configuring fixed values ​​for the grid size and contrast limiting factor in the CLAHE algorithm, and performing contrast enhancement on the brightness component of each pixel in the background image through the configured CLAHE algorithm (i.e., the CLAHE algorithm configured with the grid size and contrast limiting factor), while keeping the color component unchanged.

[0094] The background image processed in this way can enhance the contrast and avoid color distortion.

[0095] S33, converting the contrast-enhanced background image back into RGB color space to obtain an enhanced background image.

[0096] S4, denoising the foreground image and performing image enhancement on the denoised foreground image using an improved CLAHE algorithm, wherein the improved CLAHE algorithm adaptively adjusts the value of the enhancement parameter according to the image quality index when performing image enhancement on the denoised foreground images of different frames.

[0097] The foreground image is the primary target for target detection. To prevent the subsequent CLAHE algorithm from amplifying noise, the embodiment of the present invention first performs denoising on the foreground image. Specifically, the embodiment of the present invention can select the Mallat denoising method for denoising, which can be implemented through the following steps:

[0098] S41 , performing discrete wavelet transform (DWT) in row and column directions to decompose the foreground image into sub-bands of different scales, obtaining a low-frequency sub-band and three high-frequency sub-bands.

[0099] The low-frequency subband contains the main information of the foreground image, while the high-frequency subband mainly contains the details and noise of the foreground image. For the foreground image f, the discrete wavelet transform can be expressed as: f→(cA, cH, cV, cD). cA (Approximation coefficients) is the approximation coefficient of the low-frequency subband, cH (Horizontal detail coefficients), cV (Vertical detail coefficients), and cD (Diagonal detail coefficients) are the detail coefficients of the high-frequency subband in the horizontal, vertical, and diagonal directions, respectively.

[0100] This step is implemented through the following steps:

[0101] S411 performs a one-dimensional wavelet transform on the foreground image rows. Specifically, the foreground image is first represented as a two-dimensional matrix f(x, y), where the elements of the two-dimensional matrix are grayscale, x represents the rows of the foreground image, and y represents the columns of the foreground image. Then, a one-dimensional discrete wavelet transform is performed on each row of the foreground image to decompose it into low-frequency components and high-frequency components.

[0102] For the low-frequency components (row direction), each row of pixels is filtered and downsampled using a wavelet low-pass filter, with the result represented as Lr. For the high-frequency components (row direction), each row of pixels is filtered and downsampled using a wavelet high-pass filter, with the result represented as Hr. After this operation, each row of f(x, y) is decomposed into two parts: the low-frequency component Lr(x, y) and the high-frequency component Hr(x, y).

[0103] S412 , performing one-dimensional wavelet transform on the foreground image columns, specifically: performing one-dimensional wavelet transform on the low-frequency components and high-frequency components of the foreground image after the one-dimensional wavelet transform in the row direction, respectively.

[0104] Among them, the specific method of one-dimensional wavelet transform in the column direction of low-frequency components is:

[0105] Perform a column-wise discrete wavelet transform on Lr(x, y) to obtain two new subbands, Lc(x, y) and Hc(x, y). Lc(x, y) is the low-frequency portion of the low-frequency component in the column direction, i.e., the final approximate coefficient cA of the low-frequency subband. Hc(x, y) is the high-frequency portion of the low-frequency component in the column direction, i.e., the horizontal detail coefficient cH.

[0106] The specific method of one-dimensional wavelet transform in the column direction of high-frequency components is:

[0107] Perform a column-wise discrete wavelet transform on Hr(x, y) to obtain two new subbands, Lc′(x, y) and Hc′(x, y). Lc′(x, y) is the low-frequency portion of the high-frequency component in the column direction, i.e., the vertical detail coefficient cV. Hc′(x, y) is the high-frequency portion of the high-frequency component in the column direction, i.e., the diagonal detail coefficient cD.

[0108] Through the above steps, f(x, y) is decomposed into the following four subbands. The low-frequency subband is obtained by L(x, y), which represents the approximate information of the foreground image and contains the main structure and overall shape of the foreground image. The horizontal detail subband is obtained by Hc(x, y), which contains the horizontal detail information of the foreground image and reflects the changes in the foreground image in the horizontal direction. The vertical detail subband is obtained by Lc′(x, y), which contains the vertical detail information of the foreground image and reflects the changes in the foreground image in the vertical direction. The diagonal detail subband is obtained by Hc′(x, y), which contains the diagonal detail information of the foreground image and reflects the changes in the foreground image in the diagonal direction.

[0109] S42, performing threshold processing on the detail coefficients of the three high-frequency sub-bands to remove noise therein.

[0110] Specifically, the threshold processing method adopts a hard threshold to retain larger wavelet coefficients, has no impact on the main features of the signal, is simple to calculate, and has a fast execution speed.

[0111] This step is expressed as:

[0112]

[0113] Among them, d is the detail coefficient, which are cH, cV, cD respectively. is the detail coefficient after threshold processing, λ is the threshold; α is the standard deviation of the noise, which can be estimated by cD, the formula is Median is the median; N is the length of the signal, which is equal to the number of pixels in the foreground image, that is, the width of the foreground image multiplied by the height.

[0114] S43 , performing inverse discrete wavelet transform (IDWT) on the three high-frequency sub-bands and the low-frequency sub-band after the threshold processing, and reconstructing the denoised foreground image.

[0115] Furthermore, after denoising, S4 uses an improved CLAHE algorithm to enhance the denoised foreground images of different frames. The improved CLAHE algorithm adaptively adjusts the enhancement parameters based on image quality indicators. Image quality indicators are important objective evaluation indicators that reflect the difficulty of object detection tasks.

[0116] In one specific embodiment, the image quality indicator includes at least one of grayscale variance, Brenner function value, and gradient energy. Grayscale variance is an indicator that measures the degree of dispersion of the grayscale value distribution of the denoised foreground image and can reflect the contrast of the denoised foreground image. The Brenner function is mainly used to measure image clarity and reflects the image clarity by calculating the sum of the squares of the grayscale value differences between adjacent pixels. Gradient energy measures the intensity of edges and textures in the denoised foreground image and is the sum of the squares of the gradient values ​​of each pixel in the denoised foreground image.

[0117] Specifically, the grayscale variance σ of the denoised foreground image is 2 Calculated by the following formula (2):

[0118]

[0119] In formula (2), I(i, j) represents the grayscale value of the denoised foreground image at position (i, j); μ is the average grayscale value of the denoised foreground image, and the calculation formula is: H and W are the height and width of the denoised foreground image, respectively.

[0120] The Brenner function value B of the denoised foreground image is calculated using the following formula (3):

[0121]

[0122] The gradient energy E of the denoised foreground image is calculated using the following formula (4):

[0123]

[0124] In formula (4), G x (i,j) and G y (i, j) represents the horizontal gradient value and vertical gradient value of the denoised foreground image at position (i, j), respectively.

[0125] In a specific embodiment, when the image quality index includes grayscale variance, Brenner function value, and gradient energy, the image quality index of the denoised foreground image is calculated by the following steps:

[0126] S4-1, normalize the grayscale variance, Brenner function value and gradient energy.

[0127] Specifically, during normalization, the minimum and maximum values ​​of each image quality index are found for normalization (the grayscale variance, Brenner function value, and gradient energy of the foreground image after all original images are denoised are calculated, and the minimum and maximum values ​​of each image quality index are determined from all grayscale variances, Brenner function values, and gradient energies), and then normalized using the following formula:

[0128] normalize=(value-min_value) / (max_value-min_value);

[0129] Among them, normalize represents the normalization result of a certain image quality indicator, value represents the value of the image quality indicator, min_value represents the minimum value of the image quality indicator, and max_value represents the maximum value of the image quality indicator.

[0130] S4-2: Multiply the normalized grayscale variance, Brenner function value, and gradient energy by their corresponding weights to obtain the image quality index of the denoised foreground image.

[0131] Specifically, this step is expressed by the formula:

[0132] score=(σ 2 _weight×normalize_σ 2 )+(B_weight×normalize_B)+

[0133] (E_weight×normalize_E);

[0134] Among them, σ 2 _weight represents the weight of grayscale variance, normalize_σ 2 Represents the normalized result of grayscale variance, B_weight represents the weight of the Brenner function value, normalize_B represents the normalized result of the Brenner function value, E_weight represents the weight of the gradient energy, and normalize_E represents the normalized result of the gradient energy.

[0135] In a specific embodiment, a particle swarm optimization algorithm is used to determine the values ​​of enhancement parameters of the foreground images after denoising of different frames.

[0136] Specifically, because the original images of adjacent frames in the coal mine excavation working face video are highly similar, the optimal enhancement parameters in the improved CLAHE algorithm are also similar. When setting the initial range of the particle swarm optimization algorithm, the result of the optimization of the previous frame of the original image can be used as a reference to narrow the search range and accelerate convergence.

[0137] For example, prev_optimal_params is the optimal enhancement parameters for the previous frame. If prev_optimal_params does not exist (i.e., for the first original frame), the search range is initialized to the global range. If this parameter exists, a smaller search range is defined with it as the center, plus or minus a fixed value, to narrow the search range.

[0138] According to the determined search range, the particle swarm optimization algorithm updates prev_optimal_params to the optimal enhancement parameters of the original image of this frame after finding the optimal enhancement parameters. The improved CLAHE algorithm performs image enhancement on the foreground image after denoising the original image of this frame according to the optimal enhancement parameters.

[0139] In summary, the embodiment of the present invention can adaptively enhance the foreground image according to the image quality index of the foreground image through the improved CLAHE algorithm. The enhanced foreground image has high contrast, low noise, and clear edges, and can be better applied to the target detection task of the underground excavation working face.

[0140] It should be noted that, in order to speed up the processing, the embodiment of the present invention can utilize multi-threading in parallel to perform image enhancement on the foreground image and the background image, and use GPU acceleration.

[0141] S5, fusing the enhanced background image and the enhanced foreground image to obtain a target image corresponding to the original image.

[0142] In a specific embodiment, when S5 is implemented, the enhanced background image and the enhanced foreground image are fused by the following formula (5) to obtain a target image corresponding to the original image:

[0143] I final (i,j)=w(i,j)·I f (i,j)+(1-w(i,j))·I b (i,j)(5);

[0144] In formula (5), I final (i, j) represents the grayscale value of the target image at position (i, j), I f (i, j) represents the grayscale value of the enhanced foreground image at position (i, j), I b(i, j) represents the grayscale value of the enhanced background image at position (i, j); w(i, j) represents the weight at position (i, j), and its value range is between 0 and 1.

[0145] By fusing the enhanced background image and the enhanced foreground image in this way, a smooth fusion is achieved in the transition region between the foreground image and the background image. The weight of the transition region is generated using a Gaussian function to ensure smooth fusion.

[0146] S6, input the target image into the pre-trained improved YOLOv8n model for key target detection.

[0147] The improved YOLOv8n model described in the embodiment of the present invention uses the DWR module to construct the DWR-improved C2f module, and uses the DWR-improved C2f module to replace the original C2f module of the neck network in the YOLOv8n model. The SEAM module is used to replace the second Conv convolution module in the detection head of the YOLOv8n model to build a SEAM-improved detection head, and MPDIoU is used to replace the original CIoU of the YOLOv8n model as the loss function. That is, the improved YOLOv8n model described in the embodiment of the present invention includes a backbone network, a neck network and a SEAM (Separated and Enhancement Attention Module) improved detection head connected in sequence. The neck network includes the DWR-improved C2f module, and the DWR-improved C2f module uses the DWR module to replace the Bottleneck residual module in the C2f module; the SEAM-improved detection head uses the SEAM module to replace the second Conv convolution module of the detection head; the loss function of the improved YOLOv8n model is MPDIoU, where:

[0148]

[0149] MPDIoU loss = 1 - MPDIoU (8);

[0150] In formula (6) to formula (8), B pred and B gt Denote the predicted box and the true annotation box respectively, w and h are the width and height of the image respectively, d1 and d2 are the distances between the upper left corner and the lower right corner of the predicted box and the true annotation box respectively.

[0151] The three improvements of the improved YOLOv8n model are described in detail below:

[0152] Improvement point 1: The C2f module contains many Bottleneck residual modules. Connecting these Bottleneck residual modules in series can realize the extraction and fusion of features of different scales. However, the Bottleneck residual module causes the network to superimpose a large amount of information at the same position, that is, the high-frequency position, and generates many redundant features in the process of obtaining key features, which may cause the loss of key information. Multi-target detection of the excavation working face is often carried out under harsh working conditions. The loss of information will seriously reduce the accuracy of target detection. Therefore, the embodiment of the present invention uses the DWR (Dilation-wise Residual) module to replace the original Bottleneck residual module to obtain the DWR-improved C2f module, such as Figure 2 As shown in Figure 2. The DWR module is designed in a residual manner. Within the residual, a two-step method is used to effectively extract multi-scale context information, and then the feature maps generated by the multi-scale receptive fields are fused. The DWR module is applied to the deep layer of the network, and the multi-branch structure is used to adapt to the requirements of receptive fields of different sizes in a layer. For each branch, as shown in Figure 2. Figure 3 As shown, the DWR module first uses a regular 3x3 convolution combined with a batch normalization (BN) layer and a ReLU layer to complete feature extraction, obtaining regional residual features (RR) of different region sizes. The RR features are then divided into three groups, and morphological filtering is performed on the regional features of different sizes using dilated depthwise convolutions with multiple dilation rates. This is called semantic residual (SR). By increasing the flexibility of the dilation rate, the network is more adaptable to objects of different scales in terms of receptive field size and density. BN is then performed on the feature map, and all feature maps are merged using point-by-point convolution to generate a final residual corresponding to the input feature map. Finally, the final residual is added to the input feature map to construct a stronger and more comprehensive feature representation. In object detection tasks, especially for multi-scale objects, the DWR module can effectively increase the network's receptive field range, better capture the details and contextual information of the target, and thus improve detection accuracy and robustness.

[0153] Improvement point 2: Due to the small space of the tunneling scene, the large number of people, the large size of the tunneling machine, and the fixed camera angle, there will be serious occlusion when detecting the people and tools in the tunneling scene, which will greatly increase the difficulty of detection and reduce the accuracy of detection. Therefore, the embodiment of the present invention introduces the SEAM module into the detection head to construct a SEAM-improved detection head with occlusion perception, such as Figure 4 As shown in the figure, the SEAM improved detection head uses the SEAM module to replace the second Conv (convolution module) of the original detection head of the YOLOv8n model. Figure 5As shown in the figure, the first part of the SEAM module is a depthwise separable convolution with residual connections. Depthwise separable convolution operates depthwise, meaning that the convolution is channel-wise. While depthwise separable convolution can learn the importance of different channels and reduce the number of parameters, it ignores the information relationship between channels. To compensate for this loss, the outputs of the convolutions at different depths are then combined through a 1x1 point-by-point convolution. A two-layer fully connected network (i.e., two fully connected layers) is then used to fuse the information from each channel, allowing the network to strengthen the connections between all channels. The hope is that this model can compensate for the aforementioned loss in occluded scenarios by leveraging the relationship between occluded and unoccluded objects learned in the previous step. The outputs learned by the fully connected layers are then subjected to an exponential function, expanding the range from [0, 1] to [1, e]. This exponential normalization provides a monotonic mapping, making the result more tolerant to positional errors. Finally, the output of the SEAM module is multiplied by the original features as attention, enabling the model to more effectively handle alignment errors, local aliasing, and feature loss caused by occlusion. The CSMM module in the SEAM module uses different patches to obtain multi-scale features and uses depthwise separable convolution to learn the correlation between spatial dimensions and channels.

[0154] Improvement 3: MPDIoU replaces the CIoU used in the YOLOv8n model. Most existing bounding box regression loss functions cannot optimize when the predicted box and the ground-truth bounding box have the same aspect ratio but completely different width and height values. MPDIoU incorporates all relevant factors considered in existing loss functions, such as overlapping or non-overlapping area, center point distance, and width and height deviation, while simplifying the calculation process. It directly minimizes the distance between the top left and bottom right corners of the predicted bounding box and the ground-truth bounding box, which can improve the training effect of bounding box regression, increase convergence speed, and improve regression accuracy.

[0155] In summary, the improved YOLOv8n model proposed in the embodiment of the present invention has a larger receptive field, can better extract contextual information, and can enhance the detection accuracy of multi-target occlusion, while also accelerating the convergence speed and regression accuracy.

[0156] It should be noted that before executing step S6, the improved YOLOv8n model needs to be trained first. For the specific implementation method of training the improved YOLOv8n model, please refer to the existing model training method, which will not be elaborated in detail in the embodiment of the present invention.

[0157] In order to verify the effectiveness of the improved CLAHE algorithm and the improved YOLOv8n model proposed in the embodiment of the present invention, some images are selected for comparative experiments, such as Figure 6As shown in Figure 1, (a) and (b) are schematic diagrams of the original image and the result of adaptively adjusting the enhancement parameters using the improved CLAHE algorithm. The comparison of the enhanced image and the original image indicators is shown in Table 1 below:

[0158] Table 1

[0159]

[0160] As shown in Table 1, the three function values ​​of the enhanced image are much higher than those of the original image, indicating that the improved CLAHE algorithm proposed in this embodiment of the present invention can effectively solve the problems of unclear images, low contrast, and blurred edge information caused by weak light and dusty fog in coal mines, achieving the effect of image enhancement and making the model's reasoning results more accurate.

[0161] like Figure 7 As shown in , it is a comparison chart of target detection effects of different target detection models. Figure 7 (a) and Figure 7 As can be seen from the second and third rows of images in (b), the improved YOLOv8n model can detect occluded targets without missing any detections and has a higher confidence level for the detected targets. However, it can also detect the targets in the first row with extreme dust and fog. Figure 7 In Figure (c), after adding adaptive image enhancement parameters to the improved CLAHE algorithm, not only does the image become clearer and the targets in the first row of images are detected, but the confidence level of targets in the second and third rows of images is also further improved. This demonstrates the effectiveness of each component of the method proposed in this embodiment of the present invention.

[0162] To verify the superiority of the method proposed in this embodiment of the present invention, the accuracy of the method is compared with that of the YOLOv5s, YOLOv7, YOLOv8n, and YOLOv8s models. The mAP is shown in Table 2 below:

[0163] Table 2

[0164]

[0165] mAP (Mean Average Precision) comprehensively reflects the detection performance of a model across different categories and is an important indicator for measuring the performance of object detection models. As shown in Table 2, the method proposed in this embodiment outperforms other popular object detection algorithms, demonstrating the overall effectiveness of the proposed method.

[0166] In summary, the method provided by the embodiment of the present invention has the following characteristics:

[0167] 1. The particle swarm optimization algorithm is used to adaptively determine the enhancement parameters of the improved CLAHE algorithm, which can improve the enhancement effect of each original image and avoid the problems caused by over-enhancement and under-enhancement.

[0168] 2. Image quality index is an important objective evaluation indicator that reflects the difficulty of the target detection task. Using image quality index as an indicator for finding the optimal enhancement parameters of the improved CLAHE algorithm can obtain images that are comprehensively optimized in terms of brightness distribution, edge clarity, image texture, etc., which are more suitable for subsequent key target detection tasks.

[0169] 3. Because the original images of adjacent frames usually do not change much, the optimal enhancement parameters of the previous frame are referenced during the parameter optimization process, which narrows the search range of the current frame and improves the optimization efficiency and stability.

[0170] 4. The improved YOLOv8n model proposed in this embodiment utilizes the DWR module to construct a DWR-modified C2f module, replacing the original C2f module in the neck network. In key object detection tasks, especially for multi-scale objects, the DWR-modified C2f module can effectively increase the network's receptive field, better capturing object details and contextual information, thereby improving detection accuracy and robustness.

[0171] 5. The SEAM module is used to replace the second Conv convolution module in the YOLOv8n model detection head to build a SEAM-improved detection head, so that the improved YOLOv8n model can more effectively handle problems such as alignment errors, local aliasing, and feature missing caused by occlusion.

[0172] 6. Using MPDIoU instead of CIoU in the original YOLOv8n model as the loss function directly minimizes the distance between the upper left and lower right corners of the predicted bounding box and the actual annotated bounding box, which can improve the training effect of bounding box regression and increase the convergence speed and regression accuracy.

[0173] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting key targets in a coal mine excavation working face, characterized in that: include: S1, after obtaining the video collected from the coal mine excavation working face, the video is frame-processed to obtain original images frame by frame; S2, using the color-based K-means clustering algorithm to segment the original image into foreground and background, and obtain the foreground image and background image of the original image; S3, performing image enhancement on the background image using a CLAHE algorithm, wherein the value of an enhancement parameter of the CLAHE algorithm is a fixed value when performing image enhancement on background images of different frames; S4, denoising the foreground image and performing image enhancement on the denoised foreground image using an improved CLAHE algorithm, wherein the improved CLAHE algorithm adaptively adjusts the value of the enhancement parameter according to the image quality index when performing image enhancement on the denoised foreground images of different frames; and using a particle swarm optimization algorithm to determine the value of the enhancement parameter of the denoised foreground images of different frames; S5, fusing the enhanced background image and the enhanced foreground image to obtain a target image corresponding to the original image; S6, input the target image into a pre-trained improved YOLOv8n model for key target detection; the improved YOLOv8n model includes a backbone network, a neck network and a SEAM-improved detection head connected in sequence; the neck network includes a DWR-improved C2f module, and the DWR-improved C2f module uses the DWR module to replace the Bottleneck residual module in the C2f module; the SEAM-improved detection head uses the SEAM module to replace the second Conv convolution module of the detection head.

2. The method for detecting key targets in a coal mine excavation working face according to claim 1, wherein: When S2 is specifically implemented, it includes: S21, converting the original image from RGB color space to Lab color space; S22, randomly select two points C 1 and C 2 as the initial centroid, where the centroid is the point representing the center of the two clusters; S23, for pixels in the original image x i , calculate its distance from the two centroids respectively, and assign it to the cluster to which the nearest centroid belongs according to its distance from the two centroids; S24, calculate the average value of all data points of each cluster and update the centroid to this average value; S25, repeating steps S23 and S24 until the distance between the current centroid and the previous centroid is less than a preset centroid convergence standard value, and determining the current centroid as the target centroid; S26, according to the positions and color distributions of the two target centroids, the two clusters are divided into a foreground cluster and a background cluster, and the pixels in the foreground cluster and the background cluster are converted back to the RGB color space to obtain the foreground image and the background image of the original image.

3. The method for detecting key targets in a coal mine excavation working face according to claim 2, wherein: The S23 calculates the pixel x i With one of the centroids distance When , the Euclidean distance is used, which is expressed as: （1）； In formula (1), x i =[ x i1 , x i2 , x i3 ], x i1 、 x i2 and x i3 They are x i The values ​​of L, a, and b in the Lab color space; C j =[ C j1 , C j2 , C j3 ], indicating the j A centroid.

4. The method for detecting key targets in a coal mine excavation working face according to claim 1, wherein: The enhancement parameters of the CLAHE algorithm include the grid size and the contrast limiting factor. When S3 is specifically implemented, it includes: S31, converting the background image from the RGB color space to the Lab color space, wherein the Lab color space of each pixel includes a brightness component and two color components; S32, configuring fixed values ​​for the grid size and contrast limiting factor in the CLAHE algorithm, and performing contrast enhancement on the brightness component of each pixel in the background image using the configured CLAHE algorithm while keeping the color component unchanged; S33, converting the contrast-enhanced background image back into RGB color space to obtain an enhanced background image.

5. The method for detecting key targets in a coal mine excavation working face according to claim 1, wherein: When performing denoising on the foreground image, S4 includes: S41, performing discrete wavelet transform in the row and column directions to decompose the foreground image into sub-bands of different scales, obtaining a low-frequency sub-band and three high-frequency sub-bands; S42, performing threshold processing on the detail coefficients of the three high-frequency sub-bands to remove noise therein; S43, performing inverse discrete wavelet transform on the three high-frequency sub-bands and the low-frequency sub-band after the threshold processing, and reconstructing the denoised foreground image.

6. The method for detecting key targets in a coal mine excavation working face according to claim 1, wherein: The image quality index includes at least one of grayscale variance, Brenner function value and gradient energy; Grayscale variance of the denoised foreground image Calculated by formula (2): （2）； In formula (2), I ( i , j ) indicates the foreground image after denoising at position ( i , j ) at the gray value; μ is the average grayscale value of the foreground image after denoising, and the calculation formula is ; H and W are the height and width of the denoised foreground image respectively; The Brenner function value of the denoised foreground image B Calculated by formula (3): （3）； The gradient energy of the denoised foreground image E Calculated by formula (4): （4）； In formula (4), and Respectively represent the foreground image after denoising at position ( i , j ) at the horizontal and vertical gradient values, , .

7. The method for detecting key targets in a coal mine excavation working face according to claim 1 or 6, characterized in that: When the image quality index includes grayscale variance, Brenner function value, and gradient energy, the image quality index of the denoised foreground image is calculated by the following steps: S4-1, normalize the grayscale variance, Brenner function value and gradient energy; S4-2: Multiply the normalized grayscale variance, Brenner function value, and gradient energy by their corresponding weights to obtain the image quality index of the denoised foreground image.

8. The method for detecting key targets in a coal mine excavation working face according to claim 1, wherein: In the specific implementation of S5, the enhanced background image and the enhanced foreground image are fused by formula (5) to obtain the target image corresponding to the original image: （5）； In formula (5), Indicates that the target image is at position ( i , j ), Indicates that the enhanced foreground image is at position ( i , j ), Indicates that the enhanced background image is at position ( i , j ) at the gray value; Indicates the position ( i , j ), the value range is between 0 and 1.

9. The method for detecting key targets in a coal mine excavation working face according to claim 1, wherein: The loss function of the improved YOLOv8n model is MPDIoU, where: （6）； （7）； （8）； In formula (6) to formula (8), and Represent the predicted box and the true annotation box respectively, w and h are the width and height of the image, d 1 and d 2 are the distances between the upper left corner and lower right corner of the predicted box and the true annotation box, respectively.

Images