Method for detecting abnormity of key part of coal mine elevator

Abstract

The invention discloses an anomaly detection method for key parts of a coal mine hoist, which comprises the following steps of: recording operation videos of five key parts of the coal mine hoist and extracting images to construct a picture set; and carrying out brightness enhancement on the images by utilizing a trained image enhancement model, and making a training and testing data set. And training an anomaly detection model based on the convolutional neural network, wherein the model can identify the state of the key part of the elevator. In practical application, the key part of the elevator is monitored in real time by using the enhanced image and the anomaly detection model. By comparing the detected position coordinate of the key part with the center coordinate under the normal condition, the abnormal state can be found in time. Once abnormality is detected, the system stops the operation of the elevator immediately, and gives an audible and visual alarm to prompt a specific abnormal position. According to the method, the detection precision is remarkably improved, the false detection rate is reduced, the operation and maintenance cost of the underground coal mine elevator is greatly reduced, and meanwhile the overhaul and maintenance efficiency and safety are improved.

Description

Technical Field

[0001] The invention belongs to the technical field of coal mine safety, and in particular relates to a method for detecting abnormalities at key parts of a coal mine hoist. Background Art

[0002] As a core component of the coal production system, the coal mine hoist is responsible for transporting coal and ore mined underground to the surface, which is crucial to ensuring the efficient operation of the coal mine. Its operating status directly affects the production efficiency and safety of the coal mine. Once the hoist fails, it will not only significantly reduce production efficiency, but may also cause serious safety accidents and threaten the lives of miners. Traditional preventive maintenance methods usually rely on regular disassembly and inspection, which is not only time-consuming and labor-intensive, but frequent disassembly and assembly may also cause new mechanical damage, thereby increasing maintenance costs.

[0003] At present, the detection methods for key components of coal mine hoists mainly include contact and non-contact methods. The former covers means such as vibration analysis and temperature monitoring. These methods have high measurement accuracy and can meet the monitoring needs of various equipment, but they often need to be carried out when the equipment is in operation, which may interfere with the normal operation of the equipment, and the cost of installing and maintaining vibration sensors is high. The latter includes video monitoring and laser scanning technology. The advantages of these methods are that they can reduce direct interference with the operation of the equipment. With the advancement of image processing technology, methods for real-time monitoring of key parts of hoists using lasers or video images have begun to attract attention; while the existing methods for real-time monitoring of key parts of hoists using lasers or video images are usually only for the detection of a specific part, which has certain limitations. In addition, the current technical solutions rely on the deployment of a large number of sensors to achieve monitoring of the hoist, which not only increases the cost of installation and maintenance, but also brings potential safety hazards in the connection method. In addition, the use of numerous sensors significantly increases the complexity of data processing, which in turn affects the overall efficiency and performance of the system. Summary of the invention

[0004] The purpose of the embodiment of the present invention is to provide a method for detecting abnormalities in key parts of a coal mine hoist, so as to realize real-time monitoring without relying on a large number of sensors; through the synchronous detection of multiple key parts, adaptive control of the operating state of the hoist is realized, thereby reducing the operation and maintenance costs and the safety risks of manual inspections, and preventing accidents.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is a method for detecting abnormalities in key parts of a coal mine hoist, which specifically includes the following steps:

[0006] S1, constructing a sample set of images of key parts of coal mine hoists;

[0007] S2, establishing an image enhancement model to perform image enhancement on a sample set of images of key parts of the elevator to obtain an enhanced image set;

[0008] S3, establish a label value table of key parts of coal mine hoist and mark the category and center point coordinates of each image in the enhanced image set, construct all images and label data into a coal mine hoist key part anomaly detection image dataset, and divide the coal mine hoist key part anomaly detection image dataset into a training set and a test set;

[0009] S4. Construct an abnormality detection model for key parts of coal mine hoists;

[0010] S5. Train the anomaly detection model for key parts of coal mine hoists;

[0011] S6. Carry out real-time online detection of key parts of coal mine hoists.

[0012] Furthermore, the specific process of S1 is: installing cameras at key positions of the coal mine hoist, recording uninterrupted videos for 24 hours a day, capturing a picture every 1 second of the captured video, and selecting N pictures of each key position to form an image sample set ImageSet.

[0013] Furthermore, the key parts of the coal mine hoist include a coal mine hoist sheave, a roller, an upper wellhead safety door, a lower wellhead safety door and a tail rope.

[0014] Furthermore, the specific process of S2 is as follows:

[0015] S201, construct an image enhancement model, wherein the image enhancement model includes a U-Net generator module and a discriminator module:

[0016] S2011. The CA attention mechanism is introduced in the U-Net generator module. The U-Net generator consists of 8 convolutional layers, each of which consists of two 3×3 convolutional layers, a BN layer and a LeakReLU layer. In the upsampling stage, the standard deconvolution layer is replaced by a bilinear upsampling layer plus a convolutional layer.

[0017] S2012, the discriminator uses a global-local dual discriminator to guide global and local information. Both the global and local discriminators use PatchGAN for image authenticity identification, and VGG Net is used to calculate the self-feature preservation loss;

[0018] S202, after the image enhancement model is constructed, a part of images are extracted from the image sample set constructed in S1 to form a training set, and the training set generator and the discriminator are used to perform adversarial training to achieve global optimization;

[0019] S203, input the image sample set constructed in S1 into the trained image enhancement model to perform brightness enhancement, and obtain an enhanced image set.

[0020] Furthermore, the training process of the generator and the discriminator in S202 is specifically as follows:

[0021] S2021. Initialize the weights W and bias B of the generator G and discriminator D networks;

[0022] S2022, extracting n samples from the training set, and the generator generates n samples according to a predefined noise distribution;

[0023] S2023, fix the generator G and train the discriminator D; the training process of the discriminator D is as follows: the local discriminator randomly cuts 5 patches from the input generated image and the real image each time, and lets the discriminator perform identification, and learns to distinguish by comparing the local areas of the real image and the generated image. After each identification, the parameters of the discriminator are adjusted according to the result of the loss function;

[0024] S2024, after cyclically updating the discriminator D k times, update the weight, bias, and size and position of the attention map of the generator G once, normalize the illumination channel I of the input RGB image to [0, 1], use 1-I as the self-regularized attention map, adjust the size of the attention map to fit the size of each intermediate feature map, and multiply it with all the intermediate feature maps and the output image to obtain a generated map;

[0025] S2025. After multiple update iterations, until the discriminator D cannot distinguish whether the image comes from the real training data set or is generated by the generator G, the loss function reaches the minimum value and the training is completed.

[0026] Furthermore, the loss function includes the discriminator loss Generator Loss and the self-feature preservation loss L SFP (I L ):

[0027] Global discriminator loss:

[0028] Local Discriminator Loss:

[0029] Global generator loss:

[0030] Local generator loss:

[0031] Among them, D represents the discriminator, G represents the generator; D Ra is used to measure the two input xr 、x f The difference between the functions, C represents the discriminator network, σ represents the sigmoid function, and the sigmoid function is replaced by the least squares GAN loss. r and x f To sample from real pictures and generated pictures respectively; and Respectively represent in x f Obey the generated image distribution And the real picture distribution In the case of , the expected value of the corresponding expression; Represent the distribution of randomly selected small blocks in real pictures and generated pictures respectively,

[0032] From feature preservation loss:

[0033] Among them, I L represents the input low-light image, G(I L ) represents the enhanced output of the generator; φ a,b represents the feature map extracted by VGGNet of the discriminator; a represents the ath maximum pooling layer, b represents the bth convolutional layer after the ath maximum pooling layer; W a,b , H a,b is the width and height of the extracted feature map; a and b represent each pixel point of the feature map traversed from the width and height respectively, where a traverses from 1 to W a,b , b traverses from 1 to H a,b .

[0034] Furthermore, the anomaly detection model for key parts of coal mine hoists in S4 includes a preliminary feature extraction network, an enhanced feature extraction network and a classification regression layer;

[0035] The preliminary feature extraction network includes ten layers, which are the first CBS module layer, the second CBS module layer, the third C2f_ODConv module layer, the fourth CBS module layer, the fifth C2f_ODConv module layer, the sixth CBS module layer, the seventh C2f_ODConv module layer, the eighth CBS module layer, the ninth C2f_ODConv module layer, and the tenth SPPF module layer. The C2f_ODConv module layers are all C2f modules that introduce ODConv.

[0036] The enhanced feature extraction network includes sixteen layers, which are, in order, a first CBS module layer, a second upsampling module layer, a third Concat module layer, a fourth C2f_ODConv module layer, a fifth upsampling module layer, a sixth Concat module layer, a seventh C2f_ODConv module layer, an eighth GAM attention module layer, a ninth CBS module layer, a tenth Concat module layer, an eleventh C2f_ODConv module layer, a twelfth GAM attention module layer, a thirteenth CBS module layer, a fourteenth Concat module layer, a fifteenth C2f_ODConv module layer, and a sixteenth GAM attention module layer; the GAM attention module is used to enhance the representation of important information in the feature map, and the preliminary features of the three dimensions output by the preliminary feature extraction network are processed by the enhanced feature extraction network to obtain and output enhanced features of the three dimensions;

[0037] The classification and regression layer includes three Detect layers, which perform final processing by strengthening the features output by the feature extraction network to classify and locate the target.

[0038] Furthermore, the specific process of S5 is as follows:

[0039] S501, initialize all unknown matrix parameters W of each layer in the anomaly detection model of key parts of coal mine hoists [i] ,B [i] , i∈(1,3,5,6) is a random value, i represents the index of the layer, set the number of iterations to S, the learning rate to LR, and set the input A j is the first sample A in the training data set j =A0, where subscript j represents the jth image sample, and A0 represents the first image sample in the training data set;

[0040] S502, inputting the key part image sample into the key part anomaly detection model of the coal mine hoist into the layer A [0] Input image sample A j ;

[0041] S503, gradually calculating the estimated value of the key parts detection of the coal mine hoist through the key parts abnormality detection model

[0042] S504: Use input image sample A j The corresponding true value Y and estimated value Calculate and output the cross entropy loss function

[0043] S505, calculate each parameter W in each layer of the convolutional neural network [i] and B [i] The change of ΔW[i] and ΔB [i] , where i∈(1,3,5,6);

[0044] S506, use LR, ΔW [i] and ΔB [i] Update all parameters W in the convolutional neural network with the value of [i] and B [i] The value of W [i] (S+1)=W [i] -LR*ΔW [i] , B [i] (S+1)=B [i] -LR*ΔB [i] , where i∈(1,3,5,6);

[0045] S507: Determine whether it is the last sample in the training data set. If it is not the last sample, select the next sample as A. j , and jump to S502, if it is the last sample, jump to S508;

[0046] S508: Determine whether the iteration is finished. If not, reselect the first sample of the training data set as A. j , reduce the number of iterations S, jump to S502, if the iteration is completed, jump to S509;

[0047] S509, set the detection accuracy threshold T of the key parts detection model of the coal mine hoist, input all the image samples in the test data set into the convolutional neural network in turn, and output the estimated value And compare it with the true value Y to calculate the accuracy of the key parts detection of the coal mine hoist in Indicates the number of samples in the test data set whose estimated value is the same as the true value, ∑Num(Y) indicates the number of all samples in the test data set, if ACC<T, go to S501, if ACC≥T, go to S510;

[0048] S510, all parameters W of the convolutional neural network [i] and B [i] Fixed and saved, where i∈(1,3,5,6), the coal mine hoist key parts position coordinate detection model training is completed.

[0049] Furthermore, the specific process of S6 is as follows:

[0050] S601. Under normal circumstances, the position coordinates of the sheave, drum, upper wellhead safety door, lower wellhead safety door and tail rope are: (X Normal_wheel ,Y Normal_wheel )、(XNormal_roller ,Y Normal_roller )、(X Normal_upside ,Y Normal_upside )、(X Normal_downside ,Y Normal_downside )、(X Normal_rope ,Y Normal_rope );

[0051] S602, with the normal value as the center, the coordinate thresholds of the center points of the crown wheel, the drum, the upper wellhead safety door, the lower wellhead safety door and the tail rope are set to be δ wheel (δ wheel ≥0)、δ roller (δ roller ≥0)、δ upside (δ upside ≥0)、δ roller (δ roller ≥0) and δ rope (δ rope ≥0);

[0052] S603, obtaining real-time video of the sheave, roller, upper wellhead safety door, lower wellhead safety door and tail rope captured by the coal mine camera, reading each frame of the video in sequence, and inputting the real-time image into the trained image enhancement model to enhance the image brightness;

[0053] S604, the enhanced real-time image is transmitted to the detection model for detection, and the categories of the five key parts of the elevator and the coordinates of their center points are marked in the video: (X Centre_roller ,Y Centre_roller )、(X Centre_upside ,Y Centre_upside )、(X Centre_downside ,Y Centre_downside )、(X Centre_rope ,Y Centre_rope );

[0054] S605, compare the position coordinates output by the detection model with the coordinates of the center of the sheave, drum, upper wellhead safety door, lower wellhead safety door and tail rope under normal conditions and calculate their average difference:

[0055] Sky Wheel: If |f wheel |>δ wheel , the detection result of the sky wheel is judged to be "abnormal". If |f wheel |≤δ wheel , the test result of the sky wheel is determined to be "normal";

[0056] roller: If |f roller |>δ roller, the drum test result is judged to be "abnormal", if |f roller |≤δ roller , the test result of the drum is determined to be "normal";

[0057] Upper wellhead safety door: If |f upside |>δ upside , then the detection result of the upper wellhead safety door is judged to be "abnormal". If |f upside |≤δ upside , the detection result of the upper wellhead safety door is determined to be "normal";

[0058] Downhole safety door: If |f downside |>δ downside , then the detection result of the wellhead safety door is judged to be "abnormal". If |f downside |≤δ downside , the detection result of the wellhead safety door is determined to be "normal";

[0059] Tail rope: If |f rope |>δ rope , the tail rope detection result is judged to be "abnormal", if |f rope |≤δ rope , the detection result of the tail rope is judged to be "normal".

[0060] Compared with the prior art, the beneficial effects of the present invention include:

[0061] The present invention is aimed at the underground environment of coal mines, and aims to automatically identify and adaptively control abnormal conditions of key components of the hoist (including drums, sheaves, tail ropes, and upper and lower wellhead safety doors) through image and video detection technology. This method reduces the reliance on traditional sensors, simplifies the equipment configuration and installation process, and thus effectively reduces the cost of acquisition, installation, and operation. With the objectivity, efficiency, and stability of image and video technology, the subjective errors caused by manual inspections can be significantly reduced, and the detection process can be unmanned and intelligent. Without introducing additional hardware, the system can continuously monitor the aforementioned key components. Once an abnormality is detected, the hoist operation is immediately terminated and an alarm is triggered, thereby enhancing operational safety, accelerating the troubleshooting process, improving overall production efficiency, and effectively avoiding safety risks in manual inspections.

[0062] The present invention effectively improves the detection accuracy of the detection model by performing brightness enhancement processing on the video image. The detection model is based on YOLOv8, and integrates the GAM attention mechanism and the ODConv module to enhance the model's capabilities in feature extraction and feature fusion. Compared with the original YOLOv8 model, the improved model of the present invention achieves 6.5%, 15.4% and 17% performance gains in accuracy, mean average precision (mAP) and frames per second (FPS), respectively. The present invention can accurately identify the abnormal state of key components of the hoist (including but not limited to drums, sheaves, tail ropes and upper and lower wellhead safety doors), thereby significantly reducing the possibility of potential safety accidents. Therefore, the present invention not only improves the reliability and efficiency of detection, but also greatly promotes the safety and production efficiency of underground operations in coal mines, and reduces the uncertainty and safety hazards caused by manual inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 It is a flowchart of the overall process of an embodiment of the present invention;

[0065] Figure 2 is a flow chart of making a sample set of images of key parts of a hoist according to an embodiment of the present invention;

[0066] Figure 3 is a flow chart of performing image enhancement on a sample set of images of key parts of a hoist according to an embodiment of the present invention;

[0067] Figure 4 is a flow chart of making a training data set and a test data set for key parts of a hoist according to an embodiment of the present invention;

[0068] Figure 5 It is a structural block diagram of an abnormal state detection model for key parts of a coal mine hoist according to an embodiment of the present invention;

[0069] Figure 6 is a flow chart of an embodiment of the present invention for training a coal mine hoist key part abnormal state detection model;

[0070] Figure 7 This is a workflow diagram of the phase of real-time detection of abnormalities in key parts of a coal mine hoist according to an embodiment of the present invention;

[0071] Figure 8 This is an image brightness enhancement effect diagram of an embodiment of the present invention;

[0072] Fig. 9 It is a schematic diagram of the real-time abnormal detection result of an implementation manner of the present invention. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] like Figure 1 The main process of the method for detecting abnormalities in key parts of a coal mine hoist in this embodiment includes six parts: acquiring images, image enhancement, building a data set, building a detection model, training the detection model, and real-time detection of abnormalities.

[0075] In some specific implementations, the purpose of acquiring images is to construct a sample set of images of key parts of the elevator. The specific process is as follows: Figure 2 :First, install cameras at the five key parts of the coal mine hoist, namely the sheave, drum, upper wellhead safety door, lower wellhead safety door and tail rope, and record 24-hour uninterrupted video, recorded as Video wheel 、Video roller 、Video upside 、Video downside and Video rope ; For each video, capture an image every 1s, and select N images of five key parts from the images to form an image set ImageSet.

[0076] In some specific implementations, after collecting the image, it is necessary to perform image enhancement on the sample set of images of key parts of the elevator. The specific process is as follows: Figure 3 :

[0077] S201, construct an image enhancement model based on EnlightenGAN, the image enhancement model includes a U-Net generator module and a discriminator module, and the construction process is as follows:

[0078] S2011, the CA attention mechanism is introduced in the U-Net generator. The U-Net generator consists of 8 convolutional layers, each of which consists of two 3×3 convolutional layers, a BN layer and a LeakReLU layer. In the upsampling stage, in order to reduce the chessboard effect, a bilinear upsampling layer plus a convolutional layer is used to replace the standard deconvolution layer.

[0079] S2012, a global-local dual discriminator is used to guide global and local information. Both the global and local discriminators use PatchGAN for image authenticity identification. VGG Net is used when calculating the self-feature preservation loss to preserve image features.

[0080] S202, after the model is built, a part of images are extracted from the image set ImageSet to form a training set, and the generator and the discriminator are trained against each other to achieve the global optimum. The training steps are as follows:

[0081] S2021. Initialize the weights W and bias B of the generator G and discriminator D networks;

[0082] S2022, extracting n samples from the training set, and the generator generates n samples according to a predefined noise distribution;

[0083] S2023. Fix the generator G and train the discriminator D. The local discriminator randomly cuts 5 patches from the input generated image and the real image each time, and lets the discriminator identify them by comparing the local areas of the real image and the generated image. In each iteration, the discriminator tries to identify which patches are from the real image and which are from the generated image. After each identification, the parameters of the discriminator are adjusted according to the results of the loss function; in this way, the discriminator gradually becomes better at identifying unreal areas in the generated image;

[0084] S2024, after cyclically updating the discriminator D k times, update the weight, bias, and size and position of the attention map of the generator G once, normalize the illumination channel I of the input RGB image to [0, 1], use 1-I as the self-regularized attention map, adjust the size of the attention map to fit the size of each intermediate feature map, and multiply it with all the intermediate feature maps and the output image to obtain a generated map;

[0085] In this embodiment, the intermediate feature map refers to the output result of each intermediate layer in the generator network, that is, the interim result of the input image after being processed by the generator in the process of forming the final output image. In view of the fact that the feature maps of different levels in the U-Net architecture may have different sizes, in order to ensure the effective application of the attention mechanism, the original attention map needs to be adjusted (resized) to a size that matches the intermediate feature map corresponding to each layer. Subsequently, the adjusted attention map is element-by-element multiplied with each intermediate feature map and the final generated image, so as to enhance the performance effect of the dark area. This spatial adaptive adjustment of the attention map is intended to achieve an accurate spatial correspondence between it and each feature map, thereby promoting the authenticity and naturalness of the generated image details.

[0086] S2025. As the training progresses, the generator and the discriminator are constantly updated. The generator learns how to produce more realistic images, while the discriminator learns how to better distinguish between real and fake images. This adversarial training ultimately leads to continuous improvement in the quality of images generated by the generator. After multiple updates and iterations, until the discriminator D cannot distinguish whether the image comes from the real training data set or is generated by the generator G, the loss function reaches the minimum value, indicating that the model training is complete.

[0087] In some specific embodiments, the loss function includes the discriminator loss Generator Loss and the self-feature preservation loss L SFP (I L ):

[0088] Global discriminator loss:

[0089] Local Discriminator Loss:

[0090] Global generator loss:

[0091] Local generator loss:

[0092] Among them, D represents the discriminator, G represents the generator; D Ra is a function that measures the value of two inputs x r 、x f The difference between C represents the discriminator network, σ represents the sigmoid function, and the sigmoid function is replaced by the least squares GAN loss. r and x f Sample from real pictures and generated pictures respectively; and Respectively represent in xf Obey the generated image distribution P fake and the real image distribution P real , the expected value of the corresponding expression.

[0093] From feature preservation loss:

[0094] Among them, I L represents the input low-light image, G(I L ) represents the enhanced output of the generator. a,b represents the feature map extracted from the pre-trained VGG-16 model. a represents the ath maximum pooling layer, b represents the bth convolutional layer after the ath maximum pooling layer; W a,b and H a,b is the width and height of the extracted feature map; a and b represent each pixel point of the feature map traversed from the width and height respectively. Specifically, a traverses from 1 to W i,j , b traverses from 1 to H i,j .

[0095] S203, input the image set ImageSet into the trained EnlightenGAN model for brightness enhancement, and obtain the enhanced image set ImageSet-G.

[0096] In some specific implementations, constructing a data set includes constructing a training data set and a test data set for key parts of a hoist. The specific process is as follows: Figure 4 :

[0097] A label value table of the five key parts of the mine hoist is established, and the categories and center point coordinates of the head sheave, drum, upper shaft safety door, lower shaft safety door and tail rope are annotated for each image in the enhanced image set ImageSet-G. All images and label data together constitute the image dataset Datasets for anomaly detection of key parts of the hoist. The dataset is randomly divided into a training set and a test set in a ratio of 8:2. Both the training set and the test set contain five images of the head sheave, drum, upper shaft safety door, lower shaft safety door and tail rope.

[0098] In some specific implementations, building a detection model is to build a coal mine hoist key part anomaly detection model based on improved YOLOv8, and the coal mine hoist key part anomaly detection model includes a preliminary feature extraction network, an enhanced feature extraction network and a classification regression layer, and its structure is as follows Figure 5 :

[0099] S401, the preliminary feature extraction network includes ten layers, which are the first CBS module layer, the second CBS module layer, the third C2f_ODConv module layer, the fourth CBS module layer, the fifth C2f_ODConv module layer, the sixth CBS module layer, the seventh C2f_ODConv module layer, the eighth CBS module layer, the ninth C2f_ODConv module layer, and the tenth SPPF module layer. The C2f_ODConv module layer is a C2f module that introduces ODConv. ODConv can process multi-scale information and enhance the ability of feature extraction. When the picture is input into the preliminary feature extraction network, preliminary features of three dimensions are extracted and output through the preliminary feature extraction network.

[0100] S402, the enhanced feature extraction network includes sixteen layers, which are the first CBS module layer, the second upsampling module layer, the third Concat module layer, the fourth C2f_ODConv module layer, the fifth upsampling module layer, the sixth Concat module layer, the seventh C2f_ODConv module layer, the eighth GAM attention module layer, the ninth CBS module layer, the tenth Concat module layer, the eleventh C2f_ODConv module layer, the twelfth GAM attention module layer, the thirteenth CBS module layer, the fourteenth Concat module layer, the fifteenth C2f_ODConv module layer, and the sixteenth GAM attention module layer; the GAM attention module is used to enhance the representation of important information in the feature map, and the preliminary features of the three dimensions output by the preliminary feature extraction network are processed by the enhanced feature extraction network to obtain and output enhanced features of the three dimensions;

[0101] S403, the classification and regression layer includes three Detect layers, which perform final processing by strengthening the features output by the feature extraction network to classify and locate the target: the classification task is to determine whether there are abnormalities in the key parts of the coal mine hoist in the image, while the regression task is to accurately locate the location where the abnormality occurs and then obtain the final detection result.

[0102] In this embodiment, the input of the detection model is the elevator key part abnormality detection image dataset Datasets, and the final output detection results include the category labels of the key parts contained in the dataset images and the center point coordinates of the key parts.

[0103] In some specific implementations, the detection model training is specifically to train the abnormality detection model of the key parts of the coal mine hoist based on the improved YOLOv8. The specific process is as follows Figure 6 :

[0104] S501, initialize all unknown matrix parameters W in each layer of the convolutional neural network (improved YOLOv8 coal mine hoist key part anomaly detection model) [i] ,B [i] , i∈(1,3,5,6) is a random value, the number of iterations is set to S, the learning rate is LR, and the input A is set j is the first sample A in the training data set j =A0, where subscript j represents the jth image sample, and A0 represents the first image sample in the training data set;

[0105] S502: Inputting the key part image samples into the convolutional neural network layer A [0] Input image sample A j ;

[0106] S503, gradually calculating through the convolutional neural network to obtain the estimated value of the key parts of the elevator

[0107] S504: Use input image sample A j The corresponding true value Y and estimated value Calculate and output the cross entropy loss function

[0108] i represents the index of the layer, i.e. the i-th layer;

[0109] S505, calculate each parameter W in each layer of the convolutional neural network [i] and B [i] The change value ΔW [i] and ΔB [i] , where i∈(1,3,5,6);

[0110] S506, use LR, ΔW [i] and ΔB [i] Update all parameters W in the convolutional neural network with the value of [i] and B [i] The value of W [i] (S+1)=W [i] -LR*ΔW [i] , B [i] (S+1)=B [i] -LR*ΔB [i] , where i∈(1,3,5,6);

[0111] S507: Determine whether it is the last sample in the training data set. If it is not the last sample, select the next sample as A. j , that is, A j =A j+1, jump to S502, if it is the last sample, jump to S508;

[0112] S508: Determine whether the iteration is finished (S=0). If it is not finished (S≠0), reselect the first sample of the training data set as A. j , that is, A j =A0, number of iterations S=S-1, jump to S502, if the iteration ends (S=0), jump to S509;

[0113] S509, set the detection accuracy threshold T of the key parts detection model of the coal mine hoist, input all the image samples in the test data set into the convolutional neural network in sequence, and output the estimated value And compare it with the true value Y to calculate the accuracy of the key parts detection of the coal mine hoist in Indicates the number of samples in the test data set whose estimated value is the same as the true value, ∑Num(Y) indicates the number of all samples in the test data set, if ACC<T, go to S501, if ACC≥T, go to S510;

[0114] S510, all parameters W of the convolutional neural network [i] and B [i] Fixed and saved, where i∈(1,3,5,6), the key position coordinate detection model training of coal mine hoist is completed;

[0115] In some specific implementations, the real-time abnormality detection is specifically to perform real-time online detection on key parts of the coal mine hoist; the specific process is as follows Figure 7 :

[0116] S601. Under normal circumstances, the position coordinates of the sheave, drum, upper wellhead safety door, lower wellhead safety door and tail rope are: (X Normal_wheel ,Y Normal_wheel )、(X Normal_roller ,Y Normal_roller )、(X Normal_upside ,Y Normal_upside )、(X Normal_downside ,Y Normal_downside )、(X Normal_rope ,Y Normal_rope );

[0117] S602, with the normal value as the center, the coordinate thresholds of the center points of the crown wheel, the drum, the upper wellhead safety door, the lower wellhead safety door and the tail rope are set to be δ wheel (δ wheel ≥0)、δ roller (δ roller ≥0)、δ upside (δ upside≥0)、δ roller (δ roller ≥0) and δ rope (δ rope ≥0);

[0118] S603, obtaining real-time videos of the sheave, drum, upper wellhead safety door, lower wellhead safety door, and tail rope captured by five monitoring cameras in the coal mine, reading each frame of the video in sequence, and inputting the real-time image into the trained EnlightenGAN model for image brightness enhancement;

[0119] S604, the enhanced real-time image is transmitted to the detection model for detection, and the categories of the five key parts of the elevator and the coordinates of their center points are marked in the video: (X Centre_roller ,Y Centre_roller )、(X Centre_upside ,Y Centre_upside )、(X Centre_downside ,Y Centre_downside )、(X Centre_rope ,Y Centre_rope );

[0120] S605, compare the position coordinates output by the detection model with the coordinates of the center of the sheave, drum, upper wellhead safety door, lower wellhead safety door and tail rope under normal conditions and calculate their average difference:

[0121] Sky Wheel: If |f wheel |>δ wheel , the detection result of the sky wheel is judged to be "abnormal". If |f wheel |≤δ wheel , the test result of the sky wheel is determined to be "normal";

[0122] roller: If |f roller |>δ roller , the drum test result is judged to be "abnormal", if |f roller |≤δ roller , the test result of the drum is determined to be "normal";

[0123] Upper wellhead safety door: If |f upside |>δ upside , then the detection result of the upper wellhead safety door is judged to be "abnormal". If |f upside |≤δ upside , the detection result of the upper wellhead safety door is determined to be "normal";

[0124] Downhole safety door: If |f downside |>δ downside, then the detection result of the wellhead safety door is judged to be "abnormal". If |f downside |≤δ downside , the detection result of the wellhead safety door is determined to be "normal";

[0125] Tail rope: If |f rope |>δ rope , the tail rope detection result is judged to be "abnormal", if |f rope |≤δ rope , the detection result of the tail rope is judged to be "normal".

[0126] If the inspection results of the sheave, drum, upper wellhead safety door, lower wellhead safety door and tail rope are determined to be "abnormal", the coal mine hoist stops running and issues an audible and visual alarm corresponding to the abnormal part; if the inspection results of the sheave, drum, upper wellhead safety door, lower wellhead safety door and tail rope are determined to be "normal", the process goes to step S603, and the process is repeated in this way to continuously perform real-time abnormal inspections on the five key parts of the coal mine hoist, namely the sheave, drum, upper wellhead safety door, lower wellhead safety door and tail rope, and control their operation.

[0127] Table 1 Experimental results

[0128] Model P R mAP FPS Faster-RCNN 0.7541 0.9459 0.832 12 CenterNet 0.825 0.5766 0.776 56 YOLOv8 0.845 0.805 0.901 53 Improving YOLOv8 0.975 0.908 0.964 62

[0129] Table 1 is a graph showing the performance comparison of each model. The model is trained using a training data set, the prediction value is calculated through forward propagation, the model parameters are updated through back propagation, and the model performance is evaluated on the validation set. The indicators used include: precision P, recall R, mean average precision mAP, and frame rate FPS. Combining the data, it can be seen that the improved YOLOv8 detection model has the highest indicators compared to other models. Further comparison with the original YOLOv8 model shows that the mAP of YOLOv8 has increased by 6.5% and the precision has increased by 15.4%. The experimental results show that the detection model proposed by the present invention has better performance than other models, with higher accuracy and detection rate. Figure 8 This is the effect of using the EnlightGAN model to enhance the image brightness. Compared with the original image, the enhanced image has a uniform brightness distribution and clearer details. Before feeding the image into the detection model, enhancing the brightness of the image can improve the detection accuracy. Fig. 9 As part of the output results of online real-time detection, the method proposed in the present invention can accurately and efficiently output the labels of the corresponding key parts and determine the location where the abnormality exists.

[0130] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0131] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for detecting abnormalities in key parts of a coal mine hoist, characterized in that: The specific steps include: S1. Construct an image sample set of key parts of coal mine hoist; S2, establishing an image enhancement model to perform image enhancement on a sample set of images of key parts of the elevator to obtain an enhanced image set; S3, establish a label value table of key parts of coal mine hoist and mark the category and center point coordinates of each image in the enhanced image set, construct all images and label data into a coal mine hoist key part anomaly detection image dataset, and divide the coal mine hoist key part anomaly detection image dataset into a training set and a test set; S4. Construct an abnormality detection model for key parts of coal mine hoists; S5. Train the anomaly detection model for key parts of coal mine hoists; S6. Carry out real-time online detection of key parts of coal mine hoists.

2. A method for detecting abnormalities in key parts of a coal mine hoist according to claim 1, characterized in that: The specific process of S1 is: install cameras at key positions of the coal mine hoist, record uninterrupted videos for 24 hours a day, capture a picture of the captured video every 1 second, and select N pictures of each key position to form an image sample set ImageSet.

3. A method for detecting abnormalities in key parts of a coal mine hoist according to any one of claims 1 and 2, characterized in that: The key parts of the coal mine hoist include a coal mine hoist sheave, a roller, an upper wellhead safety door, a lower wellhead safety door and a tail rope.

4. A method for detecting abnormalities in key parts of a coal mine hoist according to claim 1, characterized in that: The specific process of S2 is: S201, construct an image enhancement model, wherein the image enhancement model includes a U-Net generator module and a discriminator module: S2011. The CA attention mechanism is introduced in the U-Net generator module. The U-Net generator consists of 8 convolutional layers, each of which consists of two 3×3 convolutional layers, a BN layer and a LeakReLU layer. In the upsampling stage, the standard deconvolution layer is replaced by a bilinear upsampling layer plus a convolutional layer. S2012, the discriminator uses a global-local dual discriminator to guide global and local information. Both the global and local discriminators use PatchGAN for image authenticity identification, and VGG Net is used to calculate the self-feature preservation loss; S202, after the image enhancement model is constructed, a part of images are extracted from the image sample set constructed in S1 to form a training set, and the training set generator and the discriminator are used to perform adversarial training to achieve global optimization; S203, input the image sample set constructed in S1 into the trained image enhancement model to perform brightness enhancement, and obtain an enhanced image set.

5. A method for detecting abnormalities in key parts of a coal mine hoist according to claim 4, characterized in that: The training process of the generator and the discriminator in S202 is specifically as follows: S2021. Initialize the weights W and bias B of the generator G and discriminator D networks; S2022, extracting n samples from the training set, and the generator generates n samples according to a predefined noise distribution; S2023, fix the generator G and train the discriminator D; the training process of the discriminator D is as follows: the local discriminator randomly cuts 5 patches from the input generated image and the real image each time, and lets the discriminator perform identification, and learns to distinguish by comparing the local areas of the real image and the generated image. After each identification, the parameters of the discriminator are adjusted according to the result of the loss function; S2024, after cyclically updating the discriminator D k times, update the weight, bias, and size and position of the attention map of the generator G once, normalize the illumination channel I of the input RGB image to [0, 1], use 1-I as the self-regularized attention map, adjust the size of the attention map to fit the size of each intermediate feature map, and multiply it with all the intermediate feature maps and the output image to obtain a generated map; S2025. After multiple update iterations, until the discriminator D cannot distinguish whether the image comes from the real training data set or is generated by the generator G, the loss function reaches the minimum value and the training is completed.

6. A method for detecting abnormalities in key parts of a coal mine hoist according to claim 5, characterized in that: The loss function includes the discriminator loss Generator Loss and the self-feature preservation loss L SFP (I L ): Global discriminator loss: Local Discriminator Loss: Global generator loss: Local generator loss: Among them, D represents the discriminator, G represents the generator; D Ra is used to measure the two inputs x r 、x f The difference between the functions, C represents the discriminator network, σ represents the sigmoid function, and the sigmoid function is replaced by the least squares GAN loss. r and x f To sample from real pictures and generated pictures respectively; and Respectively represent in x f Obey the generated image distribution And the real picture distribution In the case of , the expected value of the corresponding expression; Represent the distribution of randomly selected small blocks in real pictures and generated pictures respectively, From feature preservation loss: Among them, I L represents the input low-light image, G(I L ) represents the enhanced output of the generator; φ a,b represents the feature map extracted by VGGNet of the discriminator; a represents the ath maximum pooling layer, b represents the bth convolutional layer after the ath maximum pooling layer; W a,b , H a,b is the width and height of the extracted feature map; a and b represent each pixel point of the feature map traversed from the width and height respectively, where a traverses from 1 to W a,b , b traverses from 1 to H a,b .

7. A method for detecting abnormalities in key parts of a coal mine hoist according to claim 1, characterized in that: The anomaly detection model for key parts of coal mine hoists in S4 includes a preliminary feature extraction network, an enhanced feature extraction network and a classification regression layer; The preliminary feature extraction network includes ten layers, which are the first CBS module layer, the second CBS module layer, the third C2f_ODConv module layer, the fourth CBS module layer, the fifth C2f_ODConv module layer, the sixth CBS module layer, the seventh C2f_ODConv module layer, the eighth CBS module layer, the ninth C2f_ODConv module layer, and the tenth SPPF module layer. The C2f_ODConv module layers are all C2f modules that introduce ODConv. The enhanced feature extraction network includes sixteen layers, which are, in order, a first CBS module layer, a second upsampling module layer, a third Concat module layer, a fourth C2f_ODConv module layer, a fifth upsampling module layer, a sixth Concat module layer, a seventh C2f_ODConv module layer, an eighth GAM attention module layer, a ninth CBS module layer, a tenth Concat module layer, an eleventh C2f_ODConv module layer, a twelfth GAM attention module layer, a thirteenth CBS module layer, a fourteenth Concat module layer, a fifteenth C2f_ODConv module layer, and a sixteenth GAM attention module layer; the GAM attention module is used to enhance the representation of important information in the feature map, and the preliminary features of the three dimensions output by the preliminary feature extraction network are processed by the enhanced feature extraction network to obtain and output enhanced features of the three dimensions; The classification and regression layer includes three Detect layers, which perform final processing by strengthening the features output by the feature extraction network to classify and locate the target.

8. A method for detecting abnormalities in key parts of a coal mine hoist according to claim 1, characterized in that: The specific process of S5 is as follows: S501, initialize all unknown matrix parameters W of each layer in the anomaly detection model of key parts of coal mine hoists [i] ,B [i] , i∈(1,3,5,6) is a random value, i represents the index of the layer, set the number of iterations to S, the learning rate to LR, and set the input A j is the first sample A in the training data set j =A0, where subscript j represents the jth image sample, and A0 represents the first image sample in the training data set; S502, inputting the key part image sample into the key part anomaly detection model of the coal mine hoist into the layer A [0] Input image sample A j ; S503, gradually calculating the estimated value of the key parts detection of the coal mine hoist through the key parts abnormality detection model S504: Use input image sample A j The corresponding true value Y and estimated value Calculate and output the cross entropy loss function S505, calculate each parameter W in each layer of the convolutional neural network [i] and B [i] The change of ΔW [i] and ΔB [i] , where i∈(1,3,5,6); S506, use LR, ΔW [i] and ΔB [i] Update all parameters W in the convolutional neural network with the value of [i] and B [i] The value of W [i] (S+1)=W [i] -LR*ΔW [i] , B [i] (S+1)=B [i] -LR*ΔB [i] , where i∈(1,3,5,6); S507: Determine whether it is the last sample in the training data set. If it is not the last sample, select the next sample as A. j , and jump to S502, if it is the last sample, jump to S508; S508: Determine whether the iteration is finished. If not, reselect the first sample of the training data set as A. j , reduce the number of iterations S, jump to S502, if the iteration is completed, jump to S509; S509, set the detection accuracy threshold T of the key parts detection model of the coal mine hoist, input all the image samples in the test data set into the convolutional neural network in sequence, and output the estimated value And compare it with the true value Y to calculate the accuracy of the key parts detection of the coal mine hoist in Indicates the number of samples in the test data set whose estimated value is the same as the true value, ∑Num(Y) indicates the number of all samples in the test data set, if ACC<T, go to S501, if ACC≥T, go to S510; S510, all parameters W of the convolutional neural network [i] and B [i] Fixed and saved, where i∈(1,3,5,6), the coal mine hoist key parts position coordinate detection model training is completed.

9. A method for detecting abnormalities in key parts of a coal mine hoist according to claim 1, characterized in that: The specific process of S6 is as follows: S601. Under normal circumstances, the position coordinates of the sheave, drum, upper wellhead safety door, lower wellhead safety door and tail rope are: (X Normal_wheel ,Y Normal_wheel )、(X Normal_roller ,Y Normal_roller )、(X Normal_upside ,Y Normal_upside )、(X Normal_downside ,Y Normal_downside )、(X Normal_rope ,Y Normal_rope ); S602, with the normal value as the center, the coordinate thresholds of the center points of the crown wheel, the drum, the upper wellhead safety door, the lower wellhead safety door and the tail rope are set to be δ wheel (δ wheel ≥0)、δ roller (δ roller ≥0)、δ upside (δ upside ≥0)、δ roller (δ roller ≥0) and δ rope (δ rope ≥0); S603, obtaining real-time video of the sheave, roller, upper wellhead safety door, lower wellhead safety door and tail rope captured by the coal mine camera, reading each frame of the video in sequence, and inputting the real-time image into the trained image enhancement model to enhance the image brightness; S604, the enhanced real-time image is transmitted to the detection model for detection, and the categories of the five key parts of the elevator and the coordinates of their center points are marked in the video: (X Centre_roller ,Y Centre_roller )、(X Centre_upside ,Y Centre_upside )、(X Centre_downside ,Y Centre_downside )、(X Centre_rope ,Y Centre_rope ); S605, compare the position coordinates output by the detection model with the coordinates of the center of the sheave, drum, upper wellhead safety door, lower wellhead safety door and tail rope under normal conditions and calculate their average difference: Sky Wheel: If |f wheel |>δ wheel , then the detection result of the sky wheel is judged to be "abnormal". If |f wheel |≤δ wheel , the detection result of the sky wheel is determined to be "normal"; roller: If |f roller |>δ roller , the drum test result is judged to be "abnormal", if |f roller |≤δ roller , the detection result of the drum is determined to be "normal"; Upper wellhead safety door: If |f upside |>δ upside , then the detection result of the upper wellhead safety door is judged to be "abnormal", if |f upside |≤δ upside , the detection result of the upper wellhead safety door is determined to be "normal"; Wellhead safety door: If |f downside |>δ downside , then the detection result of the wellhead safety door is judged to be "abnormal". If |f downside |≤δ downside , the detection result of the wellhead safety door is determined to be "normal"; Tail rope: If |f rope |>δ rope , then the detection result of the tail rope is judged to be "abnormal", if |f rope |≤δ rope , the detection result of the tail rope is determined to be "normal".

Images