Abnormal early warning method and system for fully mechanized coal mining face of thin coal seam

Abstract

The invention relates to the technical field of fully mechanized coal mining face early warning, and discloses a thin seam fully mechanized coal mining face abnormity early warning method and system, and the method comprises the steps: building an improved YOLOv5s visual abnormity monitoring model according to collected small sample abnormal data; enriching the initial top plate abnormal image and the initial hydraulic support abnormal image to obtain an expanded top plate abnormal image and an expanded hydraulic support abnormal image; taking the initial top plate abnormal image and the extended top plate abnormal image together as a top plate abnormal image data set, taking the initial hydraulic support abnormal image and the extended hydraulic support abnormal image together as a hydraulic support abnormal image data set, and performing model training to construct an improved YOLOv5s visual anomaly monitoring model; and inputting the defogged image into the model, and when the monitoring result is that the top plate is in an abnormal state or the hydraulic support is in an abnormal state, generating an early warning signal. Based on the small sample abnormal data, the problems of early warning accuracy and timeliness in the thin coal seam space are effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of early warning for fully mechanized mining working faces, and in particular to an abnormal early warning method and system for fully mechanized mining working faces in thin coal seams. Background Art

[0002] Comprehensive coal mining, the full name of which is comprehensive mechanized coal mining, refers to the combination of main equipment such as coal mining machines, flexible scraper conveyors and self-moving hydraulic supports in the coal mining face, and the main production processes of coal mining are all mechanized. Specifically, comprehensive coal mining includes the main processes of coal breaking, coal loading, coal transportation, support, goaf area treatment, return tunnel transportation, and excavation. These processes are all realized in a mechanized way to form a continuous operation. The main advantages of this coal mining method include high yield, low cost, high efficiency, and high economic benefits.

[0003] The intelligent comprehensive mining system is based on fully mechanized comprehensive mining, and achieves the purpose of intelligent remote monitoring by adding intelligent means such as remote control and intelligent data processing. In particular, the mining conditions of the comprehensive mining face of coal mines are complex, with a large number of equipment and harsh environment. Due to the lack of monitoring of equipment and environment, accidents frequently occur in the comprehensive mining face of coal mines.

[0004] The intelligent comprehensive mining system has a matching inspection robot. By installing image sensors on the inspection robot, it can collect images of equipment, environment and personnel in the underground comprehensive mining face and upload them to the centralized control center. The staff will centrally process the image data and check whether there are any abnormal conditions on the face. Because there are many monitoring scenes required for the comprehensive mining face, in order to avoid monitoring blind spots, a certain number of inspection robots are generally used to inspect each scene. For general comprehensive mining faces with relatively large passage spaces, although monitoring blind spots can be avoided by increasing the number of inspection robots, the amount of data that needs to be processed increases.

[0005] For thin coal seam fully mechanized mining faces with narrow passage space, there is no space for a sufficient number of inspection robots or other monitoring equipment to be installed. Therefore, the existing image data cannot avoid monitoring blind spots. In order to adapt to the narrow space under the thin coal seam, the intelligent fully mechanized mining system has far fewer auxiliary installation equipment on the thin coal seam fully mechanized mining face than the ordinary coal seam fully mechanized mining face. For example, in order to save space, auxiliary equipment such as guard plates in some positions will not be installed, which undoubtedly increases the possibility of abnormal events. In addition, the abnormal situations obtained after image data processing are basically batch-based centralized review, which inevitably leads to omissions and untimely responses.

[0006] In order to carry out fully mechanized coal mining more safely, an abnormal warning system is generally set up in the fully mechanized coal mining face, so that timely warning can be given when an abnormality occurs to avoid further expansion of losses. For example, the patent scheme CN115909623A-a fully mechanized coal mining face abnormal warning method, system, electronic equipment and medium, by collecting real-time video of the fully mechanized coal mining face underground; intercepting the image in the real-time video of the fully mechanized coal mining face to obtain the fully mechanized coal mining face image; defogging the fully mechanized coal mining face image to obtain the defogging image; constructing an improved Y0L0v5s visual anomaly detection model; inputting the defogging image into the trained improved Y0L0v5s visual anomaly detection model to obtain the detection result, and generating an alarm signal when the detection result is abnormal, so as to improve the safety of the fully mechanized coal mining face.

[0007] However, although this solution has improved the safety of the fully mechanized mining face to a certain extent, it is still lacking because the detection images are limited to personnel and tow cables. Only when personnel and / or tow cables are abnormal will the detection results be abnormal. In fact, when personnel and tow cables are abnormal, irreparable losses have almost occurred on the fully mechanized mining face. This early warning method still has a large early warning loophole. Summary of the invention

[0008] The present invention provides an abnormal warning method for a thin coal seam fully mechanized mining working face, which is used to solve the problem of untimely and inaccurate warning caused by the unsuitability of the existing intelligent fully mechanized mining system and its supporting inspection robots and related equipment in a limited space under complex structural thin coal seam geological conditions.

[0009] Solution 1: A method for abnormal early warning of a thin coal seam fully mechanized mining working face, comprising the following steps: Step 1: Collect real-time video of the thin coal seam fully mechanized mining face through the inspection robot and the camera fixedly installed underground; Step 2: capture corresponding frame images from the real-time video to obtain real-time images; classify the real-time images according to the shooting content to obtain real-time images of the top plate whose shooting content includes the top plate content and real-time images of the hydraulic support whose shooting content includes the hydraulic support content; Step 3, respectively defogging the roof real-time image and the hydraulic support real-time image to obtain a roof defogging image and a hydraulic support defogging image; Step 4, constructing an improved YOLOv5s visual anomaly monitoring model based on the collected small sample anomaly data; the visual anomaly monitoring model includes a roof monitoring unit and a hydraulic support monitoring unit with the same structure; before constructing the improved YOLOv5s visual anomaly monitoring model, first process the collected small sample anomaly data, the small sample anomaly data includes an initial roof anomaly image and an initial hydraulic support anomaly image; enrich the initial roof anomaly image and the initial hydraulic support anomaly image respectively to obtain an extended roof anomaly image and an extended hydraulic support anomaly image; use the initial roof anomaly image and the extended roof anomaly image together as a roof anomaly image dataset, use the initial hydraulic support anomaly image and the extended hydraulic support anomaly image together as a hydraulic support anomaly image dataset, perform model training to construct an improved YOLOv5s visual anomaly monitoring model; Step 5, inputting the defogging image into the trained improved YOLOv5s visual anomaly monitoring model to obtain monitoring results; the monitoring results are the normal state of the roof, the normal state of the hydraulic support, the abnormal state of the roof and the abnormal state of the hydraulic support; Step six: when the monitoring result is that the roof is in an abnormal state or the hydraulic support is in an abnormal state, an early warning signal is generated.

[0010] Solution 2: The present invention also provides a thin coal seam fully mechanized mining working face abnormality warning system, which is used to implement the aforementioned thin coal seam fully mechanized mining working face abnormality warning method, including a central processing unit and a collection module, an interception module, a classification module, a defogging module, a construction module, a monitoring module and an early warning module respectively connected to the central processing unit; The acquisition module is used to collect real-time video of underground fully mechanized mining working faces in thin coal seams; The interception module is used to intercept the image in the real-time video of the fully-mechanized mining working face of the thin coal seam to obtain the real-time image of the fully-mechanized mining working face; A classification module, used for classifying the real-time image into a roof real-time image and a hydraulic support real-time image according to the shooting content; A defogging module is used to defog the real-time image of the roof and the real-time image of the hydraulic support, respectively, to obtain a defogging image of the roof and a defogging image of the hydraulic support; A building module for building an improved YOLOv5s visual anomaly monitoring model using the collected small sample anomaly data; The monitoring module is used to input the defogging image into the trained improved YOLOv5s visual anomaly monitoring model to obtain monitoring results; the monitoring results are the normal state of the roof, the normal state of the hydraulic support, the abnormal state of the roof and the abnormal state of the hydraulic support; The early warning module is used to generate an early warning signal when the monitoring result is an abnormal state of the roof or an abnormal state of the hydraulic support.

[0011] The working principle and advantages of the present invention are: Based on the characteristics that the comprehensive mining working face of thin coal seams is narrow and cannot accommodate too many auxiliary equipment, the existing underground cameras and the cameras on the inspection robots are used to collect real-time video. There are both inspection robots that can move underground and fixed cameras installed in key positions, which effectively overcome the problems of monitoring blind spots and narrow spaces.

[0012] An improved YOLOv5s visual anomaly monitoring model is constructed through the collected small sample abnormal data, which targets the underground anomaly monitoring of the roof and hydraulic support that are most likely to cause destructive effects. It can effectively avoid the roof stability problem and the collision problem caused by the slow response of the hydraulic support frame in the special environment of thin coal seams, which are most likely to cause safety incidents. While overcoming the accuracy problems that may be caused by small sample abnormal data, the calculation amount is limited to a certain range to ensure timely and effective monitoring and early warning.

[0013] The biggest difference from the existing technology is that it makes up for the limitation of use in thin coal seam fully mechanized mining working faces. By monitoring the abnormality of the roof and hydraulic supports, it breaks through the limitation of the working space in thin coal seams where the system is very small and difficult to arrange. At the same time, the cost of intelligent transformation of this system is low and the scalability is strong. The invention effectively improves the safety of fully mechanized mining working faces in thin coal seams BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 This is a flow chart of an abnormal early warning method for a fully mechanized mining face in a thin coal seam in Embodiment 1 of the present invention; Figure 2 This is a framework diagram of an abnormal warning system for a thin coal seam fully mechanized mining working face in an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The following is a further detailed description through specific implementation methods: The symbols in the drawings of the specification include: a central processing unit 1, a collection module 2, a capture module 3, a classification module 4, a defogging module 5, a construction module 6, a monitoring module 7, and an early warning module 8.

[0015] The present invention provides early warning for abnormalities in the mining process of thin coal seams with an average coal thickness of less than 1.3 meters, especially with an average coal thickness of 0.98 meters, to solve the current problem that the monitoring and early warning capabilities of thin coal seams are insufficient, either there are monitoring blind spots and all scenes cannot be monitored; or the required installation space is large and cannot adapt to the narrow space under the thin coal seams; or the image data is continuously collected by fixed cameras fixedly installed in various scenes, but the data processing volume is large and the response is not timely. The present invention can timely detect abnormal conditions of the fully-mechanized mining working face of thin coal seams and give an alarm in time, thereby improving the safety of the fully-mechanized mining working face of thin coal seams.

[0016] Embodiment 1 like Figure 1 As shown, the abnormal warning method for a fully mechanized mining working face in a thin coal seam in this embodiment includes the following contents: The first step is to collect real-time video of the underground comprehensive mining working face through inspection cameras on a small number of inspection robots and fixed cameras fixed in key scenes; among them, the number of inspection robots used in this embodiment is 1, generally not more than 3, to avoid too many inspection robots from occupying space.

[0017] The second step is to capture corresponding frame images from the real-time video of the fully mechanized mining face to obtain real-time images.

[0018] The third step is to classify the roof real-time images and hydraulic support real-time images from the real-time images.

[0019] In the real-time image, the image containing the roof content is the roof real-time image, the image containing the hydraulic support is the hydraulic support real-time image, and the image containing both the roof content and the hydraulic support content is both the roof real-time image and the hydraulic support real-time image. It should be noted that because the inspection cameras on the inspection robot and the fixed cameras installed in various key scenes are limited by the installation position and shooting angle of the cameras, the contents contained in the real-time images are not all the same. Some real-time images may be mainly shot at the coal seam, some may be shot at the coal mining machine, some may be shot at the hydraulic support, and some may be shot at the scraper conveyor. Of course, more may be images that contain multiple devices at the same time from a certain angle. Compared with the conventional method that does not classify the real-time images, this embodiment extracts the real-time images containing the roof content to form the roof real-time image through the shooting content, and extracts the real-time images containing the hydraulic support content to form the hydraulic support real-time image. In this way, these two types of real-time images can be identified and processed in a targeted manner later, while other types of real-time images can be roughly processed through random sampling and other methods, which can reduce the amount of data processing for real-time images without affecting the warning effect.

[0020] Specifically, when classifying the roof real-time image and the hydraulic support real-time image, the process is as follows: (1) Data collection: A large number of images (more than 20,000) of roof panels and hydraulic supports were collected.

[0021] (2) Data labeling: Use existing labeling tools (such as LabelImg, VIA, Label Studio, etc.) to select the top plate and hydraulic support in the image, specify the category label and save the labeling results, for example: an XML file, JSON file or CSV file containing the information of the selected area.

[0022] (3) Data preparation: Organize the labeled data into the format required by YOLOv5.

[0023] (4) Model training: Use the labeled data to train the existing general YOLOv5 model. Adjust the model parameters, set the initial learning rate to 0.01, the batch size to 16, and the number of iterations to 100. After model training, the YOLOv5 model for real-time image classification of roof and hydraulic support is obtained.

[0024] When the YOLOv5 model is used to classify real-time images, the confidence threshold is set to 0.75 and the overlap threshold is set to 0.3 for classification. If the detected category is the detection frame of "roof plate" and the confidence is greater than or equal to 0.75, the real-time image is marked as the real-time image of the roof plate, and the classification obtains the real-time image of the roof plate; if the detected category is the detection frame of "hydraulic support" and the confidence is greater than or equal to 0.75, the real-time image is marked as the real-time image of the hydraulic support, and the classification obtains the real-time image of the hydraulic support; if the detected categories are both the detection frames of "roof plate" and "hydraulic support", and the confidence of each is greater than or equal to 0.75, the real-time image is marked as the real-time image of the roof plate and the real-time image of the hydraulic support. Through the accurate classification of real-time images, the accuracy and efficiency of the real-time image classification processing can be ensured, thereby improving the overall performance of the abnormal early warning system of the thin coal seam fully mechanized mining working face.

[0025] The fourth step is to defog the roof real-time image and the hydraulic support real-time image respectively to obtain the roof defog image and the hydraulic support defog image.

[0026] The fifth step is to build an improved YOLOv5s visual anomaly monitoring model based on the collected small sample anomaly data; the visual anomaly monitoring model in this embodiment includes a roof monitoring unit for monitoring the stability of the roof by identifying roof cracks, and a hydraulic support monitoring unit for monitoring the hydraulic support shifting condition by the displacement of the hydraulic support.

[0027] The sixth step is to input the defogging image into the trained improved YOLOv5s visual anomaly monitoring model to obtain the monitoring results; the monitoring results are the normal state of the roof, the normal state of the hydraulic support, the abnormal state of the roof and the abnormal state of the hydraulic support; wherein, the defogging image of the roof is input into the roof monitoring unit, and the obtained roof monitoring result is the normal state of the roof or the abnormal state of the roof; the defogging image of the hydraulic support is input into the hydraulic support monitoring unit, and the obtained hydraulic support monitoring result is the abnormal state of the hydraulic support or the normal state of the hydraulic support.

[0028] In this embodiment, because the real-time video of the underground fully mechanized mining working face is collected, the image input into the trained improved YOLOv5s visual anomaly monitoring model is also a continuous real-time defogging image, that is, a real-time defogging video after the real-time video is processed.

[0029] The seventh step is to generate an early warning signal when the monitoring result indicates that the roof is in an abnormal state or the hydraulic support is in an abnormal state. It should be noted that the structures of the roof monitoring unit and the hydraulic support monitoring unit are the same. They are both improved model structures based on the general model structure of YOLOv5s. The two monitoring units process the corresponding real-time images respectively, and the monitoring results can be obtained more quickly. This model has two monitoring units, and abnormal conditions are judged from two dimensions. Compared with single-dimensional judgment, the monitoring results are more accurate.

[0030] In this embodiment, the defogging process of the roof image and the hydraulic support image is the same, that is, the roof real-time image and the hydraulic support real-time image are defogged by using existing technical means to obtain corresponding defogging images.

[0031] The training process of the improved YOLOv5s visual anomaly monitoring model in this embodiment is to train the roof monitoring unit and the hydraulic support monitoring unit separately, and specifically includes: a1. Obtain abnormal sample data.

[0032] Get the historical monitoring video of the abnormal time period: select the underground fully mechanized mining face where the abnormal situation has occurred. The time period before and after the abnormal situation is 1 hour before (i.e. 60 minutes) and 0.5 hour after (i.e. 30 minutes) the time when the abnormal situation is discovered, totaling 1.5 hours (i.e. 90 minutes). Capture the screen from this historical monitoring video as the sample abnormal data.

[0033] a2, capture and classify historical images.

[0034] The historical monitoring video of the underground fully mechanized mining face within the time period is intercepted to obtain the historical image.

[0035] Video capture: Use ffmpeg or other video processing tools to capture images from the historical surveillance videos within the above time period. The video capture frequency is to store one image every 1 second to ensure the continuity and representativeness of the image. For example, 90 minutes of video will capture 90 × 60 / 1 = 5400 images.

[0036] Image classification: The captured historical images are classified according to the shooting content. The initial abnormal roof image and the initial abnormal hydraulic support image are obtained by identifying abnormal conditions in the historical roof images and the historical hydraulic support images.

[0037] Among them, the screening process of the initial roof abnormal image and the initial hydraulic support abnormal image is as follows: In this embodiment, the abnormal conditions are graded according to the severity of the accidents that may be caused by the abnormal conditions of the roof and the hydraulic support, and the abnormal levels are divided into the following levels from the smallest to the largest possible impact of the accident: 1. Minor abnormality: The local ballast of the roof is minor and does not affect normal operation. Some parts of the hydraulic support leak slightly, but the support can maintain the stability of the working face. The support pressure gauge reading deviates slightly from the standard range, but is still within the acceptable range. The communication or control system has a short-term failure, which recovers quickly or is easy to repair manually.

[0038] 2. General abnormality: There are obvious cracks or small-scale collapses in the roof, and strengthening support measures need to be taken. Multiple hydraulic supports have malfunctions in synchronization or control functions, affecting the advancement speed of the working face. The support columns have continuous leakage in multiple places, and timely maintenance or replacement of parts is required. The monitoring module of the intelligent comprehensive mining system of the working face shows continuous abnormal data, and the cause needs to be verified on site.

[0039] 3. Serious abnormality: There is a risk of large-scale collapse of the roof, and the operation must be stopped immediately for emergency support. The hydraulic support is unstable over a large area and cannot effectively support the roof, resulting in interruption of work on the working surface. Damage to key structural parts of the support, such as severe bending and fracture of the column, poses a safety hazard. Failure of key components of the control system makes it impossible to operate the support group normally, affecting production safety.

[0040] 4. Crisis abnormality: The roof collapse actually occurs, directly threatening the life safety of the workers. The hydraulic support system fails as a whole, resulting in the loss of effective support for the working face, facing the risk of major safety accidents. Severe equipment damage is accompanied by signs of secondary disasters such as fire and flood. The emergency plan must be activated immediately, personnel must be evacuated, and the emergency must be reported.

[0041] According to the above four abnormal situations, the initial roof abnormality images are screened from the corresponding roof historical images, including roof slight abnormality images, roof general abnormality images, roof serious abnormality images and roof crisis abnormality images. The screening process completes the following judgments through existing image recognition: 1. Slightly abnormal image of the roof: The inspection robot or fixed camera captures a local area on the bottom surface of the roof that occupies more than 5% of the entire bottom surface area and has a color difference with the surrounding color, and it is identified that there is slag falling in the color difference area, but all working indicators of the intelligent comprehensive mining system are normal and do not affect normal operations. The corresponding roof historical image is determined to be a slightly abnormal image of the roof.

[0042] 2. General abnormal image of the roof: There are obvious cracks in the roof, the width of the cracks is less than or equal to 1 cm, the length of the cracks is less than or equal to 2% of the length of the roof, or collapse occurs in a local area where the local bottom surface area of ​​the roof accounts for less than 5% of the total bottom surface area, and strengthened support measures need to be taken; the working face monitoring system displays continuous abnormal data, and the cause needs to be verified on site; the corresponding roof historical image is determined to be a general abnormal image of the roof.

[0043] 3. Severely abnormal roof image: There is a risk of large-scale collapse in a local area of ​​the roof that accounts for more than 5% and less than 10% of the entire bottom area. Operations must be stopped immediately for emergency support. Key components of the control system fail and the support group cannot be operated normally, affecting production safety. The corresponding roof historical image is determined to be a severely abnormal roof image.

[0044] 4. Abnormal roof crisis image: When the roof collapses and the roof structure is destroyed, the lives of the workers are directly threatened. The emergency plan must be activated immediately to evacuate the personnel and report the emergency. The corresponding roof historical image is determined to be an abnormal roof crisis image.

[0045] Similarly, according to the above four abnormal situations, the initial hydraulic support abnormal images are screened from the corresponding hydraulic support historical images, including hydraulic support slight abnormal images, hydraulic support general abnormal images, hydraulic support serious abnormal images and hydraulic support crisis abnormal images. The screening process completes the following judgments through existing image recognition: 1. Slightly abnormal image of hydraulic support: individual components within 3 parts of the hydraulic support have slight leakage, and the leakage volume is less than or equal to 5 drops / minute or less than or equal to 0.1 liters / hour, but can maintain the stability of the working face support; the reading of the support pressure gauge deviates slightly from the standard range, and the deviation percentage does not exceed ±5%, and the percentage of the reading deviates from the standard value is within a certain range and is still within an acceptable range; the communication or control system has a brief failure, which quickly recovers by itself or is easy to repair manually; the corresponding hydraulic support historical image is judged to be a slightly abnormal image of the hydraulic support.

[0046] 2. General abnormal image of hydraulic support: more than 3 hydraulic supports have synchronization or control function failures, affecting the advancement speed of the working face; the support column has more than 3 consecutive leaks, and the leakage volume at any one place is greater than 5 drops / minute or greater than 0.1 liters / hour, requiring timely maintenance or replacement of parts; the working face monitoring system shows continuous abnormal data, and the cause needs to be verified on site; the corresponding hydraulic support historical image is determined to be a general abnormal image of the hydraulic support.

[0047] 3. Severely abnormal images of hydraulic supports: The hydraulic supports are unstable over a large area, that is, the number of supports affected by the instability accounts for more than 32% of the total number of supports, and the top plate cannot be effectively supported, resulting in interruption of work on the working face; key structural parts of the supports are damaged, such as severe bending and fracture of the columns, posing a safety hazard; key components of the control system fail, and the support group cannot be operated normally, affecting production safety; the corresponding historical images of the hydraulic supports are determined to be severely abnormal images of the hydraulic supports.

[0048] 4. Hydraulic support crisis abnormal image: The overall failure of the hydraulic support system leads to the loss of effective support for the working face, which faces the risk of major safety accidents; serious equipment damage is accompanied by signs of secondary disasters such as fire and flood; the emergency plan must be activated immediately, personnel must be evacuated and the emergency must be reported; the hydraulic support historical image with the corresponding content is determined to be a hydraulic support crisis abnormal image.

[0049] Before constructing the improved YOLOv5s visual anomaly monitoring model, the collected small sample anomaly data are first processed, and the small sample anomaly data include the initial roof anomaly image and the initial hydraulic support anomaly image; after the initial roof anomaly image and the initial hydraulic support anomaly image are collected, the initial roof anomaly image and the initial hydraulic support anomaly image are respectively enriched to obtain the extended roof anomaly image and the extended hydraulic support anomaly image; the initial roof anomaly image and the extended roof anomaly image are used together as the roof anomaly image data set, and the model training is performed to construct the improved YOLOv5s visual anomaly monitoring model; the initial hydraulic support anomaly image and the extended hydraulic support anomaly image are used together as the hydraulic support anomaly image data set, and the model training is performed to construct the improved YOLOv5s visual anomaly monitoring model.

[0050] The method of obtaining an extended initial roof abnormality image through an initial roof abnormality image, and obtaining an extended hydraulic support abnormality image through an initial hydraulic support abnormality image is as follows: 1. Initial image acquisition: According to the aforementioned method, the initial roof anomaly image and the initial hydraulic support image are acquired respectively as the initial images.

[0051] 2. Transformation parameter definition: Set a series of image transformation parameters, including rotation angle (θ), translation distance (x, y), scaling (s), horizontal or vertical flip flag (flip) and color adjustment parameters (hue, saturation, brightness).

[0052] 3. Image transformation operation: For each initial image, any two of the following five transformation parameter combinations are used to form an extended image. The specific implementation may include: (1) Use random rotation (θ ∈ [-35°, 35°]) to change the image perspective.

[0053] (2) Implement random translation (x ∈ [-12%, 12%] image width, y ∈ [-12%, 12%] image height) to simulate different viewing angles.

[0054] (3) Perform random scaling (s ∈ [0.9, 1.1]) to adjust the image size while maintaining content recognition.

[0055] (4) Randomly flip horizontally or vertically (flip ∈ {0, 1}) to increase sample diversity.

[0056] (5) Adjust the color balance by randomly changing the hue (hue ∈ [-10, 10]), saturation (saturation ∈ [0.9, 1.1]) and brightness (brightness ∈ [0.9, 1.1]).

[0057] 4. Data preservation and integration: The extended image generated after the transformation is merged with the initial image to form an expanded training data set, ensuring that all transformations are consistent with the actual application scenarios and that the image annotation information remains accurately corresponding.

[0058] 5. Iteration and evaluation: Use the expanded data sets mentioned above, i.e., the roof abnormality image data set formed by the initial roof abnormality images and the extended roof abnormality images, and the hydraulic support abnormality image data set formed by the initial hydraulic support abnormality images and the extended hydraulic support abnormality images, to conduct multiple rounds of training on the existing YOLOv5s model, and periodically evaluate the performance of the model on the validation set until the predetermined performance indicators are met to obtain an improved YOLOv5s visual anomaly monitoring model.

[0059] Through the above steps, the limitations of small sample image datasets in training accurate fault recognition models are effectively solved, and the generalization ability and recognition accuracy of the model are improved.

[0060] Since the roof monitoring unit and the hydraulic support monitoring unit are in a parallel relationship, the data sets used for training the two unit models are different. The initial roof abnormality images used for training the roof monitoring unit are obtained from more than 20,000 real roof images of underground work, and the initial hydraulic support abnormality images used for training the hydraulic support monitoring unit are obtained from more than 25,000 real images of hydraulic supports working underground. Among them, because multiple hydraulic supports involve changes in the current position during the frame shifting process, and some images may not be able to accurately identify the displacement of the hydraulic support due to the limitation of the shooting angle, so without increasing more calculations, for the sake of training accuracy, the initial hydraulic support images as the basic source of training data are obtained from slightly more real hydraulic support images than the roof real images.

[0061] Because the construction of thin coal seam comprehensive mining working faces is not common in itself, the underground conditions of thin coal seams are more demanding than those of ordinary coal seams. Because the space is narrower, there is also a concern about whether the coal seam supported by the roof will crack or even collapse. Therefore, in the space underground in thin coal seams, the intelligent comprehensive mining system is rarely used for construction operations, which leads to fewer records of abnormal situations when using the intelligent comprehensive mining system for thin coal seam mining. Therefore, the abnormal data that can be input as sample data is actually obtained very little, and in conventional methods, it is difficult to train an accurate model with small sample abnormal data. This embodiment processes and expands small sample abnormal data so that the visual anomaly monitoring model finally trained has sufficiently accurate computing power and calculation results, which can meet the recognition of abnormal situations underground in thin coal seams.

[0062] a3, according to the previous step a2, defogging the roof historical images of the corresponding time period including the initial roof abnormal image and the initial roof normal image to obtain the roof defogging historical image, and defogging the hydraulic support historical images including the initial hydraulic support abnormal image and the initial hydraulic support normal image to obtain the hydraulic support defogging historical image; a4, mark the crack-prone areas in the roof defogging historical image and the preset monitoring crack areas of the thin coal seam fully mechanized mining working face, and obtain the marked roof historical image; mark the frame moving deformation area in the hydraulic support defogging historical image and the preset monitoring deformation area of ​​the thin coal seam fully mechanized mining working face, and obtain the marked hydraulic support historical image. The specific marking can be carried out by existing technical means, which will not be repeated here.

[0063] a5, based on the annotated roof historical image, compare the crack-prone area and the monitored crack area to determine the monitoring result of the roof historical image; Specifically, when the roof monitoring unit is used to judge the stability of the roof, first, the roof monitoring result is obtained by judging whether there are cracks in the roof. When the crack-prone area in the real-time image of the roof obtained by real-time shooting is compared with the monitored crack area on the corresponding image of the preset thin coal seam fully mechanized mining working face, cracks appear, then the roof monitoring result is an abnormal state of the roof. On the contrary, if there are no cracks in the crack-prone area, the roof monitoring result is a normal state of the roof. Through image comparison, the corresponding images of the four levels in the real-time image of the roof and the pre-stored initial roof abnormal image are compared in order from high to low levels to judge the abnormal level of the current abnormal state of the roof, and perform the corresponding level of alarm and repair work. In this way, abnormal states with high damage levels can be promptly alarmed and handled more quickly.

[0064] When the hydraulic support monitoring unit is used to judge whether the hydraulic support frame shifting is normal and whether it does not affect the normal operation of the coal mining machine, first, the hydraulic support monitoring result is obtained by judging the displacement of the hydraulic support frame shifting. Specifically, when the frame shifting deformation area in the real-time image of the hydraulic support obtained by real-time shooting is compared with the monitoring deformation area on the corresponding image of the preset thin coal seam fully mechanized mining working face, if the hydraulic support at the corresponding position has not shifted, that is, the position of a hydraulic support in the frame shifting deformation area in the whole row of hydraulic supports is different from the position of a corresponding hydraulic support in the monitoring deformation area in the whole row of hydraulic supports, then the hydraulic support monitoring result is the abnormal state of the hydraulic support, on the contrary, if the hydraulic support at the corresponding position has shifted, then the hydraulic support monitoring result is the normal state of the hydraulic support. Then, when the monitoring result is the abnormal state of the hydraulic support, the corresponding images of the four levels in the real-time image of the hydraulic support and the initial abnormal image of the hydraulic support are further compared from high to low levels, the abnormal level of the current abnormal state of the hydraulic support is judged, and the corresponding level of alarm and repair work is performed. In this way, the abnormal state with a high damage level can be alarmed and handled more quickly.

[0065] The roof monitoring unit and the hydraulic support monitoring unit together constitute a two-dimensional visual anomaly monitoring model.

[0066] In this embodiment, during the process of roof abnormality monitoring, the cracks are monitored and located by using the improved YOLOv5s, and existing technical means such as the Elf Annotation Assistant can be used to draw a closed crack-prone area with a polygonal frame, and the center point coordinates of the monitoring and positioning rectangular frame are extracted. When there are no cracks in the crack-prone area in the monitoring video, that is, the center point coordinates of the cracks do not exist or fall outside the crack-prone area, the crack monitoring rectangular frame is marked in green, indicating that there are no cracks or the cracks do not affect the continued operation; when the center point coordinates of the cracks in the monitoring video fall within the crack-prone area, the crack monitoring rectangular frame is marked in red, indicating that cracks that may affect safe operation appear in the crack-prone area, and the existing technical means are used to generate an early warning signal for early warning. In the process of monitoring the abnormality of the hydraulic support, the improved YOLOv5s model trained by the hydraulic support data set is used to monitor and locate the real-time image of the hydraulic support corresponding to the hydraulic support video, and the existing technical means such as the wizard annotation assistant are used to draw the closed operation area of ​​the current hydraulic support that needs to be moved with a polygonal frame, and the center point coordinates of the monitoring and positioning rectangular frame are extracted. When the center point coordinates of a hydraulic support that needs to be moved in the monitoring video fall outside the preset hydraulic support frame displacement (including the position point at the center point away from the hydraulic support frame displacement), it is indicated as a normal frame shift, and the rectangular frame of the hydraulic support monitoring is marked green; when the center point coordinates of the hydraulic support that should be moved in the monitoring video fall within the distance area from the hydraulic support frame displacement (excluding the position point at the center point away from the hydraulic support frame displacement), it means that the hydraulic support has not moved to the moving position, which may affect the coal mining machine to pass and cause a collision, or it has not moved back to the original position after the coal mining machine passes, which may affect the support of the coal seam on the roof. At this time, the rectangular frame of the hydraulic support monitoring is marked red, and an early warning signal is generated for early warning. like Figure 2 As shown, the abnormal warning system for thin coal seam fully mechanized mining working face that implements the above warning method includes a central processing unit 1 and a collection module 2, an interception module 3, a classification module 4, a defogging module 5, a construction module 6, a monitoring module 7 and a warning module 8 respectively connected to the central processing unit.

[0067] Acquisition module 2 is used to capture real-time video of the underground comprehensive mining face in thin coal seams; The acquisition module in this embodiment includes inspection cameras installed on inspection robots and fixed cameras installed at key positions of key scenes underground. It should be noted that a fixed camera only means that it cannot be moved underground, but a fixed camera can be rotated to capture 360-degree shooting angle images at a fixed position.

[0068] The interception module 3 is used to intercept the image in the real-time video of the fully mechanized mining working face of the thin coal seam to obtain the real-time image of the fully mechanized mining working face.

[0069] Classification module 4 is used to classify the real-time image into the real-time image of the roof and the real-time image of the hydraulic support according to the shooting content; it should be noted that the hydraulic support in this embodiment mainly refers to a structure connected to the roof to connect between the roof and the bottom plate, and together with the roof and the bottom plate, supports the underground space to form a fully mechanized mining working face. Multiple hydraulic supports are arranged to form a row of movable support columns used to support the direction of the coal mining machine. When a hydraulic support is in the forward direction of the coal mining machine, before the coal mining machine arrives, the frame is moved backward in the width direction perpendicular to the arrangement direction of the hydraulic support (i.e., the direction of the coal mining machine), providing the coal mining machine with a wider space for coal mining operations. When the coal mining machine leaves, the frame is moved forward in the width direction perpendicular to the arrangement direction of the hydraulic support, so as to support the roof and the coal seam above more forward after the coal mining machine leaves, and provide a more stable space for fully mechanized mining working face of thin coal seams. The frame moving function of the hydraulic support is a common function of the hydraulic support in the intelligent fully mechanized mining system.

[0070] The defogging module 5 is used to defog the roof real-time image and the hydraulic support real-time image respectively to obtain a roof defogging image and a hydraulic support defogging image; Building module 6, for building an improved YOLOv5s visual anomaly monitoring model using the collected small sample anomaly data; Monitoring module 7, used for inputting the defogging image into the trained improved YOLOv5s visual anomaly monitoring model to obtain monitoring results; the monitoring results are the normal state of the roof, the normal state of the hydraulic support, the abnormal state of the roof and the abnormal state of the hydraulic support; The early warning module 8 is used to generate an early warning signal when the monitoring result is that the roof is in an abnormal state or the hydraulic support is in an abnormal state.

[0071] This embodiment can monitor the fully mechanized mining face under the thin coal seam in real time for the intelligent fully mechanized mining system arranged in the thin coal seam. It uses the small sample abnormal data that can be collected to build an improved YOLOv5s visual abnormality monitoring model suitable for the current application scenario on the basis of the original YOLOv5s visual abnormality monitoring model, which can overcome the defects of small samples, provide early warning and alarm for the fully mechanized mining of thin coal seams in a timely and accurate manner, and provide a safer working environment for the fully mechanized mining construction of thin coal seams. In addition to being applied to abnormal early warning of thin coal seams, this solution is also applicable to early warning of support structures in other narrow spaces, effectively increasing the safety of the corresponding working environment.

[0072] Embodiment 2 Different from the first embodiment, in this embodiment, when the roof monitoring unit is used to judge the stability of the roof, the roof monitoring result is obtained by judging whether the roof cracks appear and the size of the roof cracks. When the crack-prone area in the real-time image of the roof obtained by real-time shooting is compared with the monitoring crack area on the corresponding image of the preset thin coal seam fully mechanized mining working face, cracks appear, and the width of the crack is greater than the preset crack width on the monitoring crack area, that is, the width of the crack is less than or equal to 1 cm, and the length of the crack is less than or equal to 2% of the length of the roof, but the crack is in a specific area, then the roof monitoring result is that the roof is abnormal and in an emergency state. The roof is abnormal and the emergency state is a specific situation in the general abnormality of the roof. The specific area is based on the specific construction environment to affect the stress of each area of ​​the roof, and the division of certain stress areas of the roof is carried out in advance. The stress in the specific area is relatively balanced, and it can play a certain role in extending the service life by repairing after the cracks appear. Therefore, when cracks are identified in a specific area, and the cracks are within the above repairable range, in addition to the early warning and shutdown operation according to the abnormal state of the roof, it is also necessary to perform emergency treatment on the crack-prone area where the cracks appear.

[0073] In this embodiment, by spraying polymer chemical reinforcement materials onto the crack-prone areas where cracks appear, this embodiment adopts FSS (Fast Setting System) reinforcement materials, and utilizes its rapid solidification characteristics to suture the cracks that have already appeared on the roof. It can quickly penetrate into small cracks and solidify in a short time to form a high-strength adhesive, effectively reinforce the rock formation, and can win precious time for subsequent processing, which is conducive to alleviating the instability of the coal mine roof or the need for emergency reinforcement, effectively curbing the expansion of damage, and ensuring the safety of the thin coal seam comprehensive mining working face.

[0074] Embodiment 3 Different from the first embodiment, in this embodiment, when classifying the real-time images of the roof and the real-time images of the hydraulic support, the process is as follows: First, image preprocessing is performed: grayscale conversion, converting the color image of the real-time image into a grayscale image: converting the RGB image into a grayscale image, the formula is: I_gray = 0.299R + 0.587G + 0.114B, where R, G, and B represent the pixel values ​​of the red channel, green channel, and blue channel respectively, and I_gray represents the grayscale value.

[0075] Then, feature extraction is performed: the Canny edge detection algorithm is used to extract image edges, and the SIFT (Scale-Invariant Feature Transform) algorithm is used to detect feature points.

[0076] Finally, feature matching and classification: feature matching algorithms (such as BFMatcher or FLANN) are used to match feature points in each real-time image, and compared with the preset hydraulic support image and the preset roof image in the preset image. Feature matching can help determine the similarity between the two images. If the similarity of the matching result reaches 89% or more, the corresponding real-time image is matched and classified into the corresponding image, that is, when the similarity between a frame of real-time image and the preset hydraulic support image is greater than or equal to 89%, the real-time image is a real-time image of the hydraulic support; if the similarity with the preset roof image is greater than or equal to 89%, the real-time image is a real-time image of the roof; if the similarity between the real-time image and the preset roof image and the preset roof image is greater than or equal to 89%, the real-time image is both a real-time image of the hydraulic support and a real-time image of the roof.

[0077] This embodiment can quickly and accurately complete real-time image classification without relying on a deep learning model.

[0078] Embodiment 4 In this embodiment, when collecting small sample abnormal data, the few collected photos of abnormal faults or abnormal defects are processed, and corresponding three-dimensional abnormal pictures are formed through three-dimensional modeling. The three-dimensional abnormal pictures are then rendered to form simulated pictures that simulate real scenes. The initial top plate abnormal image and the initial hydraulic support abnormal image are both simulated pictures. This setting enables the photos actually taken to be rendered to simulate real scenes after three-dimensional modeling, and can effectively capture the local details of defects or abnormal faults, while eliminating the interference details caused by shooting, so that the subsequent expansion of abnormal data is more accurate and has smaller errors.

[0079] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field are aware of all the common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement the scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for the ordinary technicians in the relevant field to implement this application. It should be pointed out that for the technicians in this field, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A method for abnormal early warning of a fully mechanized mining face in a thin coal seam, characterized in that: The following steps are involved: Step 1: Collect real-time video of the thin coal seam fully mechanized mining face through the inspection robot and the camera fixedly installed underground; Step 2: capture corresponding frame images from the real-time video to obtain real-time images; classify the real-time images according to the shooting content to obtain real-time images of the top plate whose shooting content includes the top plate content and real-time images of the hydraulic support whose shooting content includes the hydraulic support content; Step 3, respectively defogging the roof real-time image and the hydraulic support real-time image to obtain a roof defogging image and a hydraulic support defogging image; Step 4, constructing an improved YOLOv5s visual anomaly monitoring model based on the collected small sample anomaly data; the visual anomaly monitoring model includes a roof monitoring unit and a hydraulic support monitoring unit with the same structure; before constructing the improved YOLOv5s visual anomaly monitoring model, first process the collected small sample anomaly data, the small sample anomaly data includes an initial roof anomaly image and an initial hydraulic support anomaly image; enrich the initial roof anomaly image and the initial hydraulic support anomaly image respectively to obtain an extended roof anomaly image and an extended hydraulic support anomaly image; use the initial roof anomaly image and the extended roof anomaly image together as a roof anomaly image dataset, use the initial hydraulic support anomaly image and the extended hydraulic support anomaly image together as a hydraulic support anomaly image dataset, perform model training to construct an improved YOLOv5s visual anomaly monitoring model; Step 5, inputting the defogging image into the trained improved YOLOv5s visual anomaly monitoring model to obtain monitoring results; the monitoring results are the normal state of the roof, the normal state of the hydraulic support, the abnormal state of the roof and the abnormal state of the hydraulic support; Step six, when the monitoring result is an abnormal state of the roof or an abnormal state of the hydraulic support, an early warning signal is generated.

2. The abnormal warning method for fully mechanized mining working face in thin coal seams according to claim 1 is characterized in that: In step five, the defogging image of the roof is input into the roof monitoring unit, and the obtained roof monitoring result is the normal state of the roof or the abnormal state of the roof; the defogging image of the hydraulic support is input into the hydraulic support monitoring unit, and the obtained hydraulic support monitoring result is the normal state of the hydraulic support or the abnormal state of the hydraulic support.

3. The abnormal warning method for fully mechanized mining working face in thin coal seams according to claim 1 is characterized in that: In step 2, the real-time images are classified according to the shooting content. When the real-time images of the roof containing the roof content and the real-time images of the hydraulic support containing the hydraulic support content are obtained, the YOLOv5 model is used for classification, and the confidence threshold is set to 0.75 and the overlap threshold is set to 0.3 for classification, and the real-time images of the roof and the real-time images of the hydraulic support are obtained respectively.

4. The abnormal warning method for fully mechanized mining working face in thin coal seams according to claim 1 is characterized in that: In step 2, the real-time images are classified according to the shooting content, and when the real-time images of the roof whose shooting content includes the roof content and the real-time images of the hydraulic support whose shooting content includes the hydraulic support content are obtained, the following contents are adopted: First, the color image of the real-time image is converted into a grayscale image. The conversion formula is I_gray = 0.299R + 0.587G + 0.114B, where R, G, and B represent the pixel values ​​of the red channel, green channel, and blue channel respectively, and I_gray represents the grayscale value; Then, the Canny edge detection algorithm is used to extract the image edges, and the SIFT algorithm is used to detect feature points; Finally, the feature matching algorithm is used to match the feature points in the real-time image, and compared with the preset hydraulic support image and the preset roof image respectively, and the similarity of the matching results is calculated; if the similarity of the matching result reaches 89% or more, the corresponding real-time image is classified as follows: if the similarity with the preset hydraulic support image is greater than or equal to 89%, it is classified as a real-time image of a hydraulic support; if the similarity with the preset roof image is greater than or equal to 89%, it is classified as a real-time image of a roof; if the similarity with both the preset hydraulic support image and the preset roof image is greater than or equal to 89%, it is classified as both a real-time image of a hydraulic support and a real-time image of a roof.

5. The abnormal warning method for fully mechanized mining working face in thin coal seams according to claim 1 is characterized in that: In step 4, in the visual anomaly monitoring model, the roof monitoring unit monitors the stability of the roof by identifying the cracks in the roof, and the hydraulic support monitoring unit monitors the displacement of the hydraulic support by the displacement of the hydraulic support.

6. The abnormal warning method for fully mechanized mining working face in thin coal seams according to claim 5 is characterized in that: In step 5, the roof monitoring unit is used to determine whether there are roof cracks to obtain the roof monitoring result; when cracks appear in the crack-prone area in the real-time image of the roof obtained by real-time shooting compared with the monitoring crack area on the corresponding image of the preset thin coal seam fully mechanized mining working face, the roof monitoring result is an abnormal state of the roof; On the contrary, if there are no cracks in the crack-prone area, the roof monitoring result is that the roof is in a normal state; The monitoring result of the hydraulic support is obtained by judging the displacement of the hydraulic support by using the hydraulic support monitoring unit; when the deformation area of ​​the hydraulic support in the real-time image obtained by real-time shooting is compared with the monitoring deformation area on the corresponding image of the preset thin coal seam fully mechanized mining working face, if the hydraulic support at the corresponding position has not been moved, the monitoring result of the hydraulic support is that the hydraulic support is in an abnormal state; On the contrary, if the hydraulic support at the corresponding position is moved, the monitoring result of the hydraulic support is that the hydraulic support is in a normal state.

7. The abnormal warning method for fully mechanized mining working face in thin coal seams according to claim 1 is characterized in that: The initial roof abnormality image and the initial hydraulic support abnormality image are respectively obtained by the following steps to obtain an extended roof abnormality image and an extended hydraulic support abnormality image: For each initial image, any two of the following five transformation parameter combinations are used to form the corresponding extended image: a. Use random rotation (θ ∈ [-35°, 35°]) to change the image perspective; b. Implement random translation (x ∈ [-12%, 12%] image width, y ∈ [-12%, 12%] image height) to simulate different viewing angles; c. Perform random scaling (s ∈ [0.9, 1.1]) to adjust the image size and maintain content recognition; d. Randomly flip horizontally or vertically (flip ∈ {0, 1}) to increase sample diversity; e. Adjust the color balance by randomly changing the hue (hue ∈ [-10, 10]), saturation (saturation ∈ [0.9,1.1]) and brightness (brightness ∈ [0.9, 1.1]).

8. The abnormal warning method for fully mechanized mining working face in thin coal seams according to claim 1 is characterized in that: According to the possible accidents caused by abnormal conditions of the roof and hydraulic support, the abnormal conditions are graded, and the abnormal conditions are divided into four levels from small to large in terms of the possible impact of the accident. The corresponding small sample abnormal data are: The initial roof abnormality image includes a roof slight abnormality image, a roof general abnormality image, a roof serious abnormality image and a roof crisis abnormality image; The initial hydraulic support image includes a hydraulic support slight abnormality image, a hydraulic support general abnormality image, a hydraulic support serious abnormality image and a hydraulic support crisis abnormality image.

9. The abnormal warning method for fully mechanized mining working face in thin coal seams according to claim 8 is characterized in that: The slightly abnormal roof image is an abnormal image in which color difference and chipping occur in an area of ​​the local bottom surface of the roof that accounts for more than 5% of the entire bottom surface area, but all working indicators of the intelligent fully mechanized mining system are normal; The general abnormal images of the roof are: the roof has cracks with a width of less than or equal to 1 cm and a length of less than or equal to 2% of the length of the roof, or the area of ​​the local bottom surface of the roof that accounts for less than 5% of the entire bottom surface area collapses, and the abnormal images require strengthening support measures; The serious abnormal image of the roof is: an abnormal image in which the area of ​​the local bottom surface of the roof accounts for more than 5% and less than 10% of the entire bottom surface area and there is a risk of collapse, and emergency support is required; The roof crisis abnormal image is an abnormal image in which the roof collapse actually occurs, directly threatening the life safety of the workers; The slightly abnormal image of the hydraulic support is: an abnormal image in which the hydraulic support has a slight leakage of less than 3 parts, and the leakage amount is less than or equal to 5 drops / minute or 0.1 liter / hour; The general abnormal images of the hydraulic supports are: more than 3 hydraulic supports have synchronization or control function failures, or more than 3 consecutive leaks from the columns, and the leakage rate at any one of them is greater than 5 drops / minute or 0.1 liter / hour; The serious abnormal images of the hydraulic support are: abnormal images showing that the hydraulic support is unstable over a large area, the instability ratio exceeds 32%, or key structural parts are damaged; The hydraulic support crisis abnormal image is an abnormal image in which the hydraulic support system fails as a whole and faces the risk of a major safety accident.

10. An abnormal warning system for a fully mechanized mining face in a thin coal seam, characterized in that: Used to implement the abnormal warning method for a fully mechanized mining face in a thin coal seam as described in claim 1, comprising a central processing unit and a collection module, an interception module, a classification module, a defogging module, a construction module, a monitoring module and a warning module respectively connected to the central processing unit; The acquisition module is used to collect real-time video of the fully mechanized mining face in thin coal seams; The interception module is used to intercept the image in the real-time video of the fully-mechanized mining working face of the thin coal seam to obtain the real-time image of the fully-mechanized mining working face; A classification module, used for classifying the real-time image into a roof real-time image and a hydraulic support real-time image according to the shooting content; A defogging module is used to defog the roof real-time image and the hydraulic support real-time image respectively to obtain a roof defogging image and a hydraulic support defogging image; A building module for building an improved YOLOv5s visual anomaly monitoring model using the collected small sample anomaly data; The monitoring module is used to input the defogging image into the trained improved YOLOv5s visual anomaly monitoring model to obtain monitoring results; the monitoring results are the normal state of the roof, the normal state of the hydraulic support, the abnormal state of the roof and the abnormal state of the hydraulic support; The early warning module is used to generate an early warning signal when the monitoring result is an abnormal state of the roof or an abnormal state of the hydraulic support.

Images