Scraper conveyor chain breakage monitoring system based on AI video recognition

Abstract

The invention discloses a scraper conveyor chain breakage monitoring system based on AI video recognition, and belongs to the technical field of scraper conveyor chain breakage detection. The scraper conveyor chain breakage monitoring system comprises an image acquisition module, an image distribution module, a chain breakage diagnosis module, a centralized control module and a visual platform. According to the invention, a solution is provided for video integrated control management, abnormal condition video identification and video linkage control.

Description

Technical Field

[0001] The present invention relates to the technical field of scraper conveyor chain break detection, and more specifically to a scraper conveyor chain break monitoring system based on AI video recognition. Background Art

[0002] Scraper conveyor is a material conveying equipment commonly used in coal mining working faces. It transports materials from one place to another by driving chains and scraper assemblies. It has the advantages of large transportation energy and strong adaptability. It is widely used in underground coal mine working faces.

[0003] Scraper conveyors are key transportation equipment in coal mine production. During the operation of scraper conveyors on underground coal mining faces, chain breaks often occur due to jamming damage, overload impact fatigue deformation, wear and other reasons. If not discovered in time, it will be very difficult to deal with the chain break (including cleaning the goods on the conveyor, opening the skylight, and taking off the chain). Not only is it unsafe, it also seriously affects the normal production of coal mines and increases labor intensity. At the same time, due to the long-term operation, high-load work and complex and harsh working environment of scraper conveyors, especially after the chain rusts and corrodes, it is easy to break the chain, which leads to production interruption and equipment damage. If the scraper conveyor breaks the chain and is not corrected in time, there will be a greater risk for the subsequent transportation process. In the face of emergencies such as chain breakage, manual monitoring is not only costly, but also energy-limited. Traditional equipment maintenance methods are usually based on regular inspections, which have the risk of production interruption and cannot detect potential problems in time.

[0004] Therefore, how to provide a scraper conveyor chain break monitoring system based on AI video recognition that can solve the above problems is an issue that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a scraper conveyor chain break monitoring system based on AI video recognition, which realizes video intelligent AI recognition monitoring and promptly issues an alarm and locks the on-site equipment when a scraper conveyor chain is found to be broken.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A scraper conveyor chain break monitoring system based on AI video recognition, comprising:

[0008] Image acquisition module, image distribution module, link break diagnosis module, centralized control module and visualization platform;

[0009] Wherein, the image acquisition module acquires the conveying video of the scraper conveyor;

[0010] The image distribution module is connected to the image acquisition module, and is used to sample the acquired high-definition conveying video, obtain the image of the working area of ​​the scraper conveyor, and pre-process it as a data set;

[0011] The chain break diagnosis module uses the YOLOv8 target detection algorithm to identify the broken chain position of the scraper conveyor chain in the image through a chain break monitoring model trained and optimized based on a data set, and divides the working state of the chain into normal and broken chain states;

[0012] The centralized control module performs locking control based on the working status of the chain divided by the chain break diagnosis module;

[0013] The visualization platform realizes the visualization monitoring of the scraper conveyor chain breakage and the visualization operation of the centralized control system.

[0014] Preferably, the image acquisition module is composed of a plurality of cameras, which are mounted on a bracket at the tail of the scraper conveyor to collect conveying video of the scraper conveyor.

[0015] Preferably, the image distribution module is connected to the image acquisition module through SDK and standard RTSP protocol, the collected conveying video is sampled every 5 frames, the image of the working area of ​​the scraper conveyor is obtained, the image is annotated using the LabelImg tool, and the data set is expanded using data enhancement as the data set.

[0016] Preferably, data augmentation is used to expand the data set, including:

[0017] Flip, rotate, scale, crop, randomize, and adjust brightness, contrast, and saturation.

[0018] Preferably, using data enhancement to expand the data set also includes: adding random occlusion to the image, and using a data enhancement method such as Mosaic and Mixup image aliasing to further expand the data set.

[0019] Preferably, the centralized control module includes: a control unit and an alarm system;

[0020] When the chain break monitoring model identifies a chain break state, the control unit will stop the scraper conveyor, and the alarm system will transmit the stop state to the visualization platform for display and warning.

[0021] Preferably, the visualization platform is based on a B / S architecture.

[0022] Preferably, the visualization platform comprises:

[0023] Alarm notification unit, video monitoring unit, intelligent monitoring unit, equipment management unit, system management unit, real-time display and query functions in a visual form

[0024] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a scraper conveyor chain break monitoring system based on AI video recognition. Aiming at the actual situation of the scraper conveyor on the coal mining face, based on machine learning and computer vision technology, the scraper conveyor chain break state analysis, identification and alarm linkage are performed, and the specific beneficial effects are:

[0025] (1) Enhance the real-time and stability of video acquisition and monitoring and improve the accuracy of the video recognition platform through the image acquisition module and image distribution module;

[0026] (2) The image distribution module and chain break diagnosis module are used to group and classify images to construct a scraper conveyor chain break video set, and the scraper conveyor diagnosis model is trained to improve the accuracy of the chain break identification model.

[0027] (3) Linkage control of the centralized control module and the visualization platform to establish a fault detection and protection mechanism for the equipment operation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0029] Figure 1 is a system block diagram of the present invention;

[0030] Figure 2 It is a schematic diagram of the system working process of the present invention;

[0031] Figure 3 A schematic diagram of the network structure of YOLOv8 provided in an embodiment of the present invention;

[0032] Figure 4 A schematic diagram of the error (CIoU) between the predicted box and the calibrated box provided in an embodiment of the present invention;

[0033] FIG5( a) is a schematic diagram of first image data annotation provided by an embodiment of the present invention;

[0034] FIG5( b ) is a schematic diagram of annotating second image data provided by an embodiment of the present invention;

[0035] FIG5( c ) is a schematic diagram of annotating third image data provided by an embodiment of the present invention;

[0036] Figure 6 A schematic diagram of label information distribution provided by an embodiment of the present invention;

[0037] Figure 7 A schematic diagram of the training results of the link break monitoring model provided in an embodiment of the present invention;

[0038] Figure 8 A schematic diagram of the link break detection effect provided by an embodiment of the present invention;

[0039] Fig. 9 A schematic diagram of a confusion matrix of a broken link provided by an embodiment of the present invention;

[0040] Fig.10 A schematic diagram of a locking control flow chart provided by an embodiment of the present invention;

[0041] Fig.11 A schematic diagram of a login page of a visualization platform provided in an embodiment of the present invention;

[0042] Fig.12 A schematic diagram of the system main page of the visualization platform provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] The purpose of the present invention is to provide a scraper conveyor chain break monitoring system based on AI video recognition, including: an image acquisition module, an image distribution module, a chain break diagnosis module, a centralized control module and a visualization platform; wherein the image acquisition module acquires the conveying video of the scraper conveyor; the image distribution module is connected to the image acquisition module, and is used to sample the acquired high-definition conveying video, obtain the image of the working area of ​​the scraper conveyor and pre-process it as a data set; the chain break diagnosis module uses the YOLOv8 target detection algorithm, and identifies the broken chain part of the scraper conveyor chain in the image through the chain break monitoring model trained and optimized based on the data set, and divides the working state of the chain, including normal and broken chain state; the centralized control module performs locking control based on the working state of the chain divided by the chain break diagnosis module; the visualization platform realizes the visual monitoring of the scraper conveyor chain break and the visual operation of the centralized control system. Provide a solution for realizing the video integrated control management, abnormal situation video recognition and video linkage control of the scraper conveyor chain break monitoring system based on AI video recognition.

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] See also Figure 1 The embodiment of the present invention discloses a scraper conveyor chain break monitoring system based on AI video recognition, comprising:

[0047] Image acquisition module, image distribution module, link break diagnosis module, centralized control module and visualization platform;

[0048] Among them, the image acquisition module collects the conveying video of the scraper conveyor; the image distribution module is connected to the image acquisition module, and is used to sample the collected high-definition conveying video, obtain the image of the working area of ​​the scraper conveyor and pre-process it as a data set; the chain break diagnosis module uses the YOLOv8 target detection algorithm, and identifies the broken chain part of the scraper conveyor chain in the image through the chain break monitoring model trained and optimized based on the data set, and divides the working state of the chain, including normal and broken chain state; the centralized control module performs locking control based on the working state of the chain divided by the chain break diagnosis module; the visualization platform realizes the visual monitoring of the scraper conveyor chain break and the visual operation of the centralized control system.

[0049] Specifically, the image acquisition and distribution module: accesses the camera through SDK and standard RTSP protocol to obtain images of the working area of ​​the scraper conveyor. These images are pre-processed through the image processing library OpenCV, including operations such as annotation, normalization, and image enhancement. They are used for subsequent target detection and chain status analysis.

[0050] Chain break monitoring model: Using the YOLO (You Only Look Once) target detection series algorithm, the model is trained and adjusted to achieve automatic identification of the broken parts of the scraper conveyor chain and classify the working status of the chain, including normal and broken chain status. The YOLO algorithm can detect targets in the image in real time and efficiently, providing fast and accurate data for the monitoring system.

[0051] Locking control: The centralized control system outputs the locking control signal to complete the locking control of the scraper conveyor, or the system directly outputs high and low control signals to the scraper conveyor to complete the locking control.

[0052] Analysis application and visualization: The front-end page design, back-end function implementation, and data storage construction are completed through relevant technologies. The Web service is integrated with the working face centralized control system into a chain break detection system presented in the form of a visualization platform to ensure that each component works together and realizes the efficient operation of the scraper conveyor chain break monitoring system.

[0053] Specifically, the chain break monitoring and locking control system helps ensure the reliable operation of industrial scraper conveyors in harsh working environments, reduce maintenance and downtime, improve production efficiency, reduce maintenance costs, reduce potential hazards, protect the safety of workers and equipment, and ensure continuous production.

[0054] See also Figure 2 First, the conveying video of the scraper conveyor is collected through the image acquisition module, and the image distribution module uses RSTP+OpenCV and other technologies to obtain and process image data. Then the front-end page is designed and the Java Spring Security framework is used for security control. Among them, in the background function implementation part, it is necessary to apply the trained detection model to the image data after data preprocessing, and store the data through the MySQL database. Initially, if it is detected that the detection algorithm is not deployed, a new chain break monitoring model is constructed and adjusted and optimized to meet the needs. When the chain break monitoring model meets the deployment requirements, it is loaded into the chain break diagnosis module, and the centralized control module implements the functions of locking control and alarm signal sending through the MODBUS-TCP protocol. Finally, the analysis results are displayed on the visualization platform and the whole process ends.

[0055] In a specific embodiment, the image acquisition module of the scraper conveyor chain break monitoring system is composed of a plurality of high-definition cameras, which are installed on a bracket at the tail of the scraper conveyor to collect conveying videos of the scraper conveyor.

[0056] Specifically, the image acquisition module should be compatible with common camera brands as much as possible, and not rely on a certain brand of a certain type of camera. At the same time, the control signal output should support output using commonly used protocols, including but not limited to modbustcp, resful protocols, etc.

[0057] Specifically, it supports the use of ordinary non-computing cameras in coal mines to reduce hardware investment to a certain extent, so as to achieve the highest cost-effectiveness of software and hardware.

[0058] In a specific embodiment, the camera is the KBA12(A) mining intrinsically safe camera produced by Ping An Kaicheng Company, which is used for front-end image acquisition and alarm control output. The main functions of the product are:

[0059] Supports Ethernet electrical port, optical port, RJ11 and other transmission modes, and the number of interfaces can be customized.

[0060] The video signal is suitable for long-distance transmission, and the effective transmission distance of the RJ11 interface can reach 2000 meters.

[0061] Starlight-level ultra-wide dynamic high-definition camera and infrared auxiliary lighting functions are suitable for low-light environments underground and meet the needs of long-distance, high-quality video monitoring at the production site.

[0062] The electronic fence function can set the warning area and implement local voice warnings for people who break into the area.

[0063] The voice pickup and intercom function enables real-time audio and video synchronous communication with the ground monitoring center through the camera.

[0064] Specifically, the camera monitoring point is installed at the rear of the scraper. The camera supports docking through SDK and standard rtsp protocol. The chain break monitoring fault diagnosis module has strong real-time requirements and obtains the real-time image of the camera through the integrated camera SDK. The explosion-proof shell of the camera can play a certain protective role and increase the service life of the camera.

[0065] Specifically, the steps for connecting to the camera screen through the SDK are as follows:

[0066] Step 1: SDK installation and configuration

[0067] Visit the Hikvision official website, find the download page for the corresponding camera SDK, and select the appropriate SDK version to download based on the required platform and compilation environment. Unzip the downloaded SDK file and install it according to the installation instructions provided. The installation steps may vary depending on the operating system. Configure the environment: Add the SDK path to the operating system's environment variables to ensure that the library files and header files can be correctly found when compiling and running the code.

[0068] Step 2: Environment setup and project creation

[0069] Open an integrated development environment (IDE), such as Visual Studio, and configure the SDK environment, including the paths of library files and header files. If you need to use it with other libraries such as OpenCV, you also need to configure the corresponding header and library file paths. Create a new project for subsequent SDK docking and code writing.

[0070] Step 3: SDK connection steps

[0071] Load the DLL function library to ensure that the function library provided by Hikvision can be called correctly. Initialize the SDK, call the NET_DVR_Init() function or similar functions (according to the SDK version and documentation) to initialize the entire network SDK system, including memory pre-allocation and other operations. Set the device information and log in, call the login function (such as NET_DVR_Login_V40()), provide the device's IP address, port number (default is 554), user name and password to log in to the device. Perform specific operations. After successful login, call the corresponding function according to specific needs to perform real-time preview, capture and other operations.

[0072] The image processing library OPENCV is used to process the collected camera images, including image contrast enhancement and image filtering. The video frames processed by OPENCV are clearer and sent to the chain break monitoring module for processing.

[0073] In a specific embodiment, the chain break monitoring model in the chain break diagnosis module is the core of the entire system. It automatically detects whether the chain state of the scraper conveyor is abnormal such as chain break based on the input image data, and sends the detection results to the upper system for subsequent processing. The main functions of this module include:

[0074] 1. YOLOv8 online detection model

[0075] Based on the YOLOv8 detection model that is pre-trained and fine-tuned on the broken chain image dataset, it can automatically detect whether there is chain breakage, chain derailment or other chain abnormalities in the scraper conveyor in the image data according to the input image.

[0076] 2. Model application service

[0077] The detection results of the YOLOv8 online detection model are saved in the database and a real-time interface is provided to feedback to the upper-level locking control module and application system calls.

[0078] 3. Model training and deployment management:

[0079] The detection model is trained and optimized using image data collected by the camera system. The continuously collected data is used to smoothly upgrade the model while keeping the system running.

[0080] Specifically, YOLOv8 is an advanced target monitoring algorithm that can efficiently process large-scale data sets. Considering the real-time, demand and accuracy requirements of scraper conveyor chain break monitoring, YOLOv8 is selected as the scraper conveyor chain break monitoring algorithm.

[0081] Specifically, the network structure of YOLOv8 is as follows Figure 3 As shown in Figure 1, it uses a similar backbone network to YOLOv5, but with some changes on CSPLayer, now called the C2f module. The C2f module (a cross-stage partial bottleneck layer structure with two convolutions) combines high-level features and contextual information to improve detection accuracy.

[0082] Specifically, YOLOv8 uses an anchor-free model and a decoupled Head module to independently handle target, classification, and regression tasks. This design allows each branch to focus on its task, improving the overall accuracy of the model. The output layer of YOLOv8 uses the Sigmoid function as the activation function for the target score, indicating the probability that the bounding box contains an object. The Softmax function is used to calculate the category probability, indicating the probability that the object belongs to each possible category.

[0083] Specifically, as mentioned before, YOLOv8 uses a new architecture that combines FAN and PAN modules. FPN is used to generate feature maps of multiple scales and resolutions, while PAN is used to aggregate features from different layers of the network to improve accuracy. The combined FAN and PAN modules outperform YOLOv5, which uses a modified version of the CSPDarknet architecture. This modified version of CSPDarknet is based on Cross-Stage Partial Connections (CSP), which improves the information flow between different parts of the network.

[0084] Specifically, YOLOv8 includes more advanced post-processing techniques than YOLOv5, which is a set of algorithms applied to the predicted bounding boxes and objectness scores generated by the neural network. These techniques help improve detection results, eliminate redundant detections, and improve the overall accuracy of predictions. YOLOv8 uses Soft-NMS, a variation of the NMS technique used in YOLOv5. Soft-NMS applies a soft threshold to overlapping bounding boxes instead of discarding them directly. NMS removes overlapping bounding boxes and only keeps the bounding boxes with the highest objectness scores.

[0085] Specifically, the output head refers to the last layer of the neural network, which is used to predict the location and category of objects in the image. In the YOLO architecture, there are usually several output heads responsible for predicting different aspects of the detected object, such as bounding box coordinates, class probabilities, and object scores. These output heads are usually connected to the last few layers of the neural network and are trained to output a set of values ​​that can be used to locate and classify objects in the image. The number and type of output heads used can vary depending on the specific object detection algorithm and the needs of the task at hand. YOLOv5 has 3 output heads, while YOLOv8 has 1 output head. YOLOv8 does not have small, medium and large anchor boxes but instead uses an anchor-free detection mechanism that directly predicts the center of the object instead of offsetting it from a known anchor box, reducing the number of boxes predicted and speeding up the post-processing process.

[0086] In a specific embodiment, the classification loss function in YOLOv8 is VFL loss (Varifocal Loss), and the regression loss takes the form of bounding box positioning loss and DFL loss (DistributionFocal Loss).

[0087] Specifically, the bounding box positioning loss L box ,include:

[0088] The error between the predicted box and the calibrated box (CIoU) is used. IoU is the intersection over union ratio of the predicted box and the true box, and is an indicator for measuring the accuracy of target object detection. The larger the IoU, the higher the overlap between the predicted box and the true box. Compared with IOU, CIOU is more sensitive to the target size, so using CIOU instead of IOU can improve the accuracy of small target object detection. The schematic diagram of CIOU is as follows Figure 4 As shown. In, is the Euclidean distance between the center points of the predicted box and the true box, and c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box.

[0089] Among them, w gt and h gt is the width and height of the real box, w and h are the width and height of the predicted box. The υ variable in the formula is a penalty term, which indicates the difference in aspect ratio between the predicted box and the real box. If they are the same, υ = 0; if the difference is larger, υ is larger. α is the υ weight value.

[0090] Specifically, the classification loss L vfl ,include:

[0091] To calculate the confidence of the model, we use the Varifocal Loss function to solve the problem of uneven positive and negative samples and propose an asymmetric weighted operation:

[0092]

[0093] Among them, q is the label, q is the IoU value for positive samples, otherwise q is 0. That is, when it is a positive sample, it is a general binary cross entropy BCE loss function, and when it is a negative sample, it is a Focal loss function. Varifocal Loss is no longer symmetrical in its treatment of positive and negative samples. It highlights the positive sample as the main body by considering the different importance of the two.

[0094] Specifically, the confidence loss L dfl ,include:

[0095] Because the interval points near the true coordinates have a higher probability, the Distribution Focal Loss, referred to as DFL, is proposed:

[0096] DFL(Si ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 ))

[0097] Where S is the probability output value. As can be seen from the formula, in y i When it is very close to y and the S probability output is large, L dfl is small, making the distribution close to the center of the annotation. Therefore, L dfl This enables the proposed network to focus on values ​​near the target y more quickly, increasing their probability and accelerating convergence.

[0098] In a specific embodiment, all data samples used are collected by the monitoring system at the rear tail position of the scraper conveyor, and multiple on-site data collection points are set to collect the conveying video of the scraper conveyor. The collected video is sampled every 5 frames, and a total of 20,837 images are obtained as a data set, and the data set is randomly divided into a training set and a validation set at a ratio of 8:2. The training set and validation set have 16,669 and 4,168 images, respectively, of which there are 734 and 191 broken chain images, respectively. The test set consists of 20 groups of broken chain conveying scenes and 20 groups of normal conveying scenes, with a total of 16,136 images, including 412 broken chain images.

[0099] Use the LabelImg tool to label the image data, such as Figure 5(a)-Figure 5(c) The corresponding label information and its distribution are shown in Figure 6 shown.

[0100] Specifically, if the number of samples is too small, the generalization of the model will be poor, which may lead to overfitting and affect the detection performance. The target features extracted by the convolutional neural network are fixed, so data enhancement can be used to expand the data set. The data augmentation methods used in the detection of broken chain of scraper conveyor include flipping, rotation, scaling, cropping, random occlusion, brightness, contrast, and saturation adjustment. Flipping is one of the most common image enhancement methods, mainly including horizontal flipping and vertical flipping. Vertical flipping is equivalent to rotating the original image 180° around the center, and the target detection object is the same as the original. When a chain break occurs, the chain has various shapes. In order to make the model better able to cope with such a situation, we simulate the various shapes of the chain when the chain breaks by rotating the image. The scaling operation is divided into expansion and contraction. The size of the image will increase after expansion, and it needs to be cut to the original size after transformation. Cropping is to randomly crop an area in the original image and then adjust it to the size of the original image. The cropping operation can enlarge the smaller detection target in the original image and simulate the detection of the same object at different shooting distances. You can also cut off part of the original object to simulate the situation where the appearance of the object to be detected is incomplete due to occlusion, thereby enhancing the model's learning ability and target discrimination ability.

[0101] Since the scraper conveyor is exposed to the outside and the lighting conditions are uncertain, the brightness adjustment can be used to simulate different lighting environments. By increasing the brightness, the scene with high light intensity is simulated, and vice versa, the low-intensity light scene is simulated to enhance the learning ability of the model. The contrast and saturation of the image are changed to simulate the model in different scenes and noisy scenes to improve the robustness of the model.

[0102] Furthermore, considering the intrusion of people and animals, bad weather, and dust, random occlusion is added to the image to improve the robustness of the model. Data augmentation not only supplements the data with a small number of samples and alleviates the problem of data imbalance, but also reduces the cost of annotating data.

[0103] The proportion of chains in an image is generally small. When detecting small targets, after multiple feature extraction and downsampling operations, their feature information is often easily lost, which reduces the accuracy of the model. In order to better obtain feature information, it is necessary to change the size of the input image, but this operation will increase the amount of calculation of the model and consume more memory and resources. If the downsampling operation is restricted, it will not be conducive to extracting deep information. In order to solve this problem, image aliasing data enhancement methods such as Mosaic and Mixup are used to further expand the data set, and the recognition rate can be improved by only increasing a small amount of calculation.

[0104] Mosaic refers to randomly selecting four images in the data set, and after scaling, stitching them together in the order of upper left, lower left, upper right, and lower right to obtain a new image of uniform size. The new image contains richer background information than the original one, and the stitching operation also increases the number of samples (batch size) passed to the model for training at a single time, making the subsequent batch normalization (BatchNormalization) calculation less dependent on the batch size itself. Mixup data enhancement is no longer based on stitching but on aliasing. The core idea of ​​Mixup is to randomly select two original images and mix them in a certain proportion to generate a new image.

[0105] In a specific embodiment, the link break monitoring model training includes:

[0106] According to the special needs of scraper conveyor chain monitoring, the hyperparameters of the YOLOv8 algorithm were adjusted. In the training stage, the YOLOv8 pre-trained weights were used, the batch-size was set to 16, the epoch was set to 200, the SGD gradient descent optimization algorithm was used, the input image pixel size was 640×640, the initialization learning rate was set to 0.01, the weight decay coefficient was 0.000005, and the rest of the hyperparameters used the YOLOv8 default configuration.

[0107] The constructed YOLOv8 model (chain break monitoring model) is trained using the collected image data, and the performance of the chain break monitoring model in the scraper conveyor chain break monitoring task is improved by continuously optimizing the chain break monitoring model. During the training process, this embodiment uses the chain break data with labeled information as positive samples and the normal data without labeled information as background negative samples. The training results are shown in Figure 2. Figure 7 As shown, we can see that during the training process, the losses gradually decrease and finally converge, and all indicators on the validation set gradually increase.

[0108] In a specific embodiment, the evaluation of the chain break monitoring model includes:

[0109] For chain break monitoring of scraper conveyors, the performance of identifying and locating the area where the chain break occurs is a crucial indicator. The evaluation elements include the detection model correctly identifying the broken chain conveying as broken chain conveying (TP), the detection model incorrectly identifying the normal conveying as broken chain conveying (FP), and the detection model incorrectly identifying the broken chain conveying as normal conveying (FN). This embodiment evaluates the model from the perspectives of detection accuracy, false detection, and overall model performance, so precision (Precision, P), recall (Recall, R), mAP50, and mAP50-95 are selected as evaluation indicators. The recall rate is used to evaluate the detection model's ability to correctly identify images in which real chain breaks occur. The higher the recall rate, the lower the false detection rate. mAP50 and mAP50-95 are used to evaluate the performance of object detection models, where n represents the total number of categories, AP i Represents the average precision (AveragePrecision) of each category i. The AP of each category is calculated as the area under the precision-recall curve.

[0110]

[0111] In terms of performance evaluation, the model was tested in 20 groups of broken chain conveying scenarios and 20 groups of normal conveying scenarios. The test results are shown in Table 1. It can be seen that the overall recognition accuracy rate reached 95.1%, the recall rate reached 85.2%, the mAP50 reached 92.8%, and the mAP50-95 reached 54.4%. This shows that after sufficient training, the system has a high recognition accuracy rate for broken chain anomalies, meets the detection accuracy requirements of broken chains of industrial scraper conveyors, and has high practical value. Figure 8 shown.

[0112] Table 1 Test set detection results

[0113]

[0114] The confusion matrix for broken links is as follows Fig. 9 As shown, it can be observed that among the 412 broken chain images, 336 were correctly detected and 74 were identified as broken chains. The system did not mistake the background as broken chains, which means that for the broken chain data, the false detection rate of the model is 0%.

[0115] Specifically, the results on the experimental data show that the target detection algorithm based on YOLOV8 is selected as the core technology for link break fault detection. By training the model on the collected data set, better fault detection effects can be achieved, which meets the technical indicators of technology research and development.

[0116] In a specific embodiment, when the chain break monitoring module detects a chain break or other potential dangerous situation, the centralized control module will output a control signal through the I / O port through the linkage logic preset by the centralized control module to quickly take measures to cut off the power supply to stop the scraper conveyor, and implement locking control to avoid further damage and protect the safety of the operator. The locking control process is as follows: Fig.10 Shown

[0117] In a specific embodiment, the visualization function is realized through a visualization platform. Specifically: a visualization platform is developed based on a B / S architecture, and the visualization platform includes: an alarm notification unit, a video monitoring unit, an intelligent monitoring unit, an equipment management unit, and a system management unit. After the alarm notification, video monitoring, intelligent monitoring, equipment management, system management and other functions of the chain break monitoring system are integrated, they can be displayed and queried in real time on terminals such as desktop computers and industrial large-screen TVs through a visualization service.

[0118] The homepage of the visualization platform uses chart visualization technology to display camera information, recognition task information, common event information, and alarm event information in different dimensions, allowing users to quickly and conveniently access information on key points of concern. In addition, important alarm events support pop-up red light alarms, and alarm events that receive device linkage locking signals also have linkage locking disposal pop-up information to provide timely reminders of key information.

[0119] Specifically, the event menu is divided into common events, alarm events, and control events according to the attributes of the events. All types of events contain rich recorded information pictures or historical video data. Alarm events support the handling function of alarm events and support docking with business handling processes. Control events are used to record the control signals sent by edge control terminals to facilitate event backtracking.

[0120] Specifically, the video monitoring unit supports viewing the real-time video, historical video, and identification video of the docking camera, and supports searching the camera by camera name and camera group. The video screen supports 1-grid, 4-grid, 9-grid, and 16-grid display modes.

[0121] Specifically, the intelligent monitoring unit includes several core functions of the platform, including recognition task configuration, recognition details viewing, event type configuration, and algorithm configuration. The recognition task configuration function can flexibly bind intelligent recognition tasks of specific scenes to ordinary cameras. At the same time, the camera recognition area can be configured through rectangular boxes, polygons, lines, etc. The recognition details menu item is convenient for viewing the marked video stream after algorithm recognition. Event type configuration supports configuring event attributes and alarm levels, and alarm events support custom home page alarm reminder methods. The algorithm configuration module supports flexible expansion and deletion of recognition algorithms for the platform.

[0122] The device management unit includes two core functions: camera management and computing power device management. The camera management module and the streaming media server platform are automatically integrated to facilitate the rapid entry of camera point information. The computing power device management supports the centralized management of all server resources and edge devices with AI computing power in the platform, and can display the resource usage of the device in real time on the web page, so as to flexibly expand the computing power resource equipment according to the amount of recognition tasks.

[0123] The system management unit includes common management of users, permissions, logs, roles, etc. The permission module can decentralize and hierarchically view all data on the monitoring platform according to different roles. It also supports flexible binding of user and camera permissions.

[0124] In a specific embodiment, the data calculation and operation of the scraper conveyor chain break monitoring system based on AI video recognition provided by the present invention are local, so that the data does not leave the mine. At the same time, a multi-level and comprehensive security protection system should be established, including security protection measures for strategy, physical security, peripheral layer, network layer, host layer, application layer and data layer.

[0125] In a specific embodiment, a scraper conveyor chain break monitoring system based on AI video recognition provided by the present invention is deployed in a containerized manner and operates continuously and stably.

[0126] In a specific embodiment, the software development part of the scraper conveyor chain break monitoring system based on AI video recognition provided by the present invention realizes the management, display, alarm and recording functions of the front-end AI high-definition camera. It includes:

[0127] The chain-breaking fault diagnosis algorithm obtains the monitoring screen of the front-end camera in real time to monitor the chain-breaking situation of the scraper conveyor. When the AI ​​algorithm recognizes that a fault has occurred, on the one hand, it sends a locking control signal to the scraper conveyor for locking control to avoid causing large economic and property losses, and on the other hand, it stores the alarm event and control event information. Therefore, it is necessary to develop a platform management system that allows managers to understand the operation of the scraper conveyor in real time through the camera screen, and at the same time query the recognition results of the AI ​​algorithm.

[0128] The B / S architecture has a strong distributed feature, allowing users to query, browse and process business anytime and anywhere. This feature makes the B / S architecture very suitable for application scenarios that require cross-regional and cross-device access. The maintenance of the B / S architecture is very simple. Since the client mainly accesses through the browser, it only needs to change the web page to achieve synchronous update for all users. This centralized maintenance method greatly improves maintenance efficiency.

[0129] By using Java as the main programming language for system development, the system can achieve cross-platform features, making it portable across different operating systems. Java's powerful ecosystem and widely used framework provide the system with a wealth of tools and libraries, simplifying the development process and improving the maintainability of the code.

[0130] In the system architecture, Java can be used to develop backend services, process data collected by sensors, implement disconnection monitoring algorithms, and provide external API interfaces. Its multithreading and concurrency features help process real-time data and improve the system's response speed. In addition, Java's security and stability make it an ideal choice for processing industrial-grade applications.

[0131] Spring Security is a powerful security solution for the Spring Framework, focusing on providing comprehensive authentication, authorization, and security capabilities for Java applications.

[0132] By integrating Spring Security, the system can realize secure authentication, authorization and session management of user identity. This is particularly important for the scraper conveyor chain break monitoring system, because the system may involve sensitive data and key operations, and it must ensure that only authorized users can access and execute specific functions.

[0133] Spring Security provides flexible configuration options that enable the system to customize various security policies, including role-based access control, password encryption, and secure logging. Its integration also helps prevent various common security attacks, such as cross-site scripting (XSS) and cross-site request forgery (CSRF).

[0134] By adopting Spring Security, the scraper conveyor chain break monitoring system can effectively respond to security challenges at the system design level, ensuring the privacy of user data and the stability of the system. This further enhances the credibility and practicality of the system in industrial environments.

[0135] Nginx architecture is a high-performance web server and reverse proxy server. Its design goal is to solve the C10k problem, that is, to maintain high performance when there are more than 10,000 connections at the same time. Nginx is known for its event-driven, asynchronous non-blocking processing method, which can effectively handle a large number of concurrent connections.

[0136] Nginx can not only provide powerful reverse proxy and load balancing functions, optimize front-end and back-end communication, but also accelerate page loading through efficient static resource services. Nginx's security features and dynamic module support enhance the stability and flexibility of the system, making it an indispensable technical component for integrating multiple functions.

[0137] The main uses include reverse proxy, forwarding client requests to the backend application server, and load balancing, distributing traffic to multiple servers to improve system performance and availability. Nginx also handles static files well, supports file caching and multiple protocols, and is suitable for providing fast and efficient static content. Its reverse proxy caching function can cache static or dynamic content on the Nginx server, reducing the burden on the backend server and improving response speed.

[0138] Nginx supports SSL / TLS protocols, which can be used for secure data transmission, providing encryption and authentication functions. Its modular design allows users to choose different modules according to their needs, achieving flexible expansion and customization. As open source software, Nginx can run on a variety of operating systems, including Linux, Unix, Windows, etc. Overall, Nginx is a powerful, flexible and high-performance web server and reverse proxy server, which is widely used to build high-performance, reliable and scalable web services and applications.

[0139] Specifically, the front-end technology Vue, as a modern JavaScript framework, is mainly used for the development of front-end user interfaces. By adopting Vue, the system can implement a responsive user interface, provide a smooth user experience, and simplify the development and maintenance of front-end components.

[0140] The front-end part of the system can use Vue to build an interactive monitoring interface. The modular development of Vue components improves the maintainability of the front-end code. Vue's data binding and componentization features make data interaction with the back-end Java service more convenient, achieving seamless integration of the front-end and back-end.

[0141] Specifically, the message middleware Redis is a high-performance memory database and cache system. The main purpose of using Redis in the system is to speed up data access and improve the response speed of the system.

[0142] One of the main uses of Redis is as a cache storage for the system, especially for storing frequently accessed data, such as real-time data collected by sensors. By storing this data in memory, the system can quickly retrieve and update it, reducing the pressure on the backend database and improving overall read and write performance.

[0143] In addition, Redis can also be used to implement distributed locks, message queues and other functions, which provides strong support for concurrent processing and real-time notifications in the system. By utilizing the characteristics of Redis, the system can more effectively handle large-scale data and concurrent requests, improving the scalability and stability of the system.

[0144] Specifically, the MySQL architecture was selected as the relational database management system, which is responsible for storing and managing the system's persistent data. It plays a key role in the system, providing reliable data storage and efficient query processing, and is suitable for scenarios that require complex queries and transaction processing.

[0145] MySQL is used to store various key data, including user information, historical monitoring data, system configuration, etc. By using MySQL, the system can achieve long-term data preservation, support query and analysis of historical data, and provide comprehensive data support for system users. MySQL's transaction processing mechanism ensures the consistency and reliability of data during system operation. It also supports advanced SQL query language, allowing the system to retrieve and analyze data more flexibly.

[0146] Specifically, the login interface is as follows: Fig.11 As shown, enter the system login page, enter the user name, corresponding password and verification code entered in the background management system, and then log in to the system main interface.

[0147] Specifically, the system main interface is as follows Fig.12 As shown, according to the left-middle-right layout design, the left side contains three blocks: system status, common event statistics, and common event records; the middle contains two blocks: identification video and status statistics information; the right side contains three blocks: alarm statistics, alarm type, and alarm events.

[0148] Specifically, alarm notification visualization: alarm events are used to display the return results of the working face centralized control system defined as alarm events (broken chain events) in the intelligent monitoring event configuration. According to the severity of the event, the alarm events triggered at different locations can be set to different alarm levels, and different reminder methods can be configured. Pop-up window reminders appear in the lower right corner of the home page, sound reminders appear on the home page, and lock screen reminders appear in the middle of the home page.

[0149] Video monitoring visualization: The video inspection menu includes three secondary submenus: real-time video, video playback, and recognition video. Real-time video plays the real-time video stream of the camera entered on the camera management page, video playback plays the historical video stream recorded by the camera entered on the camera management page, and recognition video plays the analysis video stream of the algorithm recognition task configured on the recognition configuration page.

[0150] Intelligent monitoring visualization: The intelligent monitoring menu includes four secondary submenus: identification details, identification configuration, event configuration, and algorithm configuration. Identification details is used to display the video stream and uploaded events of the identification task at the same time. The identification configuration page includes the issuance, suspension, deletion, and restart operations of the identification task. Event configuration is used to configure the event information reported by the algorithm. Algorithm configuration is used to configure the algorithm types supported by the system. Event configuration and algorithm configuration are functions for management users and are not open to ordinary users.

[0151] Device management visualization: The device management menu includes three secondary submenus: camera group management, camera management, and inference device management. Camera group management is used to query, edit, add or delete camera group information. Camera management is used to record detailed information of each camera (such as location, status, etc.). Inference device management is used to record, query, add or delete, and edit inference device information (such as resources, type, etc.).

[0152] System management includes four secondary submenus: user management, role management, log management, and homepage management. User management is responsible for the maintenance of system platform users, including adding, deleting, and binding camera permissions for users. Role management configures the role to which the user belongs and configures specific menu permissions. Log management records the operation records of the platform. Homepage management is used to configure the location information of the homepage scene base map.

[0153] In the Web-based B / S architecture, the user interface and application logic run on the user's browser, while data processing and storage are concentrated on the server. This separation design makes the application easier to maintain and upgrade, while also providing convenient deployment and strong compatibility. By using Nginx to achieve load balancing, the performance and availability of Web applications are further improved.

[0154] Compared with Python, Java has stronger performance, multi-threading and concurrency support, and a rich ecosystem, making it an ideal choice for building large-scale enterprise applications. On the front end, lightweight Vue technology is used to build an interactive Web user interface to provide users with a good user experience. On the back end, mature technologies are used, such as the Spring Security module in the Spring framework, to handle user authentication, authorization, and session management to protect the application from unauthorized access and attacks. In terms of data storage, the fast read and write capabilities provided by Redis and MySQL for storing large amounts of persistent data are used to achieve efficient management and storage of data. This combination not only provides good performance, but also meets the application's needs for real-time and persistent data.

[0155] In a specific embodiment, the linkage control of a scraper conveyor chain break monitoring system based on AI video recognition provided by the present invention includes:

[0156] The experiment was conducted in the laboratory of Zhangjiakou Coal Machinery Plant. In this embodiment, a scraper conveyor was selected to install a camera at the tail of the machine, and an AI analysis server was deployed next to the scraper conveyor. The camera and the server were interconnected through the network.

[0157] In the test environment of this embodiment, the joint adjustment of the scraper conveyor is artificially disconnected, the disconnected chains are placed into different shapes, and different objects are placed to cover the chains. Through collection and cleaning, 20 groups of normal videos and 20 groups of abnormal data sets are sorted out.

[0158] The collected data was cleaned and labeled for model training, and three versions of chain break recognition models were trained successively. The accuracy of the chain break recognition model was continuously improved. Finally, the model was tested in 20 groups of broken chain conveying scenarios and 20 groups of normal conveying scenarios. The overall recognition accuracy reached 95.1%, the recall rate reached 85.2%, the mAP50 reached 92.8%, and the mAP50-95 reached 54.4%. This shows that after sufficient training, the system has a high recognition accuracy for chain break anomalies in an experimental environment, which meets the detection accuracy requirements of industrial scraper conveyors and has high practical value.

[0159] Specific test indicators and results:

[0160] Test indicators:

[0161] The scraper conveyor chain break diagnosis accuracy rate reaches over 80%, and the fault protection shutdown delay is less than 15 seconds.

[0162] 40 simulation scenarios, 20 of which are disconnection scenarios;

[0163] The collected videos are used for AI algorithm analysis and judgment. According to the 80% accuracy requirement, 16 or more groups of broken links are required for correct judgment;

[0164] The number of non-broken link scenarios that are mistakenly judged as broken links should not exceed 4 groups

[0165] The timing starts from the moment the chain break scene appears and disappears in the camera's video. The AI ​​software should present a shutdown and lockout prompt message on the software main screen within 15 seconds (synchronously output a shutdown and lockout signal).

[0166] Test results:

[0167] Completed the collection of 40 sets of test data for simulation scenarios, including 20 sets of normal scenarios and 20 sets of disconnected scenarios

[0168] Completed the testing of the chain-break fault detection algorithm for 20 chain-break scenarios, and all 20 chain-break scenarios were identified with an accuracy rate of 100%;

[0169] Completed 20 sets of normal scenario tests on the broken link fault detection algorithm with no false alarms and a false alarm rate of 0%;

[0170] Completed the performance test of the chain break scene detection algorithm. The timing starts when the chain break scene appears and disappears in the camera video. The AI ​​software displays the shutdown lock prompt information on the software main screen within 3 seconds and outputs the shutdown lock signal synchronously. The fault protection shutdown delay is less than 15 seconds

[0171] Specifically, in summary, the present invention provides a scraper conveyor chain break monitoring system based on AI video recognition, which can effectively detect and promptly alarm abnormal scraper conveyor chain breakage by real-time analysis of high-definition video images. For the 40 sets of simulated scene videos, the AI ​​detection algorithm has a 100% accuracy rate in diagnosing scraper conveyor chain breakage, and the linkage technology of the working face centralized control system has a 3-second delay for fault protection shutdown, which meets the project test index requirements.

[0172] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0173] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A scraper conveyor chain break monitoring system based on AI video recognition, characterized in that: include: Image acquisition module, image distribution module, link break diagnosis module, centralized control module and visualization platform; Wherein, the image acquisition module acquires the conveying video of the scraper conveyor; The image distribution module is connected to the image acquisition module, and is used to sample the collected conveying view, obtain the image of the working area of ​​the scraper conveyor, and pre-process it as a data set; The chain break diagnosis module uses the YOLOv8 target detection algorithm to identify the broken chain position of the scraper conveyor chain in the image through a chain break monitoring model trained and optimized based on a data set, and divides the working state of the chain into normal and broken chain states; The centralized control module performs locking control based on the working status of the chain divided by the chain break diagnosis module; The visualization platform realizes the visualization monitoring of the scraper conveyor chain breakage and the visualization operation of the centralized control system.

2. According to claim 1, a scraper conveyor chain break monitoring system based on AI video recognition is characterized in that: The image acquisition module is composed of a plurality of cameras, which are installed on a bracket at the tail of the scraper conveyor to collect the conveying video of the scraper conveyor.

3. According to claim 1, a scraper conveyor chain break monitoring system based on AI video recognition is characterized in that: The image distribution module is connected to the image acquisition module through SDK and standard RTSP protocol, and the collected conveying video is sampled every 5 frames to obtain the image of the working area of ​​the scraper conveyor. The image is annotated using the LabelImg tool, and the data set is expanded using data enhancement as the data set.

4. According to claim 3, a scraper conveyor chain break monitoring system based on AI video recognition is characterized in that: Use data augmentation to expand the dataset, including: Flip, rotate, scale, crop, randomize, and adjust brightness, contrast, and saturation.

5. According to claim 4, a scraper conveyor chain break monitoring system based on AI video recognition is characterized in that: The data set is expanded using data augmentation, which also includes: adding random occlusion to the image, and using image aliasing data augmentation methods such as Mosaic and Mixup to further expand the data set.

6. The scraper conveyor chain break monitoring system based on AI video recognition according to claim 1 is characterized in that: The centralized control module includes: a control unit and an alarm system; When the chain break monitoring model identifies a chain break state, the control unit will stop the scraper conveyor, and the alarm system will transmit the stop state to the visualization platform for display and warning.

7. The scraper conveyor chain break monitoring system based on AI video recognition according to claim 1 is characterized in that: The visualization platform is based on B / S architecture.

8. The scraper conveyor chain break monitoring system based on AI video recognition according to claim 1 is characterized in that: The visualization platform includes: The alarm notification unit, video monitoring unit, intelligent monitoring unit, equipment management unit, and system management unit provide real-time display and query functions in a visual form.

Images