Method and equipment for monitoring abnormity of tail rope of vertical shaft hoisting system based on machine vision
By introducing a dual monitoring mechanism of example segmentation and abnormal detection of machine vision technology in tail rope monitoring, the shortcomings of existing tail rope monitoring methods in terms of accuracy, real-time and reliability are solved, and efficient and accurate monitoring of tail rope abnormalities is achieved.
Patent Information
- Application Number
- CN202411726848.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-04-11
AI Technical Summary
The existing tail rope monitoring methods have shortcomings in detection accuracy, real-timeness and reliability, especially in complex environments, and it is difficult to accurately identify subtle damage and abnormalities.
Using a dual monitoring mechanism based on machine vision, combined with instance segmentation and abnormal detection technology, the tail rope image is collected through a high-speed camera, and fine segmentation and abnormal identification are used for deep learning models to achieve efficient monitoring of the tail rope.
It significantly improves the accuracy and recall rate of tail rope abnormality detection, reduces the occurrence of missed and false alarms, improves the accuracy and real-time monitoring, and ensures the safety and reliability of the system.
Smart Images
Figure CN119810500A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of mining machinery and equipment monitoring, and in particular to a method and equipment for monitoring tail rope abnormality of a vertical shaft hoisting system based on machine vision. Background Art
[0002] In the mine hoisting and transportation system, the tail rope is a key component, and its operating condition is directly related to the safety and production efficiency of the equipment. However, there are many problems with the existing tail rope inspection methods. Traditionally, manual inspection of the tail rope has long working hours, low efficiency, poor reliability, and is difficult to achieve real-time and accurate monitoring. This not only increases safety risks, but may also result in tail rope abnormalities not being discovered in time, leading to equipment failures or even safety accidents.
[0003] There are two main types of existing tail rope detection methods: one is a mechanical monitoring method based on limit switch linkage, and the other is a visual target detection method. The former relies on mechanical structure. Although it has achieved partial automation, it has problems such as high maintenance cost, heavy system burden, and low safety. Although the latter is easy to implement, it is often difficult to accurately identify the damage characteristics of the tail rope, especially for minor damage. In addition, the complex environment of the mine is prone to false alarms and missed detections in the visual method, which limits the accuracy and reliability of its practical application. Therefore, how to improve the monitoring accuracy of tail rope anomalies and reduce missed alarms and false alarms has become a technical problem that needs to be solved urgently. Summary of the invention
[0004] The present invention proposes a method and device for monitoring tail rope anomalies in a shaft hoisting system based on machine vision, aiming to solve the defects of existing tail rope monitoring methods, especially to improve the accuracy and real-time performance of monitoring. By using machine vision technology, the present invention can perform fine image segmentation and anomaly detection on the tail rope, thereby realizing efficient monitoring of the tail rope.
[0005] Technical issues to be solved
[0006] The present invention mainly solves the technical problem that tail rope damage in mine hoisting systems is difficult to detect in time. Existing methods have deficiencies in detection accuracy, real-time performance and reliability, especially in monitoring minor damage and in complex environments. Therefore, the present invention introduces a dual monitoring mechanism of instance segmentation and anomaly detection to ensure that abnormal features of the tail rope can be accurately detected, improve monitoring effects, and reduce the occurrence of missed reports and false reports.
[0007] Technical Solution
[0008] The present invention proposes a tail rope abnormality monitoring method based on machine vision. First, an intrinsically safe high-speed camera is used to collect tail rope images, and deep learning technology is used to perform instance segmentation to accurately separate the tail rope target in the image. Then, by training an anomaly detection model, abnormal features on the tail rope surface, such as loose and broken tail rope strands, are identified. When the system detects an abnormality in the shape or surface of the tail rope, an alarm is immediately issued and the image is stored for further analysis and model training.
[0009] Beneficial effects of the present invention:
[0010] By combining instance segmentation and anomaly detection, the present invention can significantly improve the accuracy and recall rate of tail rope anomaly detection and reduce the time and workload of manual inspections compared to traditional single detection methods. At the same time, the method is suitable for complex mine environments, can reduce the occurrence of false alarms and missed detections, and ensure the safety and reliability of system operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0012] Figure 1 It is a schematic diagram of the tail rope detection workflow of the shaft hoisting system according to an embodiment of the present invention;
[0013] Figure 2 A schematic diagram of image instance segmentation according to an embodiment of the present invention;
[0014] Figure 3 Schematic diagram of the original image and result of anomaly detection according to an embodiment of the present invention. DETAILED DESCRIPTION
[0015] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.
[0016] It should be noted that the references to "one embodiment", "an embodiment", "an exemplary embodiment", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).
[0017] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.
[0018] Example 1
[0019] The present invention proposes a new method for dual monitoring of tail ropes based on instance segmentation and anomaly detection. As shown in the flow chart, the invention uses an intrinsically safe high-speed camera to pre-collect the tail rope running images of the vertical shaft hoisting system, and performs tail rope multi-point annotation on the collected images to establish a segmented image data set. First, an instance segmentation model is trained offline using the segmented image data set to segment the tail rope target in the image in detail. After the instance segmentation training is completed, the segmentation results are divided into a normal data set of the tail rope surface as a training set, and an anomaly detection model is trained. The abnormal data set of the tail rope surface, such as the tail rope broken strand, is used as a verification data set. After the offline training of the model is completed, the images collected in real time by the high-speed camera are sent to the instance segmentation model. Through calculation, it can be monitored whether the shape of the tail rope changes, such as whether it is scattered. If the shape is abnormal, an alarm is sent in time; at the same time, the segmentation results are sent to the anomaly detection model to monitor whether the tail rope surface has abnormal changes, such as the identification of the tail rope broken strand. If it is identified, an alarm is sent and the image is stored for further training of the model, otherwise it continues to monitor.
[0020] 1. Image acquisition
[0021] Since the tail rope runs at a high speed, the present invention uses an intrinsically safe high-speed camera to collect images. The camera is installed at the rope splitting position, with the lens facing the wide surface of the tail rope, and a fill light is added when necessary to ensure that the tail rope is clearly visible in the image field.
[0022] Instance segmentation dataset: The instance segmentation dataset contains a large number of tail rope datasets, including images of tail ropes with normal shapes and images of tail ropes with abnormal shapes. Use the deep learning image labeling software CVAT for labeling (the labeling steps are described in detail below). First, upload the image to be labeled to the labeling platform, create a label weisheng, select the polygonal box drawing method, and operate the mouse to label along the outer periphery of the tail rope. The advantage of using polygonal box labeling is that the edge of the tail rope can be carefully marked, and the model can learn the edge features of the tail rope later to improve accuracy. (What are the advantages of using polygonal box labeling? Can different labeling methods be expanded to improve accuracy?)
[0023] Anomaly detection dataset: The anomaly detection dataset is an instance segmentation model that identifies the tail rope mask and classifies and places these tail rope mask images according to whether the surface is abnormal or not, without the need for labeling.
[0024] 2. Training instance segmentation model
[0025] The collected original images are segmented and annotated, and the instance segmentation model is trained. Since the present invention monitors the tail rope in operation, the precision and real-time performance of the tail rope are required to be high. Since YOLACT++ can run at real-time speed on edge devices, YOLACT++ is used as the instance segmentation model. Through instance segmentation, the tail rope is classified at the pixel level on the basis of target detection (the specific segmentation method is described in detail below) to segment the tail rope in the image. Figure 2 As shown, the tail rope mask is segmented from the image and represented by different colors.
[0026] It should be noted that no matter which neural network model is used to implement instance segmentation of the collected image, it does not affect the essence of this solution.
[0027] 3. Training anomaly detection model
[0028] After the tail rope is segmented out based on the instance segmentation model, the tail rope mask in the anomaly detection dataset is divided and the anomaly detection model is trained. The normal samples on the tail rope surface are divided into the training set, and the abnormal samples on the tail rope surface are divided into the validation set. The anomaly detection model is trained with the training set samples, and the model effect is evaluated with the validation set samples. The anomaly detection model uses the latest RealNet anomaly detection algorithm. The RealNet network structure is simpler than PatchCore, the model inference time is faster, and it is more suitable for running at near real-time speeds on small edge devices. Through anomaly detection, possible anomalies on the tail rope surface are detected. Figure 3 As shown, broken wires are detected in the image.
[0029] 4. Tail rope abnormality judgment standard
[0030] After training the instance segmentation and anomaly detection models, we obtain the high-speed camera video stream in real time and enter the model reasoning stage. Based on the instance segmentation model in step 2, we can obtain the edge coordinates of the tail rope from the whole image. Along the Y-axis, from top to bottom, we can obtain pairs of coordinate values (x1, y j ) and (x2,y j ). Thus, we will get a series of coordinate differences along the X direction, and we can also find a set of width values of the tail rope from top to bottom ||x|| l1 The actual width C of the normal tail rope, considering the segmentation effect, we set an error σ, then under normal tail rope shape, we can get:
[0031] x|| l1 +σ≤C Formula 1
[0032] When the tail rope has an abnormal shape, such as loose strands, we can get:
[0033] ||x|| l1 +σ≥C Formula 2
[0034] Then, from the anomaly detection model in step 3, we can feed the instance segmentation result mask into the RealNet anomaly detection model to identify the surface abnormal features in the mask, so that we can identify the possible breakage of the tail rope.
[0035] Real-time online monitoring
[0036] Steps 1-4 collect images offline, establish image data sets, and train image instance segmentation models and anomaly detection models. The training of these models is completed offline, so sufficient learning and training can be carried out. During real-time online detection, the wellbore tail rope image is collected through an intrinsically safe high-speed camera, and the image is input into the YOLACT++ network model obtained in step 2 for instance segmentation to obtain the tail rope segmentation result mask in the image. The situation of the tail rope loose strands can be calculated by formula 1 and formula 2 in step 4 (please describe the calculation formula and algorithm in detail). If it is identified as loose strands, an alarm is issued and the image is saved, otherwise monitoring continues. The obtained mask image is then identified using the RealNet anomaly detection model trained in step 3 to determine whether the tail rope is broken. If it is identified as broken strands, an alarm is issued and the image is saved, otherwise monitoring continues.
[0037] 6. Equipment used in the present invention:
[0038] A vertical shaft hoisting tail rope abnormality monitoring device based on machine vision includes image acquisition equipment, communication transmission equipment, data processing equipment, data storage equipment, etc.
[0039] The image acquisition equipment is an intrinsically safe high-speed camera in coal mines, which can collect high-definition images during the operation of the tail rope. The surface of the camera needs to be made of dust-proof material to avoid the adhesion of dust in the mine, which affects the image quality.
[0040] Communication transmission equipment includes equipment required for wireless transmission methods such as 5G and industrial wifi, as well as equipment required for wired transmission methods such as USB3.0 and Ethernet.
[0041] The edge processing device uses an AI edge computing server, which has a certain amount of computing power and memory to complete image instance segmentation based on the YOLACT++ model and image anomaly detection tasks based on RealNet.
[0042] The data storage device is an SD card or a hard disk with a large storage capacity, which is used to store the image identified as abnormal tail rope.
[0043] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.
[0044] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for monitoring tail rope abnormality in a shaft hoisting system based on machine vision, characterized in that: The following steps are involved: Step 1: Use an intrinsically safe high-speed camera to collect the running image of the tail rope. The camera is installed at the rope splitting position, with the lens facing the wide surface of the tail rope; Step 2: Process the collected image through the instance segmentation model to segment the edge of the tail rope; Step 3: Based on the segmentation result, calculate the width change of the tail rope and determine whether the shape of the tail rope is abnormal; Step 4: If the tail rope shape is detected to be abnormal, an alarm signal is issued.
2. The method for monitoring tail rope abnormality of a shaft hoisting system based on machine vision according to claim 1, characterized in that: The instance segmentation model is a YOLACT++ model pre-trained using deep learning image labeling software.
3. The method for monitoring tail rope abnormality of a shaft hoisting system based on machine vision according to claim 2 is characterized in that: The method further includes inputting the segmentation result into an abnormality detection model to determine whether there is an abnormality on the surface of the tail rope, such as a broken strand of the tail rope. If an abnormality is detected, an alarm signal is issued and an abnormal image is saved.
4. The method for monitoring tail rope abnormality of a shaft hoisting system based on machine vision according to claim 3 is characterized in that: The anomaly detection model is a deep learning model based on the RealNet algorithm, and the model is trained and evaluated using training set samples and validation set samples respectively.
5. A method for monitoring tail rope abnormality of a shaft hoisting system based on machine vision according to claim 3 or 4, characterized in that: The tail rope abnormality detection includes the loose strand detection of the tail rope, by obtaining the coordinate value of the tail rope along the Y-axis direction, calculating the width of the tail rope, and judging whether the tail rope is loose according to the preset width error range.
6. The method for monitoring tail rope abnormality of a shaft hoisting system based on machine vision according to claim 5, characterized in that: The sending of the alarm signal includes sending the alarm information to the control system through a wireless communication device, and the wireless communication device includes a 5G or industrial WiFi module.
7. The method for monitoring tail rope abnormality of a shaft hoisting system based on machine vision according to claim 1, characterized in that: The image acquisition device is a dust-proof intrinsically safe high-speed camera in coal mines, which can ensure clear images in high dust environments.
8. A device for implementing the method according to any one of claims 1 to 7, characterized in that: include: Image acquisition equipment, used to collect high-definition images during the operation of the tail rope; data processing equipment for running instance segmentation models and anomaly detection models to monitor the shape and surface anomalies of the tail rope; Communication transmission equipment, used to transmit images and alarm signals to the control system; The data storage device is used to store the identified abnormal images.
9. The device according to claim 8, characterized in that The data processing device is a server with AI edge computing capabilities, which can process the monitoring data of the tail rope in real time. The image acquisition device is installed at the rope dividing wooden position and is equipped with a fill light device to ensure the image clarity of the tail rope in a low-light environment.