Equipment control method and device based on foreign matter identification and prediction of underground coal mine belt

Abstract

The invention discloses an equipment control method and device based on recognition and prediction of foreign matters on a belt under a coal mine, and the method comprises the steps: collecting a video image in the operation process of the belt in real time through image collection equipment which is installed above the belt, and the image collection equipment comprises an industrial camera; using an image processing algorithm and a deep learning algorithm to identify and position foreign matters in the video image; when the foreign matter is recognized, early warning is triggered, and the inspection equipment is controlled to go to an alarm point for confirmation and recheck; if it is confirmed that the foreign matter exists, according to the position and the property of the foreign matter, the measures of adjusting the running speed of the conveying belt, linkage underground broadcast broadcasting and pushing alarm information are executed. The invention solves the problems of large calculated amount, low precision, insufficient adaptability and large potential safety hazard of the existing underground coal mine belt foreign matter detection.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of mine equipment control, and particularly relates to a device control method and device based on coal mine underground belt foreign matter identification prediction. BACKGROUND

[0002] The coal mine underground belt foreign matter detection based on machine vision and image processing usually uses an industrial camera or other image acquisition device installed above the belt to collect video images in real time during the belt operation, and uses image processing technology and deep learning algorithms to automatically identify and locate foreign matters in the images.

[0003] However, the existing technology has obvious deficiencies. On the one hand, it has a large amount of calculation, resulting in low system operation efficiency; on the other hand, the detection accuracy is limited, making it difficult to accurately identify foreign matters of various types and sizes. In addition, the existing technology lacks adaptability, and the detection effect will be greatly reduced in different coal mine underground environments, such as changes in lighting conditions, high dust concentration, etc. At the same time, the linkage control mechanism is imperfect, and when foreign matters are detected, it cannot effectively cooperate with other devices, making it difficult to take timely and accurate disposal measures. These problems not only affect the safe operation efficiency of the belt conveyor, but also have great safety hazards. Therefore, there is an urgent need for a new technology to solve these problems and improve the efficiency and accuracy of the system to ensure the safety of coal mine production. SUMMARY

[0004] To this end, the present application provides a device control method and device based on coal mine underground belt foreign matter identification prediction, which solves the problems of large amount of calculation, low precision, and insufficient adaptability of existing coal mine underground belt foreign matter detection, and has great safety hazards.

[0005] To achieve the above purpose, the present application provides the following technical solution: a device control method based on coal mine underground belt foreign matter identification prediction, comprising the following steps:

[0006] Using an image acquisition device installed above the belt to collect video images in real time during the belt operation, the image acquisition device comprising an industrial camera;

[0007] Using image processing algorithms and deep learning algorithms to identify and locate foreign matters in the video images;

[0008] When foreign matters are identified, triggering an early warning and controlling a patrol device to go to the alarm point for confirmation and review;

[0009] If foreign matters are confirmed to exist, adjusting the running speed of the conveyor belt, broadcasting in the underground, and pushing alarm information measures are performed according to the position and nature of the foreign matters.

[0010] As an optimal solution of the device control method based on the coal mine underground belt foreign matter identification and prediction, the step of identifying and positioning the foreign matter in the video image comprises:

[0011] In combination with the transportation route, the belt speed and the moving track, the real-time position of the foreign matter in the monitoring area is dynamically displayed, the time when the foreign matter reaches the lap joint point, the middle of the belt or the machine head is predicted, and the dynamic icon, the track line or the countdown mode is displayed.

[0012] As an optimal solution of the device control method based on the coal mine underground belt foreign matter identification and prediction, when the foreign matter is identified, the step of triggering the early warning and controlling the inspection device to go to the alarm point for confirmation and review comprises:

[0013] In combination with the belt transportation route, the running speed and the current position of the robot, the starting time of the robot is judged, and the control instruction is issued in advance to schedule the inspection robot to the specified position;

[0014] When it is detected that there is a foreign matter on the belt, if the robot is located in front of the foreign matter and the distance satisfies the shooting time ≥3s, the robot is controlled to go to the foreign matter; if the robot is located behind the foreign matter or the distance is insufficient, the robot of the next belt is scheduled to perform the data acquisition and identification task.

[0015] As an optimal solution of the device control method based on the coal mine underground belt foreign matter identification and prediction, when the robot of the current belt is scheduled, the following conditions need to be met:

[0016] t1≤(S 本 -S 本面 ) / v 机

[0017] 2v 本 ≤(-v 机 ×t1+S 本 )-(v 本 ×t1+S 物 )

[0018] In the formula, S 物 is the position of the foreign matter to be shot, S 本 is the position of the robot of the current belt when the foreign matter is identified, S 本面 is the end position of the motion range of the robot of the current belt, v 机 is the maximum speed of the robot, v 本 is the speed of the belt, and t1 is the time length for the robot of the current belt to move to a safe distance.

[0019] As an optimal solution of the device control method based on the coal mine underground belt foreign matter identification and prediction, when the robot of the next belt is scheduled, the following conditions need to be met:

[0020] (S 接 -S 物 ) / v本 ≤t2≤(S 下 -S 下面 ) / v 机

[0021] 2v 下 ≤(-v 机 ×t2+S 下 )-[v 下 ×(t2-(S 接 -S 物 ) / v 本 )+S 接 ]

[0022] Where S 接 is the belt overlap position, S 下 For the next belt robot position, S 下面 is the end position of the next robot motion range, v 下 is the speed of the next belt, and t2 is the time it takes for the next robot to move to the safe distance.

[0023] As a preferred solution for equipment control methods based on foreign object identification and prediction on underground coal mine belts, this method dynamically adjusts equipment parameters and operating modes based on the type and size of foreign objects and the operating status of the belt during the adjustment of the conveyor belt speed. It initiates an emergency shutdown procedure when a foreign object exceeding a preset size is detected, and initiates an early warning and notifies the operator when a foreign object within the preset size is detected.

[0024] When the robot confirms a foreign object, it sends out an alarm signal through a sound alarm, SMS notification, and a pop-up window on the monitoring platform, and captures images and records the alarm information.

[0025] As a preferred solution for the equipment control method based on foreign body identification and prediction of coal mine belts, the equipment control process also includes:

[0026] Improving the image processing algorithm includes: establishing a sample database, storing the collected video images containing foreign objects and the corresponding edge features, texture features, and color feature information as sample data in the database, and optimizing the threshold segmentation parameters and edge detection operator parameters in the image processing algorithm using the sample data;

[0027] The deep learning algorithm is improved by storing the collected video images containing foreign objects and the corresponding foreign object type, location, and property information as sample data in the database, and using the sample data to train the deep learning model:

[0028]

[0029] Where L is the loss function, N is the number of samples, and L iL(i) is the loss function of the i-th sample, and theta is the parameter of the deep learning model, and the parameter of the deep learning model is optimized by minimizing the loss function L.

[0030] The application also provides a device control device based on coal mine underground belt foreign matter identification prediction, comprising:

[0031] An image acquisition module is configured to acquire video images in real time during belt operation by using an image acquisition device installed above the belt, wherein the image acquisition device comprises an industrial camera.

[0032] A foreign matter identification module is configured to identify and locate foreign matters in the video images by using image processing algorithms and deep learning algorithms.

[0033] A patrol device control module is configured to trigger an early warning and control a patrol device to go to an alarm point for confirmation and review when foreign matters are identified.

[0034] A disposal execution module is configured to execute measures of adjusting the running speed of the belt, broadcasting in linkage with underground, and pushing alarm information if it is confirmed that there are foreign matters according to the position and nature of the foreign matters.

[0035] As a preferred scheme of the device control device based on coal mine underground belt foreign matter identification prediction, in the foreign matter identification module, the moving track is analyzed in combination with the transportation route and the belt speed, the real-time position of the foreign matter in the monitoring area is dynamically displayed, the time when the foreign matter reaches the lap joint point, the middle of the belt or the head is predicted, and the dynamic icon, the track line or the countdown mode is displayed.

[0036] As a preferred scheme of the device control device based on coal mine underground belt foreign matter identification prediction, in the patrol device control module, the robot start-up and moving time are judged in combination with the belt transportation route, the running speed and the current position of the robot, and the control instruction is issued in advance to schedule the patrol robot to the specified position.

[0037] When it is detected that there are foreign matters on the belt, if the robot is located in front of the foreign matters and the distance satisfies the shooting time of ≥3s, the robot is controlled to go to the foreign matters, and if the robot is located behind the foreign matters or the distance is insufficient, the robot of the next belt is scheduled to perform the data acquisition and identification task.

[0038] The following conditions need to be met when the robot is scheduled:

[0039] t1≤(S 本 -S 本面 ) / v 机

[0040] 2v 本 ≤(-v 机 ×t1+S 本 )-(v 本xt1 + S 物 )

[0041] In the formula, S 物 is the position of the foreign matter when it needs to be photographed, S 本 is the position of the local robot when the foreign matter is identified, S 本面 is the end position of the movement range of the local robot, v 机 is the maximum speed of the robot, v 本 is the speed of the belt, and t1 is the time length for the local robot to move to a safe distance.

[0042] When scheduling the next belt robot, the following conditions need to be met:

[0043] (S 接 -S 物 ) / v 本 ≤t2≤(S 下 -S 下面 ) / v 机

[0044] 2v 下 ≤(-v 机 ×t2+S 下 )-[v 下 ×(t2-(S 接 -S 物 ) / v 本 )+S 接 ]

[0045] In the formula, S 接 is the position of the belt overlap, S 下 is the position of the next belt robot, S 下面 is the end position of the movement range of the next robot, v 下 is the speed of the next belt, and t2 is the time length for the next robot to move to a safe distance.

[0046] As an optimal solution of the device control device based on the identification and prediction of foreign matters in the coal mine underground belt, in the disposal execution module, the device parameters and working mode are dynamically adjusted according to the type, size and running state of the foreign matter; when a foreign matter exceeding the preset size is detected, an emergency shutdown program is started, and when a foreign matter not exceeding the preset size is detected, a warning is started and the operator is notified; when the robot confirms the foreign matter, an alarm signal is sent through sound alarm, SMS notification, and pop-up window of the monitoring platform, and the image is captured and the alarm information is recorded.

[0047] As an optimal solution of the device control device based on the identification and prediction of foreign matters in the coal mine underground belt, it further comprises:

[0048] An image processing optimization module is configured to improve the image processing algorithm, including: establishing a sample database, storing the collected video images containing foreign matters and corresponding edge features, texture features, and color feature information as sample data into the database, and using the sample data to optimize the threshold segmentation parameters and edge detection operator parameters in the image processing algorithm.

[0049] A deep learning optimization module is configured to improve the deep learning algorithm, including: storing the collected video images containing foreign matters and corresponding foreign matter types, positions, and property information as sample data into the database, and using the sample data to train the deep learning model:

[0050]

[0051] In the formula, L is a loss function, N is the number of samples, L i L (θ) is the loss function of the i-th sample, and θ is the parameter of the deep learning model.

[0052] The beneficial effects of the present application are as follows:

[0053] First, when foreign matters are identified, a pre-warning is triggered and the inspection equipment is automatically controlled to go to review, which can reduce the false alarm problem caused by single camera misidentification. Through the robot on-site data collection and comprehensive research and judgment, the reliability of foreign matter detection can be improved, and the misjudgment or omission in the prior art caused by single detection mechanism can be avoided. In addition, the linkage control mechanism can automatically adjust the speed of the conveyor belt, broadcast or push the alarm information according to the position and nature of the foreign matter, realize the timely disposal of the foreign matter, and avoid the damage of the foreign matter to the belt such as scratching and tearing.

[0054] Second, combined with the transportation route and the belt speed, the position of the foreign matter is dynamically tracked and the time when it reaches the key node (such as the lap joint point and the machine head) is predicted, so that the inspection equipment can be dispatched in advance or the belt running state can be adjusted to ensure that the foreign matter is timely confirmed, solve the problem of imperfect linkage control in the prior art, and improve the response efficiency and adaptability of the system.

[0055] Third, the automatic monitoring and foreign matter treatment of the belt conveyor can be realized, which reduces the labor cost and reduces the safety hazards caused by the contact of personnel with foreign matters. When large foreign matters are detected, an emergency shutdown program is automatically started to avoid serious accidents such as belt breakage; when small foreign matters are detected, a pre-warning is started and the operator is notified to realize graded disposal. Compared with the prior art, the present application effectively improves the safety operation efficiency of the belt conveyor and provides a powerful guarantee for the safety production in the coal mine. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only exemplary, and for those skilled in the art, other drawings can be obtained from the provided drawings without creative labor.

[0057] The structures, proportions, sizes, etc. shown in the specification are only used to cooperate with the content disclosed in the specification, to be understood and read by those skilled in the art, and are not used to limit the conditions that the present application can be implemented, so they do not have technical significance. Any modification of structure, change of proportion relationship or adjustment of size, without affecting the effects and purposes that the present application can produce, should still fall within the scope of the technical content disclosed by the present application.

[0058] Figure 1 The device control method flowchart based on coal mine underground belt foreign matter identification prediction provided for the embodiments of the present application;

[0059] Figure 2 The belt foreign matter real-time monitoring and early warning interface provided for the embodiments of the present application;

[0060] Figure 3 The foreign matter real-time position tracking and prediction interface provided for the embodiments of the present application;

[0061] Figure 4 The control inspection device confirmation schematic diagram provided for the embodiments of the present application;

[0062] Figure 5 The control inspection device confirmation interface provided for the embodiments of the present application;

[0063] Figure 6 The abnormal alarm interface provided for the embodiments of the present application;

[0064] Figure 7 The device linkage control interface provided for the embodiments of the present application;

[0065] Figure 8 The device control algorithm flowchart based on coal mine underground belt foreign matter identification prediction provided for the embodiments of the present application;

[0066] Figure 9 The device control device architecture schematic diagram based on coal mine underground belt foreign matter identification prediction provided for the embodiments of the present application. DETAILED DESCRIPTION

[0067] The following describes embodiments of the present application by specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the disclosure. Obviously, the described embodiments are part of the embodiments of the present application, not all. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0068] Embodiment 1

[0069] Referring to Figure 1 Embodiment 1 of the present application provides a device control method based on coal mine underground belt foreign matter identification and prediction, comprising the following steps:

[0070] S1, using an image acquisition device installed above the belt, real-time acquisition of video images in the belt running process, the image acquisition device comprising an industrial camera;

[0071] S2, using image processing algorithm and deep learning algorithm, identifying and positioning foreign matter in the video image;

[0072] S3, when foreign matter is identified, triggering an early warning and controlling the inspection device to go to the alarm point for confirmation and review;

[0073] S4, if it is confirmed that there is foreign matter, according to the position and nature of the foreign matter, adjusting the running speed of the conveyor belt, linkage underground broadcast, and pushing alarm information measures are executed.

[0074] In this embodiment, in step S1, the industrial camera converts the belt running picture into an electrical signal based on the principle of machine vision through a CCD or CMOS image sensor, and then forms a digital image through analog-to-digital conversion. Installed above the belt can avoid belt obstruction, ensuring a full view of the working face. Real-time acquisition can capture foreign matter dynamics, providing continuous data for subsequent processing.

[0075] In this embodiment, in step S2, the image processing algorithm extracts the target area in the image through grayscale, filter denoising (such as Gaussian filter), threshold segmentation (such as Otsu algorithm), and then uses edge detection (Canny operator) and morphological operation (dilation, erosion) to outline the foreign matter outline, achieving preliminary positioning.

[0076] Among them, the deep learning algorithm adopts YOLO, Faster R-CNN and other target detection models, automatically extracts semantic features (such as shape, texture) in the image through convolutional neural network (CNN), and generates candidate boxes combined with region proposal network (RPN), to realize accurate identification of foreign matter type (such as anchor rod, large piece of coal) and position. Among them, the model needs to be trained in advance on the labeled data set to learn the feature difference between foreign matter and background. Referring to Figure 2, by automatically identifying and locating foreign objects in the image, such as large pieces of coal, gangue, anchor rods, channel steel, etc.; once foreign objects are found, timely warning will be issued, the video screen of the alarm area will be magnified and displayed, and the initial location information of the hidden danger will be displayed to the monitoring personnel in the form of fixed-point markings.

[0077] In a possible embodiment, in step S2, the step of identifying and locating the foreign object in the video image includes:

[0078] Combined with the transport route and belt speed analysis, the moving trajectory can be dynamically displayed to dynamically display the real-time position of foreign objects in the monitoring area. The time when foreign objects reach the joint point, the middle of the belt or the head of the machine can be predicted and displayed in the form of dynamic icons, trajectory lines or countdown.

[0079] Specifically, based on the kinematic formula s=v×t, the current position of the foreign body S 物 , belt speed v 本 Substitute and calculate the time t to reach each key node = (S 节点 -S 物 ) / v 本 . Through data visualization technology, the trajectory and countdown are converted into a graphical interface, helping monitoring personnel to intuitively grasp the dynamics of foreign objects and provide a basis for early disposal.

[0080] See also Figure 3 , using big data analysis technology combined with transportation routes, belt speeds, etc. to analyze movement trajectories, dynamically display the real-time location of hidden dangers in the monitoring area, analyze and predict the time when hidden dangers reach specific key nodes (such as overlap points, middle of belts, and belt conveyor heads), and provide intuitive graphical displays, such as dynamic icons, trajectory lines, key node countdowns, etc., to facilitate monitoring personnel to intuitively understand the movement of hidden dangers.

[0081] In this embodiment, in step S3, when a foreign object is identified, the steps of triggering an early warning and controlling the inspection equipment to go to the alarm point for confirmation and review specifically include:

[0082] Combined with the belt transport route, running speed and the current position of the robot, the robot startup and movement time is determined, and the control command is issued in advance to dispatch the inspection robot to the specified position; robot scheduling is a path planning problem that needs to meet kinematic constraints (such as the maximum speed v 机 ) and time window requirements. By establishing a time-space coordinate system, the robot position S 本 , belt speed v 本 , foreign body position S 物 Converted into time parameters to ensure that the foreign object is in the field of view when the robot reaches the shooting point.

[0083] When detecting the foreign matter on the current section, if the robot is in front of the foreign matter and the distance meets ≥3s shooting time, the robot is controlled to move to the foreign matter; if the robot is behind the foreign matter or the distance is insufficient, the robot on the next section is dispatched to perform the data acquisition and identification task. The ≥3s shooting time is based on ergonomics to ensure that the robot has enough time to adjust the posture and complete image acquisition. If the current robot cannot meet the requirements, the next section robot is dispatched to move to the target position through the belt overlap position S 接 The time for the foreign matter to move to the next section is calculated (S 接 -S 物 ) / v 本 , and the next section robot is dispatched to move to the target position in advance to form a relay detection mechanism

[0084] Referring to Figure 4 and Figure 5 , since the inspection robot can only move within the current section and its speed is much smaller than the belt speed, when detecting the foreign matter on the current section, if the robot is in front of the foreign matter and the distance meets ≥3s shooting time (i.e. route ①), the detection is successful; if the robot is behind the foreign matter or the distance is insufficient (the meeting time is less than 3s), the foreign matter continues to move forward to the next section, and the robot on the next section is dispatched to perform the data acquisition and image identification task, i.e. route ②.

[0085] Wherein, when dispatching the current robot, the following conditions need to be met:

[0086] t1≤(S 本 -S 本面 ) / v 机

[0087] 2v 本 ≤(-v 机 ×t1+S 本 )-(v 本 ×t1+S 物 )

[0088] In the formula, S 物 is the position of the foreign matter to be photographed, S 本 is the position of the current robot when the foreign matter is identified, S 本面 is the end position of the movement range of the current robot, v 机 is the maximum speed of the robot, v 本 is the belt speed, and t1 is the time for the current robot to move to a safe distance.

[0089] Wherein, when dispatching the next section robot, the following conditions need to be met:

[0090] (S 接 -S 物 ) / v本 ≤ t2 ≤ (S 下 -S 下面 ) / v 机

[0091] 2v 下 ≤(-v 机 × t1 + S 下 )- [v 下 × (t2 - (S 接 -S 物 ) / v 本 )+ S 接 ]

[0092] In the formula, S 接 is the belt overlap position, S 下 is the next belt robot position, S 下面 is the next robot motion range end position, v 下 is the next belt speed, and t2 is the time length for the next robot to move to a safe distance.

[0093] Specifically, referring to Figure 4 , since the speed of the inspection robot is much smaller than the belt speed, when an object is identified to be photographed, the robot on the current belt is preferentially dispatched to execute. If the current position of the robot can meet the requirements, in order to confirm the hidden danger as soon as possible, the robot is directly controlled to go to the specified position, that is, route ①, at this time, the following conditions must be met simultaneously:

[0094] t1 ≤ (S 本 -S 本面 ) / v 机 ensure that the robot has enough time to start and reach the specified position; 2v 本 ≤(-v 机 × t1 + S 本 )-(v 本 × t1 + S 物 ) ensure that there is at least 2s time for photographing and collecting before the two parties meet;

[0095] Otherwise, the inspection robot on the next belt needs to be dispatched to take a photograph to confirm the object through route ②, at this time, the following conditions must be met simultaneously:

[0096] (S 接 -S 物 ) / v 本 ≤ t2 ≤ (S 下 -S 下面 ) / v 机 ensure that the object has reached the next belt, and the robot on the next belt has enough time to start and reach the specified position; 2v 下 ≤(-v 机 × t2 + S 下 )- [v下 x(t2-t1) 接 -S 物 ) / v 本 )+S 接 ]to ensure that there is at least 2s time for shooting and collecting before the two parties meet.

[0097] In this embodiment, in the process of adjusting the running speed of the conveying belt in step S4, the equipment parameters and working mode are dynamically adjusted according to the type, size and running state of the foreign matter; when a foreign matter exceeding the preset size is detected, an emergency shutdown program is started; when a foreign matter not exceeding the preset size is detected, a warning is started and the operator is notified;

[0098] When the robot confirms the foreign matter, an alarm signal is sent through a sound alarm, a short message notification, a pop-up window of a monitoring platform, and a snapshot image and alarm information are recorded.

[0099] Referring to Figure 6 , specifically, by establishing a foreign matter risk level model, the size (such as a diameter > 10 cm) and type (such as a metal anchor rod) of the foreign matter are taken as trigger conditions. The emergency shutdown program cuts off the motor control loop through a safety relay, which meets the requirements of the coal mine safety regulations; the warning mechanism is prompted through an audible and visual signal and a human-machine interface (HMI), and is manually intervened for processing to realize a graded response. An event-driven mechanism is adopted, when the robot returns a confirmation signal, the alarm module (audio output), the short message module (GSM communication) and the monitoring software (interface refresh) are triggered at the same time through multiple threads, and the image and data (such as a timestamp and foreign matter coordinates) at the alarm time are stored in a database for subsequent tracing and model optimization.

[0100] Referring to Figure 7 , in a possible embodiment, the equipment parameters and working mode are dynamically adjusted to realize intelligent collaborative control according to the type, size and running state of the foreign matter and the belt conveying system. For example, when a large foreign matter is detected, the emergency shutdown program is preferentially started; when a small foreign matter is detected, the warning program is started and the operator is notified to handle.

[0101] In a possible embodiment, the device control process further includes:

[0102] The image processing algorithm is improved, including: establishing a sample database, storing the collected video images containing foreign matters and the corresponding edge features, texture features and color feature information as sample data in the database, and using the sample data to optimize the threshold segmentation parameters and edge detection operator parameters in the image processing algorithm;

[0103] For example, based on the supervised learning method in machine learning, the parameter optimization model is trained by labeling samples (such as foreign object edge coordinates and texture histogram). The optimal threshold of Otsu threshold segmentation is adjusted using cross-validation method, or the high and low thresholds of Canny operator are optimized by gradient descent method, so that the algorithm maintains stable performance in different light (such as the change of light and shade in underground tunnel) and dust environment.

[0104] Among them, the improvement of the deep learning algorithm includes: storing the collected video images containing foreign objects and the corresponding foreign object type, position and property information into the database as sample data, and training the deep learning model using the sample data:

[0105]

[0106] In the formula, L is the loss function, N is the number of samples, L i The loss function L of the i-th sample is θ, and the parameter of the deep learning model is optimized by minimizing the loss function L. The stochastic gradient descent (SGD) algorithm is adopted, and the cross-entropy loss function (classification task) or smooth L1 loss (regression task) is used as the optimization target. The weight parameters θ of the convolutional layer and the fully connected layer are updated by the back propagation algorithm. With the continuous accumulation of sample data (such as newly added anchor rod and sleeper images), the recognition accuracy of rare foreign objects is gradually improved, solving the problem of insufficient adaptability in the prior art.

[0107] Referring to Figure 8 The business flowchart of coal mine underground belt foreign object identification and prediction and equipment control presents the complete process from data preparation to foreign object disposal, which can assist in understanding the execution logic of the technical scheme:

[0108] First, data preparation (left branch):

[0109] Region extraction / data labeling: manually label foreign object regions (such as anchor rod and large coal position) in belt scene images, provide "label data" for model training, solve the problem of "model not knowing what foreign object is"; train foreign object identification model: train deep learning model (such as YOLO) with labeled data, let the model learn the mapping relationship between foreign object features and categories / positions, provide algorithm basis for subsequent detection.

[0110] Second, real-time detection (upper half of the main process):

[0111] Camera captures video images: industrial camera continuously captures belt running pictures, converts physical scene into digital image; uses foreign object identification model for detection: calls the trained model to analyze the image, extracts features based on convolutional neural network (CNN), and judges whether there is a foreign object, replacing manual visual inspection, solving the problem of "low efficiency and easy to miss detection" of manual detection.

[0112] Third, foreign matter confirmation (main process section) :

[0113] Whether there is foreign matter: the model outputs the detection result, if it is determined that there is no foreign matter, the image collection cycle is repeated to continuously monitor; the inspection equipment is started to review: if it is determined that there is foreign matter, the inspection robot is automatically dispatched to the alarm point, the authenticity of the foreign matter is confirmed through secondary data collection (multi-angle image, sensor information), and the problem of "model misjudgment" (such as false positive alarm caused by dust and light interference) is solved.

[0114] Fourth, linkage disposal (main process lower half + left branch) :

[0115] Confirm whether there is foreign matter: if the review shows that there is no foreign matter, return to the image collection link, if it is confirmed that there is foreign matter, enter the disposal process; configure disposal logic: preset the response rules (such as shutdown, speed reduction, alarm level) of different foreign matters (such as large gangue and metal anchor rod) in advance, so that the system "knows how to dispose"; linkage equipment control: according to the danger degree of the foreign matter, automatically trigger the belt speed reduction / shutdown (such as large foreign matter directly shutdown to avoid tearing the belt); sound and light / alarm broadcast: underground broadcast, sound and light alarm are triggered synchronously to alert on-site personnel; information push: send alarm information (SMS, platform notification) to managers according to preset rules (such as level, responsibility), realize the cooperation of "unattended + someone responds".

[0116] Embodiment 2

[0117] Reference Figure 9 , the embodiment 2 of the present application provides a device control device based on coal mine underground belt foreign matter identification prediction, comprising:

[0118] The image acquisition module 100 is used for acquiring video images in the running process of the belt in real time by using the image acquisition device installed above the belt, and the image acquisition device comprises an industrial camera.

[0119] The foreign matter identification module 200 is used for identifying and positioning the foreign matter in the video image by using an image processing algorithm and a deep learning algorithm.

[0120] The inspection equipment control module 300 is used for triggering an early warning and controlling the inspection equipment to go to the alarm point for confirmation and review when the foreign matter is identified.

[0121] The disposal execution module 400 is used for executing the measures of adjusting the running speed of the belt, linkage underground broadcast broadcast, and pushing alarm information according to the position and nature of the foreign matter if it is confirmed that there is foreign matter.

[0122] In the foreign matter recognition module 200, the moving track is analyzed in combination with the transportation route and the belt speed, the real-time position of the foreign matter in the monitoring area is dynamically displayed, the time when the foreign matter reaches the lap joint point, the middle of the belt or the machine head is predicted, and the dynamic icon, the track line or the countdown mode is displayed.

[0123] In the inspection equipment control module 300, the robot is judged to be turned on and moved in combination with the belt transportation route, the running speed and the current position of the robot, the control instruction is issued in advance to schedule the inspection robot to the specified position;

[0124] When it is detected that there is a foreign matter on the belt, if the robot is located in front of the foreign matter and the distance satisfies ≥3s shooting time, the robot is controlled to go to the foreign matter; if the robot is located behind the foreign matter or the distance is insufficient, the robot of the next belt is scheduled to perform the data acquisition and recognition task;

[0125] When the robot of the current belt is scheduled, the following conditions need to be met:

[0126] t1≤(S 物 -S 本 ) / v 本面

[0127] 2v 机 ≤(-v 本 ×t1+S 接 )-(v 物 ×t1+S 本 )

[0128] In the formula, S 下 is the position of the foreign matter to be shot, S 下面 is the position of the robot of the current belt when the foreign matter is recognized, S 机 is the end position of the movement range of the robot of the current belt, v 下 is the maximum speed of the robot, v 机 is the belt speed, and t1 is the time length for the robot of the current belt to move to a safe distance.

[0129] When the robot of the next belt is scheduled, the following conditions need to be met:

[0130] (S 下 -S 下 ) / v 接 ≤t2≤(S 物 -S 本 ) / v 接

[0131] 2v 接 ≤(-v 下 ×t2+S 下面 )-[v 下 ×(t2-(S i -S ) / v 本 )+S 接 ]

[0132] In the formula, S 接 is the belt overlap position, S 下 is the next part of the belt robot position, S 下面 is the next part of the robot motion range end position, v 下 is the next belt speed, and t2 is the time length for the next part of the robot to move to a safe distance.

[0133] In the treatment execution module 400 in this embodiment, the device parameters and working mode are dynamically adjusted according to the foreign matter type, size and belt running state; when foreign matter exceeding the preset size is detected, an emergency shutdown program is started, and when foreign matter not exceeding the preset size is detected, a warning is started and the operator is notified; when the robot confirms the foreign matter, an alarm signal is sent through a sound alarm, an SMS notification, a pop-up window of a monitoring platform, and a snapshot image and alarm information are recorded.

[0134] In this embodiment, it also includes:

[0135] The image processing optimization module 500 is configured to improve the image processing algorithm, including: establishing a sample database, storing the collected video images containing foreign matter and the corresponding edge features, texture features and color feature information as sample data in the database, and optimizing the threshold segmentation parameters and edge detection operator parameters in the image processing algorithm by using the sample data.

[0136] The deep learning optimization module 600 is configured to improve the deep learning algorithm, including: storing the collected video images containing foreign matter and the corresponding foreign matter type, position and property information as sample data in the database, and training the deep learning model by using the sample data.

[0137]

[0138] In the formula, L is a loss function, N is the number of samples, L i (θ) is the loss function of the i-th sample, and θ is the parameter of the deep learning model. The parameter of the deep learning model is optimized by minimizing the loss function L.

[0139] It should be noted that the information interaction, execution process and the like between the modules of the above device are based on the same concept as the method embodiment in Embodiment 1 of the present application, and the technical effects brought by them are the same as those of the method embodiment of the present application. For specific content, refer to the description in the foregoing method embodiment of the present application, which will not be repeated here.

[0140] Embodiment 3

[0141] The embodiment 3 of the present application provides a non-transitory computer readable storage medium, which stores program codes of a device control method based on coal mine underground belt foreign matter identification prediction, and the program codes include instructions for executing the device control method based on coal mine underground belt foreign matter identification prediction of the embodiment 1 or any possible implementation manner thereof.

[0142] The computer readable storage medium can be any available medium or a data storage device such as a server, data center, etc. integrated with one or more available medium sets that can be accessed by a computer. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0143] Embodiment 4

[0144] The embodiment 4 of the present application provides an electronic device, which comprises a memory and a processor.

[0145] The processor and the memory complete mutual communication through a bus; the memory stores program instructions that can be executed by the processor, and the processor calling the program instructions can execute the device control method based on coal mine underground belt foreign matter identification prediction of the embodiment 1 or any possible implementation manner thereof.

[0146] Specifically, the processor can be implemented by hardware or software, when implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor, which is implemented by reading software codes stored in a memory, and the memory can be integrated in the processor or exist independently outside the processor.

[0147] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner.

[0148] It should be apparent to those skilled in the art that the modules or steps of the application described above can be implemented with general computing devices, which can be centralized on a single computing device or distributed over a network of multiple computing devices, and optionally implemented with program code executable by a computing device, which can be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described can be performed in a different order than shown or described, or made into individual integrated circuit modules or multiple modules or steps made into a single integrated circuit module. Thus, the application is not limited to any particular combination of hardware and software.

[0149] While the application has been described in detail above with general and specific embodiments, it can be apparent to those skilled in the art that modifications and improvements can be made to the application without departing from the spirit thereof. Such modifications and improvements are intended to fall within the scope of the application.

Claims

1. A device control method based on foreign body identification and prediction of belts in underground coal mines, characterized in that: The following steps are involved: Using an image acquisition device installed above the belt to capture real-time video images of the belt during its operation, the image acquisition device includes an industrial camera; Using image processing algorithms and deep learning algorithms to identify and locate foreign objects in the video images; When a foreign object is identified, an early warning is triggered and the inspection equipment is controlled to go to the alarm point for confirmation and review; If the presence of foreign matter is confirmed, measures such as adjusting the conveyor belt speed, linking underground broadcasts, and pushing alarm information will be implemented according to the location and nature of the foreign matter.

2. The equipment control method based on foreign body identification and prediction of coal mine belts according to claim 1 is characterized in that: The steps of identifying and locating foreign objects in the video image include: Combined with the transport route and belt speed analysis, the moving trajectory can be dynamically displayed to dynamically display the real-time position of foreign objects in the monitoring area. The time when foreign objects reach the joint point, the middle of the belt or the head of the machine can be predicted and displayed in the form of dynamic icons, trajectory lines or countdown.

3. The equipment control method based on foreign body identification and prediction of coal mine belts according to claim 1 is characterized in that: When a foreign object is identified, the steps to trigger an early warning and control the inspection equipment to go to the alarm point for confirmation and review include: Combined with the belt transport route, operating speed and the current position of the robot, the robot startup and movement time is determined, and control instructions are issued in advance to dispatch the inspection robot to the designated location; When a foreign object is detected on the main belt, if the robot is in front of the foreign object and the distance meets the shooting time of ≥3s, the main robot is controlled to move forward; if the robot is behind the foreign object or the distance is insufficient, the robot of the next belt is dispatched to perform data collection and identification tasks.

4. The equipment control method based on foreign body identification and prediction of coal mine underground belt according to claim 3 is characterized in that: When dispatching the headquarters robot, the following conditions must be met: t1≤(S 本 -S 本面 ) / v 机 2v 本 ≤(-v 机 ×t1+S 本 )-(v 本 ×t1+S 物 ) Where S 物 The position of the foreign object when shooting is required, S 本 The position of the robot when a foreign object is detected, S 本面 is the end position of the robot's motion range, v 机 is the maximum speed of the robot, v 本 is the belt speed, and t1 is the time it takes for the robot to move to the safe distance.

5. The equipment control method based on foreign body identification and prediction of coal mine underground belt according to claim 3 is characterized in that: The following conditions must be met when scheduling the next belt robot: (S 接 -S 物 ) / v 本 ≤t2≤(S 下 -S 下面 ) / v 机 2v 下 ≤(-v 机 ×t2+S 下 )-[v 下 ×(t2-(S 接 -S 物 ) / v 本 )+S 接 ] Where S 接 is the belt overlap position, S 下 For the next belt robot position, S 下面 is the end position of the next robot motion range, v 下 is the speed of the next belt, and t2 is the time it takes for the next robot to move to the safe distance.

6. The equipment control method based on foreign body identification and prediction of coal mine belts according to claim 1 is characterized in that: During the process of adjusting the conveyor belt speed, the equipment parameters and operating mode are dynamically adjusted according to the type and size of the foreign matter and the belt running status; when a foreign matter exceeding the preset size is detected, the emergency stop program is activated; when a foreign matter within the preset size is detected, an early warning is activated and the operator is notified; When the robot confirms a foreign object, it sends out an alarm signal through a sound alarm, SMS notification, and a pop-up window on the monitoring platform, and captures images and records the alarm information.

7. The equipment control method based on foreign body identification and prediction of coal mine underground belt according to claim 1 is characterized in that: The equipment control process also includes: Improving the image processing algorithm includes: establishing a sample database, storing the collected video images containing foreign objects and the corresponding edge features, texture features, and color feature information as sample data in the database, and optimizing the threshold segmentation parameters and edge detection operator parameters in the image processing algorithm using the sample data; The deep learning algorithm is improved by storing the collected video images containing foreign objects and the corresponding foreign object type, location, and property information as sample data in the database, and using the sample data to train the deep learning model: Where L is the loss function, N is the number of samples, and L i (θ) is the loss function of the i-th sample, θ is the parameter of the deep learning model, and the parameters of the deep learning model are optimized by minimizing the loss function L.

8. An equipment control device based on foreign body identification and prediction of belts in underground coal mines, characterized in that: include: An image acquisition module is used to acquire video images of the belt in real time during its operation using an image acquisition device installed above the belt, wherein the image acquisition device includes an industrial camera; A foreign object recognition module, configured to identify and locate foreign objects in the video image using image processing algorithms and deep learning algorithms; Inspection equipment control module, used to trigger an early warning and control the inspection equipment to go to the alarm point for confirmation and review when a foreign object is identified; The disposal execution module is used to adjust the conveyor belt speed, link underground broadcasting, and push alarm information measures based on the location and nature of the foreign matter if the presence of foreign matter is confirmed.

9. The equipment control device based on foreign body identification and prediction of coal mine belts according to claim 8, characterized in that: In the foreign object recognition module, the movement trajectory is analyzed in combination with the transportation route and belt speed, and the real-time position of the foreign object in the monitoring area is dynamically displayed. The time when the foreign object reaches the overlap point, the middle of the belt or the head of the machine is predicted and displayed in the form of dynamic icons, trajectory lines or countdown; In the inspection equipment control module, the robot startup and movement time is determined based on the belt transport route, operating speed and current position of the robot, and a control instruction is issued in advance to dispatch the inspection robot to the designated position; When a foreign object is detected on the belt, if the robot is in front of the foreign object and the distance meets the shooting time of ≥3s, the robot will be controlled to move towards it; if the robot is behind the foreign object or the distance is insufficient, the robot of the next belt will be dispatched to perform data collection and recognition tasks; When dispatching the headquarters robot, the following conditions must be met: t1≤(S 本 -S 本面 ) / v 机 2v 本 ≤(-v 机 ×t1+S 本 )-(v 本 ×t1+S 物 ) Where S 物 The position of the foreign object when shooting is required, S 本 The position of the robot when a foreign object is detected, S 本面 is the end position of the robot's motion range, v 机 is the maximum speed of the robot, v 本 is the belt speed, t1 is the time it takes for the robot to move to the safe distance; The following conditions must be met when scheduling the next belt robot: (S 接 -S 物 ) / v 本 ≤t2≤(S 下 -S 下面 ) / v 机 2v 下 ≤(-v 机 ×t2+S 下 )-[v 下 ×(t2-(S 接 -S 物 ) / v 本 )+S 接 ] Where S 接 is the belt overlap position, S 下 For the next belt robot position, S 下面 is the end position of the next robot motion range, v 下 is the speed of the next belt, and t2 is the time it takes for the next robot to move to the safe distance; In the disposal execution module, the equipment parameters and working mode are dynamically adjusted according to the type and size of the foreign object and the running status of the belt; when a foreign object exceeding the preset size is detected, the emergency shutdown program is activated; when a foreign object not exceeding the preset size is detected, an early warning is activated and the operator is notified; when the robot confirms the foreign object, an alarm signal is issued through a sound alarm, SMS notification, and a pop-up window on the monitoring platform, and an image is captured and the alarm information is recorded.

10. The equipment control device based on foreign body identification and prediction of coal mine belts according to claim 8, characterized in that: Also includes: An image processing optimization module is used to improve the image processing algorithm, including: establishing a sample database, storing the collected video images containing foreign objects and the corresponding edge features, texture features, and color feature information as sample data in the database, and using the sample data to optimize the threshold segmentation parameters and edge detection operator parameters in the image processing algorithm; A deep learning optimization module is used to improve the deep learning algorithm, including: storing the collected video images containing foreign objects and the corresponding foreign object type, location, and property information as sample data in the database, and using the sample data to train the deep learning model: Where L is the loss function, N is the number of samples, and L i (θ) is the loss function of the i-th sample, θ is the parameter of the deep learning model, and the parameters of the deep learning model are optimized by minimizing the loss function L.

Images