Conveyor belt position abnormity detection method, device, equipment and medium

Abstract

The invention provides a conveyor belt position anomaly detection method, a conveyor belt position anomaly detection device, computing equipment and a computer readable storage medium. The conveying belt runs on a plurality of carrier rollers, and the method comprises the following steps: acquiring a running image of the conveying belt; the carrier rollers exposed out of the two sides of the conveying belt are recognized from the running images, recognition frames are obtained, each recognition frame comprises the image of one carrier roller, the recognition frame located on the first side of the conveying belt is a first recognition frame, and the recognition frame located on the second side of the conveying belt is a second recognition frame; and judging whether the position of the conveying belt is abnormal or not according to the number and the area of the first recognition frames and the second recognition frames. According to the method and the device, the abnormal position condition of the conveying belt can be accurately identified, and false alarms are reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a conveyor belt position anomaly detection method, a conveyor belt position anomaly detection device, a computing device, and a computer-readable storage medium. Background Art

[0002] In the art, there are technologies for detecting position anomalies of conveyor belts (for example, deviation of the coal conveyor belt). Existing detection technologies include traditional detection technologies based on contact switches, infrared rays or lasers, as well as artificial intelligence detection technologies based on semantic segmentation models or target detection models. However, these technologies are not able to detect position anomalies of conveyor belts very well. Especially when the working conditions of the conveyor belts are harsh or complex, the conveyor belts may shake or displace significantly during operation. Existing detection technologies often judge this normal shaking as position anomalies, resulting in false alarms, causing the detection effect to fail to meet the needs of actual work. Summary of the invention

[0003] To this end, the present application is dedicated to providing a conveyor belt position anomaly detection method, a conveyor belt position anomaly detection device, a computing device and a computer-readable storage medium, which can accurately identify the position anomaly of the conveyor belt and reduce the occurrence of false alarms.

[0004] On the one hand, the present application provides a method for detecting abnormal position of a conveyor belt, wherein the conveyor belt runs on multiple rollers, and the method includes: acquiring a running image of the conveyor belt; identifying the rollers exposed on both sides of the conveyor belt from the running image to obtain identification frames, wherein each identification frame includes an image of a roller, the identification frame located on the first side of the conveyor belt is the first identification frame, and the identification frame located on the second side of the conveyor belt is the second identification frame; judging whether the position of the conveyor belt is abnormal based on the number and area of ​​the first identification frames and the second identification frames.

[0005] According to this aspect, by identifying the rollers in the image and drawing identification frames of the rollers in the image, it is determined whether the position of the conveyor belt is abnormal based on the number and area of ​​the identification frames on both sides of the conveyor belt, and the position abnormality can be determined with a more comprehensive judgment standard. Such a judgment method allows the determination of whether the conveyor belt is deviating when the conveyor belt is shaking, because even if the conveyor belt is shaking, the exposed rollers on both sides of the conveyor belt can be more accurately identified. In other words, when the conveyor belt is shaking, the number and area of ​​the first and second identification frames of the conveyor belt will not change significantly, but will remain stable and balanced for a period of time. In this way, by counting the area and number of identification frames on both sides of the conveyor belt, the deviation of the conveyor belt can be accurately determined, reducing the occurrence of false alarms.

[0006] In a possible implementation of the present application, obtaining a running image of the conveyor belt includes: obtaining a running image of the conveyor belt by means of a movable photographing device, wherein the photographing device is disposed above the conveyor belt and moves between the front and rear ends of the conveyor belt, thereby photographing the entire conveyor belt.

[0007] According to this implementation, by using a movable camera to capture the running image of the conveyor belt, the running condition of the entire long conveyor belt can be inspected to ensure that any positional abnormalities of the entire conveyor belt can be monitored and reported, thereby ensuring the safe operation of the entire conveyor belt.

[0008] In a possible implementation of the present application, rollers exposed on both sides of the conveyor belt are identified from the running image to obtain an identification frame, including: training a roller identification model; and identifying the rollers in the running image through the trained roller identification model.

[0009] According to this implementation method, the running image is identified through a trained roller recognition model (i.e., an artificial intelligence model), which can more accurately identify rollers of different shapes (when the roller area covered by the running conveyor belt changes, the roller shape will also change and differ), thereby improving the accuracy of roller recognition and thereby improving the accuracy of conveyor belt position abnormality detection.

[0010] In a possible implementation of the present application, training a roller recognition model includes: collecting images of a conveyor belt when it is running as training images; marking rollers in the training images, wherein rollers located on a first side of the conveyor belt are marked as first-type rollers, and rollers located on a second side of the conveyor belt are marked as second-type rollers; inputting the marked training images into the roller recognition model, training the roller recognition model, and obtaining a trained roller recognition model.

[0011] According to this implementation method, by marking two types of rollers in the training image, the trained model can accurately distinguish the rollers on both sides of the conveyor belt in the image, which is conducive to accurately counting the number and area of ​​the identification boxes of the rollers on both sides, thereby improving the accuracy of judging abnormal conveyor belt position.

[0012] In a possible implementation of the present application, the roller recognition model adopts the YOLOv8 model.

[0013] According to this implementation method, the inventors of the present application discovered in research and practice that the YOLOv8 model is an artificial intelligence model with good recognition performance, which is particularly suitable for detecting the roller images in the present application and can have a good recognition accuracy rate, thereby helping to improve the accuracy of conveyor belt position abnormality judgment.

[0014] In a possible implementation of the present application, judging whether the position of the conveyor belt is abnormal is based on the number and area of ​​the first identification box and the second identification box, including: judging whether the number and area of ​​the first identification box and the second identification box meet the judgment condition, wherein the judgment condition includes that the number of first identification boxes is less than a first number threshold, the average area of ​​the first identification boxes is less than the first area threshold, the number of second identification boxes is greater than the second number threshold and the average area of ​​the second identification boxes is greater than the second area threshold, or the number of second identification boxes is less than the first number threshold, the average area of ​​the second identification boxes is less than the first area threshold, the number of first identification boxes is greater than the second number threshold and the average area of ​​the first identification boxes is greater than the second area threshold; if so, it is determined that there is an abnormality in the position of the conveyor belt.

[0015] According to this implementation, when either the first identification frame or the second identification frame is less in number and smaller in area, it means that the rollers on this side of the conveyor belt are not well exposed and are over-covered by the conveyor belt. At the same time, when the other of the first identification frame and the second identification frame is more in number and larger in area, it means that more of the rollers on the other side of the conveyor belt have been exposed and the conveyor belt does not cover this side normally. Combined with the area and number of the identification frames on both sides, it can be determined that the conveyor belt has shifted or deviated to one side, resulting in the exposed roller portion being different from the normal situation. By using this judgment and statistical method to determine the abnormal position of the conveyor belt, the accuracy of the judgment can be improved and false alarms can be avoided.

[0016] In a possible implementation of the present application, the operation image includes multiple operation images taken continuously, and whether the position of the conveyor belt is abnormal is judged according to the number and area of ​​the first identification box and the second identification box, including: judging whether the number and area of ​​the first identification box and the second identification box in each of the multiple operation images meet the judgment conditions; if so, it is determined that there is an abnormality in the position of the conveyor belt.

[0017] According to the present implementation, by judging the position abnormality of the conveyor belt based on a plurality of continuously shot running images, the position abnormality of the conveyor belt can be judged over a longer time span, so that an alarm will be triggered only when the position abnormality judgment result is maintained for a period of time, thereby avoiding abnormalities in a short period of time or misidentification being identified as dangerous situations in which the conveyor belt deviates, thereby reducing the occurrence of false alarms and improving the accuracy of alarms.

[0018] On the other hand, the present application provides a conveyor belt position abnormality detection device, the device including: an acquisition module, used to acquire a running image of the conveyor belt; an identification module, used to identify rollers exposed on both sides of the conveyor belt from the running image to obtain identification frames, wherein each identification frame includes an image of a roller, the identification frame located on the first side of the conveyor belt is a first identification frame, and the identification frame located on the second side of the conveyor belt is a second identification frame; a judgment module, used to judge whether the position of the conveyor belt is abnormal based on the number and area of ​​the first identification frames and the second identification frames.

[0019] In a possible implementation of the present application, the acquisition module is further configured to: acquire the running image of the conveyor belt through a movable shooting device, wherein the shooting device is arranged above the conveyor belt and moves between the front and rear ends of the conveyor belt to shoot the entire conveyor belt through movement.

[0020] In a possible implementation of the present application, the recognition module is further configured to: collect images of the conveyor belt when it is running as training images; mark the rollers in the training images, wherein the rollers located on the first side of the conveyor belt are marked as first-type rollers, and the rollers located on the second side of the conveyor belt are marked as second-type rollers; input the marked training images into the roller recognition model, train the roller recognition model, and obtain a trained roller recognition model.

[0021] In a possible implementation of the present application, the recognition module is further configured to: recognize rollers in the running image by using a trained roller recognition model.

[0022] In a possible implementation of the present application, the roller recognition model adopts the YOLOv8 model.

[0023] In a possible implementation of the present application, the judgment module is further configured to: judge whether the number and area of ​​the first identification box and the second identification box meet the judgment condition, wherein the judgment condition includes that the number of first identification boxes is less than a first number threshold, the average area of ​​the first identification box is less than the first area threshold, the number of second identification boxes is greater than the second number threshold and the average area of ​​the second identification box is greater than the second area threshold, or the number of second identification boxes is less than the first number threshold, the average area of ​​the second identification box is less than the first area threshold, the number of first identification boxes is greater than the second number threshold and the average area of ​​the first identification box is greater than the second area threshold; if so, it is determined that there is an abnormality in the position of the conveyor belt.

[0024] In a possible implementation of the present application, the operation image includes multiple operation images taken continuously, and the judgment module is further configured to: judge whether the number and area of ​​the first identification box and the second identification box in each of the multiple operation images meet the judgment conditions; if so, it is determined that there is an abnormality in the position of the conveyor belt.

[0025] On the other hand, the present application provides a computing device, the computing device including a processor and a memory, the processor being used to execute a computer program stored in the memory to implement the above-mentioned conveyor belt position abnormality detection method.

[0026] On the other hand, the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and the computer program is used to execute the above-mentioned conveyor belt position abnormality detection method.

[0027] On the other hand, the present application provides a computer program product, including program code, which enables a computer to implement the above-mentioned conveyor belt position abnormality detection method when the computer program product is run on the computer.

[0028] Any of the conveyor belt position abnormality detection devices, computing devices, computer-readable storage media or computer program products provided above are used to execute the conveyor belt position abnormality detection method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding schemes in the corresponding methods provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The specific implementation of the present application is described in detail below with reference to the accompanying drawings, wherein:

[0030] Figure 1 A schematic diagram showing the structure of a conveyor belt position abnormality detection method according to an embodiment of the present application;

[0031] Figure 2 Show according to Figure 1 A schematic diagram of an operation image in a conveyor belt position abnormality detection method according to an embodiment;

[0032] Figure 3 Show according to Figure 1 A schematic diagram of a running image with an identification frame in a method for detecting abnormal position of a conveyor belt according to an embodiment;

[0033] Figure 4 Show according to Figure 1 A schematic diagram of a running image showing position abnormality in a conveyor belt position abnormality detection method according to an embodiment;

[0034] Figure 5 A schematic diagram showing a flow chart of a method for detecting abnormal position of a conveyor belt according to an embodiment of the present application;

[0035] Figure 6 A schematic diagram showing a flow chart of a method for detecting abnormal position of a conveyor belt according to another embodiment of the present application;

[0036] Figure 7A schematic diagram showing a flow chart of a method for detecting abnormal position of a conveyor belt according to an embodiment of the present application;

[0037] Figure 8 A schematic diagram showing the structure of a conveyor belt position abnormality detection device according to an embodiment of the present application is shown;

[0038] Fig. 9 A schematic diagram of the structure of a computing device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0039] In order to make those skilled in the art understand the concept and thought of the present application more clearly, the present application is described in detail below in conjunction with specific embodiments. It should be understood that the embodiments provided herein are only a part of all embodiments that the present application may have. After reading the specification of the present application, those skilled in the art have the ability to make improvements, transformations, or replacements to part or all of the following embodiments, and these improvements, transformations, or replacements are also included in the scope of the present application.

[0040] In this document, the terms "one", "an" and other similar words are not intended to indicate that there is only one of the things described, but rather that the relevant description is only for one of the things described, and the things described may have one or more. In this document, the terms "comprises", "includes" and other similar words are intended to indicate logical relationships, and cannot be regarded as indicating spatial structural relationships. For example, "A includes B" is intended to indicate that B logically belongs to A, but does not mean that B is spatially located inside A. In addition, the meanings of the terms "comprises", "includes" and other similar words should be regarded as open, rather than closed. For example, "A includes B" is intended to indicate that B belongs to A, but B does not necessarily constitute the whole of A, and A may also include other elements such as C, D, and E.

[0041] In this document, the terms "first", "second" and other similar words are not intended to imply any order, quantity and importance, but are only used to distinguish different elements. In this document, the terms "embodiment", "present embodiment", "one embodiment" and "an embodiment" do not mean that the relevant description is only applicable to a specific embodiment, but rather that these descriptions may also be applicable to one or more other embodiments. Those skilled in the art should understand that in this document, any description made for a certain embodiment can be replaced, combined, or otherwise combined with the relevant description in one or more other embodiments, and the new embodiment produced by the replacement, combination, or other combination is easily conceivable by those skilled in the art and belongs to the scope of protection of this application.

[0042] In the various embodiments of the present application, a conveyor belt may refer to a conveyor belt used to convey articles or goods, which may be a belt made of a flexible material, or a chain conveyor belt made of a rigid material, etc. In the various embodiments of the present application, an abnormal position of a conveyor belt may refer to a situation in which the running position of the conveyor belt deviates from the allowed range under normal circumstances, resulting in a situation in which the operation of the conveyor belt may be dangerous or harmful, such as possibly causing goods to fall, the conveyor belt to be damaged, the system to be interrupted, and other undesirable situations. Conveyor belt position abnormality detection may refer to monitoring, judging and analyzing the situation in which the conveyor belt position is abnormal. When it is determined that the running position of the conveyor belt is abnormal and may cause a dangerous situation, a judgment result is given, the judgment result is notified to the staff, and a report or alarm is issued.

[0043] For example, in the coal production process, the coal conveyor belt is an important conveying equipment, and its operating status directly affects the efficiency and safety of coal transportation. Belt deviation is one of the common faults in the coal conveyor scenario. If it is not discovered and corrected in time, it may cause belt wear and tear, or even cause serious accidents such as fire. Early belt deviation detection mainly relied on simple mechanical switches, such as lever-type or roller-type contact switches. When the belt deviates from the normal position, it will touch these switches, thereby sounding an alarm or triggering a shutdown mechanism. With the advancement of optoelectronic technology, non-contact detection equipment based on infrared or laser principles has emerged. Such equipment can more efficiently detect the position change of the belt edge. However, the above technologies all have problems such as low sensitivity, high false alarm rate, and high installation and maintenance costs. In recent years, with the development of advanced technologies such as the Internet of Things (IoT), artificial intelligence (AI), and machine vision, belt deviation detection has become more intelligent. Belt deviation detection technology based on deep learning models can detect the running status of the belt in real time, accurately and quickly determine whether the belt is deviating, and issue an alarm in time after an abnormality is found, so as to achieve efficient inspection of coal conveyor belt deviation.

[0044] In one embodiment of the present application, a method for detecting belt deviation based on a deep learning model includes: using a semantic segmentation model based on deep learning to extract the belt area in the image in real time from the video and image captured by a fixed-point camera, and determining whether the belt is deviating based on the deviation value between the preset area and the model extracted area.

[0045] In one embodiment of the present application, another method for detecting belt deviation based on a deep learning model includes: using a deep learning-based target detection model to detect the belt area in the image in real time in the video and image captured by a fixed-point camera, and judging whether the belt is deviating based on whether the model's prediction result exceeds a threshold.

[0046] Regardless of whether the above two methods are based on semantic segmentation models or target detection models, they both require a large number of coal conveyor belt videos collected by robots or cameras. After extracting frames, a data set is made. The belt or roller area in the image is annotated according to the model requirements, and the appropriate model network and parameters are selected. After multiple rounds of training, the final deep learning model is obtained.

[0047] At present, the coal conveyor belt will produce irregular vibrations during operation, and the belt video collected by the robot in motion will also shake and deviate. When using the first method mentioned above, the belt area extracted by the model will randomly and frequently exceed the preset area, so this method has a high false alarm rate and cannot be applied to the inspection scene of the moving robot. When facing the shaking video, the belt area predicted by the model of the second method will also exceed the preset size and centerline position, resulting in the problem of random false alarms. Therefore, the detection stability of the above two methods in the moving robot scenario is poor and cannot guarantee high recognition accuracy.

[0048] According to some embodiments of the present application, for the detection of coal conveyor belt deviation in the scenarios of motion robots and fixed-point cameras, the above two methods are affected by video jitter and preset position offset, and there are problems of low recognition accuracy and easy false alarms. A coal conveyor belt deviation detection method based on target detection is proposed, which detects the roller areas at both ends of the belt in real time, and jointly determines whether the belt is deviated based on the number and area of ​​the rollers. The embodiments of the present application can effectively reduce the impact of video jitter on the recognition results, avoid false alarms caused by preset position offsets, and ensure recognition accuracy in the scenarios of motion robots and fixed-point cameras.

[0049] According to some embodiments of the present application, in order to intelligently inspect whether the coal conveyor belt is deviating on a motion robot and reduce the impact of video jitter and preset position offset, a coal conveyor belt deviation detection method based on a target detection model is proposed. By counting the number and area of ​​rollers at both ends of the belt, it is determined whether the belt in the area is deviating. The embodiments of the present application use coal conveyor videos collected by motion robots or fixed-point cameras, which can predict the belt status in real time in different coal conveyor scenes and provide a preview of the recognition results, with strong recognition accuracy and stability.

[0050] According to some embodiments of the present application, a method for detecting belt deviation of a coal conveyor belt based on a target detection model is provided. With the help of a high-resolution visible light camera, the belt deviation can be detected in motion or at a fixed point in different scenarios, thereby ensuring the operating safety of the coal transport belt.

[0051] According to some embodiments of the present application, in order to effectively solve the problem that when a motion robot inspects the coal conveyor, it is difficult to accurately detect whether the belt is deviating due to the influence of video jitter and preset position offset, a detection method that can accurately and efficiently detect the deviation of the entire belt is proposed. By continuously expanding the video images of the coal conveyor and retraining the target detection model, the embodiments of the present application have strong stability and robustness when detecting the deviation of the coal conveyor belt in different scenarios.

[0052] Figure 1 A schematic diagram of the structure of a conveyor belt position abnormality detection method according to an embodiment of the present application is shown.

[0053] like Figure 1 As shown, according to this embodiment, firstly, a running image is captured for the running conveyor belt. An example of a running image may be Figure 2 The image of the belt running on the coal conveyor is shown. Figure 2 It can be seen that the belt is used to transport coal, and multiple roller pairs arranged in a V shape are set under the belt. The roller pairs are exposed on both sides of the belt for a short period of time. By analyzing the condition of the rollers exposed on both sides of the belt, it can be determined whether the belt is deviated or abnormal in position.

[0054] After the running image is acquired, the exposed rollers on both sides of the conveyor belt in the running image can be identified. The result of the identification can be an identification frame, that is, the exposed roller segments in the image are framed in the form of a square in the recognition result image. An example of the recognition result image can be Figure 3 The image shown. Figure 3 In the figure, a small section of rollers exposed on both sides of the belt are framed by rectangular identification frames. As can be seen from the figure, the identification frame on the left side (or far end) of the belt is larger, and the identification frame on the right side (or near end) of the belt is smaller, because the roller area on the left side of the belt is larger and the roller area on the right side of the belt is smaller when observed from this angle. Based on this feature, it is necessary to distinguish the roller identification frames on the left and right sides of the belt, and analyze, judge and count them separately.

[0055] After the roller is identified, the identification frame on one side of the conveyor belt is determined as the first identification frame, and the identification frame on the other side of the conveyor belt is determined as the second identification frame. Based on the number and area of ​​the first and second identification frames, it can be determined whether the running position of the conveyor belt is abnormal. The image of the abnormal conveyor belt position can be, for example, Figure 4 The image shown. Figure 4It can be seen that the roller identification frames identified from the image are all located on the right side (or far end) of the belt. The rollers on the left side of the belt are completely covered by the belt and cannot be identified from the image. Therefore, there is no identification frame on the left side of the belt. Based on this information, it can be determined that the belt position has deviated, and the situation can be notified to the staff or an alarm can be issued.

[0056] Figure 5 A schematic flow chart of a method for detecting abnormal position of a conveyor belt according to an embodiment of the present application is shown.

[0057] According to this embodiment, the conveyor belt position abnormality detection method includes steps S510 to S530, and each step is described in detail below.

[0058] S510, acquiring a running image of a conveyor belt, where the conveyor belt runs on a plurality of rollers.

[0059] In this embodiment, the conveyor belt can be a long conveyor belt for transporting any articles or goods, such as a belt or chain conveyor belt. During operation, the upper part of the conveyor belt is used to place or load goods, and the lower part of the conveyor belt is supported by rollers, and the rotation of the rollers can drive the movement of the conveyor belt above the rollers. Of course, the rollers can be actively rotating rollers or passively rotating rollers. In the latter case, the movement of the conveyor belt can drive the rotation of the rollers, and the power of the conveyor belt can be provided by other driving devices.

[0060] In this embodiment, the running image of the conveyor belt may refer to an image of the conveyor belt in a normal operating state. In a normal operating state, the conveyor belt is located above the rollers, and goods are loaded on the conveyor belt. Of course, in some cases, there may be no goods loaded on the conveyor belt, for example, the loading of goods is interrupted at this time, or the conveyor belt is being debugged at this time, and the conveyor belt is in a trial operation stage. In the running image, the conveyor belt is on the rollers, and the conveyor belt and the rollers below it can be photographed at the same time. At this time, it is possible to determine whether the position of the conveyor belt is abnormal by observing and analyzing the condition of the rollers.

[0061] As an example, in order to obtain a running image of the conveyor belt, a movable shooting device can be used to obtain a running image of the conveyor belt, wherein the shooting device is set above the conveyor belt and moves between the front and rear ends of the conveyor belt to shoot the entire conveyor belt.

[0062] In this example, the movable photographing device is, for example, a mobile robot equipped with a camera. Of course, the movable photographing device may be other types of devices, such as a mobile cart equipped with a camera or a camera head located above the conveyor belt. Other types of mobile photographing devices are also conceivable. By photographing the running image of the conveyor belt through a movable photographing device, an overall image of a conveyor belt of a certain length can be obtained. By moving the photographing device, images of the entire conveyor belt length direction from one end of the conveyor belt to the other end can be photographed, so that the conveyor belt can be monitored and observed over its entire length, and any abnormal position of any part of the conveyor belt can be found.

[0063] Specifically, a mobile robot equipped with a visible light camera (resolution of 1920×1080, frame rate of 25HZ) is located above the coal conveyor belt. The camera selects a suitable angle and magnification to aim at the belt and then starts recording video. The robot moves forward along the belt, fully recording the entire belt and obtaining a clear video of the belt and rollers. If there are multiple belts, the above steps can be repeated for video recording.

[0064] S520, identifying rollers exposed on both sides of the conveyor belt from the running image to obtain identification frames, wherein each identification frame includes an image of a roller, the identification frame located on the first side of the conveyor belt is the first identification frame, and the identification frame located on the second side of the conveyor belt is the second identification frame.

[0065] In this embodiment, since the rollers are located below the conveyor belt and are partially covered by the conveyor belt, when the running image of the conveyor belt is taken from above the conveyor belt, only the roller parts exposed from both sides of the conveyor belt, that is, a small section of the rollers, can be photographed. The recognition model can be trained to identify the roller parts exposed on both sides of the conveyor belt from such running images, and frame the identified roller parts by using recognition frames. Each recognition frame only contains the image of one (a section of) roller. In this way, the number of recognition frames corresponds to the number of exposed rollers, and the area of ​​the recognition frame corresponds to the size or proportion of the exposed part of the roller. The two sides of the conveyor belt can refer to the two sides in the width direction of the conveyor belt, that is, the two sides in the direction perpendicular to the moving direction of the conveyor belt. The recognition frame on one side of the conveyor belt can be the first recognition frame, and the recognition frame on the other side of the conveyor belt can be the second recognition frame. By distinguishing the first and second recognition frames, identifying and analyzing them respectively, the exposure of the rollers can be determined, and then whether the position of the conveyor belt is abnormal.

[0066] Specifically, the video stream transmitted by the motion robot equipped with a camera can be extracted to obtain an image, and the image can be sent to the generated inference model to obtain the recognition frame coordinates and scores of all rollers in the frame image. Then, the recognition frame coordinates with scores below the threshold are filtered out, and NMS (non-maximum suppression) processing is performed to filter out overlapping and interfering recognition frames.

[0067] S530: Determine whether the position of the conveyor belt is abnormal based on the number and area of ​​the first identification frame and the second identification frame.

[0068] In this embodiment, the number of identification boxes corresponds to the number of rollers exposed on both sides of the conveyor belt. When the number of identification boxes is small, it means that the number of rollers exposed on that side is small, and more rollers are completely covered by the conveyor belt, so that the rollers cannot be seen or identified from the running image. When the number of identification boxes is large, it means that the number of rollers exposed on that side is large, and the conveyor belt may have shifted to the other side, causing all the rollers on that side to be exposed, so that more rollers can be identified from the running image. The area of ​​the identification box corresponds to the size or proportion of the exposed part of the rollers on both sides of the conveyor belt. When the area of ​​the identification box is large, it means that the length of the rollers exposed on that side is large, and the conveyor belt may have shifted to the other side. When the area of ​​the identification box is small or zero, it means that the exposed part of the rollers on that side is small or cannot be exposed, and the conveyor belt may have shifted to that side.

[0069] As an example, in order to determine whether the position of the conveyor belt is abnormal based on the number and area of ​​the first identification box and the second identification box, it can be determined whether the number and area of ​​the first identification box and the second identification box meet the judgment conditions, wherein the judgment conditions include that the number of first identification boxes is less than a first number threshold, the average area of ​​the first identification boxes is less than the first area threshold, the number of second identification boxes is greater than the second number threshold and the average area of ​​the second identification boxes is greater than the second area threshold, or the number of second identification boxes is less than the first number threshold, the average area of ​​the second identification boxes is less than the first area threshold, the number of first identification boxes is greater than the second number threshold and the average area of ​​the first identification boxes is greater than the second area threshold; if so, it is determined that there is an abnormality in the position of the conveyor belt.

[0070] In this example, when the number of first identification frames is small and the area is small, it means that the rollers on the first side are covered to a large extent, and the conveyor belt may deviate to the first side. At this time, if the number of second identification frames is large and the area is large, it means that the rollers on the second side are exposed to a large extent, further proving the possibility of the conveyor belt deviating to the first side. Conversely, when the number of second identification frames is small and the area is small, it means that the rollers on the second side are covered to a large extent. Combined with the recognition result that the number of first identification frames is large and the area is large, it can be further judged that the conveyor belt may deviate to the second side.

[0071] Specifically, the number and average area of ​​the identification frames of the near-end roller and the far-end roller can be calculated respectively. If the number of identification frames of the far-end roller is 0, and the average area of ​​the identification frames of the near-end roller is greater than the threshold value 1, it is determined that the belt in the frame image is deviating toward the far end; if the number of identification frames of the near-end roller is 0, and the average area of ​​the identification frames of the far-end roller is greater than the threshold value 2, it is determined that the belt in the frame image is deviating toward the near end; if the number of identification frames of the rollers at both ends is not zero, it is determined that the belt in the frame image is normal. In actual situations, the belt deviation judgment standard can be flexibly adjusted according to needs. For example, if the number of identification frames of the roller at one end is between 1 and 3, and the average area of ​​the identification frame at the roller end is less than the threshold value 3, and the average area of ​​the identification frame of the roller at the other end is greater than the threshold value 4, it is determined that the belt is deviating toward the other end.

[0072] As an example, the operation image includes a plurality of operation images taken continuously. At this time, in order to determine whether the position of the conveyor belt is abnormal according to the number and area of ​​the first identification frame and the second identification frame, it can be determined whether the number and area of ​​the first identification frame and the second identification frame in each of the plurality of operation images meet the determination conditions; if so, it is determined that the position of the conveyor belt is abnormal.

[0073] In this example, by identifying and analyzing multiple (multi-frame) running images taken continuously, the time span of analysis and judgment can be extended, and the reliability and accuracy of judgment can be improved. If it can be determined in multiple running images taken continuously that the conveyor belt has shifted to the first side or the second side, it can be determined with a high probability that the conveyor belt has indeed been abnormally positioned at this time, requiring staff intervention or an alarm.

[0074] Specifically, when multiple consecutive frames determine that the belt is deviating to the same end, it is determined that the belt in this area has a deviation problem, and the detection program immediately outputs a deviation alarm; when multiple consecutive frames determine that the belt is normal, the detection program outputs that the belt is normal. Each time the detection program outputs a result, it saves the image of the roller detection result of that frame.

[0075] Specifically, the belt deviation detection results can be pushed to the information center in real time, and the roller detection result image can be displayed; when the belt deviation is detected, an alarm is immediately sent to the information center to prompt manual intervention and timely correction.

[0076] Figure 6 A schematic flow chart of a method for detecting abnormal position of a conveyor belt according to an embodiment of the present application is shown.

[0077] According to this embodiment, the conveyor belt position abnormality detection method includes steps S610 to S640, and each step is described in detail below.

[0078] S610, acquiring a running image of a conveyor belt, where the conveyor belt runs on a plurality of rollers.

[0079] For details about S610, see above Figure 5 The detailed description of S510 in the embodiment will not be repeated here.

[0080] S620, training a roller recognition model.

[0081] In this embodiment, the roller recognition model may be an artificial intelligence model that can recognize rollers or a part of rollers from the running image of the conveyor belt. The roller recognition model can be trained with a large number of annotated running images, so that it has the ability to accurately recognize rollers from various running images. Recognizing rollers through an artificial intelligence model can make the identification of rollers more widely applicable, so that rollers appearing in various conveyor belt running conditions can be identified, avoiding the problems of inaccuracy and high false alarm rate caused by detection achieved by traditional detection methods (such as contact switches, lasers, infrared rays, etc.).

[0082] As an example, in order to train a roller recognition model, you can first collect images of the conveyor belt when it is running as training images; then, label the rollers in the training images, where the rollers located on the first side of the conveyor belt are labeled as first-type rollers, and the rollers located on the second side of the conveyor belt are labeled as second-type rollers; finally, input the labeled training images into the roller recognition model, train the roller recognition model, and obtain a trained roller recognition model.

[0083] In this example, in order to train the roller recognition model, images of the conveyor belt can be collected during its actual production operation. The conveyor belt images used for training and the conveyor belt operation images used for identification should have correspondence or consistency, so as to ensure that the trained model can be used for the recognition of the operation image. After the training image is collected, the image can be annotated manually or with an annotation tool to mark out the rollers or roller parts therein. When annotating, the rollers on both sides of the conveyor belt should be distinguished. Because the rollers on both sides of the conveyor belt need to be analyzed and calculated separately during subsequent identification and analysis, they need to be annotated separately during the annotation in the training stage. After the annotation is completed, the annotated training images can be input into the artificial intelligence model so that the artificial intelligence model can continuously learn the characteristics and features of the rollers, so that the recognition model has better recognition ability and recognition accuracy, and finally a trained roller recognition model that can be put into use is obtained.

[0084] Specifically, the collected belt video can be framed, and 1 frame can be retained for every 5 frames (adjusted according to the robot's moving speed), and about 1,000 images can be obtained. All rollers in the image can be annotated using image annotation tools, and the rollers on both sides of the belt can be divided into two categories: the rollers close to the robot side are proximal rollers, and the rollers on the other side of the belt are distal rollers. The rollers in all images are annotated in these two categories. The annotated images are divided into training sets and validation sets in a ratio of 8:2 to complete the preparation of the coal conveyor belt dataset.

[0085] As an example, the roller recognition model adopts the YOLOv8 model.

[0086] In this example, the YOLOv8 model is a new version of the YOLO (You Only Look Once) series of real-time object detectors with many advanced features in the field of object detection. The YOLOv8 model is known for its cutting-edge performance in terms of accuracy and speed. It can quickly process image or video data and is very suitable for application scenarios with high real-time requirements. The YOLOv8 model has been improved and innovated on the basis of previous YOLO versions, introducing new features and optimizations, and adopting a more advanced backbone and neck architecture to better balance the relationship between accuracy and speed.

[0087] Specifically, the prepared dataset can be used to train the target detection model YOLOv8. YOLOv8 is a SOTA (State-of-the-Art Model, industry-leading model) model. The backbone network and Neck structure are replaced with a C2f (Channel to Feature) structure with richer gradient flow, and the number of channels is adjusted for different scale models, which greatly improves the model performance. The head part uses the current mainstream decoupled head structure to separate the classification and regression heads, and also changes from Anchor-Based to Anchor-Free. The loss function Loss calculation adopts the Task Aligned Assigner positive and negative sample allocation strategy, and introduces the Distribution Focal Loss function. The last 10 epochs of model training can effectively improve the model accuracy by turning off Mosiac enhancement. The YOLOv8m version model with moderate model parameters is used while ensuring accuracy and inference speed. Model parameter settings: image input size is 640×640, pre-trained model is based on COCO dataset, optimizer is SGD, initial learning rate is 0.01, learning rate optimization strategy is linear, warm_up is 1000 at most, batch_size is 16, training rounds are 400 epochs. After training, the model's optimal mean average precision mAP (mean average precision) is 95.45%, and the model is exported as an inference model for subsequent use.

[0088] S630. Identify the rollers exposed on both sides of the conveyor belt from the running image through the trained roller recognition model to obtain recognition frames, wherein each recognition frame includes an image of a roller, the recognition frame located on the first side of the conveyor belt is the first recognition frame, and the recognition frame located on the second side of the conveyor belt is the second recognition frame.

[0089] In this embodiment, the rollers are identified from the running image by the trained roller identification model. Since the rollers have been divided into two categories according to their locations on the first and second sides of the conveyor belt during the labeling and training process, the roller identification model can automatically identify the rollers located on the first and second sides of the conveyor belt during the identification process, distinguish the two types of rollers, and obtain the first identification frame and the second identification frame respectively. For other details about S630, see above. Figure 5 The detailed description of S520 in the embodiment will not be repeated here.

[0090] S640: Determine whether the position of the conveyor belt is abnormal based on the number and area of ​​the first identification frame and the second identification frame.

[0091] For details on S640, see above Figure 5 The detailed description of S530 in the embodiment will not be repeated here.

[0092] Figure 7 A schematic flow chart of a method for detecting abnormal position of a conveyor belt according to an embodiment of the present application is shown.

[0093] like Figure 7 As shown, according to this embodiment, the belt video of the coal conveyor can be first collected by a shooting robot. After the video is collected, the video is framed, and the rollers in each frame image are marked after the frame is drawn, so as to prepare a data set of the coal conveyor. Then, the YOLOv8 target detection model is trained using the data set and the trained optimal model is saved. Then, in practical applications, the video stream in actual production captured by the shooting robot is framed cyclically, and the extracted image frames are model tested. The number and area mean of the rollers on both sides of the belt are calculated according to the detection results of the frame image, so as to judge whether the number of rollers on one side of the belt is 0 and whether the area mean of the rollers on the other side is greater than the threshold. If the judgment result is yes, it can be determined that the belt in the frame image has deviated; if the judgment result is no, it can be determined that the belt in the frame image is normal. Then, the detection results of n frames are continuously recorded. If the images of n consecutive frames all show that the belt is deviating, the result of the belt deviation is output, and the staff is notified or an alarm is triggered; if the results of n consecutive frames do not all show that the belt is deviating, the result of the belt is normal is output, and no alarm is triggered.

[0094] Based on the above Figure 5 The method embodiment described above, the present application embodiment also provides a conveyor belt position abnormality detection device, the structural diagram of which is as follows Figure 8 The device is used to perform the above Figure 5 The various steps in .

[0095] According to this embodiment, the conveyor belt position abnormality detection device 800 includes an acquisition module 810, an identification module 820 and a judgment module 830. The acquisition module 810 is used to acquire the running image of the conveyor belt. The identification module 820 is used to identify the rollers exposed on both sides of the conveyor belt from the running image to obtain an identification frame, wherein each identification frame includes an image of a roller, the identification frame located on the first side of the conveyor belt is the first identification frame, and the identification frame located on the second side of the conveyor belt is the second identification frame. The judgment module 830 is used to judge whether the position of the conveyor belt is abnormal based on the number and area of ​​the first identification frame and the second identification frame.

[0096] It should be noted that Figure 8The conveyor belt position abnormality detection device 800 provided in the embodiment shown in the figure is only used as an example to illustrate the division of the above-mentioned functional modules when executing the method. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Figure 5 The embodiments of the conveyor belt position abnormality detection method shown belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0097] Fig. 9 It is a schematic diagram of the hardware structure of a computing device 900 provided in an embodiment of the present application.

[0098] See also Fig. 9 The computing device 900 includes a processor 910, a memory 920, a communication interface 930, and a bus 940, wherein the processor 910, the memory 920, and the communication interface 930 are connected to each other via the bus 940. The processor 910, the memory 920, and the communication interface 930 may also be connected in other connection modes besides the bus 940.

[0099] Among them, the memory 920 can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, optical storage, hard disk, etc.

[0100] The processor 910 may be a general-purpose processor, which may be a processor that performs specific steps and / or operations by reading and executing the contents stored in a memory (e.g., the memory 920). For example, the general-purpose processor may be a central processing unit (CPU). The processor 910 may include at least one circuit to execute Figure 5 The illustrated embodiment provides all or part of the steps of the method for detecting conveyor belt position abnormality.

[0101] Among them, the communication interface 930 includes input / output (I / O) interfaces, physical interfaces, and logical interfaces for interconnecting devices within the computing device 900, as well as interfaces for interconnecting the computing device 900 with other devices (such as other computing devices or user devices). The physical interface may be an Ethernet interface, an optical fiber interface, an ATM interface, etc. The communication interface 930 may be connected to an external input device and an output device. For example, the input device may be a microphone or a microphone array for capturing voice input signals; it may be a communication network connector for receiving collected input signals from the cloud or other devices; it may also include, for example, a keyboard, a mouse, and the like. The output device may output various information to the outside, including determined distance information, direction information, and the like. The output device may include, for example, a display, a speaker, a printer, a communication network, and a remote output device connected thereto, and the like.

[0102] The bus 940 may be any type of communication bus for interconnecting the processor 910 , the memory 920 , and the communication interface 930 , such as a system bus.

[0103] The above devices may be arranged on independent chips, or at least partially or completely on the same chip. Whether to arrange each device independently on different chips or to integrate them on one or more chips often depends on the needs of product design. The embodiments of the present application do not limit the specific implementation form of the above devices.

[0104] Fig. 9 The computing device 900 shown is merely exemplary. During implementation, the computing device 900 may further include other components, which are not listed one by one herein.

[0105] An embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps of the conveyor belt position abnormality detection method according to various embodiments of the present application described above in this specification.

[0106] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0107] The concepts, principles and ideas of the present application are described in detail above in conjunction with specific implementation methods (including embodiments and examples). Those skilled in the art should understand that the implementation methods of the present application are more than the several forms given above. After reading the application documents, those skilled in the art can make any possible improvements, replacements and equivalent forms to the steps, methods, devices and components in the above-mentioned implementation methods, and these improvements, replacements and equivalent forms should be deemed to fall within the scope of the present application. The scope of protection of the present application shall be subject only to the claims.

Claims

1. A method for detecting abnormal position of a conveyor belt, characterized in that: The conveyor belt runs on a plurality of rollers, and the method comprises: Acquiring a running image of the conveyor belt; Identify the rollers exposed on both sides of the conveyor belt from the running image to obtain identification frames, wherein each of the identification frames includes an image of the roller, the identification frame located on the first side of the conveyor belt is a first identification frame, and the identification frame located on the second side of the conveyor belt is a second identification frame; Whether the position of the conveyor belt is abnormal is determined according to the number and area of ​​the first identification frame and the second identification frame.

2. The method according to claim 1, characterized in that The step of obtaining the running image of the conveyor belt comprises: The running image of the conveyor belt is acquired by a movable photographing device, wherein the photographing device is arranged above the conveyor belt and moves between the front and rear ends of the conveyor belt, so as to photograph the entire conveyor belt by moving.

3. The method according to claim 1, characterized in that The rollers exposed on both sides of the conveyor belt are identified from the running image to obtain an identification frame, including: Train the roller recognition model; The rollers in the running image are identified by using the trained roller identification model.

4. The method according to claim 3, characterized in that The training roller identification model comprises: Collecting images of the conveyor belt when it is running as training images; marking the rollers in the training image, wherein the rollers located on the first side of the conveyor belt are marked as first-type rollers, and the rollers located on the second side of the conveyor belt are marked as second-type rollers; The labeled training image is input into the roller recognition model to train the roller recognition model to obtain the trained roller recognition model.

5. The method according to claim 3, characterized in that: The roller recognition model adopts the YOLOv8 model.

6. The method according to claim 1, characterized in that The step of judging whether the position of the conveyor belt is abnormal according to the number and area of ​​the first identification frame and the second identification frame includes: Determine whether the number and area of ​​the first identification box and the second identification box meet a determination condition, wherein the determination condition includes that the number of the first identification boxes is less than a first number threshold, the average area of ​​the first identification boxes is less than the first area threshold, the number of the second identification boxes is greater than a second number threshold, and the average area of ​​the second identification boxes is greater than the second area threshold, or the number of the second identification boxes is less than the first number threshold, the average area of ​​the second identification boxes is less than the first area threshold, the number of the first identification boxes is greater than the second number threshold, and the average area of ​​the first identification boxes is greater than the second area threshold; If so, it is determined that there is an abnormality in the position of the conveyor belt.

7. The method according to claim 6, characterized in that The operation image includes a plurality of operation images taken continuously, and judging whether the position of the conveyor belt is abnormal according to the number and area of ​​the first identification frame and the second identification frame includes: Determine whether the number and area of ​​the first identification frame and the second identification frame in each of the plurality of operation images meet the determination condition; If so, it is determined that there is an abnormality in the position of the conveyor belt.

8. A conveyor belt position abnormality detection device, characterized in that: The device comprises: An acquisition module, used for acquiring a running image of the conveyor belt; an identification module, configured to identify the rollers exposed on both sides of the conveyor belt from the running image, and obtain identification frames, wherein each of the identification frames includes an image of the roller, the identification frame located on the first side of the conveyor belt is a first identification frame, and the identification frame located on the second side of the conveyor belt is a second identification frame; The judging module is used to judge whether the position of the conveyor belt is abnormal according to the number and area of ​​the first identification frame and the second identification frame.

9. A computing device, characterized in that The computing device includes a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the conveyor belt position abnormality detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the conveyor belt position abnormality detection method according to any one of claims 1 to 7.

Images