A belt blockage abnormality automatic identification method and device

Abstract

This invention relates to the field of computer vision and image processing, and discloses an automatic identification method and apparatus for conveyor belt blockage anomalies. The method includes: inputting a first video frame group and a second video frame group of the conveyor belt into a material region segmentation model to obtain a first material region segmentation image group and a second material region segmentation image group; calculating the material width percentage in the first and second material region segmentation image groups, and comparing the changes in the material width percentage; if any change in the material width percentage is greater than or equal to a preset threshold, then a blockage is determined to have occurred; if the changes in the material width percentage are all less than the preset threshold, then calculating the material offset percentage in the first and second material region segmentation image groups, and comparing the changes in the material offset percentage; if any change in the material offset percentage is greater than or equal to a preset threshold, then a blockage is determined to have occurred. This achieves automatic identification of conveyor belt blockage anomalies.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing, specifically to a method and apparatus for automatic identification of conveyor belt blockage. Background Technology

[0002] Belt conveyors are a common tool in material transportation. Because belt conveyors can only travel in a straight line, multiple conveyors are usually used in coordination. This results in two belts exchanging materials, which refers to the process where material falls from the upstream belt into the head funnel and then onto the downstream belt conveyor. In actual production, material falling into the head funnel during the exchange sometimes cannot pass smoothly, leading to blockages. Blockages can cause a large amount of material to spill onto the ground. If the blockage is not detected in time, it may result in minor issues such as needing to clean up the spilled material, or even serious safety accidents.

[0003] Existing belt conveyor monitoring processes, especially in automated material handling processes, often encounter the following technical problems:

[0004] First, existing video recognition solutions typically use only one camera to identify the boundary of the material drop area at the head of the conveyor belt. During material transportation, the dust content at the head of the conveyor belt can be quite high, leading to inaccurate boundary determination and affecting the recognition effect.

[0005] Second, relying on manual video viewing or simple mechanical sensors makes it impossible to react in time the moment a blockage occurs.

[0006] Third, monitoring methods based on simple threshold settings are easily affected by environmental changes or equipment vibration, resulting in false alarms and unnecessary downtime and maintenance.

[0007] Fourth, although many conveyor belt sites have existing monitoring cameras, the existing monitoring systems cannot make full use of these cameras, requiring additional equipment and modifications. Summary of the Invention

[0008] The summary section of this invention provides a brief overview of the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0009] This invention proposes an automatic identification method and device for conveyor belt blockage abnormalities to solve one or more of the technical problems mentioned in the background section above.

[0010] In a first aspect, the present invention provides an automatic identification method for belt blockage anomalies, comprising: inputting a first video frame group of the belt conveyor into a pre-deployed material region segmentation model to obtain a first material region segmentation image group; and inputting a second video frame group of the belt conveyor into a pre-deployed belt blockage identification model to obtain a second material region segmentation image group.

[0011] Obtain the material width percentage of each first material region segmentation image in the first material region segmentation image group to obtain the first material width percentage group; obtain the material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain the second material width percentage group;

[0012] The difference between the first material width percentage group and the second material width percentage group is calculated to obtain the material width percentage change group; the ratio between the material width percentage change group and the first material width percentage group is calculated to obtain the material width percentage change amount group; if any material width percentage change amount in the material width percentage change amount group is greater than or equal to the preset material width percentage change threshold, it is determined that material blockage has occurred.

[0013] If the percentage change in material width of each material width in the material width percentage change group is less than the preset percentage change threshold for material width, then the percentage of material offset of each first material region segmentation image in the first material region segmentation image group is obtained to obtain the first material offset percentage group; the percentage of material offset of each second material region segmentation image in the second material region segmentation image group is obtained to obtain the second material offset percentage group.

[0014] The difference between the first material offset percentage group and the second material offset percentage group is calculated to obtain the material offset percentage change group; the ratio between the material offset percentage change group and the first material offset percentage group is calculated to obtain the material offset percentage change amount group; if any material offset percentage change amount in the material offset percentage change amount group is greater than or equal to the preset material offset percentage change threshold, then a blockage is determined to have occurred.

[0015] Optionally, the belt conveyor includes a feeding end area and a discharging end area; the first video file corresponding to the first video frame group is captured by the first camera, and the second video file corresponding to the second video frame group is captured by the second camera. The first camera has corresponding first camera installation environment information, and the second camera has corresponding second camera installation environment information. The first camera installation environment information and the second camera installation environment information are determined through the following steps:

[0016] The system acquires first environmental dust information in the feeding end area through a preset dust sensor and identifies the first environmental dust information as feeding end dust information; it acquires first ambient light information in the feeding end area through a preset light sensor and identifies the first ambient light information as feeding end light information; and it determines the first camera installation environment information based on the feeding end dust information and feeding end light information.

[0017] The second environmental dust information of the discharge end area is obtained by a preset dust sensor and is determined as the discharge end dust information; the second environmental light information of the discharge end area is obtained by a preset light sensor and is determined as the discharge end light information; the installation environment information of the second camera is determined based on the discharge end dust information and the discharge end light information.

[0018] Optionally, the first material offset percentage includes either the left offset percentage or the right offset percentage of the first material; the second material offset percentage includes either the left offset percentage or the right offset percentage of the second material, and

[0019] The difference between the left offset percentage of the first material and the left offset percentage of the second material is calculated to obtain the change value of the left offset percentage of the material. The ratio of the change value of the left offset percentage of the material to the left offset percentage of the first material is calculated to obtain the change amount of the left offset percentage of the material. If the change amount of the left offset percentage of the material is greater than or equal to a preset threshold, a blockage is determined to have occurred. If the change amount of the left offset percentage of the material is less than the preset threshold, the difference between the right offset percentage of the first material and the right offset percentage of the second material is calculated to obtain the change value of the right offset percentage of the material. The ratio of the change value of the right offset percentage of the material to the right offset percentage of the first material is calculated to obtain the change amount of the right offset percentage of the material. If the change amount of the right offset percentage of the material is greater than or equal to a preset threshold, a blockage is determined to have occurred.

[0020] Optionally, before inputting the first group of video frames from the belt conveyor into a pre-deployed material region segmentation model to obtain the first group of segmented material region images, the method further includes:

[0021] Decode the pre-stored historical belt conveyor monitoring video files to obtain historical belt conveyor video decoded files, and extract frames from the historical belt conveyor video decoded files to obtain historical belt conveyor monitoring image frame groups;

[0022] The historical belt conveyor monitoring image frame group is divided into a belt conveyor image frame training set and a belt conveyor image frame verification set according to a preset ratio;

[0023] Data annotation is performed on each historical belt conveyor monitoring image frame in the belt conveyor image frame training set to obtain the belt conveyor image frame annotation training set.

[0024] Optionally, the automatic identification method for conveyor belt blockage abnormalities of the present invention further includes:

[0025] The image frame annotation training set of the belt conveyor is input into the semantic segmentation model, and the semantic segmentation model is iteratively trained to obtain the first training model;

[0026] The first trained model is optimized to obtain the first optimized model.

[0027] Optionally, the automatic identification method for conveyor belt blockage abnormalities of the present invention further includes:

[0028] Input the image frame verification set of the belt conveyor into the first optimization model to obtain the first optimization verification result set;

[0029] The cross-union ratio (CUI) of the first optimized verification result set is compared with the preset CUI threshold to obtain the evaluation result set;

[0030] If the proportion of the evaluation results set that pass the evaluation reaches the preset evaluation threshold, then the first optimized model will be determined as the target model.

[0031] Optionally, the automatic identification method for conveyor belt blockage abnormalities of the present invention further includes:

[0032] The target model is converted to a new format to obtain a material region segmentation model, which is then deployed.

[0033] Optionally, both the first material region segmentation image and the second material region segmentation image are single-channel binary images.

[0034] In a second aspect, the present invention provides an automatic belt blockage abnormality identification device, comprising:

[0035] The video processing unit is used to input the first video frame group of the belt conveyor into a pre-deployed material area segmentation model to obtain a first material area segmentation image group; and to input the second video frame group of the belt conveyor into a pre-deployed belt blockage recognition model to obtain a second material area segmentation image group.

[0036] The material width calculation unit is used to obtain the material width percentage of each first material region segmentation image in the first material region segmentation image group to obtain the first material width percentage group; and to obtain the material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain the second material width percentage group.

[0037] The material width determination unit is used to perform a difference calculation between the first material width percentage group and the second material width percentage group to obtain a material width percentage change group; and to perform a ratio calculation between the material width percentage change group and the first material width percentage group to obtain a material width percentage change amount group; if any material width percentage change amount in the material width percentage change amount group is greater than or equal to a preset material width percentage change threshold, then a blockage is determined to have occurred.

[0038] The material offset percentage calculation unit is used to obtain the material offset percentage of each first material region segmentation image in the first material region segmentation image group if the material width percentage change amount of each material width percentage change group is less than the preset material width percentage change threshold, and obtain the material offset percentage of each second material region segmentation image in the second material region segmentation image group to obtain the second material offset percentage group.

[0039] The material offset percentage determination unit is used to perform a difference calculation between the first material offset percentage group and the second material offset percentage group to obtain a material offset percentage change group; and to perform a ratio calculation between the material offset percentage change group and the first material offset percentage group to obtain a material offset percentage change amount group; if any material offset percentage change amount in the material offset percentage change amount group is greater than or equal to a preset material offset percentage change threshold, then a blockage is determined to have occurred.

[0040] The present invention has the following advantages: by automatically monitoring the material area of ​​the conveyor belt, using two central cameras and a semantic segmentation model, it can accurately identify material blockage, avoiding the response delay and false alarm problems of manual monitoring, and improving identification efficiency and accuracy; based on real-time video analysis, material blockage can be automatically identified and triggered to stop the machine in a short time, minimizing material spillage and safety hazards; by utilizing existing camera equipment, system modifications and costs are reduced, and the feasibility of implementation is improved. Attached Figure Description

[0041] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0042] Figure 1 This is a flowchart of an automatic identification method for conveyor belt blockage abnormalities according to the present invention;

[0043] Figure 2 This is a schematic diagram of the structure of an automatic identification device for conveyor belt blockage abnormalities according to the present invention. Detailed Implementation

[0044] The invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the invention. It should be understood that the drawings and embodiments of the invention are for illustrative purposes only and are not intended to limit the scope of protection of the invention.

[0045] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0046] It should be noted that the concepts of "first" and "second" mentioned in this invention are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0047] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0048] The names of messages or information exchanged between the various devices of this invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0049] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0050] like Figure 1 The diagram shown is a flowchart of an automatic identification method and device for conveyor belt blockage abnormalities according to the present invention, which specifically includes the following steps:

[0051] Step 101: Input the first video frame group of the belt conveyor into the pre-deployed material region segmentation model to obtain the first material region segmentation image group; input the second video frame group of the belt conveyor into the pre-deployed belt blockage recognition model to obtain the second material region segmentation image group.

[0052] In some embodiments, the executing entity can be a server. The executing entity first establishes a communication connection with two pre-deployed cameras, acquiring the real-time video files corresponding to each camera. Then, it uses an open-source computer vision library to decode the two video files and extract multiple image frames to obtain a first video frame group and a second video frame group. Finally, the first video frame group is input into a pre-deployed conveyor belt blockage recognition model to obtain a first material region segmentation image group, and the second video frame group is input into the same model to obtain a second material region segmentation image group. In practice, the open-source computer vision library can be OpenCV (Open Source Computer Vision Library). An open-source computer vision library refers to an open-source software library specifically designed for computer vision and image processing tasks. A conveyor belt is a mechanical device used for continuous material transport, widely used in mining, metallurgy, coal, and other industries for conveying bulk materials or packaged goods. A video frame is a single image in a video file. A video file consists of a series of continuously played images; the number of frames played per second is called the frame rate. Video frames capture and record visual information at a specific point in time. By playing these frames rapidly and continuously, a video can present smooth motion effects. In the fields of computer vision and image processing, video frames are commonly used to analyze and process video content. Material blockage refers to the phenomenon in which materials, due to various reasons (such as accumulation, jamming, or excess), become blocked on conveyor equipment (such as belt conveyors), preventing normal flow and transport.

[0053] Step 102: Obtain the material width percentage of each first material region segmentation image in the first material region segmentation image group to obtain the first material width percentage group; obtain the material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain the second material width percentage group.

[0054] In some embodiments, the executing entity first calculates the material width percentage of each first material region segmentation image in the first material region segmentation image group, obtaining a first material width percentage group. Then, it calculates the material width percentage of each second material region segmentation image in the second material region segmentation image group, obtaining a second material width percentage group. In practice, the material width percentage can be calculated using the following formula:

[0055]

[0056] Among them, Ratio mThe percentage represents the material width. Pr represents the area of ​​the material recognition region (in pixels), and Gt represents the area of ​​the scribbled region (in pixels). The material width percentage is obtained by calculating the ratio of the intersection area of ​​the material recognition region and the scribbled region to the total area of ​​the scribbled region, and then converting this ratio to a percentage. The scribbled region is a pre-defined area in the material region segmentation image. The material recognition region is the actual area of ​​the material in the material region segmentation image. The intersection refers to the elements commonly contained in or formed by two or more sets, often used to describe the overlapping portion of two regions.

[0057] Step 103: Perform a difference calculation between the first material width percentage group and the second material width percentage group to obtain a material width percentage change group; perform a ratio calculation between the material width percentage change group and the first material width percentage group to obtain a material width percentage change amount group; if any material width percentage change amount in the material width percentage change amount group is greater than or equal to a preset material width percentage change threshold, then it is determined that a blockage has occurred.

[0058] In some embodiments, the executing entity sequentially performs a difference calculation on each first material width percentage in the first material width percentage group and each second material width percentage in the second material width percentage group to obtain a material width percentage change group. Next, it performs a ratio calculation between each material width percentage change value in the material width percentage change group and the corresponding first material width percentage in the first material width percentage group to obtain a material width percentage change amount group. Finally, it compares each material width percentage change amount in the material width percentage change amount group with a preset material width percentage change threshold. If any material width percentage change amount is greater than or equal to the preset material width percentage change threshold, it is determined that material blockage has occurred. In practice, the preset material width percentage change threshold is 20%.

[0059] Step 104: If the percentage change in material width of each material in the percentage change in material width group is less than the preset percentage change threshold in material width, then obtain the percentage of material offset of each first material region segmentation image in the first material region segmentation image group to obtain the first material offset percentage group; obtain the percentage of material offset of each second material region segmentation image in the second material region segmentation image group to obtain the second material offset percentage group.

[0060] In some embodiments, the executing entity first compares each material width percentage change in the material width percentage change group with a preset material width percentage change threshold. If all material width percentage changes are less than the preset material width percentage change threshold, then the material offset percentage of each first material region segmentation image in the first material region segmentation image group is calculated to obtain the first material offset percentage group. Next, the material offset percentage of each second material region segmentation image in the second material region segmentation image group is calculated to obtain the second material offset percentage group. In practice, the area of ​​the lined region and the material recognition region is subtracted to obtain the area of ​​the blank region. The material offset percentage is obtained by calculating the ratio of the intersection of the blank region area and the lined region area to the total area of ​​the lined region, and then performing a percentage conversion.

[0061] Step 105: Perform a difference calculation between the first material offset percentage group and the second material offset percentage group to obtain a material offset percentage change group; perform a ratio calculation between the material offset percentage change group and the first material offset percentage group to obtain a material offset percentage change amount group; if any material offset percentage change amount in the material offset percentage change amount group is greater than or equal to a preset material offset percentage change threshold, then it is determined that a blockage has occurred.

[0062] In some embodiments, the executing entity sequentially performs a difference calculation on each first material offset percentage in the first material offset percentage group and each second material offset percentage in the second material offset percentage group to obtain a material offset percentage change group. Next, it performs a ratio calculation between each material offset percentage change value in the material offset percentage change group and the corresponding first material offset percentage in the first material offset percentage group to obtain a material offset percentage change amount group. Finally, it compares each material offset percentage change amount in the material offset percentage change amount group with a preset material offset percentage change threshold. If any material offset percentage change amount is greater than or equal to the preset material offset percentage change threshold, it is determined that material blockage has occurred. In practice, the preset material offset percentage change threshold is 15%.

[0063] Optionally, the belt conveyor includes a feeding end area and a discharging end area; the first video file corresponding to the first video frame group is captured by the first camera, and the second video file corresponding to the second video frame group is captured by the second camera. The first camera has corresponding first camera installation environment information, and the second camera has corresponding second camera installation environment information. The first camera installation environment information and the second camera installation environment information are determined through the following steps:

[0064] The system acquires first environmental dust information in the feeding end area through a preset dust sensor and identifies the first environmental dust information as feeding end dust information; it acquires first ambient light information in the feeding end area through a preset light sensor and identifies the first ambient light information as feeding end light information; and it determines the first camera installation environment information based on the feeding end dust information and feeding end light information.

[0065] The second environmental dust information of the discharge end area is obtained by a preset dust sensor and is determined as the discharge end dust information; the second environmental light information of the discharge end area is obtained by a preset light sensor and is determined as the discharge end light information; the installation environment information of the second camera is determined based on the discharge end dust information and the discharge end light information.

[0066] In some embodiments, the belt conveyor includes a feed end area and a discharge end area. A camera located in the feed end area is a first camera, and a camera located in the discharge end area is a second camera. The video file captured by the first camera is a first video file, and the video file captured by the second camera is a second video file. The first camera has corresponding first camera installation environment information, and the second camera has corresponding second camera installation environment information. The first camera installation environment information and the second camera installation environment information are determined through the following steps:

[0067] The executing entity first establishes a communication connection with a pre-installed dust sensor in the feeding end area to obtain first ambient dust information, which is then identified as feeding end dust information. Subsequently, it establishes a communication connection with a pre-installed light sensor in the feeding end area to obtain first ambient light information, which is then identified as feeding end light information. Based on the feeding end dust information and feeding end light information, the installation environment information of the first camera is determined.

[0068] Next, a communication connection is established with a pre-installed dust sensor in the discharge end area to acquire second environmental dust information, which is then identified as the dust information at the discharge end. Subsequently, a communication connection is established with a pre-installed light sensor in the discharge end area to acquire second ambient light information, which is then identified as the light information at the discharge end. Based on the dust information and light information at the discharge end, the installation environment information of the second camera is determined.

[0069] In practice, the dust sensor model can be PMS5003, and the light sensor model can be TSL2561. The PMS5003 monitors environmental dust information including PM2.5 concentration, PM10 concentration, and dust concentration index. The TSL2561 monitors ambient light information including ambient light intensity, light intensity variation, and spectral distribution. High air pollution levels may affect the camera's field of view and performance; for example, a PM2.5 concentration of 76–150 micrograms per cubic meter indicates moderate pollution. In low ambient light intensity, the camera may require a longer exposure time or additional light source, while in excessively bright ambient light, the camera may overexpose, affecting image quality. For example, an ambient light intensity of 100–300 lux represents normal indoor lighting intensity.

[0070] The feed end refers to the inlet section where material enters the conveying equipment. The discharge end refers to the outlet section where material leaves the conveying equipment and enters the next process or storage area. The PMS5003 is a sensor for detecting particulate matter in the air, capable of measuring particulate matter concentration and providing related air quality information. It is suitable for monitoring PM2.5 and PM10 particulate matter concentrations in the environment. A dust sensor is a device used to detect the concentration of particulate matter (such as dust) in the air, typically using laser or light scattering technology to measure the degree to which particles block or scatter light to determine the concentration of particulate matter in the air. The TSL2561 is a sensor for measuring light intensity, providing ambient light data. A light sensor is a device capable of detecting light intensity or light radiation; it measures the ambient light intensity by converting light signals into electrical signals. PM2.5 refers to particulate matter in the air with a diameter of 2.5 micrometers or less. Because of their very small size, these particles can enter the human respiratory tract and penetrate deep into the lungs, even entering the bloodstream. PM10 refers to particulate matter in the air with a diameter of 10 micrometers or less. Although larger than PM2.5, they can still be inhaled into the respiratory tract and enter the lungs. Lux is the international unit of luminous intensity, representing the luminous flux per unit area. It is used to measure the brightness of a particular surface.

[0071] Optionally, the first material offset percentage includes either the left offset percentage or the right offset percentage of the first material; the second material offset percentage includes either the left offset percentage or the right offset percentage of the second material, and

[0072] The difference between the left offset percentage of the first material and the left offset percentage of the second material is calculated to obtain the change value of the left offset percentage of the material. The ratio of the change value of the left offset percentage of the material to the left offset percentage of the first material is calculated to obtain the change amount of the left offset percentage of the material. If the change amount of the left offset percentage of the material is greater than or equal to a preset threshold, a blockage is determined to have occurred. If the change amount of the left offset percentage of the material is less than the preset threshold, the difference between the right offset percentage of the first material and the right offset percentage of the second material is calculated to obtain the change value of the right offset percentage of the material. The ratio of the change value of the right offset percentage of the material to the right offset percentage of the first material is calculated to obtain the change amount of the right offset percentage of the material. If the change amount of the right offset percentage of the material is greater than or equal to a preset threshold, a blockage is determined to have occurred.

[0073] In some embodiments, the first material offset percentage includes a left offset percentage or a right offset percentage of the first material; the second material offset percentage includes a left offset percentage or a right offset percentage of the second material, and

[0074] The executing entity first calculates the left offset percentage of the first material and the left offset percentage of the second material, and then calculates the difference between the two left offset percentages to obtain the change value of the left offset percentage. Next, it calculates the ratio of this change value to the left offset percentage of the first material to obtain the amount of change in the left offset percentage. Then, it compares this change value to a preset threshold for the change in the left offset percentage. If the change in the left offset percentage is greater than or equal to the preset threshold, a blockage is determined to have occurred. If the change in the left offset percentage is less than the preset threshold, the entity calculates the right offset percentage of the first material and the right offset percentage of the second material, and then calculates the difference between them to obtain the change value of the right offset percentage. Finally, it compares this change value to a preset threshold for the change in the right offset percentage. If the change in the right offset percentage is greater than or equal to the preset threshold, a blockage is determined to have occurred. In practice, the material area is divided into a left material area and a right material area by a pre-marked center line, and the scribing area is divided into a left scribing area and a right scribing area by a pre-marked center line. The preset threshold for the percentage change of the left material offset and the preset threshold for the percentage change of the right material offset are both 15%.

[0075] The percentage of material offset to the left can be calculated using the following formula:

[0076]

[0077] Among them, Ratio L The left-side offset percentage is represented by Ls, where Ls represents the area of ​​the blank area on the left side of the material, and Gl represents the area of ​​the lined area on the left side. The difference between the area of ​​the lined area and the area of ​​the material's left side is calculated to obtain the area of ​​the blank area. The left-side offset percentage is then obtained by calculating the ratio of the intersection of the blank area and the lined area to the lined area, and then converting this ratio to a percentage.

[0078] The percentage of material offset to the right can be calculated using the following formula:

[0079]

[0080] Among them, Raio R The value represents the percentage offset to the right of the material. Rs represents the area of ​​the blank area to the right of the material, and Gr represents the area of ​​the lined area to the right. The area of ​​the blank area to the right is obtained by subtracting the area of ​​the lined area from the area of ​​the material to the area of ​​the blank area. The percentage offset to the right of the material is obtained by calculating the ratio of the intersection of the blank area and the lined area to the lined area.

[0081] Optionally, before inputting the first group of video frames from the belt conveyor into a pre-deployed material region segmentation model to obtain the first group of segmented material region images, the method further includes:

[0082] Decode the pre-stored historical belt conveyor monitoring video files to obtain historical belt conveyor video decoded files, and extract frames from the historical belt conveyor video decoded files to obtain historical belt conveyor monitoring image frame groups;

[0083] The historical belt conveyor monitoring image frame group is divided into a belt conveyor image frame training set and a belt conveyor image frame verification set according to a preset ratio;

[0084] Data annotation is performed on each historical belt conveyor monitoring image frame in the belt conveyor image frame training set to obtain the belt conveyor image frame annotation training set.

[0085] In some embodiments, before inputting the first group of video frames from the conveyor belt into a pre-deployed material region segmentation model to obtain the first group of segmented material region images, the executing entity first uses an open-source computer vision library to decode pre-stored historical conveyor belt monitoring video files to obtain historical conveyor belt video decoded files, and extracts multiple image frames to obtain a group of historical conveyor belt monitoring image frames. Subsequently, the historical conveyor belt monitoring image frame group is divided into a conveyor belt image frame training set and a conveyor belt image frame validation set according to a preset ratio. Finally, an open-source image annotation tool is used to annotate each historical conveyor belt monitoring image frame in the conveyor belt image frame training set to obtain an annotated conveyor belt image frame training set. In practice, the interval for extracting image frames is set to 1, the ratio for dividing the historical conveyor belt monitoring image frame group is 7:3, and the open-source image annotation tool is labelimg.

[0086] The training set is a set of data used to train a machine learning model. It includes known inputs and expected outputs. The model learns from this known data to adjust its internal parameters in order to make accurate predictions on unknown data. The validation set is a set of data used to evaluate the performance of the machine learning model, used for model selection and hyperparameter tuning during training. Hyperparameter tuning refers to the process of selecting and adjusting the hyperparameters of the model during training. Hyperparameters are parameters set by the developers before training and are not updated during training. Data annotation is the process of labeling or annotating raw data, commonly used for training machine learning and artificial intelligence models. For image data annotation, common tasks include labeling objects in images, drawing bounding boxes or polygonal regions, etc. Annotated data can be used for supervised learning model training, enabling the model to learn how to identify corresponding features or objects from unlabeled data. Labelimg is an open-source image annotation tool primarily used for annotating image datasets.

[0087] Optionally, the automatic identification method for conveyor belt blockage abnormalities of the present invention further includes:

[0088] The image frame annotation training set of the belt conveyor is input into the semantic segmentation model, and the semantic segmentation model is iteratively trained to obtain the first training model;

[0089] The first trained model is optimized to obtain the first optimized model.

[0090] In some embodiments, the execution entity first inputs each belt conveyor image annotation sample from the belt conveyor image frame annotation training set into the semantic segmentation model. On the graphics processing unit (GPU), the semantic segmentation model is trained iteratively multiple times using an open-source deep learning framework. The semantic segmentation model after multiple iterations is determined as the first training model. Subsequently, an optimizer is used to optimize the parameters of the first training model, and the optimized first training model is determined as the first optimized model. In practice, the semantic segmentation model is RTFormer, the GPU is the graphics processing unit, the open-source deep learning framework is PaddlePaddle, and the optimizer is AdamW. When training RTFormer, the number of input samples in each iteration is set to 12, the number of training epochs is set to 200, the initial learning rate is set to 0.0004, and a multinomial decay algorithm is used to adjust the learning rate.

[0091] Semantic segmentation is a computer vision task that aims to assign each pixel in an image to a specific category. Semantic segmentation models achieve fine-grained understanding of images by dividing them into regions and assigning labels to each region. A GPU is hardware specifically designed for processing graphics and image data. It is commonly used to accelerate deep learning training because it can handle a large number of computational tasks in parallel, making it more efficient than a central processing unit, especially when dealing with large-scale data. Open-source deep learning frameworks are free, open-source tool libraries that typically include functions for training, evaluating, and deploying deep learning models. Iterative training in deep learning refers to the process of gradually updating the model's weight parameters through multiple training iterations. Each iteration calculates the error and applies optimization algorithms to adjust the model parameters, thereby progressively improving the model's performance. An optimizer is an algorithm used in deep learning to update model parameters. RTFormer is a specific deep learning model architecture typically used to process sequential or time-series data. PaddlePaddle is an open-source deep learning framework designed to provide developers with efficient and flexible deep learning tools. AdamW is a variant of the Adam optimizer that uses weight decay to control model complexity, thus helping to address overfitting. It's commonly used for deep learning tasks requiring regularization. The number of samples refers to the total number of individual data instances used in a dataset. It's typically used to describe the size of the data in the training, validation, or test sets. A epoch refers to a complete training run of the neural network on the entire dataset. Each epoch contains multiple batches, each processing a fixed number of samples. The initial learning rate is the learning rate set when the model begins training; it determines the pace of parameter updates and usually needs to be adjusted based on the training task and data size. Polynomial decay is a strategy for adjusting the learning rate. During training, the learning rate gradually decreases in a polynomial fashion, allowing for more granular parameter adjustments later in the training process and preventing over-updating near the optimal solution. Weight decay is a regularization technique designed to prevent overfitting in deep learning models. Overfitting occurs when a model performs very well on training data but poorly on new, unseen data (such as validation or test sets). The test set is used to evaluate the performance of the final model. It contains data that has not been seen during training and is used to measure the model's generalization ability and practical application effect. The data distribution of the test set should be similar to the real application scenario to ensure that the evaluation results are representative.

[0092] Optionally, the automatic identification method for conveyor belt blockage abnormalities of the present invention further includes:

[0093] Input the image frame verification set of the belt conveyor into the first optimization model to obtain the first optimization verification result set;

[0094] The cross-union ratio (CUI) of the first optimized verification result set is compared with the preset CUI threshold to obtain the evaluation result set;

[0095] If the proportion of the evaluation results set that pass the evaluation reaches the preset evaluation threshold, then the first optimized model will be determined as the target model.

[0096] In some embodiments, the executing entity first inputs each belt conveyor image frame verification sample from the belt conveyor image frame verification set into the first optimization model to obtain a first optimization verification result set. Next, the intersection-union ratio (IUR) of each first optimization verification result in the first optimization verification result set is calculated and compared with a preset IUR threshold to obtain an evaluation result set. If the proportion of the evaluation result set representing passed evaluation is greater than or equal to the preset evaluation threshold, the first optimization model that passed evaluation is determined as the target model. In practice, the preset IUR threshold is 0.85, and the evaluation threshold is 80%. The IUR of each first optimization verification result can be calculated using the following formula:

[0097]

[0098] Where IoU represents the intersection-union ratio of the first optimization validation result, and Region P Region represents the predicted material region area labeled by the first optimization model in the first optimization verification results, in pixels. G This represents the actual material region area in the verified image frame of the conveyor belt, expressed in pixels. The intersection-union ratio (IUR) of the first optimized verification result is obtained by calculating the ratio of the intersection of the predicted and actual material regions to the actual material region area. The IUR is an indicator used to evaluate the performance of an image segmentation model; it measures the model's accuracy by calculating the degree of overlap between the predicted and ground truth regions.

[0099] Optionally, the automatic identification method for conveyor belt blockage abnormalities of the present invention further includes:

[0100] The target model is converted to a new format to obtain a material region segmentation model, which is then deployed.

[0101] In some embodiments, the execution entity first uses an open-source model conversion tool to convert the target model's format from pdparams to ONNX. The target model converted to ONNX format is then identified as a material region segmentation model. Subsequently, the material region segmentation model is deployed to a local server. In practice, PaddlePaddle outputs models in pdparams format. The open-source model conversion tool is paddle2onnx. pdparams is a model storage format within the PaddlePaddle framework. ONNX is an open-source format used to represent machine learning models, designed to promote model interoperability between different deep learning frameworks. paddle2onnx is an open-source tool used to convert models trained on the PaddlePaddle framework to ONNX format.

[0102] Optionally, both the first material region segmentation image and the second material region segmentation image are single-channel binary images.

[0103] In some embodiments, both the first material region segmentation image and the second material region segmentation image are single-channel binary images. In practice, a single-channel binary image includes white and black regions. The white region of a single-channel binary image represents the material region, and the black region represents the background. A single-channel binary image is a special type of image, commonly used in computer vision and image processing. A single-channel image is an image with only one color channel. Each pixel's value contains only one numerical value, representing grayscale or brightness. In a grayscale image, this value is typically an integer between 0 and 255, used to represent different grayscale levels from black to white. A binary image means that each pixel in the image has only two possible values, typically 0 and 1. A value of 0 represents black, and a value of 1 represents white.

[0104] Further reference Figure 2 This invention provides some embodiments of an automatic identification device for conveyor belt blockage, and these device embodiments are similar to... Figure 1 Corresponding to the methods described, the device can be specifically applied to various electronic devices.

[0105] like Figure 2 As shown, an automatic identification device for belt blockage includes: a video processing unit 201, a material width calculation unit 202, a material width determination unit 203, a material offset percentage calculation unit 204, and a material offset percentage determination unit 205.

[0106] The video processing unit 201 is used to input the first video frame group of the belt conveyor into a pre-deployed material area segmentation model to obtain a first material area segmentation image group; and to input the second video frame group of the belt conveyor into a pre-deployed belt blockage recognition model to obtain a second material area segmentation image group.

[0107] The material width calculation unit 202 is used to obtain the material width percentage of each first material region segmentation image in the first material region segmentation image group to obtain a first material width percentage group; and to obtain the material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain a second material width percentage group.

[0108] The material width determination unit 203 is used to perform a difference calculation between the first material width percentage group and the second material width percentage group to obtain a material width percentage change group; to perform a ratio calculation between the material width percentage change group and the first material width percentage group to obtain a material width percentage change amount group; if any material width percentage change amount in the material width percentage change amount group is greater than or equal to a preset material width percentage change threshold, then it is determined that a blockage has occurred.

[0109] The material offset percentage calculation unit 204 is used to obtain the material offset percentage of each first material region segmentation image in the first material region segmentation image group if the material width percentage change amount of each material width percentage change group is less than the preset material width percentage change threshold, and obtain the material offset percentage of each second material region segmentation image in the second material region segmentation image group to obtain the second material offset percentage group.

[0110] The material offset percentage determination unit 205 is used to perform a difference calculation between the first material offset percentage group and the second material offset percentage group to obtain a material offset percentage change group; to perform a ratio calculation between the material offset percentage change group and the first material offset percentage group to obtain a material offset percentage change amount group; if any material offset percentage change amount in the material offset percentage change group is greater than or equal to a preset material offset percentage change threshold, then it is determined that a blockage has occurred.

[0111] It is understandable that the units described in the device are related to the reference. Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the apparatus and the units contained therein, and will not be repeated here.

[0112] In these embodiments, by automating the monitoring of the material area of ​​the conveyor belt, and utilizing two central cameras and a semantic segmentation model, material blockage can be accurately identified, avoiding the response delays and false alarms of manual monitoring, thus improving identification efficiency and accuracy. Based on real-time video analysis, material blockage can be automatically identified and a shutdown can be triggered within a short time, minimizing material spillage and safety hazards. By utilizing existing camera equipment, system modifications and costs are reduced, and the feasibility of implementation is improved.

[0113] The above description is merely a selection of preferred embodiments of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to specific combinations of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. A method for automatic identification of conveyor belt blockage, characterized in that, include: The first video frame group of the belt conveyor is input into the pre-deployed material region segmentation model to obtain the first material region segmentation image group; The second video frame group of the belt conveyor is input into the pre-deployed belt blockage recognition model to obtain the second material region segmentation image group; Obtain the material width percentage of each first material region segmentation image in the first material region segmentation image group to obtain the first material width percentage group; Obtain the material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain the second material width percentage group; The difference between the first material width percentage group and the second material width percentage group is calculated to obtain the material width percentage change group; the ratio between the material width percentage change group and the first material width percentage group is calculated to obtain the material width percentage change amount group; if any material width percentage change amount in the material width percentage change amount group is greater than or equal to a preset material width percentage change threshold, it is determined that a blockage has occurred. If the percentage change in the material width of each material in the material width percentage change group is less than the preset threshold for the percentage change in the material width, then the percentage of material offset of each first material region segmentation image in the first material region segmentation image group is obtained to obtain the first material offset percentage group. Obtain the material offset percentage of each second material region segmentation image in the second material region segmentation image group to obtain the second material offset percentage group; The difference between the first material offset percentage group and the second material offset percentage group is calculated to obtain the material offset percentage change group; the ratio between the material offset percentage change group and the first material offset percentage group is calculated to obtain the material offset percentage change amount group; if any material offset percentage change amount in the material offset percentage change amount group is greater than or equal to a preset material offset percentage change threshold, then a blockage is determined to have occurred.

2. The automatic identification method for conveyor belt blockage abnormalities according to claim 1, characterized in that, The belt conveyor includes a feeding end area and a discharging end area; the first video file corresponding to the first video frame group is captured by the first camera, and the second video file corresponding to the second video frame group is captured by the second camera. The first camera has corresponding first camera installation environment information, and the second camera has corresponding second camera installation environment information. The first camera installation environment information and the second camera installation environment information are determined through the following steps: The first environmental dust information of the feeding end area is obtained by a preset dust sensor, and the first environmental dust information is determined as the feeding end dust information; the first ambient light information of the feeding end area is obtained by a preset light sensor, and the first ambient light information is determined as the feeding end light information; the first camera installation environment information is determined based on the feeding end dust information and the feeding end light information. The second environmental dust information of the discharge end area is obtained by a preset dust sensor, and the second environmental dust information is determined as the discharge end dust information; the second ambient light information of the discharge end area is obtained by a preset light sensor, and the second ambient light information is determined as the discharge end light information; the installation environment information of the second camera is determined based on the discharge end dust information and the discharge end light information.

3. The automatic identification method for conveyor belt blockage abnormalities according to claim 1, characterized in that, The first material offset percentage includes either the left offset percentage or the right offset percentage of the first material; the second material offset percentage includes either the left offset percentage or the right offset percentage of the second material, and The difference between the left offset percentage of the first material and the left offset percentage of the second material is calculated to obtain the change value of the left offset percentage of the material; the ratio of the change value of the left offset percentage of the material to the left offset percentage of the first material is calculated to obtain the change amount of the left offset percentage of the material; if the change amount of the left offset percentage of the material is greater than or equal to a preset threshold for the change of the left offset percentage of the material, it is determined that a blockage has occurred; if the change amount of the left offset percentage of the material is less than the preset threshold for the change of the left offset percentage of the material, the difference between the right offset percentage of the first material and the right offset percentage of the second material is calculated to obtain the change value of the right offset percentage of the material. The percentage change in the right offset of the material is calculated by comparing it with the percentage change in the right offset of the first material to obtain the percentage change in the right offset of the material. If the percentage change in the right offset of the material is greater than or equal to a preset threshold for the percentage change in the right offset of the material, then a blockage is determined to have occurred.

4. The automatic identification method for conveyor belt blockage abnormalities according to claim 1, characterized in that, Before inputting the first group of video frames from the belt conveyor into a pre-deployed material region segmentation model to obtain the first group of segmented material region images, the method further includes: Decode the pre-stored historical belt conveyor monitoring video file to obtain the historical belt conveyor video decoded file, and extract frames from the historical belt conveyor video decoded file to obtain historical belt conveyor monitoring image frame groups; The historical belt conveyor monitoring image frame group is divided into a belt conveyor image frame training set and a belt conveyor image frame verification set according to a preset ratio; Data annotation is performed on each historical belt conveyor monitoring image frame in the belt conveyor image frame training set to obtain the belt conveyor image frame annotation training set.

5. The automatic identification method for conveyor belt blockage abnormalities according to claim 4, characterized in that, Also includes: The labeled training set of the belt conveyor image frames is input into the semantic segmentation model, and the semantic segmentation model is iteratively trained to obtain the first training model; The first trained model is optimized to obtain the first optimized model.

6. The automatic identification method for conveyor belt blockage abnormalities according to claim 5, characterized in that, Also includes: The image frame verification set of the belt conveyor is input into the first optimization model to obtain the first optimization verification result set; The intersection-union ratio (CUI) of the first optimized verification result set is compared with the preset CUI threshold to obtain the evaluation result set; If the proportion of the evaluation results set that pass the evaluation reaches a preset evaluation threshold, then the first optimized model is determined as the target model.

7. The automatic identification method for conveyor belt blockage abnormalities according to claim 6, characterized in that, Also includes: The target model is converted to a new format to obtain a material region segmentation model, which is then deployed.

8. The automatic identification method for conveyor belt blockage abnormalities according to claim 1, characterized in that, Both the first material region segmentation image and the second material region segmentation image are single-channel binary images.

9. An automatic identification device for conveyor belt blockage, characterized in that, include: The video processing unit is used to input the first video frame group of the belt conveyor into the pre-deployed material area segmentation model to obtain the first material area segmentation image group; The second video frame group of the belt conveyor is input into the pre-deployed belt blockage recognition model to obtain the second material region segmentation image group; The material width calculation unit is used to obtain the material width percentage of each first material region segmentation image in the first material region segmentation image group, and obtain the first material width percentage group; Obtain the material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain the second material width percentage group; The material width determination unit is used to perform a difference calculation between the first material width percentage group and the second material width percentage group to obtain a material width percentage change group; to perform a ratio calculation between the material width percentage change group and the first material width percentage group to obtain a material width percentage change amount group; if any material width percentage change amount in the material width percentage change amount group is greater than or equal to a preset material width percentage change threshold, then it is determined that a blockage has occurred. The material offset percentage calculation unit is used to obtain the material offset percentage of each first material region segmentation image in the first material region segmentation image group if the material width percentage change amount of each material width percentage change group is less than the preset material width percentage change threshold, and thus obtain the first material offset percentage group. Obtain the material offset percentage of each second material region segmentation image in the second material region segmentation image group to obtain the second material offset percentage group; The material offset percentage determination unit is used to perform a difference calculation between the first material offset percentage group and the second material offset percentage group to obtain a material offset percentage change group; to perform a ratio calculation between the material offset percentage change group and the first material offset percentage group to obtain a material offset percentage change amount group; if any material offset percentage change amount in the material offset percentage change amount group is greater than or equal to a preset material offset percentage change threshold, then a blockage is determined to have occurred.

Images