Automatic identification method and device for abnormal material blockage of belt

Abstract

The invention relates to the field of computer vision and image processing, and discloses a belt material blocking abnormity automatic identification method and device, and the method comprises the steps: inputting a first video frame group and a second video frame group of a belt conveyor into a material region segmentation model, and obtaining a first material region segmentation image group and a second material region segmentation image group; calculating material width percentages of the first material area segmentation image group and the second material area segmentation image group, comparing material width percentage variations, and if any material width percentage variation is greater than or equal to a preset threshold value, judging that material blockage occurs; and if the material width percentage variation is smaller than a preset threshold value, calculating the material deviation percentage of the first material area segmentation image group and the material deviation percentage variation of the second material area segmentation image group, comparing the material deviation percentage variation, and if any material deviation percentage variation is larger than or equal to the preset threshold value, judging that material blockage occurs. Therefore, the automatic identification of the abnormal material blockage of the belt is realized.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and image processing, and in particular to a method and device for automatically identifying abnormal belt blockage. Background Art

[0002] Conveyors are a common tool for material transportation. Because they can only move in a straight line, multiple conveyors are typically required. This leads to the situation where two conveyors are interleaving materials. Interleaving refers to the process by which material passes through the upstream conveyor belt, falls into the head hopper, and then falls onto the downstream conveyor belt. In actual production, material that falls into the head hopper during interleaving is sometimes unable to pass smoothly, resulting in blockages. When blockages occur, large amounts of material are scattered on the ground. If the blockage is not discovered in time, at best, a large amount of scattered material must be cleaned up, and at worst, serious safety accidents may occur.

[0003] The existing belt conveyor monitoring process, especially the automated material transportation process, often has the following technical problems:

[0004] First, existing video recognition solutions typically use only one camera to identify the boundaries of the drop zone. During material transportation, dust builds up where the conveyor head drops material, leading to inaccurate boundary determination and impacting recognition.

[0005] Second, relying on manual video review or simple mechanical sensors makes it impossible to respond immediately when blockage occurs.

[0006] Third, monitoring methods based on simple threshold settings are easily affected by environmental changes or equipment jitter, resulting in false alarms and unnecessary downtime and maintenance.

[0007] Fourth, although many conveyor belt sites have existing monitoring cameras, the existing monitoring systems cannot fully utilize these cameras and require additional equipment and modifications. Summary of the Invention

[0008] This summary is intended to briefly introduce concepts that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0009] The present invention proposes a method and device for automatically identifying belt blockage anomalies to solve one or more of the technical problems mentioned in the above background technology section.

[0010] In a first aspect, the present invention provides a method for automatically identifying belt material blockage anomalies, comprising: inputting a first video frame group of a belt conveyor into a pre-deployed material region segmentation model to obtain a first material region segmentation image group; inputting a second video frame group of the belt conveyor into a pre-deployed belt material blockage recognition model to obtain a second material region segmentation image group;

[0011] Obtaining the material width percentage of each first material region segmentation image in the first material region segmentation image group to obtain a first material width percentage group; obtaining the material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain a second material width percentage group;

[0012] Perform a difference operation on the first material width percentage group and the second material width percentage group to obtain a material width percentage change group; perform a ratio operation on the material width percentage change group and the first material width percentage group to obtain a material width percentage change amount group; if the material width percentage change amount of any material in the material width percentage change amount group is greater than or equal to a preset material width percentage change threshold, it is determined that a material blockage has occurred;

[0013] If each material width percentage change in the material width percentage change group is less than a preset material width percentage change threshold, then obtaining the material offset percentage of each first material area segmentation image in the first material area segmentation image group to obtain a first material offset percentage group; obtaining the material offset percentage of each second material area segmentation image in the second material area segmentation image group to obtain a second material offset percentage group;

[0014] Perform a difference operation on the first material offset percentage group and the second material offset percentage group to obtain a material offset percentage change group; perform a ratio operation on the material offset percentage change group and the first material offset percentage group to obtain a material offset percentage change amount group; if any material offset percentage change amount in the material offset percentage change amount group is greater than or equal to a preset material offset percentage change threshold, it is determined that material blockage has occurred.

[0015] Optionally, the belt conveyor includes a feed end area and a discharge end area; the first video file corresponding to the first video frame group is captured by a first camera, the second video file corresponding to the second video frame group is captured by a second camera, the first camera has corresponding first camera installation environment information, and the second camera has corresponding second camera installation environment information, and the first camera installation environment information and the second camera installation environment information are determined by the following steps:

[0016] Acquire first environmental dust information of the feed end area through a preset dust sensor, and determine the first environmental dust information as the feed end dust information; acquire first environmental light information of the feed end area through a preset light sensor, and determine the first environmental light information as the feed end light information; determine the first camera installation environment information based on the feed end dust information and the feed end light information;

[0017] The second environmental dust information of the discharge end area is obtained through a preset dust sensor, and the second environmental dust information is determined as the discharge end dust information; the second environmental light information of the discharge end area is obtained through a preset light sensor, and the second environmental light information is determined as the discharge end light information; based on the discharge end dust information and the discharge end light information, the second camera installation environment information is determined.

[0018] Optionally, the first material offset percentage includes the first material left offset percentage or the first material right offset percentage; the second material offset percentage includes the second material left offset percentage or the second material right offset percentage, and

[0019] Perform a difference operation on the left-side offset percentage of the first material and the left-side offset percentage of the second material to obtain a change value of the left-side offset percentage of the material; perform a ratio operation on the change value of the left-side offset percentage of the material and the left-side offset percentage of the first material to obtain a change amount of the left-side offset percentage of the material; if the change amount of the left-side offset percentage of the material is greater than or equal to a preset change threshold value of the left-side offset percentage of the material, it is determined that a blockage has occurred; if the change amount of the left-side offset percentage of the material is less than the preset change threshold value of the left-side offset percentage of the material, perform a difference operation on the right-side offset percentage of the first material and the right-side offset percentage of the second material to obtain a change value of the right-side offset percentage of the material; perform a ratio operation on the change value of the right-side offset percentage of the material and the right-side offset percentage of the first material to obtain a change amount of the right-side offset percentage of the material; if the change amount of the right-side offset percentage of the material is greater than or equal to the preset change threshold value of the right-side offset percentage of the material, it is determined that a blockage has occurred.

[0020] Optionally, before inputting the first video frame group of the belt conveyor into a pre-deployed material region segmentation model to obtain the first material region segmentation image group, the method further includes:

[0021] Decoding a pre-stored historical belt conveyor monitoring video file to obtain a historical belt conveyor video decoding file, extracting frames from the historical belt conveyor video decoding file to obtain a historical belt conveyor monitoring image frame group;

[0022] The historical belt conveyor monitoring image frame group is divided into a belt conveyor image frame training set and a belt conveyor image frame verification set according to a preset ratio;

[0023] Data annotation is performed on each historical belt conveyor monitoring image frame in the belt conveyor image frame training set to obtain the belt conveyor image frame annotation training set.

[0024] Optionally, the method for automatically identifying belt blockage anomalies of the present invention further includes:

[0025] Inputting the belt conveyor image frame annotated training set into the semantic segmentation model, iteratively training the semantic segmentation model to obtain a first training model;

[0026] The first training model is optimized to obtain a first optimized model.

[0027] Optionally, the method for automatically identifying belt blockage anomalies of the present invention further includes:

[0028] Inputting the belt conveyor image frame verification set into the first optimization model to obtain a first optimization verification result set;

[0029] Comparing the intersection-over-union ratio of the first optimization verification result set with a preset intersection-over-union ratio threshold to obtain an evaluation result set;

[0030] If the evaluation result set indicates that the proportion of evaluations that have passed has reached a preset evaluation threshold, the first optimization model is determined as the target model.

[0031] Optionally, the method for automatically identifying belt blockage anomalies of the present invention further includes:

[0032] The target model is formatted and a material region segmentation model is obtained, which is then deployed.

[0033] Optionally, both the first material region segmentation image and the second material region segmentation image are single-channel binary images.

[0034] In a second aspect, the present invention provides an automatic identification device for belt blockage anomalies, comprising:

[0035] The video processing unit is configured to input a first video frame group of the belt conveyor into a pre-deployed material region segmentation model to obtain a first material region segmentation image group; and input a second video frame group of the belt conveyor into a pre-deployed belt blockage recognition model to obtain a second material region segmentation image group;

[0036] a material width calculation unit, configured to obtain a material width percentage of each first material region segmentation image in the first material region segmentation image group to obtain a first material width percentage group; and obtain a material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain a second material width percentage group;

[0037] a material width determination unit configured to perform a difference operation on the first material width percentage group and the second material width percentage group to obtain a material width percentage change group; perform a ratio operation on the material width percentage change group and the first material width percentage group to obtain a material width percentage change amount group; and determine that a material blockage has occurred if the percentage change amount of any material width in the material width percentage change amount group is greater than or equal to a preset material width percentage change threshold;

[0038] a material offset percentage calculation unit configured to obtain the material offset percentage of each first material region segmentation image in the first material region segmentation image group to obtain a first material offset percentage group if the material width percentage change amount of each material width in the material width percentage change group is less than a preset material width percentage change threshold; and obtain the material offset percentage of each second material region segmentation image in the second material region segmentation image group to obtain a second material offset percentage group;

[0039] The material offset percentage determination unit is used to perform a difference operation on the first material offset percentage group and the second material offset percentage group to obtain a material offset percentage change group; perform a ratio operation on the material offset percentage change group and the first material offset percentage group to obtain a material offset percentage change amount group; if the material offset percentage change amount of any material offset in the material offset percentage change amount group is greater than or equal to a preset material offset percentage change threshold, it is determined that a material blockage has occurred.

[0040] The present invention has the following beneficial effects: by automatically monitoring the material area of the belt conveyor, using two central cameras and a semantic segmentation model, it can accurately identify material blockages, avoid response delays and false alarms in manual monitoring, and improve recognition efficiency and accuracy; based on real-time video analysis, blockages can be automatically identified and triggered to stop within a short period of time, minimizing material scattering and safety hazards; by utilizing existing camera equipment, system changes and costs are reduced, and the feasibility of implementation is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the elements are not necessarily drawn to scale.

[0042] Figure 1 This is a flow chart of a method for automatically identifying abnormal belt blockage according to the present invention;

[0043] Figure 2 It is a structural schematic diagram of a device for automatically identifying abnormal belt blockage according to the present invention. DETAILED DESCRIPTION

[0044] The present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0045] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features of the embodiments of the present invention may be combined with each other.

[0046] It should be noted that the concepts of "first" and "second" mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0047] It should be noted that the modifications of "one" and "multiple" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0048] The names of the messages or information exchanged between multiple devices of the present invention are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0049] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0050] like Figure 1 FIG. 1 is a flow chart of a method and device for automatically identifying belt blockage abnormalities according to the present invention, which specifically includes the following steps:

[0051] Step 101: Input the first video frame group of the belt conveyor into a pre-deployed material area segmentation model to obtain a first material area segmentation image group; input the second video frame group of the belt conveyor into a pre-deployed belt blockage recognition model to obtain a second material area segmentation image group.

[0052] In some embodiments, the execution entity may be a server. The execution entity first establishes a communication connection with two pre-deployed cameras and obtains the real-time video files corresponding to the two cameras. Subsequently, the two video files are decoded using an open source computer vision library and multiple image frames are extracted to obtain a first video frame group and a second video frame group. Finally, the first video frame group is input into a pre-deployed belt blockage recognition model to obtain a first material region segmentation image group, and the second video frame group is input into a pre-deployed belt blockage recognition model to obtain a second material region segmentation image group. In practice, the open source computer vision library may be OpenCV (Open Source Computer Vision Library). The open source computer vision library refers to an open source software library specifically for computer vision and image processing tasks. A conveyor belt is a mechanical device used for continuous material transportation and is widely used in industries such as mining, metallurgy, and coal for conveying bulk materials or piece items. A video frame is a single image in a video file. A video file consists of a series of continuously played images, and the number of frames played per second is called the frame rate. Video frames capture and record visual information at a specific point in time. By playing these frames in rapid succession, a video can create a smooth motion effect. In the fields of computer vision and image processing, video frames are often used to analyze and process video content. Material blockage occurs when material, due to various reasons (such as accumulation, jamming, or excessive material), becomes blocked on conveying equipment (such as a conveyor belt), preventing normal flow and transportation.

[0053] Step 102: Obtain the material width percentage of each first material region segmentation image in the first material region segmentation image group to obtain a first material width percentage group; obtain the material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain a second material width percentage group.

[0054] In some embodiments, the execution entity first calculates the material width percentage of each first material region segmentation image in the first material region segmentation image group to obtain a first material width percentage group. Then, the execution entity calculates the material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain a second material width percentage group. In practice, the material width percentage can be calculated using the following formula:

[0055]

[0056] Among them, Ratio mrepresents the material width percentage, Pr represents the material recognition area in pixels, and Gt represents the ruled area in pixels. The material width percentage is calculated by calculating the ratio of the intersection of the material recognition area and the ruled area to the total area of the ruled area, and then converting the result into a percentage. The ruled area is the preset area in the material segmentation image. The material recognition area is the actual area of the material in the material segmentation image. The intersection refers to the elements or set of elements shared by two or more sets and is often used to describe the overlap between two areas.

[0057] Step 103: Perform a difference operation on the first material width percentage group and the second material width percentage group to obtain a material width percentage change group; perform a ratio operation on the material width percentage change group and the first material width percentage group to obtain a material width percentage change amount group; if any material width percentage change amount in the material width percentage change amount group is greater than or equal to a preset material width percentage change threshold, it is determined that a material blockage has occurred.

[0058] In some embodiments, the execution entity sequentially performs a difference operation on each first material width percentage in the first material width percentage group and each second material width percentage in the second material width percentage group to obtain a material width percentage change group. Next, a ratio operation is performed on each material width percentage change value in the material width percentage change group and the corresponding first material width percentage in the first material width percentage group to obtain a material width percentage change amount group. Finally, each material width percentage change amount in the material width percentage change amount group is compared with a preset material width percentage change threshold. If any material width percentage change amount is greater than or equal to the preset material width percentage change threshold, it is determined that a material blockage has occurred. In practice, the preset material width percentage change threshold is 20%.

[0059] In step 104, if each material width percentage change in the material width percentage change group is less than the preset material width percentage change threshold, the material offset percentage of each first material area segmentation image in the first material area segmentation image group is obtained to obtain a first material offset percentage group; and the material offset percentage of each second material area segmentation image in the second material area segmentation image group is obtained to obtain a second material offset percentage group.

[0060] In some embodiments, the execution entity first compares each material width percentage change in the material width percentage change group with a preset material width percentage change threshold. If all material width percentage changes are less than the preset material width percentage change threshold, the material offset percentage is calculated for each first material region segmentation image in the first material region segmentation image group to obtain a first material offset percentage group. Next, the material offset percentage is calculated for each second material region segmentation image in the second material region segmentation image group to obtain a second material offset percentage group. In practice, the area of the marked area and the material identification area is subtracted to obtain the blank area area. The material offset percentage is then obtained by calculating the ratio of the intersection of the blank area area and the marked area area to the total area of the marked area, and then performing a percentage conversion.

[0061] Step 105: Perform a difference operation on the first material offset percentage group and the second material offset percentage group to obtain a material offset percentage change group; perform a ratio operation on the material offset percentage change group and the first material offset percentage group to obtain a material offset percentage change amount group; if any material offset percentage change amount in the material offset percentage change amount group is greater than or equal to a preset material offset percentage change threshold, it is determined that material blockage has occurred.

[0062] In some embodiments, the execution entity performs a difference operation on each first material offset percentage in the first material offset percentage group and each second material offset percentage in the second material offset percentage group in sequence to obtain a material offset percentage change group. Next, a ratio operation is performed on each material offset percentage change value in the material offset percentage change group and the corresponding first material offset percentage in the first material offset percentage group to obtain a material offset percentage change amount group. Finally, each material offset percentage change amount in the material offset percentage change amount group is compared with a preset material offset percentage change threshold. If any material offset percentage change amount is greater than or equal to the preset material offset percentage change threshold, it is determined that a material blockage has occurred. In practice, the preset material offset percentage change threshold is 15%.

[0063] Optionally, the belt conveyor includes a feed end area and a discharge end area; the first video file corresponding to the first video frame group is captured by a first camera, the second video file corresponding to the second video frame group is captured by a second camera, the first camera has corresponding first camera installation environment information, and the second camera has corresponding second camera installation environment information, and the first camera installation environment information and the second camera installation environment information are determined by the following steps:

[0064] Acquire first environmental dust information of the feed end area through a preset dust sensor, and determine the first environmental dust information as the feed end dust information; acquire first environmental light information of the feed end area through a preset light sensor, and determine the first environmental light information as the feed end light information; determine the first camera installation environment information based on the feed end dust information and the feed end light information;

[0065] The second environmental dust information of the discharge end area is obtained through a preset dust sensor, and the second environmental dust information is determined as the discharge end dust information; the second environmental light information of the discharge end area is obtained through a preset light sensor, and the second environmental light information is determined as the discharge end light information; based on the discharge end dust information and the discharge end light information, the second camera installation environment information is determined.

[0066] In some embodiments, the conveyor belt includes a feed end area and a discharge end area. The camera located in the feed end area is a first camera, and the camera located in the discharge end area is a second camera. The video file captured by the first camera is a first video file, and the video file captured by the second camera is a second video file. The first camera has corresponding first camera installation environment information, and the second camera has corresponding second camera installation environment information. The first camera installation environment information and the second camera installation environment information are determined by the following steps:

[0067] The execution entity first establishes a communication connection with a pre-set dust sensor in the feed end area to obtain first environmental dust information, which is then determined as the feed end dust information. Subsequently, a communication connection is established with a pre-set light sensor in the feed end area to obtain first environmental light information, which is then determined as the feed end light information. The feed end dust information and the feed end light information are used to determine the environment information in which the first camera is installed.

[0068] Next, a communication connection is established with a pre-set dust sensor in the discharge end area to obtain second environmental dust information, which is then determined as the discharge end dust information. Subsequently, a communication connection is established with a pre-set light sensor in the discharge end area to obtain second environmental light information, which is then determined as the discharge end light information. The second camera installation environment information is determined based on the discharge end dust information and the discharge end light information.

[0069] In practice, the dust sensor model can be PMS5003, and the light sensor model can be TSL2561. The environmental dust information monitored by the PMS5003 includes PM2.5 concentration, PM10 concentration, and dust concentration index. The ambient light information monitored by the TSL2561 includes ambient light intensity, light intensity changes, and spectral distribution. High air pollution may affect the camera's field of view and performance. For example, a PM2.5 concentration of 76–150 micrograms per cubic meter is considered moderate pollution. When the ambient light intensity is low, the camera may require a longer exposure time or additional light source. In an environment with excessive ambient light, the camera may overexpose, affecting image quality. For example, an ambient light intensity of 100–300 lux is normal indoor light intensity.

[0070] The feed end refers to the inlet where materials enter the conveying equipment. The discharge end refers to the outlet where materials exit the conveying equipment and enter the next processing or storage area. The PMS5003 is a sensor for detecting particulate matter in the air. It measures particulate matter concentration and provides relevant air quality information. It is suitable for monitoring PM2.5 and PM10 concentrations in the environment. A dust sensor is a device used to detect the concentration of particulate matter (such as dust) in the air. It typically uses laser or light scattering technology to measure the degree to which particles block or scatter light to determine the concentration of particulate matter in the air. The TSL2561 is a sensor for measuring light intensity, providing ambient light data. A light sensor is a device that detects light intensity or radiation, measuring ambient light intensity by converting optical signals into electrical signals. PM2.5 refers to airborne particles with a diameter of 2.5 microns or less. Due to their extremely small size, these particles can enter the human respiratory tract, penetrate deep into the lungs, and even enter the blood circulation. PM10 refers to airborne particles with a diameter of 10 microns or less. Although larger than PM2.5, they can still be inhaled into the respiratory tract and enter the lungs. Lux is the international unit of light intensity, which represents the luminous flux per unit area. It is used to measure the brightness of a specific surface.

[0071] Optionally, the first material offset percentage includes the first material left offset percentage or the first material right offset percentage; the second material offset percentage includes the second material left offset percentage or the second material right offset percentage, and

[0072] Perform a difference operation on the left-side offset percentage of the first material and the left-side offset percentage of the second material to obtain a change value of the left-side offset percentage of the material; perform a ratio operation on the change value of the left-side offset percentage of the material and the left-side offset percentage of the first material to obtain a change amount of the left-side offset percentage of the material; if the change amount of the left-side offset percentage of the material is greater than or equal to a preset change threshold value of the left-side offset percentage of the material, it is determined that a blockage has occurred; if the change amount of the left-side offset percentage of the material is less than the preset change threshold value of the left-side offset percentage of the material, perform a difference operation on the right-side offset percentage of the first material and the right-side offset percentage of the second material to obtain a change value of the right-side offset percentage of the material; perform a ratio operation on the change value of the right-side offset percentage of the material and the right-side offset percentage of the first material to obtain a change amount of the right-side offset percentage of the material; if the change amount of the right-side offset percentage of the material is greater than or equal to the preset change threshold value of the right-side offset percentage of the material, it is determined that a blockage has occurred.

[0073] In some embodiments, the first material offset percentage includes a first material left offset percentage or a first material right offset percentage; the second material offset percentage includes a second material left offset percentage or a second material right offset percentage, and

[0074] The execution entity first calculates the left-side offset percentage of the first material and the left-side offset percentage of the second material, and performs a subtraction operation on the left-side offset percentage of the first material and the left-side offset percentage of the second material to obtain a change value of the material left-side offset percentage. Subsequently, a ratio operation is performed on the change value of the material left-side offset percentage and the left-side offset percentage of the first material to obtain a change amount of the material left-side offset percentage. Next, the change value of the material left-side offset percentage is compared with a preset material left-side offset percentage change threshold. If the change amount of the material left-side offset percentage is greater than or equal to the preset material left-side offset percentage change threshold, a material blockage is determined to have occurred. If the change amount of the material left-side offset percentage is less than the preset material left-side offset percentage change threshold, the right-side offset percentage of the first material and the right-side offset percentage of the second material are calculated, and a subtraction operation is performed on the right-side offset percentage of the first material and the right-side offset percentage of the second material to obtain a change value of the material right-side offset percentage. Finally, the change value of the material right-side offset percentage is compared with a preset material right-side offset percentage change threshold. If the change amount of the material right-side offset percentage is greater than or equal to the preset material right-side offset percentage change threshold, a material blockage is determined to have occurred. In practice, the material area is divided into the left side area and the right side area by the pre-marked center line, and the crossed-out area is divided into the left side crossed-out area and the right side crossed-out area by the pre-marked center line. The preset material left side offset percentage change threshold and the preset material right side offset percentage change threshold are both 15%.

[0075] The material left offset percentage can be calculated using the following formula:

[0076]

[0077] Among them, Ratio L Indicates the percentage of material left offset. Ls represents the area of the blank area on the left side of the material, and Gl represents the area of the ruled area on the left side. The area of the blank area on the left side is calculated by subtracting the area of the ruled area from the area of the left side of the material. The percentage of material left offset is calculated by calculating the ratio of the intersection of the blank area and the ruled area to the area of the ruled area, and then converting the result into a percentage.

[0078] The right offset percentage of the material can be calculated using the following formula:

[0079]

[0080] Among them, Raio R Indicates the right offset percentage of the material. Rs represents the area of the blank area on the right side of the material, and Gr represents the area of the right lined area. The area of the blank area on the right side is calculated by subtracting the area of the right lined area from the area of the material on the right side. The right offset percentage of the material is obtained by calculating the ratio of the intersection of the right blank area and the right lined area to the area of the right lined area, and then converting the result into a percentage.

[0081] Optionally, before inputting the first video frame group of the belt conveyor into a pre-deployed material region segmentation model to obtain the first material region segmentation image group, the method further includes:

[0082] Decoding a pre-stored historical belt conveyor monitoring video file to obtain a historical belt conveyor video decoding file, extracting frames from the historical belt conveyor video decoding file to obtain a historical belt conveyor monitoring image frame group;

[0083] The historical belt conveyor monitoring image frame group is divided into a belt conveyor image frame training set and a belt conveyor image frame verification set according to a preset ratio;

[0084] Data annotation is performed on each historical belt conveyor monitoring image frame in the belt conveyor image frame training set to obtain the belt conveyor image frame annotation training set.

[0085] In some embodiments, before inputting the first video frame group of the belt conveyor into a pre-deployed material area segmentation model to obtain the first material area segmentation image group, the execution entity first uses an open source computer vision library to decode the pre-stored historical belt conveyor monitoring video file to obtain a historical belt conveyor video decoding file, and extracts multiple image frames to obtain a historical belt conveyor monitoring image frame group. Subsequently, the historical belt conveyor monitoring image frame group is divided into a belt conveyor image frame training set and a belt conveyor image frame verification set according to a preset ratio. Finally, an open source image annotation tool is used to perform data annotation on each historical belt conveyor monitoring image frame in the belt conveyor image frame training set to obtain a belt conveyor image frame annotation training set. In practice, the interval for extracting image frames is set to 1, the ratio of dividing the historical belt conveyor monitoring image frame group is 7:3, and the open source image annotation tool is labelimg.

[0086] The training set is a set of data used to train a machine learning model. It includes known inputs and expected outputs. The model adjusts its internal parameters by learning from this known data in order to make accurate predictions on unknown data. The validation set is a set of data used to evaluate the performance of a machine learning model and is used for model selection and hyperparameter tuning during the training process. Hyperparameter tuning refers to the process of selecting and adjusting the model's hyperparameters when training a machine learning model. Hyperparameters are parameters set by the developer before training and are not updated during the training process. Data labeling is the process of labeling or annotating raw data and is commonly used for training machine learning and artificial intelligence models. For image data labeling, common tasks include labeling objects in the image, outlining bounding boxes or polygonal areas, etc. The labeled data can be used for model training in supervised learning, enabling the model to learn how to identify corresponding features or objects from unlabeled data. Labelimg is an open source image labeling tool mainly used for labeling image datasets.

[0087] Optionally, the method for automatically identifying belt blockage anomalies of the present invention further includes:

[0088] Inputting the belt conveyor image frame annotated training set into the semantic segmentation model, iteratively training the semantic segmentation model to obtain a first training model;

[0089] The first training model is optimized to obtain a first optimized model.

[0090] In some embodiments, the execution entity first inputs each belt conveyor image annotation sample in the belt conveyor image frame annotation training set into the semantic segmentation model, and on the graphics processing unit, the semantic segmentation model is iteratively trained multiple times through the open source deep learning framework, and the semantic segmentation model after the multiple iterative training is determined as the first training model. Subsequently, the optimizer is used to optimize the parameters of the first training model, and the optimized first training model is determined as the first optimized model. In practice, the semantic segmentation model is RTFormer, the graphics processing unit is GPU (Graphics Processing Unit), the open source deep learning framework is PaddlePaddle, and the optimizer is AdamW. When training RTFormer, the number of samples input in each iteration is set to 12, the number of training rounds is set to 200, the initial learning rate is set to 0.0004, and the learning rate is adjusted using a polynomial decay algorithm.

[0091] Semantic segmentation is a computer vision task whose goal is to assign each pixel in an image to a specific category. Semantic segmentation models achieve a fine-grained understanding of images by dividing them into regions and assigning labels to each region. GPUs are specialized hardware designed to process graphics and image data. They are often used to accelerate deep learning training because they can handle numerous computational tasks in parallel, making them more efficient than central processing units, especially when processing large amounts of data. Open source deep learning frameworks are free, open source tool libraries that typically include functionality for training, evaluating, and deploying deep learning models. Iterative training, in deep learning, involves gradually updating a model's weight parameters through multiple training cycles. Each iteration calculates the error and applies an optimization algorithm to adjust the model parameters, gradually improving the model's performance. An optimizer is an algorithm used in deep learning to update model parameters. RTFormer is a specific deep learning model architecture typically used to process sequence or time series data. PaddlePaddle is an open source deep learning framework designed to provide developers with efficient and flexible deep learning tools. AdamW is a variant of the Adam optimizer that controls model complexity by introducing weight decay, thereby helping to address overfitting. It is commonly used for deep learning tasks that require regularization. The number of samples refers to the total number of individual data instances used in a dataset. It is often used to describe the size of the data in the training, validation, or test sets. An epoch refers to a complete training of the neural network on the entire dataset. Each epoch consists of multiple batches, each processing a fixed number of samples. The initial learning rate is the value set at the beginning of model training. It determines the pace of parameter updates and typically needs to be adjusted based on the training task and data size. The polynomial decay algorithm is a learning rate adjustment strategy that gradually decreases the learning rate in a polynomial manner as training progresses. This facilitates more detailed parameter adjustments in the later stages of training and avoids excessive updates when approaching the optimal solution. Weight decay is a regularization technique designed to prevent overfitting in deep learning models. Overfitting occurs when a model performs very well on the training data but performs poorly on new, unseen data (such as the validation or test set). The test set is used to evaluate the performance of the final model. It contains data that has not been seen during the training process and is used to measure the model's generalization ability and actual application effect. The data distribution of the test set should be similar to the actual application scenario to ensure that the evaluation results are representative.

[0092] Optionally, the method for automatically identifying belt blockage anomalies of the present invention further includes:

[0093] Inputting the belt conveyor image frame verification set into the first optimization model to obtain a first optimization verification result set;

[0094] Comparing the intersection-over-union ratio of the first optimization verification result set with a preset intersection-over-union ratio threshold to obtain an evaluation result set;

[0095] If the evaluation result set indicates that the proportion of evaluations that have passed has reached a preset evaluation threshold, the first optimization model is determined as the target model.

[0096] In some embodiments, the execution entity first inputs each belt conveyor image frame verification sample in the belt conveyor image frame verification set into the first optimization model to obtain a first optimization verification result set. Then, the intersection-and-union ratio of each first optimization verification result in the first optimization verification result set is calculated, and compared with the preset intersection-and-union ratio threshold to obtain an evaluation result set. If the proportion of evaluation results in the evaluation result set that pass the evaluation is greater than or equal to the preset evaluation threshold, the first optimization model that passes the evaluation is determined as the target model. In practice, the preset intersection-and-union ratio threshold is 0.85, and the evaluation threshold is 80%. The intersection-and-union ratio of each first optimization verification result can be calculated by the following formula:

[0097]

[0098] Among them, IoU represents the intersection-over-union ratio of the first optimization verification result, Region P Indicates the predicted material area marked by the first optimization model in the first optimization verification result, in pixels. G Represents the area of the actual material area annotated in the belt conveyor image frame verification sample, in pixels. The intersection-over-union (IoU) ratio of the first optimization verification result is calculated by calculating the ratio of the intersection of the predicted and actual material areas to the actual area. The IoU ratio is a performance metric used to evaluate image segmentation models. It measures model accuracy by calculating the degree of overlap between the predicted and actual areas.

[0099] Optionally, the method for automatically identifying belt blockage anomalies of the present invention further includes:

[0100] The target model is formatted and a material region segmentation model is obtained, which is then deployed.

[0101] In some embodiments, the executing entity first uses an open source model conversion tool to convert the format of the target model from pdparams to ONNX, determines the target model converted to ONNX format as a material area segmentation model, and then deploys the material area segmentation model to a local server. In practice, the model format output by PaddlePaddle is pdparams. The open source model conversion tool is paddle2onnx. Among them, pdparams is a model saving format in the PaddlePaddle framework. ONNX is an open source format for representing machine learning models, which aims to promote model interoperability between different deep learning frameworks. paddle2onnx is an open source tool for converting models trained by the PaddlePaddle framework into ONNX format.

[0102] Optionally, both the first material region segmentation image and the second material region segmentation image are single-channel binary images.

[0103] In some embodiments, the first material area segmentation image and the second material area segmentation image are both single-channel binary images. In practice, the single-channel binary image includes a white area and a black area. The white area of the single-channel binary image represents the material area, and the black area of the single-channel binary image represents the background. Among them, the single-channel binary image is a special type of image, which is commonly used in the fields of computer vision and image processing. A single-channel image refers to an image with only one color channel. The value of each pixel contains only one numerical value, which represents the grayscale or brightness. In a grayscale image, this numerical value is usually an integer between 0 and 255, which is used to represent different grayscale levels from black to white. A binary image means that each pixel in the image has only two possible values, usually 0 and 1. A value of 0 represents black, and a value of 1 represents white.

[0104] Further references Figure 2 The present invention provides some embodiments of a device for automatically identifying abnormal belt blockage. Figure 1 Corresponding to the aforementioned method embodiments, the device can be specifically applied to various electronic devices.

[0105] like Figure 2 As shown, a device for automatically identifying belt material blockage anomalies includes: a video processing unit 201, a material width calculation unit 202, a material width determination unit 203, a material offset percentage calculation unit 204 and a material offset percentage determination unit 205.

[0106] The video processing unit 201 is configured to input a first video frame group of the belt conveyor into a pre-deployed material region segmentation model to obtain a first material region segmentation image group; and input a second video frame group of the belt conveyor into a pre-deployed belt blockage recognition model to obtain a second material region segmentation image group.

[0107] The material width calculation unit 202 is configured to obtain the material width percentage of each first material region segmentation image in the first material region segmentation image group to obtain a first material width percentage group; and obtain the material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain a second material width percentage group;

[0108] The material width determination unit 203 is configured to perform a difference operation on the first material width percentage group and the second material width percentage group to obtain a material width percentage change group; perform a ratio operation on the material width percentage change group and the first material width percentage group to obtain a material width percentage change amount group; and determine that a material blockage has occurred if any material width percentage change amount in the material width percentage change amount group is greater than or equal to a preset material width percentage change threshold;

[0109] The material offset percentage calculation unit 204 is configured to obtain the material offset percentage of each first material region segmentation image in the first material region segmentation image group to obtain a first material offset percentage group; and obtain the material offset percentage of each second material region segmentation image in the second material region segmentation image group to obtain a second material offset percentage group if the material width percentage change amount of each material width in the material width percentage change group is less than a preset material width percentage change threshold.

[0110] The material offset percentage determination unit 205 is used to perform a difference operation on the first material offset percentage group and the second material offset percentage group to obtain a material offset percentage change group; perform a ratio operation on the material offset percentage change group and the first material offset percentage group to obtain a material offset percentage change amount group; if any material offset percentage change amount in the material offset percentage change group is greater than or equal to a preset material offset percentage change threshold, it is determined that material blockage has occurred.

[0111] It is understood that the units described in the device are similar to those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device and the units included therein, and will not be repeated here.

[0112] In these embodiments, by automatically monitoring the material area of the belt conveyor and utilizing two central cameras and a semantic segmentation model, material blockages can be accurately identified, thus avoiding response delays and false alarms in manual monitoring and improving recognition efficiency and accuracy. Based on real-time video analysis, material blockages can be automatically identified and triggered to stop within a short period of time, minimizing material scattering and safety hazards. By utilizing existing camera equipment, system modifications and costs are reduced, thereby improving the feasibility of implementation.

[0113] The above descriptions are merely some preferred embodiments of the present invention and illustrate the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the present invention.

Claims

1. A method for automatically identifying belt blockage anomalies, characterized in that: include: Inputting a first video frame group of the belt conveyor into a pre-deployed material region segmentation model to obtain a first material region segmentation image group; Inputting the second video frame group of the belt conveyor into a pre-deployed belt jam recognition model to obtain a second material region segmentation image group; Obtaining a material width percentage of each first material region segmentation image in the first material region segmentation image group to obtain a first material width percentage group; Obtaining a material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain a second material width percentage group; Performing a difference operation on the first material width percentage group and the second material width percentage group to obtain a material width percentage change group; performing a ratio operation on the material width percentage change group and the first material width percentage group to obtain a material width percentage change amount group; if any material width percentage change amount in the material width percentage change amount group is greater than or equal to a preset material width percentage change threshold, it is determined that a material blockage has occurred; If the percentage change in the material width of each material in the material width percentage change group is less than a preset material width percentage change threshold, obtaining the material offset percentage of each first material area segmentation image in the first material area segmentation image group to obtain a first material offset percentage group; acquiring a material offset percentage of each second material region segmentation image in the second material region segmentation image group to obtain a second material offset percentage group; Perform a difference operation on the first material offset percentage group and the second material offset percentage group to obtain a material offset percentage change group; perform a ratio operation on the material offset percentage change group and the first material offset percentage group to obtain a material offset percentage change amount group; if any material offset percentage change amount in the material offset percentage change amount group is greater than or equal to a preset material offset percentage change threshold, it is determined that material blockage has occurred.

2. The method for automatically identifying belt blockage abnormality according to claim 1, characterized in that: The belt conveyor includes a feed end area and a discharge end area; the first video file corresponding to the first video frame group is captured by a first camera, and the second video file corresponding to the second video frame group is captured by a second camera, the first camera has corresponding first camera installation environment information, and the second camera has corresponding second camera installation environment information, and the first camera installation environment information and the second camera installation environment information are determined by the following steps: obtaining first environmental dust information of the feed end area through a preset dust sensor, and determining the first environmental dust information as the feed end dust information; obtaining first environmental light information of the feed end area through a preset light sensor, and determining the first environmental light information as the feed end light information; and determining first camera installation environment information based on the feed end dust information and the feed end light information; The second environmental dust information of the discharge end area is obtained through a preset dust sensor, and the second environmental dust information is determined as the discharge end dust information; the second environmental light information of the discharge end area is obtained through a preset light sensor, and the second environmental light information is determined as the discharge end light information; according to the discharge end dust information and the discharge end light information, the second camera installation environment information is determined.

3. The method for automatically identifying belt blockage abnormality according to claim 1, characterized in that: The first material offset percentage includes the first material left offset percentage or the first material right offset percentage; the second material offset percentage includes the second material left offset percentage or the second material right offset percentage, and Perform a difference operation on the left-side offset percentage of the first material and the left-side offset percentage of the second material to obtain a material left-side offset percentage change value; perform a ratio operation on the material left-side offset percentage change value and the first material left-side offset percentage to obtain a material left-side offset percentage change amount; if the material left-side offset percentage change amount is greater than or equal to a preset material left-side offset percentage change threshold, it is determined that material blockage has occurred; if the material left-side offset percentage change amount is less than the preset material left-side offset percentage change threshold, perform a difference operation on the first material right-side offset percentage and the second material right-side offset percentage to obtain a material right-side offset percentage change value; The ratio operation is performed on the percentage change value of the material right side offset and the first material right side offset percentage to obtain the percentage change amount of the material right side offset; if the percentage change amount of the material right side offset is greater than or equal to the preset material right side offset percentage change threshold, it is determined that material blockage has occurred.

4. The method for automatically identifying belt blockage abnormality according to claim 1, characterized in that: Before inputting the first video frame group of the belt conveyor into a pre-deployed material region segmentation model to obtain the first material region segmentation image group, the method further includes: Decoding a pre-stored historical belt conveyor monitoring video file to obtain a historical belt conveyor video decoding file, extracting frames from the historical belt conveyor video decoding file to obtain a historical belt conveyor monitoring image frame group; Dividing the historical belt conveyor monitoring image frame group into a belt conveyor image frame training set and a belt conveyor image frame verification set according to a preset ratio; Data annotation is performed on each historical belt conveyor monitoring image frame in the belt conveyor image frame training set to obtain a belt conveyor image frame annotation training set.

5. The method for automatically identifying belt blockage abnormality according to claim 4, characterized in that: Also includes: Inputting the belt conveyor image frame annotated training set into a semantic segmentation model, and iteratively training the semantic segmentation model to obtain a first training model; The first training model is optimized to obtain a first optimized model.

6. The method for automatically identifying belt blockage abnormality according to claim 5, characterized in that: Also includes: Inputting the belt conveyor image frame verification set into a first optimization model to obtain a first optimization verification result set; Comparing the intersection-over-union ratio of the first optimization verification result set with a preset intersection-over-union ratio threshold to obtain an evaluation result set; If the evaluation result set indicates that the proportion of evaluations that have passed has reached a preset evaluation threshold, the first optimization model is determined as the target model.

7. The method for automatically identifying belt blockage abnormality according to claim 6, characterized in that: Also includes: The target model is format-converted to obtain a material region segmentation model, and the material region segmentation model is deployed.

8. The method for automatically identifying belt blockage abnormality according to claim 1, characterized in that: The first material region segmentation image and the second material region segmentation image are both single-channel binary images.

9. An automatic identification device for belt blockage abnormality, characterized in that: include: a video processing unit, configured to input a first video frame group of the belt conveyor into a pre-deployed material region segmentation model to obtain a first material region segmentation image group; Inputting the second video frame group of the belt conveyor into a pre-deployed belt jam recognition model to obtain a second material region segmentation image group; a material width calculation unit, configured to obtain a material width percentage of each first material region segmentation image in the first material region segmentation image group to obtain a first material width percentage group; Obtaining a material width percentage of each second material region segmentation image in the second material region segmentation image group to obtain a second material width percentage group; a material width determination unit, configured to perform a difference operation on the first material width percentage group and the second material width percentage group to obtain a material width percentage change group; perform a ratio operation on the material width percentage change group and the first material width percentage group to obtain a material width percentage change amount group; and determine that a material blockage has occurred if any material width percentage change amount in the material width percentage change amount group is greater than or equal to a preset material width percentage change threshold; a material offset percentage calculation unit, configured to obtain the material offset percentage of each first material region segmentation image in the first material region segmentation image group if the material width percentage change amount of each material width in the material width percentage change group is less than a preset material width percentage change threshold, to obtain a first material offset percentage group; acquiring a material offset percentage of each second material region segmentation image in the second material region segmentation image group to obtain a second material offset percentage group; The material offset percentage determination unit is used to perform a difference operation on the first material offset percentage group and the second material offset percentage group to obtain a material offset percentage change group; perform a ratio operation on the material offset percentage change group and the first material offset percentage group to obtain a material offset percentage change amount group; if any material offset percentage change amount in the material offset percentage change amount group is greater than or equal to a preset material offset percentage change threshold, it is determined that material blockage has occurred.

Images