Belt load analysis method and device based on image recognition

Abstract

The application discloses a belt transport capacity analysis method and device based on image recognition, which utilizes a marking tool to mark two labels of "coal" and "belt" in images collected by a coal flow belt video; an instance segmentation algorithm is adopted to train the marked images, so that an instance segmentation model for identifying coal flow is obtained; a line segment is marked on the images collected by the coal flow belt video, the pixel length and the real physical length of the line segment are calculated, the unit pixel length is determined, the belt is perspective-recovered, the scaling ratio at the longitudinal pixel is calculated, and the coal flow width is obtained; the video frame is subjected to instance segmentation through the instance segmentation model, the mass of coal per unit time is calculated according to the measured average density of coal, the estimated speed of coal flow and the cross-sectional area of coal flow, and the preset granularity coal quantity is accumulated. The application realizes non-contact transport capacity estimation through image recognition, avoids artificial fraud, and improves data accuracy.

Description

Technical Field

[0001] This invention belongs to the field of coal quantity analysis technology, specifically relating to a belt conveyor capacity analysis method and device based on image recognition. Background Technology

[0002] In the course of coal mine production and operation, coal mining enterprises commonly conceal or omit the production of raw coal and crude coal in order to increase profits. The actual production data of some coal mines is derived from the advance footage reported by each shift, and the production figures are then calculated. This reliance on manual reporting and calculation makes it impossible for managers at all levels and regulatory units to obtain the true production data of coal mining enterprises, making it difficult to accurately control the production situation.

[0003] Currently, the industry primarily obtains coal mine output data through two methods: self-reporting by enterprises and measurement using belt scales. However, both methods have significant drawbacks: the data reported by enterprises may be subject to subjective motives for falsification and lacks objective and reliable verification methods; belt scale measurement is a contact-based measurement, making the equipment easily modified or interfered with, resulting in extremely low costs for falsification, and measurement errors accumulate over time, rendering it unreliable as a true reflection of coal mine output. These two methods together lead to significant blind spots in the supervision of coal mine output. Summary of the Invention

[0004] To address this issue, the present invention provides a method and apparatus for analyzing conveyor belt transport volume based on image recognition, which solves the problems of low cost of falsifying coal mine output data, data distortion, and blind spots in supervision in existing methods of obtaining coal mine output data.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a belt conveyor capacity analysis method based on image recognition, comprising the following steps:

[0006] Using annotation tools, label the images captured by the coal conveyor belt video with two types of tags: "coal" and "conveyor belt";

[0007] An instance segmentation algorithm was used to train the labeled images to obtain an instance segmentation model for identifying coal flow.

[0008] Mark line segments on the images captured by the coal conveyor belt video, calculate the pixel length and actual physical length of the line segments, determine the unit pixel length, perform perspective restoration on the belt, calculate the scaling ratio at the longitudinal pixels, and obtain the coal flow width.

[0009] Coal is classified using a classification model, and the angle of repose is determined based on the coal flow width and the classification results.

[0010] The cross-sectional area of ​​the coal flow is approximated as a trapezoid and an arc shape, and the cross-sectional area of ​​the coal flow is calculated by combining the natural angle of repose.

[0011] The video frames are segmented using the instance segmentation model. Based on the measured average coal density, the estimated coal flow velocity, and the coal flow cross-sectional area, the mass of coal per unit time is calculated using the frame time difference, and the preset particle size coal quantity is obtained by summing these values.

[0012] As a preferred method for belt conveyor capacity analysis based on image recognition, when labeling the images with "coal" and "belt", the following operations are performed on the coal conveyor belt video frame images using the labeling tool:

[0013] Pixel-level contour annotation is performed on the edges of the coal flow area and the conveyor belt body.

[0014] A mutually exclusive labeling system is established, dividing the labeled pixel set into two non-overlapping categories: "coal" and "belt". The "belt" label serves as the negative sample set for the "coal" label.

[0015] As a preferred method for belt conveyor capacity analysis based on image recognition, an instance segmentation algorithm is used to train the labeled images to obtain an instance segmentation model for identifying coal flow, specifically including:

[0016] Training samples containing coal flow target bounding boxes and masks were constructed using the labeled image dataset;

[0017] Based on the YOLOv8 algorithm, the Anchor-Free architecture and PathAggregationNetwork feature pyramid structure are used to train a deep learning model with pixel-level segmentation capability for coal flow regions. This model is then used as an instance segmentation model to perform instance-level recognition and boundary localization of coal flow targets in conveyor belt video images.

[0018] As a preferred method for belt conveyor capacity analysis based on image recognition, when calculating the coal flow width, line segment EF is marked on the image captured by the video of the coal flow belt, and the pixel length l of line segment EF is calculated. k and the actual physical length l m The unit pixel length is l. m / l k ;

[0019] Perspective reconstruction was performed by marking the coordinates of four points on the edge of the belt, and the vertical pixel y was calculated. i Scaling ratio r at the location i The true length l of the coal flow is obtained. (i) =r i ×dx i ×l m / l k ; where dx i For vertical pixels y i The pixel length of the coal.

[0020] As a preferred method for belt conveyor capacity analysis based on image recognition, the cross-sectional area of ​​the coal flow is approximated as a trapezoid and an arc shape. During the calculation of the coal flow cross-sectional area using the natural angle of repose:

[0021] The area of ​​the trapezoidal portion is calculated by the difference in the areas of the two isosceles triangles, using the following formula:

[0022]

[0023]

[0024] In the formula, S (l) Let the area be an isosceles triangle with the coal flow width l as the base and the included angle α of the idler rollers as the base angle. To correct the width l b Let l be the base of the isosceles triangle with the included angle α between the idler roller and the horizontal direction; l be the width of the coal flow recognized by the image; and α be the angle between the idler roller and the horizontal direction. m is the standard width of the belt corresponding to the top of the idler roller, and h is the height from the top of the idler roller to the bottom of the belt.

[0025] The area of ​​the arc-shaped portion is calculated by the difference between the area of ​​the sector and the area of ​​the triangle, using the following formula:

[0026]

[0027] The cross-sectional area of ​​the coal flow is:

[0028] S = S 梯形 +S 弓形

[0029] In the formula, S 扇形 The sector area corresponding to the arc-shaped portion of the coal flow is used as the minuend when calculating the arc area; S Δ β represents the area of ​​the triangle corresponding to the arc-shaped portion of the coal flow, which is subtracted when calculating the area of ​​the arc; β is the angle of repose; S represents the cross-sectional area of ​​the coal flow.

[0030] The preset rule for the natural angle of repose β is as follows:

[0031] When l ≤ 0.2l m And when the coal type is "low", β = 20°;

[0032] When 0.2l m <l<0.8l m And when the coal type is "low", β = 30°;

[0033] When l ≥ 0.8l m When the coal type is "high", β = 40°.

[0034] As a preferred method for belt conveyor capacity analysis based on image recognition, the formula for calculating the mass of coal per unit time by combining frame time difference is as follows:

[0035] M=S×Δt×v×ρ

[0036] In the formula, Δt is the time difference between the current frame and the previous frame, v is the coal flow velocity estimated by the feature point detection algorithm, and ρ is the measured average density of the coal.

[0037] The present invention also provides a belt conveyor capacity analysis device based on image recognition, comprising:

[0038] The labeling module is used to label images captured by video of the coal conveyor belt with two types of labels: "coal" and "conveyor belt".

[0039] The model training module is used to train the labeled images using an instance segmentation algorithm to obtain an instance segmentation model for identifying coal flow.

[0040] The width calculation module is used to mark line segments on the video captured image of the coal flow conveyor belt, calculate the pixel length and actual physical length of the line segments, determine the unit pixel length, perform perspective restoration of the conveyor belt, calculate the scaling ratio at the longitudinal pixels, and obtain the coal flow width.

[0041] The classification and parameter determination module is used to classify coal using a classification model and determine the angle of repose based on the coal flow width and the classification results.

[0042] The cross-sectional area calculation module is used to approximate the cross-sectional area of ​​the coal flow as a trapezoid and an arc shape, and calculate the cross-sectional area of ​​the coal flow by combining the natural angle of repose.

[0043] The transport volume calculation module is used to segment video frames into instances using the instance segmentation model, calculate the mass of coal per unit time based on the measured average coal density, estimated coal flow velocity and the cross-sectional area of ​​the coal flow, and accumulate the mass of coal with a preset particle size.

[0044] As a preferred embodiment of the image recognition-based belt conveyor capacity analysis device, the labeling module includes:

[0045] Pixel-level contour annotation is performed on the edges of the coal flow area and the conveyor belt body.

[0046] A mutually exclusive labeling system is established, dividing the labeled pixel set into two non-overlapping categories: "coal" and "belt". The "belt" label serves as the negative sample set for the "coal" label.

[0047] As a preferred embodiment of the image recognition-based belt conveyor capacity analysis device, the model training module includes:

[0048] Training samples containing coal flow target bounding boxes and masks were constructed using the labeled image dataset;

[0049] Based on the YOLOv8 algorithm, the Anchor-Free architecture and PathAggregationNetwork feature pyramid structure are used to train a deep learning model with pixel-level segmentation capability for coal flow regions. This model is then used as an instance segmentation model to perform instance-level recognition and boundary localization of coal flow targets in conveyor belt video images.

[0050] As a preferred embodiment of the image recognition-based belt conveyor capacity analysis device, the width calculation module includes:

[0051] The pixel length l of line segment EF is calculated by marking line segment EF on the image captured by the video of the coal conveyor belt. k and the actual physical length l m The unit pixel length is l. m / l k ;

[0052] Perspective reconstruction was performed by marking the coordinates of four points on the edge of the belt, and the vertical pixel y was calculated. i Scaling ratio r at the location i The true length l of the coal flow is obtained. (i) =r i ×dx i ×l m / l k ; where dx i For vertical pixels y i The pixel length of the coal.

[0053] As a preferred embodiment of the image recognition-based belt conveyor capacity analysis device, the cross-sectional area calculation module includes:

[0054] The area of ​​the trapezoidal portion is calculated by the difference in the areas of the two isosceles triangles, using the following formula:

[0055]

[0056] In the formula, S (l) Let the area be an isosceles triangle with the coal flow width l as the base and the included angle α of the idler rollers as the base angle. To correct the width l b Let l be the base of the isosceles triangle with the included angle α between the idler roller and the horizontal direction; l be the width of the coal flow recognized by the image; and α be the angle between the idler roller and the horizontal direction. m is the standard width of the belt corresponding to the top of the idler roller, and h is the height from the top of the idler roller to the bottom of the belt.

[0057] The area of ​​the arc-shaped portion is calculated by the difference between the area of ​​the sector and the area of ​​the triangle, using the following formula:

[0058]

[0059] The cross-sectional area of ​​the coal flow is:

[0060] S = S 梯形 +S 弓形

[0061] In the formula, S 扇形 The sector area corresponding to the arc-shaped portion of the coal flow is used as the minuend when calculating the arc area; S Δ β represents the area of ​​the triangle corresponding to the arc-shaped portion of the coal flow, which is subtracted when calculating the area of ​​the arc; β is the angle of repose; S represents the cross-sectional area of ​​the coal flow.

[0062] In the classification and parameter determination module, the preset rule for the natural repose angle β is as follows:

[0063] When l ≤ 0.2l m And when the coal type is "low", β = 20°;

[0064] When 0.2l m <l<0.8l m And when the coal type is "low", β = 30°;

[0065] When l ≥ 0.8l m Or, when the coal type is "high", β = 40°;

[0066] In the transport volume calculation module, the formula for calculating the mass of coal per unit time based on the frame time difference is as follows:

[0067] M=S×Δt×v×ρ

[0068] In the formula, Δt is the time difference between the current frame and the previous frame, v is the coal flow velocity estimated by the feature point detection algorithm, and ρ is the measured average density of the coal.

[0069] The beneficial effects of this invention are as follows:

[0070] First, by acquiring images and combining them with image recognition technology to estimate transport volume, this method avoids the vulnerability of traditional belt scales, which are easily manipulated through contact measurement. This technical approach eliminates the possibility of data falsification, ensuring the accuracy and reliability of production data. No complex weighing equipment is required; transport volume analysis is achieved solely through cameras and image algorithms. This results in low equipment investment and maintenance costs, making it suitable for large-scale deployment in coal mines and improving the overall economic efficiency of the solution.

[0071] Secondly, by accurately identifying coal flow edges through instance segmentation models and calculating cross-sectional areas using perspective reconstruction and geometric modeling, the pilot project achieved an hourly accuracy rate exceeding 80% and a daily accuracy rate exceeding 90%, providing reliable data support for supervision. Real-time calculation and accumulation of transport volume for video frames enables statistical analysis of coal quantities at various particle sizes, such as minutes and hours, replacing the lag in manual reporting and facilitating the digital upgrade of production management.

[0072] Third, by designing mutually exclusive label training models and fixed roller position measurements, the influence of interference factors such as belt misalignment and perspective distortion is reduced, ensuring measurement stability in complex industrial environments. Attached Figure Description

[0073] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0074] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0075] Figure 1 A schematic diagram of the conveyor belt transport capacity analysis method based on image recognition provided in an embodiment of the present invention;

[0076] Figure 2 An operational interface diagram with "coal" and "belt conveyor" labeled on the same figure is provided for embodiments of the present invention;

[0077] Figure 3 This is a schematic diagram of an instance segmentation model for identifying coal flow provided in an embodiment of the present invention;

[0078] Figure 4 The interface for calculating coal flow width provided in this embodiment of the invention;

[0079] Figure 5 A perspective view showing the coordinates of four points marked on the edge of the belt in an illustration provided by an embodiment of the present invention;

[0080] Figure 6 This is a schematic diagram of the estimated coal flow cross-sectional area provided in an embodiment of the present invention;

[0081] Figure 7This is a schematic diagram for calculating the area of ​​the trapezoidal portion provided in an embodiment of the present invention;

[0082] Figure 8 This is a schematic diagram illustrating the calculation of the area of ​​the arc-shaped portion provided in an embodiment of the present invention;

[0083] Figure 9 This is a schematic diagram of the classification interface of the classification model provided in an embodiment of the present invention;

[0084] Figure 10 This is a schematic diagram of the architecture of a belt conveyor capacity analysis device based on image recognition provided in an embodiment of the present invention. Detailed Implementation

[0085] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0086] Example 1

[0087] See Figure 1 Embodiment 1 of the present invention provides a belt conveyor capacity analysis method based on image recognition, comprising the following steps:

[0088] S1. Using annotation tools, label the images captured by the coal conveyor belt video with two types of tags: "coal" and "conveyor belt." Manual annotation clarifies the boundaries between the coal flow and the conveyor belt in the images, providing accurate sample data for subsequent model training. The labeled information serves as supervisory information, guiding the model to learn the characteristics of the coal flow and the conveyor belt, enabling the model to accurately distinguish between the two in subsequent image recognition.

[0089] S2. An instance segmentation algorithm is used to train the labeled images to obtain an instance segmentation model for coal flow identification. The instance segmentation algorithm can perform accurate pixel-level segmentation of each target instance in the image. By training on the labeled images, the instance segmentation algorithm learns the shape, texture, and other features of the coal flow, enabling the trained model to accurately identify coal flow instances in the image and segment their contours.

[0090] S3. Mark line segments on the video captured image of the coal conveyor belt, calculate the pixel length and actual physical length of the line segments, determine the unit pixel length, and perform perspective restoration on the belt. Calculate the scaling ratio at the vertical pixels to obtain the coal flow width. Due to perspective distortion in camera capture, the pixel length of objects in the image is not a simple linear relationship with their actual physical length. By marking line segments with known actual physical lengths and calculating their pixel lengths, the actual length represented by each pixel can be obtained. Perspective restoration of the belt is performed to eliminate the influence of perspective distortion, ensuring that pixels at each position in the image accurately reflect the actual physical dimensions. Then, by calculating the scaling ratio at the vertical pixels, the pixel width of the coal flow in the image is converted to the actual physical width.

[0091] S4. The coal is classified using a classification model, and the angle of repose (ARR) is determined based on the coal flow width and the classification results. Coal in different stacking states has different ARRs. The classification model categorizes the coal stacking states, and combined with the coal flow width, appropriate ARR parameters can be determined. The ARR is used in subsequent calculations of the arc-shaped portion of the coal flow cross-sectional area; different ARRs affect the shape and area of ​​the arc, thus affecting the calculation of the coal flow cross-sectional area.

[0092] S5. Approximate the cross-sectional area of ​​the coal flow as a trapezoid and an arc, and calculate the cross-sectional area of ​​the coal flow by combining the angle of repose. The cross-sectional shape of the coal flow on the conveyor belt can be approximated as consisting of a lower trapezoid and an upper arc. The trapezoidal part is formed by the support of the idler rollers, and the arc part is formed by the natural accumulation of coal. Combining parameters such as the angle of repose, calculate the areas of the trapezoid and arc using the corresponding geometric formulas, and add the two to obtain the cross-sectional area of ​​the coal flow.

[0093] S6. The video frames are segmented using the instance segmentation model. Based on the measured average coal density, estimated coal flow velocity, and the cross-sectional area of ​​the coal flow, the mass of coal per unit time is calculated using the frame time difference. These are then accumulated to obtain the preset coal quantity for a given particle size. According to the physics formula, mass equals volume multiplied by density. The volume of the coal flow can be obtained by multiplying the cross-sectional area by the velocity and then by the time. By obtaining the cross-sectional area of ​​the coal flow through the instance segmentation model, and combining it with the estimated coal flow velocity and frame time difference, the volume of coal per unit time can be calculated. Multiplying this by the average coal density yields the mass of coal per unit time. Accumulating the coal masses for each unit time period yields the coal quantity for a preset particle size (e.g., per minute, per hour, per day).

[0094] See Figure 2 In this embodiment, when labeling the "coal" and "belt" tags in step S1, the labeling tool performs the following operations on the coal conveyor belt video frame image:

[0095] Pixel-level contour annotation is performed on the edges of the coal flow area and the conveyor belt body. Pixel-level contour annotation can more accurately define the boundaries of the coal flow and the conveyor belt, enabling the model to learn more detailed features during training and improving the accuracy of model recognition. If the labeled contours are not accurate enough, the model may misidentify the edge areas of the coal flow and the conveyor belt, thus affecting the subsequent analysis results.

[0096] A mutually exclusive labeling system is established, dividing the labeled pixel set into two non-overlapping categories: "coal" and "conveyor belt." The "conveyor belt" label serves as the negative sample set for the "coal" label. This mutually exclusive labeling system ensures that each pixel belongs to only one category, avoiding label confusion. Using the conveyor belt as the negative sample set for coal allows the model to better learn the characteristics of coal flow and distinguish between the coal flow and the background (conveyor belt), thereby improving the model's accuracy in recognizing coal flow. If the labels are not mutually exclusive, the model may become confused during training, leading to a decrease in recognition performance.

[0097] See Figure 3 In this embodiment, in step S2, an instance segmentation algorithm is used to train the labeled image to obtain an instance segmentation model for identifying coal flow, specifically including:

[0098] Training samples containing bounding boxes and masks of coal flow targets are constructed using annotated image datasets. The bounding boxes are used to locate the coal flow targets in the images, and the masks are used to accurately segment the pixel regions of the coal flow targets. Constructing training samples containing this information provides more comprehensive supervision information for the instance segmentation algorithm, enabling the algorithm to better learn the position and shape features of the coal flow.

[0099] Based on the YOLOv8 algorithm, the Anchor-Free architecture and PathAggregationNetwork feature pyramid structure are used to train a deep learning model with pixel-level segmentation capability for coal flow areas by optimizing the CIoU loss function and the binary cross-entropy loss function of the mask branch. This model is then used as an instance segmentation model to perform instance-level recognition and boundary localization of coal flow targets in conveyor belt video images.

[0100] Specifically, YOLOv8's Anchor-Free architecture avoids the hassle of manually designing anchor boxes and can automatically learn the scale and positional features of targets. The PathAggregationNetwork feature pyramid structure can fuse features from different levels, improving the model's ability to recognize coal flow targets of different sizes. Optimizing the CIoU loss function can improve the accuracy of bounding box regression, enabling the model to more accurately locate coal flow targets. The binary cross-entropy loss function of the mask branch is used to optimize the pixel-level segmentation results of coal flow, enabling the model to more accurately segment the contours of the coal flow. Through the combination of these techniques, the trained model can perform instance-level recognition and boundary localization of coal flow targets in conveyor belt video images.

[0101] In this embodiment, when calculating the coal flow width in step S3, the pixel length l of line segment EF is calculated by marking line segment EF on the image captured by the coal flow conveyor video. k and the actual physical length l m The unit pixel length is l. m / l k The actual physical length l of line segment EF m It is known that its pixel length l in the image is calculated. k The ratio of the two is the actual length represented by a unit pixel. This unit pixel length is used to convert the pixel lengths of other objects in the image into their actual physical lengths.

[0102] Perspective reconstruction was performed by marking the coordinates of four points on the edge of the belt, and the vertical pixel y was calculated. i Scaling ratio r at the location i The true length l of the coal flow is obtained. (i) =r i ×dx i ×l m / l k ; where dx i For vertical pixels y i The pixel length of the coal.

[0103] See Figure 4 and Figure 5 Specifically, the coordinates of four points on the edge of the belt are marked. Using the principle of perspective transformation, the perspective-distorted image is restored to a front view, thus eliminating the effects of perspective distortion. In the perspective-restored image, the relationship between pixels at each location and their actual physical dimensions is more consistent. The vertical pixel y is calculated. i Scaling ratio r at the location i This is used to adjust the ratio between the pixel length and the actual physical length at different vertical positions. It is achieved through the scaling ratio r. i Unit pixel length l m / l k And coal in vertical pixels yi Pixel length dx i This allows us to calculate the actual physical length of the coal flow at that location, and thus obtain the width of the coal flow.

[0104] Since the belt often runs off-center, it is recommended to select the position of the idler roller as the position for calculating the coal flow width. After perspective transformation, other positions that are more accurate can also be selected as the position for calculating the coal flow width.

[0105] In this embodiment, in step S5, the cross-sectional area of ​​the coal flow is approximated as a trapezoid and an arc shape, and the cross-sectional area of ​​the coal flow is calculated in combination with the angle of repose:

[0106] The area of ​​the trapezoidal portion is calculated by the difference in the areas of the two isosceles triangles, using the following formula:

[0107]

[0108]

[0109] In the formula, S (l) Let the area be an isosceles triangle with the coal flow width l as the base and the included angle α of the idler rollers as the base angle. To correct the width l b Let l be the base of the isosceles triangle with the included angle α between the idler roller and the horizontal direction; l be the width of the coal flow recognized by the image; and α be the angle between the idler roller and the horizontal direction. m is the standard width of the belt corresponding to the top of the idler roller, and h is the height from the top of the idler roller to the bottom of the belt.

[0110] See Figure 6 , Figure 7 and Figure 8 Specifically, the supporting action of the idlers causes the conveyor belt to form a certain shape, and the lower cross-section of the coal flow on the belt can be approximated as a trapezoid. The area of ​​this trapezoid can be calculated by the difference in area between two isosceles triangles. The area S of the isosceles triangle with the coal flow width l as its base is... (l) This represents the area including the coal flow and idler support sections, adjusted for width l. b Let S be the area of ​​the isosceles triangle with the base. (lb) This represents the area of ​​the roller support portion; the difference between the two is the area of ​​the trapezoidal portion. Correction width l b The calculation takes into account the height h from the top of the idler to the bottom of the belt and the included angle α of the idler, reflecting the influence of the idler on the shape of the belt.

[0111] The area of ​​the arc-shaped portion is calculated by the difference between the area of ​​the sector and the area of ​​the triangle, using the following formula:

[0112]

[0113] The cross-sectional area of ​​the coal flow is:

[0114] S = S 梯形 +S 弓形

[0115] In the formula, S 扇形 The sector area corresponding to the arc-shaped portion of the coal flow is used as the minuend when calculating the arc area; S Δ β represents the area of ​​the triangle corresponding to the arc-shaped portion of the coal flow, which is subtracted when calculating the area of ​​the arc; β is the angle of repose; S represents the cross-sectional area of ​​the coal flow.

[0116] Specifically, when coal naturally accumulates, it forms a certain arc shape. The area of ​​this arc can be calculated by the difference between the area of ​​the sector and the area of ​​the triangle. The sector area S 扇形 The area S of the triangle is determined by the natural angle of repose β and the width of the coal flow l. Δ Similarly, the area of ​​the arc-shaped portion is determined by the natural angle of repose β and the width of the coal flow l. The difference between the two is the area of ​​the arc-shaped portion. Adding the areas of the trapezoidal portion and the arc-shaped portion gives the cross-sectional area of ​​the coal flow.

[0117] In step S4, the preset rule for the natural repose angle β is as follows:

[0118] When l ≤ 0.2l m And when the coal type is "low", β = 20°;

[0119] When 0.2l m <l<0.8l m And when the coal type is "low", β = 30°;

[0120] When l ≥ 0.8l m When the coal type is "high", β = 40°.

[0121] See Figure 9 Specifically, the angle of repose β is related to the coal's packing state and the width of the coal flow. When the coal flow width is small and the coal type is "low," the coal's packing height is low, and the angle of repose is small. When the coal flow width is moderate and the coal type is "low," the coal's packing height is moderate, and the angle of repose is moderate. When the coal flow width is large or the coal type is "high," the coal's packing height is large, and the angle of repose is large. This pre-defined rule allows for the selection of an appropriate angle of repose based on actual conditions, improving the accuracy of coal flow cross-sectional area calculations.

[0122] In this embodiment, in step S6, the formula for calculating the mass of coal per unit time based on the frame time difference is as follows:

[0123] M=S×Δt×v×ρ

[0124] In the formula, Δt is the time difference between the current frame and the previous frame, v is the coal flow velocity estimated by the feature point detection algorithm, and ρ is the measured average density of the coal.

[0125] Specifically, according to the physics formula, mass equals volume multiplied by density. The volume of a coal flow can be considered as the cross-sectional area S multiplied by the coal flow velocity v and then by the time Δt, i.e., V = S × v × Δt. Therefore, the mass M of coal per unit time is equal to the volume V multiplied by the average density ρ of the coal, i.e., M = S × v × Δt × ρ. Using this formula, the mass of coal per unit time can be calculated based on the cross-sectional area of ​​the coal flow, the coal flow velocity, the time difference between frames, and the average density of the coal. The amount of coal with a preset particle size can then be obtained by summing these values.

[0126] In coal mine production, when raw coal is transported via belt conveyor after mining, high-definition cameras are installed above the conveyor belt to capture real-time video images of the coal flow. Using the image recognition-based transport volume analysis method of this invention, pixel-level segmentation, width calculation, and cross-sectional area modeling of the coal flow in the video images are performed. Combined with coal flow velocity and density parameters, the amount of coal per unit time is automatically calculated and accumulated. This application scenario can help coal mining enterprises and regulatory units to grasp real-time production data, solving the problems of easy falsification of traditional belt scale measurements and data distortion caused by manual reporting, providing accurate quantitative basis for production management, output statistics, and regulatory verification. The practical application process of this invention is as follows:

[0127] Preliminary preparations and deployment

[0128] Equipment installation: In coal mine belt conveyor scenarios, a high-definition camera should be installed at a suitable position above the belt to ensure clear video capture of the coal flow on the belt. The installation position of the camera should ensure a stable shooting angle and avoid frequent changes.

[0129] Annotation tools and environment setup: Deploy annotation tools and deep learning frameworks (such as the environment required for YOLOv8) on the computer to provide software support for subsequent image annotation and model training.

[0130] Data Acquisition and Model Training

[0131] Image Acquisition and Labeling: Video of coal conveyor belt flow is acquired using a camera, and multiple frames are extracted from the video as samples. Labeling tools are used to perform pixel-level contour annotations on "coal" and "conveyor belt" in each frame, establishing a mutually exclusive labeling system to ensure that the labeled pixel sets of "coal" and "conveyor belt" do not intersect. "Conveyor belt" serves as a negative sample for "coal".

[0132] Model Training: Using the labeled image dataset, training samples containing coal flow target bounding boxes and masks are constructed. Based on the YOLOv8 algorithm's Anchor-Free architecture and PathAggregationNetwork feature pyramid structure, the CIoU loss function and the binary cross-entropy loss function of the mask branch are optimized to train an instance segmentation model capable of pixel-level segmentation of coal flows.

[0133] Real-time calculation of coal flow parameters

[0134] Coal flow width calculation: Mark a line segment EF with a known actual physical length on the real-time acquired video image, and calculate its pixel length l. k and the actual physical length l m Determine the unit pixel length l m / l k Simultaneously, perspective restoration was performed by marking the coordinates of four points along the belt edge, and the scaling ratio r at each vertical pixel was calculated. i This converts the pixel width of the coal flow into the actual physical width.

[0135] Determination of natural angle of repose and calculation of cross-sectional area: The coal is classified ("high" or "low") using a trained classification model, combined with the coal flow width l and the standard belt width l corresponding to the top of the idler roller. m The natural angle of repose β is determined according to preset rules. The cross-sectional area of ​​the coal flow is approximated as a trapezoid and an arc shape. Using parameters such as the included angle α of the idler rollers and the height h from the top of the idler rollers to the bottom of the belt, the areas of the trapezoid and arc shape are calculated by formula. The two are added together to obtain the cross-sectional area S of the coal flow.

[0136] Volume Calculation and Accumulation: The coal flow velocity v is estimated using a feature point detection algorithm, and the average coal density ρ is measured. An instance segmentation model is used to segment the coal flow in each frame of the video and calculate the cross-sectional area. Combined with the frame time difference Δt, the mass of coal per unit time is calculated using the formula M = S × Δt × v × ρ. The total coal volume is then accumulated at preset granularities such as minutes and hours.

[0137] System optimization and maintenance

[0138] Model updates: As factors such as coal quality and conveyor belt operation status change, new video images are collected and re-labeled periodically. The instance segmentation and classification models are then fine-tuned or retrained to maintain the model's recognition accuracy.

[0139] Parameter calibration: Regularly check physical parameters such as the idler roller angle α and the height h from the top of the idler roller to the bottom of the belt. If belt wear or changes in the position of the idler roller occur, update the parameters in a timely manner to ensure the accuracy of the cross-sectional area calculation.

[0140] Anomaly Handling: A system monitoring mechanism is set up so that when abnormal situations such as belt misalignment or camera obstruction occur, an alarm is automatically triggered and the transport volume calculation is paused. Operation resumes after the anomaly is resolved, ensuring data reliability.

[0141] The solution of this invention was applied in pilot coal mines. After three months of statistical analysis, the hourly average accuracy rate exceeded 80%, and the daily accuracy rate exceeded 90%, which was well received by the coal mining enterprises.

[0142] Example 2

[0143] See Figure 10 Embodiment 2 of the present invention provides a belt conveyor capacity analysis device based on image recognition, comprising:

[0144] The labeling module 100 is used to label "coal" and "belt" in the images captured by the video of the coal conveyor belt using the labeling tool;

[0145] The model training module 200 is used to train the labeled image using an instance segmentation algorithm to obtain an instance segmentation model for identifying coal flow.

[0146] The width calculation module 300 is used to mark line segments on the video captured image of the coal flow conveyor belt, calculate the pixel length and actual physical length of the line segments, determine the unit pixel length, perform perspective restoration of the conveyor belt, calculate the scaling ratio at the longitudinal pixels, and obtain the coal flow width.

[0147] The classification and parameter determination module 400 is used to classify coal using a classification model and determine the angle of repose based on the coal flow width and the classification results.

[0148] The cross-sectional area calculation module 500 is used to approximate the cross-sectional area of ​​the coal flow as a trapezoid and an arc shape, and calculate the cross-sectional area of ​​the coal flow by combining the natural angle of repose.

[0149] The transport volume calculation module 600 is used to perform instance segmentation on video frames through the instance segmentation model, calculate the mass of coal per unit time based on the measured average coal density, the estimated coal flow velocity and the cross-sectional area of ​​the coal flow, and accumulate the coal volume with a preset particle size.

[0150] In this embodiment, the labeling module 100 includes:

[0151] Pixel-level contour annotation is performed on the edges of the coal flow area and the conveyor belt body.

[0152] A mutually exclusive labeling system is established, dividing the labeled pixel set into two non-overlapping categories: "coal" and "belt". The "belt" label serves as the negative sample set for the "coal" label.

[0153] In this embodiment, the model training module 200 includes:

[0154] Training samples containing coal flow target bounding boxes and masks were constructed using the labeled image dataset;

[0155] Based on the YOLOv8 algorithm, the Anchor-Free architecture and PathAggregationNetwork feature pyramid structure are used to train a deep learning model with pixel-level segmentation capability for coal flow regions. This model is then used as an instance segmentation model to perform instance-level recognition and boundary localization of coal flow targets in conveyor belt video images.

[0156] In this embodiment, the width calculation module 300 includes:

[0157] The pixel length l of line segment EF is calculated by marking line segment EF on the image captured by the video of the coal conveyor belt. k and the actual physical length l m The unit pixel length is l. m / l k ;

[0158] Perspective reconstruction was performed by marking the coordinates of four points on the edge of the belt, and the vertical pixel y was calculated. i Scaling ratio r at the location i The true length l of the coal flow is obtained. (i) =r i ×dx i ×l m / l k ; where dx i For vertical pixels y i The pixel length of the coal.

[0159] In this embodiment, the cross-sectional area calculation module 500 includes:

[0160] The area of ​​the trapezoidal portion is calculated by the difference in the areas of the two isosceles triangles, using the following formula:

[0161]

[0162]

[0163] In the formula, S (l) Let the area be an isosceles triangle with the coal flow width l as the base and the included angle α of the idler rollers as the base angle. To correct the width l b Let l be the base of the isosceles triangle with the included angle α between the idler roller and the horizontal direction; l be the width of the coal flow recognized by the image; and α be the angle between the idler roller and the horizontal direction. m is the standard width of the belt corresponding to the top of the idler roller, and h is the height from the top of the idler roller to the bottom of the belt.

[0164] The area of ​​the arc-shaped portion is calculated by the difference between the area of ​​the sector and the area of ​​the triangle, using the following formula:

[0165]

[0166] The cross-sectional area of ​​the coal flow is:

[0167] S = S 梯形 +S 弓形

[0168] In the formula, S 扇形 The sector area corresponding to the arc-shaped portion of the coal flow is used as the minuend when calculating the arc area; S Δ β represents the area of ​​the triangle corresponding to the arc-shaped portion of the coal flow, which is subtracted when calculating the area of ​​the arc; β is the angle of repose; S represents the cross-sectional area of ​​the coal flow.

[0169] In the classification and parameter determination module 400, the preset rule for the natural repose angle β is as follows:

[0170] When l ≤ 0.2l m And when the coal type is "low", β = 20°;

[0171] When 0.2l m <l<0.8l m And when the coal type is "low", β = 30°;

[0172] When l ≥ 0.8l m When the coal type is "high", β = 40°.

[0173] In this embodiment, the formula for calculating the mass of coal per unit time in the transport volume calculation module 600, combined with the frame time difference, is as follows:

[0174] M=S×Δt×v×ρ

[0175] In the formula, Δt is the time difference between the current frame and the previous frame, v is the coal flow velocity estimated by the feature point detection algorithm, and ρ is the measured average density of the coal.

[0176] It should be noted that the information interaction and execution process between the modules of the above-mentioned device are based on the same concept as the method embodiment in Embodiment 1 of this application, and the resulting technical effects are the same as those in the method embodiment of this application. For details, please refer to the description in the method embodiment shown above in this application, and it will not be repeated here.

[0177] Example 3

[0178] Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium storing program code for a belt conveyor capacity analysis method based on image recognition. The program code includes instructions for executing the belt conveyor capacity analysis method based on image recognition of Embodiment 1 or any possible implementation thereof.

[0179] Computer-readable storage media can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives, SSDs).

[0180] Example 4

[0181] Embodiment 4 of the present invention provides an electronic device, including: a memory and a processor;

[0182] The processor and the memory communicate with each other via a bus; the memory stores program instructions that can be executed by the processor, and the processor can execute the image recognition-based belt conveyor capacity analysis method of Embodiment 1 or any possible implementation thereof by calling the program instructions.

[0183] Specifically, a processor can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor that reads software code stored in memory. This memory can be integrated into the processor or located outside the processor and exist independently.

[0184] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0185] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0186] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A belt conveyor capacity analysis method based on image recognition, characterized in that, Includes the following steps: Using annotation tools, label the images captured by the coal conveyor belt video with two types of tags: "coal" and "conveyor belt"; An instance segmentation algorithm was used to train the labeled images to obtain an instance segmentation model for identifying coal flow. Mark line segments on the images captured by the coal conveyor belt video, calculate the pixel length and actual physical length of the line segments, determine the unit pixel length, perform perspective restoration on the belt, calculate the scaling ratio at the longitudinal pixels, and obtain the coal flow width. Coal is classified using a classification model, and the angle of repose is determined based on the coal flow width and the classification results. The cross-sectional area of ​​the coal flow is approximated as a trapezoid and an arc shape, and the cross-sectional area of ​​the coal flow is calculated by combining the natural angle of repose. The video frames are segmented using the instance segmentation model. Based on the measured average coal density, the estimated coal flow velocity, and the coal flow cross-sectional area, the mass of coal per unit time is calculated by combining the frame time difference, and the preset particle size coal quantity is obtained by accumulating these values. An instance segmentation algorithm is used to train the labeled images to obtain an instance segmentation model for identifying coal flow, specifically including: Training samples containing coal flow target bounding boxes and masks were constructed using the labeled image dataset; Based on the YOLOv8 algorithm, the Anchor-Free architecture and Path Aggregation Network feature pyramid structure are used to train a deep learning model with pixel-level segmentation capability for coal flow areas by optimizing the CIoU loss function and the binary cross-entropy loss function of the mask branch. This model is then used as an instance segmentation model to perform instance-level recognition and boundary localization of coal flow targets in conveyor belt video images. When calculating the width of the coal flow, the pixel length of line segment EF is calculated by marking line segment EF on the image captured by the video of the coal flow conveyor belt. and actual physical length The unit pixel length is obtained as ; Perspective reconstruction was performed by marking the coordinates of four points on the edge of the belt, and the vertical pixels were calculated. Scaling ratio at the location The true length of the coal flow was obtained. ;in, Vertical pixels The pixel length of the coal; When approximating the cross-sectional area of ​​the coal flow as a trapezoid and an arc shape, and combining this with the angle of repose in calculating the cross-sectional area of ​​the coal flow: The area of ​​the trapezoidal portion is calculated by the difference in the areas of the two isosceles triangles, using the following formula: In the formula, To use the width of the coal flow The angle between the bottom edge and the idler roller Let the area of ​​the isosceles triangle be the base angle. To correct the width The angle between the bottom edge and the idler roller Let the area of ​​the isosceles triangle be the base angle. The width of the coal flow identified in the image. The angle between the idler roller and the horizontal direction. This refers to the standard width of the belt corresponding to the top of the idler roller. The height from the top of the idler roller to the bottom of the belt; The area of ​​the arc-shaped portion is calculated by the difference between the area of ​​the sector and the area of ​​the triangle, using the following formula: The cross-sectional area of ​​the coal flow is: In the formula, The sector area corresponding to the arc-shaped portion of the coal flow is used as the minuend when calculating the arc area; This represents the area of ​​the triangle corresponding to the arc-shaped portion of the coal flow, and is the part subtracted when calculating the area of ​​the arc; The angle of accumulation is natural. This indicates the cross-sectional area of ​​the coal flow.

2. The belt conveyor capacity analysis method based on image recognition according to claim 1, characterized in that, When labeling with both "coal" and "conveyor belt", the following operations are performed on the coal conveyor belt video frame image using the labeling tool: Pixel-level contour annotation is performed on the edges of the coal flow area and the conveyor belt body. A mutually exclusive labeling system is established, dividing the labeled pixel set into two non-overlapping categories: "coal" and "belt". The "belt" label serves as the negative sample set for the "coal" label.

3. The belt conveyor capacity analysis method based on image recognition according to claim 1, characterized in that, The natural angle of accumulation The default rules are: when And when the coal type is "low", ; when And when the coal type is "low", ; when Or when the coal type is "high", .

4. The belt conveyor capacity analysis method based on image recognition according to claim 1, characterized in that, The formula for calculating the mass of coal per unit time using the frame time difference is: In the formula, The time difference between the current frame and the previous frame. The coal flow velocity is estimated using a feature point detection algorithm. This represents the average density of the coal as measured.

5. A belt conveyor capacity analysis device based on image recognition, employing the belt conveyor capacity analysis method based on image recognition as described in any one of claims 1 to 4, characterized in that, include: The labeling module is used to label images captured by video of the coal conveyor belt with two types of labels: "coal" and "conveyor belt". The model training module is used to train the labeled images using an instance segmentation algorithm to obtain an instance segmentation model for identifying coal flow. The width calculation module is used to mark line segments on the video captured image of the coal flow conveyor belt, calculate the pixel length and actual physical length of the line segments, determine the unit pixel length, perform perspective restoration of the conveyor belt, calculate the scaling ratio at the longitudinal pixels, and obtain the coal flow width. The classification and parameter determination module is used to classify coal using a classification model and determine the angle of repose based on the coal flow width and the classification results. The cross-sectional area calculation module is used to approximate the cross-sectional area of ​​the coal flow as a trapezoid and an arc shape, and calculate the cross-sectional area of ​​the coal flow by combining the natural angle of repose. The transport volume calculation module is used to segment video frames into instances using the instance segmentation model, calculate the mass of coal per unit time based on the measured average coal density, estimated coal flow velocity and the cross-sectional area of ​​the coal flow, and accumulate the mass of coal with a preset particle size.

6. The belt conveyor capacity analysis device based on image recognition according to claim 5, characterized in that, In the label annotation module: Pixel-level contour annotation is performed on the edges of the coal flow area and the conveyor belt body. A mutually exclusive labeling system is established, dividing the labeled pixel set into two non-overlapping categories: "coal" and "belt". The "belt" label serves as the negative sample set for the "coal" label.

7. The belt conveyor capacity analysis device based on image recognition according to claim 5, characterized in that, In the model training module: Training samples containing coal flow target bounding boxes and masks were constructed using the labeled image dataset; Based on the YOLOv8 algorithm, the Anchor-Free architecture and Path Aggregation Network feature pyramid structure are used to train a deep learning model with pixel-level segmentation capability for coal flow areas by optimizing the CIoU loss function and the binary cross-entropy loss function of the mask branch. This model is then used as an instance segmentation model to perform instance-level recognition and boundary localization of coal flow targets in conveyor belt video images. In the width calculation module: The pixel length of line segment EF is calculated by marking line segment EF on the image captured by the video of the coal conveyor belt. and actual physical length The unit pixel length is obtained as ; Perspective reconstruction was performed by marking the coordinates of four points on the edge of the belt, and the vertical pixels were calculated. Scaling ratio at the location The true length of the coal flow was obtained. ;in, Vertical pixels The pixel length of the coal.

8. The belt conveyor capacity analysis device based on image recognition according to claim 5, characterized in that, In the cross-sectional area calculation module: The area of ​​the trapezoidal portion is calculated by the difference in the areas of the two isosceles triangles, using the following formula: In the formula, To use the width of the coal flow The angle between the bottom edge and the idler roller Let the area of ​​the isosceles triangle be the base angle. To correct the width The angle between the bottom edge and the idler roller Let the area of ​​the isosceles triangle be the base angle. The width of the coal flow identified in the image. The angle between the idler roller and the horizontal direction. This refers to the standard width of the belt corresponding to the top of the idler roller. The height from the top of the idler roller to the bottom of the belt; The area of ​​the arc-shaped portion is calculated by the difference between the area of ​​the sector and the area of ​​the triangle, using the following formula: The cross-sectional area of ​​the coal flow is: In the formula, The sector area corresponding to the arc-shaped portion of the coal flow is used as the minuend when calculating the arc area; This represents the area of ​​the triangle corresponding to the arc-shaped portion of the coal flow, and is the part subtracted when calculating the area of ​​the arc; The angle of accumulation is natural. Represents the cross-sectional area of ​​the coal flow; In the classification and parameter determination module, the natural angle of accumulation The default rules are: when And when the coal type is "low", ; when And when the coal type is "low", ; when Or when the coal type is "high", ; In the transport volume calculation module, the formula for calculating the mass of coal per unit time based on the frame time difference is as follows: In the formula, The time difference between the current frame and the previous frame. The coal flow velocity is estimated using a feature point detection algorithm. This represents the average density of the coal as measured.

Images