A logistics belt conveying line blockage detection method and system based on deep learning and machine vision fusion

Abstract

This invention discloses a method and system for detecting blockages in logistics conveyor belts based on the fusion of deep learning and machine vision, relating to the field of logistics technology. The key technical solutions include the following steps: acquiring video streams of the conveyor belt area using industrial cameras deployed at monitoring points on the conveyor belt, and preprocessing the video streams; inputting the preprocessed video streams into a pre-trained semantic segmentation model to obtain binary mask images of the same size; comparing the image features of the binary mask images during the risk period with a root cause feature library to obtain root cause matching coefficients; obtaining blockage root cause determination coefficients based on the feature causation coefficients and root cause matching coefficients; and outputting control commands after analyzing the blockage root cause determination coefficients. The effect is to provide technical support for the logistics transmission system.

Description

Technical Field

[0001] This invention relates to the field of logistics technology, and more specifically, to a method and system for detecting blockages in logistics conveyor belts based on the fusion of deep learning and machine vision. Background Technology

[0002] In the logistics industry's belt conveyor operation scenarios, the continuous and stable transport of goods is the core link to ensure sorting and transfer efficiency, while bag blockage is a major problem affecting the stable operation of the conveyor. Traditional bag blockage detection methods mostly rely on manual inspection, mechanical contact sensors, or simple image thresholding technology, which have many limitations in practical applications.

[0003] Manual inspection methods are limited by inspection frequency and subjective judgment, making real-time monitoring of blockage risks difficult. Blockages are often only detected after they have formed and caused transmission interruptions, leading to delayed fault response and significant production losses. Mechanical contact sensors are susceptible to belt vibration and cargo impact, resulting in false alarms or missed alarms, making them unsuitable for complex and ever-changing logistics scenarios. They are also sensitive to environmental factors such as changes in lighting, dust interference, and differences in cargo color. In complex industrial environments, they can easily misinterpret belt patterns and shadows as cargo accumulation, or miss blockage risks due to the lack of visual difference between cargo and belt, resulting in insufficient detection accuracy and robustness. Furthermore, traditional methods mostly only enable post-event detection of blockages, failing to provide early warnings or pinpoint the root cause. Even after a blockage occurs, manual troubleshooting is still required, leading to low efficiency and hindering rapid production recovery. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for detecting blockages in logistics conveyor belts based on the fusion of deep learning and machine vision.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for detecting bag blockages on a logistics conveyor belt based on the fusion of deep learning and machine vision, the method comprising the following steps: Video streams of the belt conveyor area are captured by industrial cameras deployed at monitoring points on the belt conveyor line, and the video streams are preprocessed. The preprocessed video stream is input into a pre-trained semantic segmentation model to obtain a binary mask image of the same size; Binary mask images are processed and analyzed to obtain a blockage warning coefficient. Monitoring periods when the blockage warning coefficient exceeds the warning coefficient threshold are marked as risk periods for the logistics conveyor belt. Extract the feature correlation degree and feature mutation parameters of the binary mask image within the risk period, and obtain the feature causation coefficient of the risk period based on the feature correlation degree and feature mutation parameters; The root cause matching coefficient is obtained by comparing the image features of the binary mask image within the risk period with the root cause feature library; the root cause determination coefficient of the blockage is obtained based on the feature causation coefficient and the root cause matching coefficient. After analyzing the root cause determination coefficient of the blockage, the control command is output.

[0006] Preferably, the binary mask image is processed and analyzed to obtain the packet blocking warning coefficient, specifically including the following steps: Image fusion features are obtained by extracting inter-frame temporal features and intra-frame spatial features from the binary mask image; the real-time operation anomaly coefficient of the belt conveyor is obtained by analyzing the image fusion features. The threshold matching coefficient is obtained based on the real-time operational anomaly coefficient and the feature threshold range; Calculate the similarity between the real-time operational anomaly coefficient and the critical value of the target threshold interval; The similarity is multiplied by the threshold weight value of the target threshold interval to obtain the threshold matching coefficient.

[0007] Preferably, the inter-frame temporal features and intra-frame spatial features of the binary mask image are extracted to obtain image fusion features. The real-time operation anomaly coefficient of the belt transmission line is obtained by analyzing the image fusion features. Specifically, the following steps are included: The inter-frame temporal features include cargo outline movement rate, pixel feature change gradient, and inter-frame feature overlap. The intra-frame spatial features include cargo stacking density, contour distortion degree, and belt area proportion; Set the weights for inter-frame temporal features and intra-frame spatial features; The image fusion features are obtained by fusing the inter-frame temporal features, inter-frame temporal feature weights, intra-frame spatial features, and intra-frame spatial feature weights. The image fusion features are input into the anomaly prediction model to obtain the fusion feature anomaly values; The real-time operation anomaly coefficient of the belt conveyor is obtained based on the fusion characteristic anomaly value and the normal operation characteristic benchmark value of the belt conveyor.

[0008] Preferably, the characteristic causative coefficient of the risk period is obtained based on the characteristic correlation degree and characteristic mutation parameters, specifically including the following steps: The feature correlation includes the correlation between cargo stacking features and belt speed features, and the correlation between profile distortion features and load features. The characteristic mutation parameters include the characteristic mutation time, mutation magnitude, and mutation duration; Set the weights for feature correlation and feature mutation parameters; The correlation degree value is calculated based on the feature correlation degree and the feature correlation degree weight; the mutation parameter value is calculated based on the feature mutation parameter and the feature mutation parameter weight. After normalizing the correlation coefficient and mutation parameter values, a weighted sum is performed to obtain the characteristic causative coefficient of the risk period.

[0009] Preferably, the root cause matching coefficient is obtained by comparing the image features of the binary mask image during the risk period with the root cause feature library, specifically including the following steps: The root cause types are classified into cargo accumulation type, belt failure type, material drop deviation type and equipment linkage type; After extracting the corresponding target feature set for each root cause type, a root cause feature library for blocking packets is constructed, and root cause feature weights are set for each target feature in the root cause feature library for blocking packets. The image features during the risk period are compared with the target feature sets of various root cause types in the root cause feature library to obtain the comprehensive feature similarity. The root cause matching coefficient is obtained by multiplying the feature similarity by the weight of the target feature corresponding to the root cause type.

[0010] Preferably, the root cause determination coefficient for packet blocking is obtained based on the characteristic causative coefficient and the root cause matching coefficient, specifically including the following steps: Determine the correlation between the feature causation coefficient and the root cause matching coefficient; Based on the correlation relationship, the trend of the feature causative coefficient is extracted to obtain the changing trend of the feature causative coefficient. The root cause matching coefficients are screened based on the changing trend of the characteristic causal coefficients to obtain the root cause matching coefficients that match the changing trend. The coupling result is obtained by coupling the changing trends of the matching root cause coefficients and the characteristic causal coefficients. Based on the characteristic mutation parameters of the binary mask image during the risk period, the root cause determination coefficient of the blockage is obtained after eliminating the deviation of the coupling results.

[0011] Preferably, determining the correlation between the feature causative coefficient and the root cause matching coefficient specifically includes the following steps: Clarify the correlation between the feature causation coefficient and the feature mutation parameters of the binary mask image within the risk period; Clarify the correlation between the feature causative coefficient and the feature correlation degree and feature mutation parameter, and determine the blockage development trend reflected by the correlation to form the first preliminary relationship corresponding to the feature causative coefficient; Clarify the correspondence between the root cause matching coefficient and various types of root cause blocking, and determine the fluctuation pattern of the root cause matching coefficient with different features in the root cause feature library based on the correspondence. Based on the fluctuation pattern, the correspondence logic between the root cause matching coefficient and the root cause type of the blockage and the actual features of the binary mask image is clarified, forming the second preliminary relationship corresponding to the root cause matching coefficient; The first and second preliminary relationships are cross-referenced and verified to form their respective corresponding associations.

[0012] Preferably, the control command is output after analyzing the root cause determination coefficient of the blockage, specifically including the following steps: Based on the direction of causal transmission according to the root cause matching coefficient; Output control commands based on the direction of the causative agent transmission.

[0013] Preferably, based on the causal transmission direction of the root cause matching coefficient, the specific steps include: Determine the gradient of the root cause matching coefficient; The direction of causative agent transmission is determined based on gradient changes; the initial location of the causative agent is confirmed based on the characteristic causative coefficient. The direction of induced causation is determined based on the causation pathway and the initial location of action.

[0014] A system for detecting bag blockages on a logistics conveyor belt based on the fusion of deep learning and machine vision includes: Acquisition module: Acquires video streams of the belt area through industrial cameras deployed at monitoring points on the belt conveyor line, and preprocesses the video streams; The first processing module inputs the preprocessed video stream into the pre-trained semantic segmentation model to obtain a binary mask image of the same size. The first analysis module processes and analyzes the binary mask image to obtain the blockage warning coefficient, and marks the monitoring period when the blockage warning coefficient exceeds the warning coefficient threshold as the risk period of the logistics belt conveyor. The second processing module extracts the feature correlation degree and feature mutation parameters of the binary mask image within the risk period, and obtains the feature causation coefficient of the risk period based on the feature correlation degree and feature mutation parameters. The comparison module compares the image features of the binary mask image within the risk period with the root cause feature library to obtain the root cause matching coefficient; and obtains the root cause determination coefficient of the packet blocking based on the feature causation coefficient and the root cause matching coefficient. Output module: After analyzing the root cause determination coefficients of the blockage, output control commands.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This invention acquires video streams of the conveyor belt area using an industrial camera and preprocesses them to effectively eliminate environmental interference from industrial lighting, dust, and vibration. A pre-trained semantic segmentation model is then used to generate high-quality binary mask images, achieving pixel-level precise separation between the conveyor belt background and the cargo foreground. By extracting and fusing inter-frame temporal features and intra-frame spatial features, a blockage warning coefficient is calculated, enabling the identification of risk periods in the early stages of blockage formation. This overcomes the limitations of traditional detection methods, which can only passively respond after a blockage occurs, significantly improving the foresight of risk prevention and control. The feature causation coefficient is calculated by extracting feature correlation and feature mutation parameters within the risk period, clearly reflecting the driving effect of various operational features on blockage formation. The image features are compared with a pre-set blockage root cause feature library to obtain a root cause matching coefficient, clarifying the degree of matching between the risk period and various root cause types. The blockage root cause determination coefficient obtained after coupled analysis can distinguish between different root causes such as cargo accumulation, conveyor belt failure, material deviation, and equipment linkage, solving the problem that traditional methods cannot locate the causes of blockages and providing direction for subsequent intervention. Provide technical support for the intelligent upgrading of logistics transmission systems. Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating the steps of a logistics conveyor belt blockage detection method based on the fusion of deep learning and machine vision, as provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a blockage detection system for a logistics conveyor belt based on the fusion of deep learning and machine vision, provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0018] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0019] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0020] Reference Figures 1-2 As shown.

[0021] The embodiments further illustrate the method and system for detecting blockages in logistics conveyor belts based on the fusion of deep learning and machine vision proposed in this invention.

[0022] A method for detecting bag blockages on a logistics conveyor belt based on the fusion of deep learning and machine vision, the method comprising the following steps: Video streams of the belt conveyor area are captured by industrial cameras deployed at monitoring points on the belt conveyor line, and the video streams are preprocessed. In the monitoring system of logistics belt conveyors, industrial cameras are deployed at monitoring points along the conveyor. These monitoring points are typically chosen to clearly cover the belt's operating area, such as behind the material discharge port, at the head or middle of the conveyor, to ensure that the captured video stream fully reflects the real-time status of the belt and the goods. The industrial cameras continuously capture video streams of the belt area at a preset frame rate. This video stream includes visual information about the position, shape, and movement of the goods during belt operation, as well as the belt's own operational status.

[0023] After video stream acquisition, preprocessing is performed. The purpose of preprocessing is to eliminate interference from the complex industrial environment on subsequent analysis and improve data quality. Specific preprocessing steps include, but are not limited to, the following: First, image denoising is performed, using filtering algorithms to remove noise points caused by dust, vibration, or electromagnetic interference from the video stream, making the image clearer; second, image enhancement is performed, adjusting parameters such as contrast and brightness to highlight the visual differences between the conveyor belt and the goods, facilitating subsequent feature extraction; third, color space conversion is performed, converting the color video stream to grayscale images or other color spaces suitable for model input to reduce computational complexity; finally, size normalization is performed, uniformly scaling each frame of the video stream to a preset size to ensure consistent data format input to the subsequent deep learning model and avoid analysis errors caused by image size differences.

[0024] The preprocessed video stream is input into a pre-trained semantic segmentation model to obtain a binary mask image of the same size; The preprocessed video stream is fed frame-by-frame into a pre-trained semantic segmentation model. This model, built on deep learning, performs pixel-level classification of the input image, dividing each pixel into either the belt background or the cargo foreground. During training, the model has learned from numerous annotated images containing belts and cargo, enabling it to accurately identify the outlines and regions of the cargo within the image. When the preprocessed video frames are input into the model, the output is a binary mask image of the exact same size as the input image. In this mask, pixels representing the belt background are assigned a value of 0, and pixels representing the cargo are assigned a value of 255, thus separating the belt region from the cargo region.

[0025] Binary mask images are processed and analyzed to obtain a blockage warning coefficient. Monitoring periods when the blockage warning coefficient exceeds the warning coefficient threshold are marked as risk periods for the logistics conveyor belt. Extract the feature correlation degree and feature mutation parameters of the binary mask image within the risk period, and obtain the feature causation coefficient of the risk period based on the feature correlation degree and feature mutation parameters; The root cause matching coefficient is obtained by comparing the image features of the binary mask image within the risk period with the root cause feature library; the root cause determination coefficient of the blockage is obtained based on the feature causation coefficient and the root cause matching coefficient. After analyzing the root cause determination coefficient of the blockage, the control command is output.

[0026] The blocking warning coefficient is obtained by processing and analyzing the binary mask image, specifically including the following steps: Image fusion features are obtained by extracting inter-frame temporal features and intra-frame spatial features from the binary mask image; the real-time operation anomaly coefficient of the belt conveyor is obtained by analyzing the image fusion features, specifically including the following steps: Inter-frame temporal features include cargo outline movement rate, pixel feature change gradient, and inter-frame feature overlap. Intra-frame spatial features include cargo stacking density, contour distortion degree, and belt area proportion; Set the weights for inter-frame temporal features and intra-frame spatial features; The image fusion features are obtained by fusing the inter-frame temporal features, inter-frame temporal feature weights, intra-frame spatial features, and intra-frame spatial feature weights. The image fusion features are input into the anomaly prediction model to obtain the fusion feature anomaly values; The real-time operation anomaly coefficient of the belt conveyor is obtained based on the fusion characteristic anomaly value and the normal operation characteristic benchmark value of the belt conveyor. The threshold matching coefficient is obtained based on the real-time operational anomaly coefficient and the feature threshold range; Calculate the similarity between the real-time operational anomaly coefficient and the critical value of the target threshold interval; The similarity is multiplied by the threshold weight value of the target threshold interval to obtain the threshold matching coefficient.

[0027] Inter-frame temporal features mainly include cargo outline movement rate, pixel feature change gradient, and inter-frame feature overlap. Cargo outline movement rate refers to the ratio of the displacement of the center point of the cargo outline in two adjacent frames to the time interval, reflecting the speed of the cargo's movement on the conveyor belt. A sudden drop in rate indicates a risk of cargo jamming or accumulation. Pixel feature change gradient refers to the magnitude of the change in pixel values ​​at the same location between adjacent frames. A larger gradient indicates a more drastic change in the cargo's shape or position, and can capture early signals of cargo blockage. Inter-frame feature overlap refers to the proportion of overlap between cargo areas in two adjacent frames. A lower overlap indicates more significant cargo movement or shape changes, while a higher overlap may indicate cargo stagnation.

[0028] Intra-frame spatial features include cargo stacking density, contour distortion degree, and belt area proportion. Cargo stacking density refers to the ratio of the number of pixels representing cargo in the binary mask image to the total number of pixels in the image. A higher ratio indicates more severe cargo stacking within the belt area, making it a core indicator for assessing the risk of cargo blockage. Contour distortion degree refers to the difference between the cargo contour and the standard cargo contour template. A greater difference indicates more significant shape deformation caused by compression and stacking of the cargo. Belt area proportion refers to the ratio of the number of pixels representing the belt in the binary mask image to the total number of pixels in the image. It reflects the effective operating area of ​​the belt. A continuously decreasing proportion indicates that the space occupied by the cargo is constantly expanding, and the risk of cargo blockage is increasing accordingly.

[0029] Weights are assigned to inter-frame temporal features and intra-frame spatial features respectively. The weights are determined based on the contribution of each feature to packet blocking warning. The inter-frame temporal features, inter-frame temporal feature weights, intra-frame spatial features, and intra-frame spatial feature weights are then fused to obtain the image fusion feature. Image fusion feature = Σ(inter-frame temporal feature i × inter-frame temporal feature weight i) + Σ(intra-frame spatial feature j × intra-frame spatial feature weight j), where i and j represent different inter-frame temporal features and intra-frame spatial features, respectively.

[0030] Image fusion features are input into a pre-trained operation anomaly prediction model. This model learns from a large amount of image fusion feature data of normal and abnormal operation states and outputs fusion feature anomaly values. These values ​​reflect the degree of deviation between the current image fusion features and the features under normal operation. Based on the fusion feature anomaly values ​​and the normal operation feature benchmark values ​​of the belt conveyor, the real-time operation anomaly coefficient of the belt conveyor is calculated. The real-time operation anomaly coefficient = fusion feature anomaly value ÷ normal operation feature benchmark value. The larger the real-time operation anomaly coefficient, the higher the degree of deviation of the current belt operation state from the normal state.

[0031] The threshold matching coefficient is obtained by comparing the real-time operational anomaly coefficient with the feature threshold interval. Then, the similarity is multiplied by a weighted threshold weight value for the target threshold interval. Similarity is calculated using algorithms such as Euclidean distance or cosine similarity; the closer the real-time operational anomaly coefficient is to the critical value of the target threshold interval, the higher the similarity. The threshold weight value is set according to the risk level corresponding to the target threshold interval; the higher the risk level, the larger the weight value. Threshold matching coefficient = Similarity × Threshold weight value.

[0032] The characteristic causative coefficients for risk periods are obtained based on characteristic correlation and characteristic mutation parameters, specifically including the following steps: Feature correlation includes the correlation between cargo stacking features and belt speed features, and the correlation between profile distortion features and load features; Characteristic mutation parameters include the time of the characteristic mutation, the magnitude of the mutation, and the duration of the mutation; Set the weights for feature correlation and feature mutation parameters; The correlation degree value is calculated based on the feature correlation degree and the feature correlation degree weight; the mutation parameter value is calculated based on the feature mutation parameter and the feature mutation parameter weight. After normalizing the correlation coefficient and mutation parameter values, a weighted sum is performed to obtain the characteristic causative coefficient of the risk period.

[0033] The correlation between features mainly includes the correlation between cargo accumulation features and belt speed features, and the correlation between profile distortion features and load features. The correlation between cargo accumulation features and belt speed features reflects the degree of influence of belt speed changes on cargo accumulation. When the belt speed decreases, the time the cargo spends on the belt increases, and the degree of accumulation intensifies. The higher the correlation, the more significant the driving effect of speed changes on cargo accumulation. The correlation between profile distortion features and load features reflects the influence of transmission line load changes on cargo profile shape. When the load increases, the squeezing effect on the cargo intensifies, and the degree of profile distortion increases accordingly. The higher the correlation, the tighter the coupling relationship between load changes and profile distortion.

[0034] The characteristic mutation parameters include the mutation time, mutation amplitude, and mutation duration. The mutation time refers to the starting point at which the characteristic value deviates from the normal operating range, used to pinpoint the critical node where packet congestion risk begins to manifest. The mutation amplitude is the difference between the characteristic value at the mutation time and the normal operating baseline value; a larger difference indicates a more severe deviation from the normal state and a stronger suddenness of packet congestion risk. The mutation duration refers to the length of time the characteristic value remains within the abnormal range; a longer duration indicates a stronger persistence of the abnormal state and a more severe development trend of packet congestion risk.

[0035] We set corresponding weights for feature correlation degree and feature mutation parameter respectively. The weights are determined based on the contribution of each parameter to the formation of packet blocking risk. The correlation degree value is calculated based on feature correlation degree and feature correlation degree weight. Correlation degree value = Σ(feature correlation degree k × feature correlation degree weight k), where k represents different feature correlation degrees.

[0036] The mutation parameter value is calculated based on the characteristic mutation parameter and the characteristic mutation parameter weight. The mutation parameter value = Σ(characteristic mutation parameter a × characteristic mutation parameter weight a), where a represents different characteristic mutation parameters.

[0037] The correlation and mutation parameter values ​​were normalized to eliminate dimensional differences and ensure consistent numerical ranges for both types of parameters. After normalization, the correlation and mutation parameter values ​​were weighted and summed to obtain the characteristic causative coefficient for the risk period. The characteristic causative coefficient = normalized correlation value × correlation weight + normalized mutation parameter value × mutation parameter weight. This coefficient comprehensively reflects the driving effect of characteristic correlation changes and abnormal mutations on packet blockage formation during the risk period, providing a core basis for subsequent determination of the root cause of packet blockage.

[0038] The root cause matching coefficient is obtained by comparing the image features of the binary mask image within the risk period with the root cause feature library. The specific steps include: The root cause types are classified into cargo accumulation type, belt failure type, material drop deviation type and equipment linkage type; After extracting the corresponding target feature set for each root cause type, a root cause feature library for blocking packets is constructed, and root cause feature weights are set for each target feature in the root cause feature library for blocking packets. The image features during the risk period are compared with the target feature sets of various root cause types in the root cause feature library to obtain the comprehensive feature similarity. The root cause matching coefficient is obtained by multiplying the feature similarity by the weight of the target feature corresponding to the root cause type.

[0039] The root causes of belt conveyor blockages can be categorized into four types: cargo accumulation, belt failure, material drop misalignment, and equipment linkage. Cargo accumulation is mainly caused by excessive stacking and jamming of goods within the belt area; belt failure stems from wear, misalignment, or malfunction of the drive unit; material drop misalignment occurs when the material drop position deviates from the belt center, resulting in uneven cargo distribution; and equipment linkage issues are caused by mismatches in the operating rhythms of upstream and downstream equipment, abnormal signal transmission, etc.

[0040] After extracting the corresponding target feature set for each root cause type, a root cause feature library for bag jamming is constructed, and a root cause feature weight is assigned to each target feature in the library. Specifically, the target feature set for cargo accumulation type mainly includes cargo accumulation density and inter-frame feature overlap, used to reflect cargo stagnation and stacking status; the target feature set for belt failure type mainly includes belt area proportion and contour distortion degree, used to reflect the impact of belt operation abnormalities on cargo shape; the target feature set for material drop type mainly includes cargo contour offset and uniformity of cargo distribution in the belt area, used to reflect material drop position deviation; and the target feature set for equipment linkage type mainly includes the matching degree between cargo contour movement rate and equipment operation rhythm, and inter-frame feature change gradient, used to reflect the manifestation of equipment linkage abnormalities. The weight of each target feature is set according to its representation strength for the corresponding root cause type; the higher the representation strength, the larger the weight value.

[0041] After constructing the root cause feature library for packet blocking, the image features of the risk period are compared with the target feature sets of various root cause types in the feature library to obtain the comprehensive feature similarity. The similarity comparison uses cosine similarity or Euclidean distance algorithms. The higher the overlap between the image features of the risk period and the target feature set of a certain type of root cause, the greater the comprehensive feature similarity, indicating that the image features of that period are more consistent with the performance characteristics of that type of root cause.

[0042] The root cause matching coefficient is obtained by multiplying the feature similarity by the target feature weight of the corresponding root cause type. The root cause matching coefficient = feature similarity × target feature weight. The root cause matching coefficient directly reflects the degree of matching between image features during the risk period and various root causes of packet blocking. The higher the coefficient, the greater the possibility that the root cause will cause the current packet blocking risk, providing a basis for the accurate determination of the root cause of subsequent packet blocking.

[0043] The root cause determination coefficient for packet blocking is obtained based on the feature causative coefficient and the root cause matching coefficient, specifically including the following steps: Determining the correlation between the feature causation coefficient and the root cause matching coefficient involves the following steps: Clarify the correlation between the feature causation coefficient and the feature mutation parameters of the binary mask image within the risk period; Clarify the correlation between the feature causative coefficient and the feature correlation degree and feature mutation parameter, and determine the blockage development trend reflected by the correlation to form the first preliminary relationship corresponding to the feature causative coefficient; Clarify the correspondence between the root cause matching coefficient and various types of root cause blocking, and determine the fluctuation pattern of the root cause matching coefficient with different features in the root cause feature library based on the correspondence. Based on the fluctuation pattern, the correspondence logic between the root cause matching coefficient and the root cause type of the blockage and the actual features of the binary mask image is clarified, forming the second preliminary relationship corresponding to the root cause matching coefficient; The first and second preliminary relationships are cross-compared and verified to form their respective corresponding association relationships; Based on the correlation relationship, the trend of the feature causative coefficient is extracted to obtain the changing trend of the feature causative coefficient. The root cause matching coefficients are screened based on the changing trend of the characteristic causal coefficients to obtain the root cause matching coefficients that match the changing trend. The coupling result is obtained by coupling the changing trends of the matching root cause coefficients and the characteristic causal coefficients. Based on the characteristic mutation parameters of the binary mask image during the risk period, the root cause determination coefficient of the blockage is obtained after eliminating the deviation of the coupling results.

[0044] Determine the correlation between the feature causation coefficient and the root cause matching coefficient. First, clarify the correspondence between the feature causation coefficient and the feature correlation degree and feature mutation parameters of the binary mask image during the risk period. Analyze the correlation pattern of the feature causation coefficient with the changes in feature correlation degree and feature mutation parameters, and determine the trend of blockage development reflected by this correlation pattern, thus forming the first preliminary relationship corresponding to the feature causation coefficient. For example, when the correlation between cargo accumulation characteristics and belt speed characteristics continues to increase, and the amplitude and duration of feature mutations increase, the feature causation coefficient will rise synchronously, reflecting a trend of gradually escalating blockage risk.

[0045] The correspondence between root cause matching coefficients and various types of packet jamming root causes is clarified. Based on this correspondence, the fluctuation pattern of root cause matching coefficients with different features in the root cause feature library is determined. Then, based on the fluctuation pattern, the correspondence logic between root cause matching coefficients and packet jamming root cause types and actual features of binary mask images is clarified, forming a second preliminary relationship corresponding to root cause matching coefficients. For example, when features such as cargo stacking density and inter-frame feature overlap are significantly abnormal, the matching coefficient of cargo stacking type root causes increases significantly, demonstrating a strong correspondence between image features and this type of root cause.

[0046] The first and second preliminary relationships are cross-checked to form their respective correlations. The comparison is then used to verify whether the changing trends of the feature causative coefficients and the fluctuation patterns of the root cause matching coefficients are consistent, ensuring that the correlation between the two conforms to the logic of actual packet blocking and avoiding mismatches between features and root causes.

[0047] Trend extraction is performed on the characteristic causative coefficients based on correlation relationships to obtain their changing trends. Trend extraction is achieved through sliding window fitting and time series analysis methods, and is used to determine whether the characteristic causative coefficients are continuously rising, fluctuating downwards, or trending towards stability, thereby clarifying the development trend of the blockage risk.

[0048] The root cause matching coefficients are screened based on the changing trend of the characteristic causative coefficients to obtain root cause matching coefficients that match the changing trend. For example, if the characteristic causative coefficients show a continuous upward trend, root cause matching coefficients that increase synchronously are selected, while coefficients that contradict the trend are excluded, ensuring that the focus is on root cause types that are consistent with the direction of the blockage development.

[0049] The coupling result is obtained by coupling the changing trends of the matching root cause coefficients and the characteristic causative coefficients. The coupling result = Σ(matching root cause coefficient m × characteristic causative coefficient changing trend value m), where m represents different matching root cause types. By integrating the degree of root cause matching with the blockage development trend, the pertinence of root cause determination is enhanced.

[0050] Based on the characteristic mutation parameters of the binary mask image during the risk period, the root cause determination coefficient of packet blocking is obtained after eliminating the bias in the coupling results. The mutation amplitude and duration of the characteristic mutation parameters are used to correct the coupling results. If the mutation amplitude is large and the duration is short, it indicates that the packet blocking is sudden, and the bias caused by trend fitting needs to be reduced; if the mutation amplitude is small and the duration is long, it indicates that the packet blocking is gradual, and the trend influence needs to be strengthened. The root cause determination coefficient of packet blocking = coupling result - (mutation amplitude × bias correction coefficient) + (mutation duration × trend strengthening coefficient). The root cause determination coefficient of packet blocking integrates the characteristic causal driving force, the degree of root cause matching, and the development characteristics of packet blocking, and can determine the core root cause of packet blocking, providing a reliable basis for the output of subsequent control commands.

[0051] After analyzing the root cause determination coefficients of the blockage, the control command is output, which specifically includes the following steps: Based on the direction of causal transmission according to the root cause matching coefficient; Output control commands based on the direction of the causative agent transmission.

[0052] Based on the causal transmission direction of the root cause matching coefficient, the specific steps include: Determine the gradient of the root cause matching coefficient; The direction of causative agent transmission is determined based on gradient changes; the initial location of the causative agent is confirmed based on the characteristic causative coefficient. The direction of induced causation is determined based on the causation pathway and the initial location of action.

[0053] The direction of causal transmission is determined based on the root cause matching coefficient, and the gradient of the root cause matching coefficient is determined. The gradient is obtained by calculating the ratio of the difference between the root cause matching coefficients at adjacent time points to the time interval. Gradient of change = (root cause matching coefficient at the next time point - root cause matching coefficient at the previous time point) ÷ time interval. A positive gradient indicates that the influence of the corresponding root cause type is increasing; a negative gradient indicates that the influence is decreasing. The absolute value of the gradient reflects the severity of the change in influence.

[0054] The direction of causal transmission is determined based on gradient changes. If the gradient of the matching coefficient of a certain type of root cause remains positive and its absolute value increases, it indicates that the influence of that type of root cause is gradually spreading, and the causal transmission direction is positive diffusion. If the gradient remains negative, it indicates that the influence of that type of root cause is gradually converging, and the causal transmission direction is negative convergence. By combining the gradient change order of the matching coefficients of different root cause types, it can be determined whether the causal transmission is from upstream to downstream or from downstream to upstream.

[0055] The initial location of the inducing factor is determined based on the characteristic causation coefficient. The characteristic causation coefficient reflects the driving effect of various characteristics on the blockage during the risk period. By judging the contribution ratio of each characteristic in the characteristic causation coefficient, the physical location corresponding to the characteristic with the highest contribution ratio is located, and this location is the initial location of the inducing factor. If the contribution ratio of the cargo stacking characteristic is the highest, and this characteristic mainly originates from the head area of ​​the conveyor belt, then the initial location of the inducing factor is determined to be the head area of ​​the conveyor belt.

[0056] The direction of causative transmission is determined based on the transmission path and initial location of the causative agent. Taking the initial location as the starting point of transmission, and combining it with the diffusion or convergence trend of the causative agent's transmission path, it is clear that the causative agent is gradually transmitted from the initial location in a specific direction, such as diffusion from the head to the tail of the conveyor belt, or convergence from the material drop outlet to the middle of the belt, thus completely determining the direction of causative agent transmission.

[0057] After identifying the direction of the cause's transmission, control commands are output based on that direction. If the cause's transmission direction is from the material drop outlet towards the middle of the belt, and the root cause type is material drop deviation, then control commands are output to adjust the material drop position and reduce the material drop rate. If the cause's transmission direction is from the head of the belt conveyor towards the tail, and the root cause type is cargo accumulation, then control commands are output to reduce the belt speed and activate the auxiliary guiding device. If the cause's transmission direction is abnormal linkage between upstream and downstream equipment, then control commands are output to synchronously adjust the operating rhythm of upstream and downstream equipment and repair the signal transmission link. Through targeted control commands, the transmission process of the blockage cause is intervened, effectively preventing the fault from escalating and ensuring the stable operation of the belt conveyor.

[0058] A system for detecting bag blockages on a logistics conveyor belt based on the fusion of deep learning and machine vision includes: Acquisition module: Acquires video streams of the belt area through industrial cameras deployed at monitoring points on the belt conveyor line, and preprocesses the video streams; The first processing module inputs the preprocessed video stream into the pre-trained semantic segmentation model to obtain a binary mask image of the same size. The first analysis module processes and analyzes the binary mask image to obtain the blockage warning coefficient, and marks the monitoring period when the blockage warning coefficient exceeds the warning coefficient threshold as the risk period of the logistics belt conveyor. The second processing module extracts the feature correlation degree and feature mutation parameters of the binary mask image within the risk period, and obtains the feature causation coefficient of the risk period based on the feature correlation degree and feature mutation parameters. The comparison module compares the image features of the binary mask image within the risk period with the root cause feature library to obtain the root cause matching coefficient; and obtains the root cause determination coefficient of the packet blocking based on the feature causation coefficient and the root cause matching coefficient. Output module: After analyzing the root cause determination coefficients of the blockage, output control commands.

[0059] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0060] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting bag blockages in a logistics conveyor belt based on the fusion of deep learning and machine vision, characterized in that, The method includes the following steps: Video streams of the belt conveyor area are captured by industrial cameras deployed at monitoring points on the belt conveyor line, and the video streams are preprocessed. The preprocessed video stream is input into a pre-trained semantic segmentation model to obtain a binary mask image of the same size; Binary mask images are processed and analyzed to obtain a blockage warning coefficient. Monitoring periods when the blockage warning coefficient exceeds the warning coefficient threshold are marked as risk periods for the logistics conveyor belt. Extract the feature correlation degree and feature mutation parameters of the binary mask image within the risk period, and obtain the feature causation coefficient of the risk period based on the feature correlation degree and feature mutation parameters; The root cause matching coefficient is obtained by comparing the image features of the binary mask image within the risk period with the root cause feature library; the root cause determination coefficient of the blockage is obtained based on the feature causation coefficient and the root cause matching coefficient. After analyzing the root cause determination coefficient of the blockage, the control command is output.

2. The method for detecting blockages in a logistics conveyor belt based on the fusion of deep learning and machine vision as described in claim 1, characterized in that, The blocking warning coefficient is obtained by processing and analyzing the binary mask image, specifically including the following steps: Image fusion features are obtained by extracting inter-frame temporal features and intra-frame spatial features from the binary mask image; the real-time operation anomaly coefficient of the belt conveyor is obtained by analyzing the image fusion features. The threshold matching coefficient is obtained based on the real-time operational anomaly coefficient and the feature threshold range; Calculate the similarity between the real-time operational anomaly coefficient and the critical value of the target threshold interval; The similarity is multiplied by the threshold weight value of the target threshold interval to obtain the threshold matching coefficient.

3. The method for detecting blockages in a logistics conveyor belt based on the fusion of deep learning and machine vision as described in claim 2, characterized in that, Image fusion features are obtained by extracting inter-frame temporal features and intra-frame spatial features from the binary mask image. Based on the image fusion features, the real-time operation anomaly coefficient of the belt transmission line is obtained through analysis. The specific steps include: The inter-frame temporal features include cargo outline movement rate, pixel feature change gradient, and inter-frame feature overlap. The intra-frame spatial features include cargo stacking density, contour distortion degree, and belt area proportion; Set the weights for inter-frame temporal features and intra-frame spatial features; The image fusion features are obtained by fusing the inter-frame temporal features, inter-frame temporal feature weights, intra-frame spatial features, and intra-frame spatial feature weights. The image fusion features are input into the anomaly prediction model to obtain the fusion feature anomaly values; The real-time operation anomaly coefficient of the belt conveyor is obtained based on the fusion characteristic anomaly value and the normal operation characteristic benchmark value of the belt conveyor.

4. The method for detecting blockages in a logistics conveyor belt based on the fusion of deep learning and machine vision as described in claim 3, characterized in that, The characteristic causative coefficients for risk periods are obtained based on characteristic correlation and characteristic mutation parameters, specifically including the following steps: The feature correlation includes the correlation between cargo stacking features and belt speed features, and the correlation between profile distortion features and load features. The characteristic mutation parameters include the characteristic mutation time, mutation magnitude, and mutation duration; Set the weights for feature correlation and feature mutation parameters; The correlation degree value is calculated based on the feature correlation degree and the feature correlation degree weight; the mutation parameter value is calculated based on the feature mutation parameter and the feature mutation parameter weight. After normalizing the correlation coefficient and mutation parameter values, a weighted sum is performed to obtain the characteristic causative coefficient of the risk period.

5. The method for detecting blockages in a logistics conveyor belt based on the fusion of deep learning and machine vision as described in claim 4, characterized in that, The root cause matching coefficient is obtained by comparing the image features of the binary mask image within the risk period with the root cause feature library. The specific steps include: The root cause types are classified into cargo accumulation type, belt failure type, material drop deviation type and equipment linkage type; After extracting the corresponding target feature set for each root cause type, a root cause feature library for blocking packets is constructed, and root cause feature weights are set for each target feature in the root cause feature library for blocking packets. The image features during the risk period are compared with the target feature sets of various root cause types in the root cause feature library to obtain the comprehensive feature similarity. The root cause matching coefficient is obtained by multiplying the feature similarity by the weight of the target feature corresponding to the root cause type.

6. The method for detecting blockages in a logistics conveyor belt based on the fusion of deep learning and machine vision as described in claim 5, characterized in that, The root cause determination coefficient for packet blocking is obtained based on the feature causative coefficient and the root cause matching coefficient, specifically including the following steps: Determine the correlation between the feature causation coefficient and the root cause matching coefficient; Based on the correlation relationship, the trend of the feature causative coefficient is extracted to obtain the changing trend of the feature causative coefficient. The root cause matching coefficients are screened based on the changing trend of the characteristic causal coefficients to obtain the root cause matching coefficients that match the changing trend. The coupling result is obtained by coupling the changing trends of the matching root cause coefficients and the characteristic causal coefficients. Based on the characteristic mutation parameters of the binary mask image during the risk period, the root cause determination coefficient of the blockage is obtained after eliminating the deviation of the coupling results.

7. The method for detecting blockages in a logistics conveyor belt based on the fusion of deep learning and machine vision as described in claim 6, characterized in that, Determining the correlation between the feature causation coefficient and the root cause matching coefficient involves the following steps: Clarify the correlation between the feature causation coefficient and the feature mutation parameters of the binary mask image within the risk period; Clarify the correlation between the feature causative coefficient and the feature correlation degree and feature mutation parameter, and determine the blockage development trend reflected by the correlation to form the first preliminary relationship corresponding to the feature causative coefficient; Clarify the correspondence between the root cause matching coefficient and various types of root cause blocking, and determine the fluctuation pattern of the root cause matching coefficient with different features in the root cause feature library based on the correspondence. Based on the fluctuation pattern, the correspondence logic between the root cause matching coefficient and the root cause type of the blockage and the actual features of the binary mask image is clarified, forming the second preliminary relationship corresponding to the root cause matching coefficient; The first and second preliminary relationships are cross-referenced and verified to form their respective corresponding associations.

8. The method for detecting blockages in a logistics conveyor belt based on the fusion of deep learning and machine vision as described in claim 7, characterized in that, After analyzing the root cause determination coefficients of the blockage, the control command is output, which specifically includes the following steps: Based on the direction of causal transmission according to the root cause matching coefficient; Output control commands based on the direction of the causative agent transmission.

9. The method for detecting blockages in a logistics conveyor belt based on the fusion of deep learning and machine vision as described in claim 8, characterized in that, Based on the causal transmission direction of the root cause matching coefficient, the specific steps include: Determine the gradient of the root cause matching coefficient; The direction of causative agent transmission is determined based on gradient changes; the initial location of the causative agent is confirmed based on the characteristic causative coefficient. The direction of induced causation is determined based on the causation pathway and the initial location of action.

10. A logistics conveyor belt blockage detection system based on the fusion of deep learning and machine vision, applied to the logistics conveyor belt blockage detection method based on the fusion of deep learning and machine vision as described in any one of claims 1-9, characterized in that, include: Acquisition module: Acquires video streams of the belt area through industrial cameras deployed at monitoring points on the belt conveyor line, and preprocesses the video streams; The first processing module inputs the preprocessed video stream into the pre-trained semantic segmentation model to obtain a binary mask image of the same size. The first analysis module processes and analyzes the binary mask image to obtain the blockage warning coefficient, and marks the monitoring period when the blockage warning coefficient exceeds the warning coefficient threshold as the risk period of the logistics belt conveyor. The second processing module extracts the feature correlation degree and feature mutation parameters of the binary mask image within the risk period, and obtains the feature causation coefficient of the risk period based on the feature correlation degree and feature mutation parameters. Comparison module: The similarity comparison between the image features of the binary mask image within the risk period and the root cause feature library is performed to obtain the root cause matching coefficient; The root cause determination coefficient of the blockage is obtained based on the characteristic causative coefficient and the root cause matching coefficient. Output module: After analyzing the root cause determination coefficients of the blockage, output control commands.

Images