A video detection system for conveyor belt misalignment and foreign object accumulation

Abstract

This invention relates to the field of machine vision inspection technology, specifically to a video detection system for conveyor belt misalignment and foreign object accumulation. The system includes a video acquisition module, an adaptive illumination compensation module, a multi-scale feature fusion module, a spatiotemporal consistency analysis module, and a decision output module. Specifically: the video acquisition module acquires the original video stream of the conveyor belt's operating area; the adaptive illumination compensation module generates an illumination-equalized image through dynamic background separation and adaptive gamma correction; the multi-scale feature fusion module extracts and generates multiple feature maps; the spatiotemporal consistency analysis module outputs the misalignment azimuth, foreign object coordinates, and confidence level; and the decision output module generates the misalignment level, foreign object type, and alarm command based on preset thresholds. This invention achieves high-precision identification and classification of conveyor belt misalignment and foreign object accumulation through multi-module collaboration, improving the intelligence level and practical response capability of the detection system.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, and in particular to a video detection system for conveyor belt misalignment and foreign object accumulation. Background Technology

[0002] With the continuous improvement of industrial automation, conveyor belts, as important equipment for material transportation on production lines, directly affect overall production efficiency and safety due to their operational stability. In actual operation, conveyor belts are prone to abnormal conditions such as belt misalignment and foreign object accumulation. If these are not detected and handled in a timely manner, they can cause not only equipment wear and material loss, but may even lead to safety accidents. Traditional conveyor belt anomaly detection mostly relies on mechanical limit devices or single-point sensors, which have low detection accuracy, slow response, and difficulty in accurately judging the direction of misalignment, the location and severity of foreign objects, thus failing to meet the needs of modern high-precision and intelligent detection.

[0003] While existing video surveillance methods can collect image information, they largely rely on manual observation or simple analysis of static images, lacking effective mechanisms for image enhancement, feature fusion, and spatiotemporal information assessment. This makes it difficult to accurately identify conveyor belt misalignment trends and foreign object accumulation under conditions of changing lighting, background interference, or dynamic object interference. Therefore, a video detection system for conveyor belt misalignment and foreign object accumulation is urgently needed to solve these problems. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides a video detection system for conveyor belt misalignment and foreign object accumulation.

[0005] A video detection system for conveyor belt misalignment and foreign object accumulation includes a video acquisition module, an adaptive illumination compensation module, a multi-scale feature fusion module, a spatiotemporal consistency analysis module, and a decision output module; wherein:

[0006] Video acquisition module: used to acquire the raw video stream of the conveyor belt running area and output a time-series image sequence;

[0007] Adaptive illumination compensation module: used to receive time-series image sequences and generate illumination-equalized images through dynamic background separation and adaptive gamma correction;

[0008] Multi-scale feature fusion module: used to receive the illumination equalization image, extract and generate edge gradient feature map, texture structure feature map and motion optical flow feature map;

[0009] Spatiotemporal consistency analysis module: Constructs a dynamic baseline based on edge gradient feature map and calculates the conveyor belt edge offset; identifies foreign object accumulation areas based on texture structure feature map and locates their coordinates; verifies the consistency between foreign object displacement and conveyor belt running direction by combining motion optical flow feature map; and outputs deviation azimuth angle, foreign object coordinates and confidence level.

[0010] Decision output module: Used to receive deviation azimuth angle, foreign object coordinates and confidence level, and generate deviation level, foreign object type and alarm command according to preset threshold.

[0011] Optionally, the video acquisition module includes a lens control unit, an image sensing unit, and a synchronization control unit; wherein:

[0012] Lens control unit: used to adjust the shooting angle and zoom level, so that the camera equipment can cover the entire conveyor belt running area;

[0013] Image sensing unit: integrates an image sensor component to receive visual information captured by the lens and convert it into a series of frame images;

[0014] Synchronization control unit: Used to set and synchronize the time interval for image acquisition, ensuring a fixed time interval between adjacent frames, thereby forming a continuous time sequence of images.

[0015] Optionally, the adaptive illumination compensation module includes a background modeling unit, a foreground extraction unit, a gamma estimation unit, and an image reconstruction unit; wherein:

[0016] Background modeling unit: used to perform sliding window averaging on the received time-series image sequence, calculate the background brightness reference value of each pixel position based on the past N frames of images, and establish a static background model;

[0017] The foreground extraction unit is used to perform pixel-by-pixel difference between the current frame image and the background model, and to mark the pixel regions with differences exceeding a set threshold as foreground regions, thereby achieving dynamic background separation.

[0018] Gamma estimation unit: Used to calculate the optimal gamma value based on the global luminance histogram distribution of the current frame image. This makes the image grayscale range within the range Internal equilibrium distribution;

[0019] Image reconstruction unit: used to perform optimal gamma-value-based processing on the foreground region. The nonlinear gamma transform is used to keep the background region unchanged, ultimately generating an image with equal illumination.

[0020] Optionally, the multi-scale feature fusion module includes an edge feature extraction unit, a texture feature extraction unit, an optical flow feature extraction unit, and a feature fusion unit; wherein:

[0021] Edge feature extraction unit: used to perform multi-scale edge detection on the image with equal illumination. By setting multiple Sobel operators with different convolution kernel sizes, it extracts the horizontal and vertical grayscale changes of the image respectively, highlighting the position and intensity of the structural edges, and forming an edge gradient feature map.

[0022] Texture feature extraction unit: Based on the local binary mode method, extract the gray-level structure of each pixel neighborhood in the illumination-equalized image, generate binary code by comparing the gray-level relationship between the center pixel and the surrounding pixels, divide the image region, and statistically analyze the LBP histogram at the region level to form a texture structure feature map.

[0023] Optical flow feature extraction unit: used to perform optical flow intensity calculation on two adjacent frames of equally lit images, obtain the motion vector of each pixel in the image between two consecutive frames, thereby characterizing the motion trend of local areas of the image and forming a motion optical flow feature map;

[0024] Feature fusion unit: It is used to align edge gradient feature maps, texture structure feature maps and motion optical flow feature maps according to their spatial positions, fuse them into a unified multi-scale feature tensor, and perform normalization processing after fusion to ensure that different feature scales are consistent.

[0025] Optionally, the spatiotemporal consistency analysis module includes an edge offset analysis unit, a foreign object localization unit, an optical flow consistency verification unit, and an index generation unit; wherein:

[0026] Edge offset analysis unit: used to receive edge gradient feature map, extract left and right edge points of conveyor belt, construct dynamic baseline of current frame, compare with set reference line, and calculate offset of both sides of conveyor belt;

[0027] Foreign object positioning unit: used to receive texture structure feature map, extract the area that is inconsistent with the normal surface texture of the conveyor belt as the foreign object accumulation area, and determine the center coordinates of the foreign object accumulation area;

[0028] Optical flow consistency verification unit: used to receive the motion optical flow feature map, analyze whether the optical flow vector direction in the foreign object area is consistent with the conveyor belt movement direction, and determine whether the foreign object is in a static accumulation state;

[0029] The index generation unit is used to generate the deviation azimuth angle based on the results of the edge offset analysis unit, output the foreign object coordinates based on the results of the foreign object positioning unit, and calculate the confidence level based on the results of the optical flow consistency verification unit as the final detection output.

[0030] Optionally, the edge offset analysis unit includes:

[0031] Edge point extraction subunit: used to scan the edge gradient feature map column by column in the vertical direction, and extract the leftmost and rightmost pixel coordinates of each column whose gradient magnitude is greater than a set threshold, which are respectively used as the left edge point and right edge point of the conveyor belt in the current frame;

[0032] Baseline construction sub-unit: used to fit all left edge points to the left dynamic edge curve, fit all right edge points to the right dynamic edge curve, and use the midline of the two edge curves as the dynamic baseline of the current frame.

[0033] Offset calculation subunit: Used to compare the dynamic baseline of the current frame with the preset ideal conveyor belt centerline point by point in the corresponding column position, calculate the lateral offset value of each column, and take the average value as the overall lateral offset.

[0034] Optionally, the foreign object positioning unit includes:

[0035] Texture anomaly detection subunit: It is used to divide the received texture structure feature map into local regions, calculate the local binary pattern histogram of each region, and compare it with the preset normal texture template of the conveyor belt. When the similarity is lower than the set threshold, the region is determined to be a texture anomaly region.

[0036] Region segmentation subunit: used to perform connectivity analysis on all adjacent abnormal pixel regions, merge them to form continuous foreign object accumulation candidate regions, remove noise blocks with an area lower than the minimum effective region threshold, and extract the bounding box of the main foreign object region.

[0037] Center coordinate extraction sub-unit: used to calculate the center point of the bounding rectangle of the main foreign object region, as the positioning coordinates of the main foreign object region.

[0038] Optionally, the optical flow consistency verification unit includes:

[0039] Optical flow vector extraction subunit: used to extract two-dimensional optical flow vectors of all pixel positions inside the foreign object accumulation region from the motion optical flow feature map, forming a set of local motion vectors of the foreign object accumulation region;

[0040] Directional similarity calculation subunit: used to calculate the directional cosine of each optical flow vector and the unit vector of the conveyor belt running direction to obtain the directional similarity of each pixel;

[0041] The static judgment subunit is used to count the proportion of pixels in the foreign object area that meet the orientation consistency threshold condition. If the proportion exceeds the set threshold, the foreign object is judged to be in a static accumulation state; otherwise, it is judged to be a moving interference object.

[0042] Optionally, the indicator generation unit includes:

[0043] Misalignment azimuth calculation subunit: Used to receive the left and right edge offset values ​​of the conveyor belt output by the edge offset analysis unit, and calculate the misalignment azimuth of the conveyor belt based on the offset difference between the left and right sides. ;

[0044] Foreign object coordinate output subunit: Used to receive the center position coordinates of the foreign object output by the foreign object positioning unit and output them as the spatial position parameters of the foreign object in the structured detection results;

[0045] Confidence assessment subunit: Receives the output from the optical flow consistency verification unit, calculates the percentage of pixels whose optical flow direction deviates from the conveyor belt direction within the foreign object region, and infers the confidence level of the foreign object's stationary state based on this percentage. .

[0046] Optionally, the decision output module includes a deviation determination subunit, a foreign object classification subunit, and an alarm generation subunit; wherein:

[0047] Deviation determination subunit: Used to receive the deviation azimuth angle and compare it with the set deviation angle level threshold set, and output the corresponding deviation level according to the interval of the absolute value of the angle, including four categories: normal, slight deviation, moderate deviation and severe deviation.

[0048] Foreign object classification subunit: used to receive the coordinates and confidence level of the foreign object, and classify it according to the confidence level and a set threshold. The comparison is performed. If the confidence level is higher than the threshold, it is determined to be a real foreign object accumulation; otherwise, it is determined to be a moving interference object, and the category of the foreign object region is determined according to the image location of the foreign object coordinates.

[0049] Alarm generation subunit: Used to combine the deviation level and foreign object type judgment results, and generate corresponding alarm instructions according to preset response rules; when the deviation level is moderate or severe, or the foreign object type is actual accumulation, it outputs an alarm instruction data packet containing alarm type, alarm area and handling suggestions.

[0050] The beneficial effects of this invention are:

[0051] This invention constructs a modular video detection system that includes video acquisition, adaptive illumination compensation, multi-scale feature fusion, spatiotemporal consistency analysis, and decision output. This system achieves a complete process for image enhancement, key feature extraction, and abnormal state identification of the conveyor belt operating area. The modules are connected through image sequences and feature maps, forming a closed-loop structure from video input to quantitative detection output. This system has stable data links and processing logic, effectively improving adaptability and robustness in complex industrial environments.

[0052] This invention, by introducing dynamic baseline construction, texture anomaly recognition, optical flow direction verification, and threshold decision-making mechanisms, achieves collaborative analysis and structured output of multiple indicators such as deviation azimuth angle, foreign object coordinates, and confidence level. The system can accurately determine the deviation level, foreign object type, and spatial location, and generate alarm commands in conjunction, significantly improving the intelligence level and response efficiency of conveyor belt anomaly detection. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram of a video detection system according to an embodiment of the present invention;

[0055] Figure 2 This is a schematic diagram of the spatiotemporal consistency analysis module in an embodiment of the present invention. Detailed Implementation

[0056] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0057] It should be noted that the use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0058] Generally, terms can be understood at least partly from their use in context. For example, depending at least partly on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in a singular sense, or a combination of features, structures, or characteristics in a plural sense. Additionally, the term "based on" can be understood not necessarily to convey an exclusive set of factors, but rather, alternatively, depending at least partly on the context, to allow for the presence of other factors that are not necessarily explicitly described.

[0059] like Figures 1-2 As shown, a video detection system for conveyor belt misalignment and foreign object accumulation includes a video acquisition module, an adaptive illumination compensation module, a multi-scale feature fusion module, a spatiotemporal consistency analysis module, and a decision output module; wherein:

[0060] Video acquisition module: used to acquire the raw video stream of the conveyor belt running area and output a time-series image sequence;

[0061] The video acquisition module includes a lens control unit, an image sensing unit, and a synchronization control unit; wherein:

[0062] Lens control unit: used to adjust the shooting angle and zoom level, so that the camera equipment can cover the entire conveyor belt running area;

[0063] Image sensing unit: integrates an image sensor component to receive visual information captured by the lens and convert it into a series of frame images;

[0064] Synchronization control unit: Used to set and synchronize the time interval of image acquisition, ensuring a fixed time interval between adjacent frames, thereby forming a continuous time-series image sequence; by setting lens control, image perception and synchronization control subunits, the video acquisition module can stably acquire continuous video data of the conveyor belt running area under different working conditions, ensuring that the subsequent processing module analyzes based on clear and time-consistent image sequences, which is conducive to improving the stability and robustness of the overall detection system.

[0065] Adaptive illumination compensation module: used to receive time-series image sequences and generate illumination-equalized images through dynamic background separation and adaptive gamma correction;

[0066] The adaptive illumination compensation module includes a background modeling unit, a foreground extraction unit, a gamma estimation unit, and an image reconstruction unit; among which:

[0067] Background modeling unit: used to perform sliding window averaging on the received time-series image sequence, calculate the background brightness reference value of each pixel position based on the past N frames of images, and establish a static background model;

[0068] The formula for calculating the background brightness baseline value is: ,in, Indicates the background model at the pixel position The brightness value at that location; Indicates the first Frame image in pixels The grayscale value at that location; This indicates the number of frames involved in the calculation within the sliding window;

[0069] The foreground extraction unit is used to perform pixel-by-pixel difference between the current frame image and the background model, and to mark the pixel regions with differences exceeding a set threshold as foreground regions, thereby achieving dynamic background separation.

[0070] The above difference calculation and binary foreground mask generation formulas are as follows:

[0071] ;

[0072] in, Indicates the foreground mask in pixels The value at this location represents the foreground (1) and the background (0). Indicates the current frame image in pixels The grayscale value at that location; This indicates the set brightness difference threshold.

[0073] Gamma estimation unit: used to calculate the optimal gamma value based on the global luminance histogram distribution of the current frame image. This makes the image grayscale range within the range Internal equilibrium distribution, where the optimal gamma value Calculated using the following formula: ,in, This represents the average grayscale value of the current frame image; This represents a logarithmic operation with the natural logarithm as the base; the value 0.5 represents the gray point mapping value in the target; the constant 255 represents the maximum gray level of the 8-bit image;

[0074] Image reconstruction unit: used to perform optimal gamma-value-based processing on the foreground region. The nonlinear gamma transform maintains the background region unchanged, ultimately generating an image with balanced illumination, improving the visual quality and feature stability of the foreground region; the above unit introduces a moving average model to construct a background brightness benchmark and combines it with threshold difference to extract the dynamic foreground, achieving robust background separation; further, the gamma parameter is adaptively adjusted in combination with the image brightness distribution, so that the image still has stable and balanced visual features under complex lighting conditions, effectively improving the accuracy of subsequent feature extraction and detection.

[0075] Multi-scale feature fusion module: used to receive the illumination equalization image, extract and generate edge gradient feature map, texture structure feature map and motion optical flow feature map;

[0076] The multi-scale feature fusion module includes an edge feature extraction unit, a texture feature extraction unit, an optical flow feature extraction unit, and a feature fusion unit; wherein:

[0077] Edge feature extraction unit: Used to perform multi-scale edge detection on the evenly lit image. By setting multiple Sobel operators with different kernel sizes, it extracts the horizontal and vertical grayscale changes of the image, highlighting the position and intensity of structural edges, and forming an edge gradient feature map; the formula for calculating the edge gradient magnitude is as follows: ,in, For pixels in the edge gradient map The gradient magnitude; For pixels The lateral gradient component at the point is obtained by horizontal convolution of the Sobel operator; For pixels The vertical gradient component at a given point is obtained by convolution in the vertical direction using the Sobel operator; Represents the square root operation;

[0078] Texture feature extraction unit: Based on the local binary mode method, extract the gray-level structure of each pixel neighborhood in the illumination-equalized image, generate binary code by comparing the gray-level relationship between the center pixel and the surrounding pixels, divide the image region, and statistically analyze the LBP histogram at the region level to form a texture structure feature map.

[0079] Optical flow feature extraction unit: This unit calculates the optical flow intensity of two adjacent frames of equally lit images, obtaining the motion vector of each pixel in the image between the two consecutive frames. This characterizes the motion trend of local regions in the image, forming a motion optical flow feature map. The formula for calculating the optical flow intensity is as follows:

[0080] ,in, For pixels The optical flow intensity at that location; For pixels The horizontal optical flow displacement component; For pixels The optical flow displacement component in the vertical direction;

[0081] The feature fusion unit aligns edge gradient feature maps, texture structure feature maps, and motion optical flow feature maps according to their spatial positions, fusing them into a unified multi-scale feature tensor. After fusion, normalization is performed to ensure consistency across different feature scales, providing a highly compatible input tensor for subsequent modules. The edge feature extraction unit enhances image boundary changes, the texture feature extraction unit captures regional grayscale structures, and the optical flow feature extraction unit reveals pixel-level dynamic displacement trends. These three types of features are uniformly fused to form a high-dimensional multi-scale information tensor, thereby enhancing the system's ability to express deviations in boundaries and changes in the morphology of foreign objects from multiple angles, improving the accuracy and stability of detection and discrimination.

[0082] Spatiotemporal consistency analysis module: Constructs a dynamic baseline based on edge gradient feature map and calculates the conveyor belt edge offset; identifies foreign object accumulation areas based on texture structure feature map and locates their coordinates; verifies the consistency between foreign object displacement and conveyor belt running direction by combining motion optical flow feature map; and outputs deviation azimuth angle, foreign object coordinates and confidence level.

[0083] The spatiotemporal consistency analysis module includes an edge offset analysis unit, a foreign object localization unit, an optical flow consistency verification unit, and an index generation unit; among which:

[0084] Edge offset analysis unit: used to receive edge gradient feature map, extract left and right edge points of conveyor belt, construct dynamic baseline of current frame, compare with set reference line, and calculate offset of both sides of conveyor belt;

[0085] Foreign object positioning unit: used to receive texture structure feature map, extract the area that is inconsistent with the normal surface texture of the conveyor belt as the foreign object accumulation area, and determine the center coordinates of the foreign object accumulation area;

[0086] Optical flow consistency verification unit: used to receive the motion optical flow feature map, analyze whether the optical flow vector direction in the foreign object area is consistent with the conveyor belt movement direction, and determine whether the foreign object is in a static accumulation state;

[0087] The index generation unit is used to generate the deviation azimuth angle based on the results of the edge offset analysis unit, output the foreign object coordinates based on the results of the foreign object positioning unit, and calculate the confidence level based on the results of the optical flow consistency verification unit as the final detection output. The above units achieve accurate extraction of conveyor belt deviation and foreign object accumulation status through joint analysis of edge, texture and optical flow features, and generate structured detection indexes to provide a reliable basis for subsequent system decision-making and alarm.

[0088] The edge offset analysis unit includes:

[0089] Edge point extraction subunit: used to scan the edge gradient feature map column by column in the vertical direction, and extract the leftmost and rightmost pixel coordinates of each column whose gradient magnitude is greater than a set threshold, which are respectively used as the left edge point and right edge point of the conveyor belt in the current frame;

[0090] Baseline construction sub-unit: used to fit all left edge points to the left dynamic edge curve, fit all right edge points to the right dynamic edge curve, and use the midline of the two edge curves as the dynamic baseline of the current frame.

[0091] Offset Calculation Subunit: This subunit compares the current frame's dynamic baseline with the preset ideal conveyor belt centerline point-by-point at corresponding column positions, calculates the lateral offset value for each column, and takes the average value as the overall lateral offset. The calculation formula is as follows: ,in, This represents the average lateral offset of the conveyor belt in the current frame. The number of columns in the image; For the first The center position of the current frame's dynamic baseline; For the first The center position of the preset ideal reference line; Indicates from column 1 to column 2. Column summation: By setting sub-units, stable extraction of edge points, high-precision fitting of dynamic boundaries and quantitative analysis of inter-frame offset are achieved. It can accurately extract structural changes on both sides of the conveyor belt under complex conditions such as local image occlusion and edge blurring, and output the lateral offset index, providing a key basis for judging the deviation trend.

[0092] The foreign object positioning unit includes:

[0093] Texture anomaly detection subunit: It is used to divide the received texture structure feature map into local regions, calculate the local binary pattern histogram of each region, and compare it with the preset normal texture template of the conveyor belt. When the similarity is lower than the set threshold, the region is determined to be a texture anomaly region.

[0094] Region segmentation subunit: used to perform connectivity analysis on all adjacent abnormal pixel regions, merge them to form continuous foreign object accumulation candidate regions, remove noise blocks with an area lower than the minimum effective region threshold, and extract the bounding box of the main foreign object region.

[0095] Center coordinate extraction sub-unit: Used to calculate the center point of the bounding rectangle of the main foreign object region, which serves as the positioning coordinate of the main foreign object region. The calculation formula is as follows: ; ,in, The coordinates of the center of the foreign object accumulation area; These represent the minimum and maximum values ​​of the bounding box of the foreign object region in the horizontal direction. The minimum and maximum values ​​of the bounding box of the foreign object region in the vertical direction are defined. All boundary values ​​are calculated based on the pixel positions extracted from the image coordinate system. The above sub-units effectively realize the automatic identification of abnormal surface structure regions by performing similarity detection between local texture features and normal templates and combining connected region segmentation methods. At the same time, the system calculates the center position of the foreign object by combining the circumscribed boundary, enabling the system to accurately complete the spatial positioning and region calibration of foreign object accumulation without relying on color or shape.

[0096] The optical flow conformance verification unit includes:

[0097] Optical flow vector extraction subunit: used to extract two-dimensional optical flow vectors of all pixel positions inside the foreign object accumulation region from the motion optical flow feature map, forming a set of local motion vectors of the foreign object accumulation region;

[0098] Directional similarity calculation subunit: This unit calculates the directional similarity of each pixel by performing a directional cosine comparison between each optical flow vector and the unit vector representing the conveyor belt's running direction. The calculation formula is as follows:

[0099] ,in, For the first The cosine value of the consistency between the optical flow direction of each pixel and the conveyor belt direction, ranging from... ; For the first A two-dimensional optical flow vector of each pixel; The standard motion direction unit vector set for the conveyor belt; the numerator is the vector dot product, and the denominator is the product of the vector magnitudes.

[0100] Static judgment sub-unit: used to statistically determine whether the orientation consistency threshold condition is met within the foreign object region. The pixel ratio of ) The ratio is set as a directional consistency threshold. If the ratio exceeds the set threshold, the foreign object is judged to be in a static accumulation state; otherwise, it is judged to be a moving interference object. The above sub-unit extracts the optical flow information inside the foreign object area and calculates the similarity between its direction and the running direction of the conveyor belt. Combined with the overall consistency ratio, it makes a judgment, effectively distinguishing between truly accumulated static foreign objects and dynamic interference targets that have been temporarily stationary, thus improving the system's ability to accurately identify abnormal states in complex motion scenarios.

[0101] The indicator generation unit includes:

[0102] Misalignment azimuth calculation subunit: Used to receive the left and right edge offset values ​​of the conveyor belt output by the edge offset analysis unit, and calculate the misalignment azimuth of the conveyor belt based on the offset difference between the left and right sides. The calculation formula is as follows:

[0103] ,in, These represent the average lateral offset of the right and left edges, respectively. The height range of the longitudinal projection of the conveyor belt in the image; This represents the arctangent function, used to calculate the angular offset corresponding to the edge offset difference;

[0104] Foreign Object Coordinate Output Subunit: Used to receive the center position coordinates of the foreign object output by the foreign object positioning unit. And output it as the spatial location parameter of the foreign object in the structured detection results;

[0105] Confidence assessment subunit: Receives the output from the optical flow consistency verification unit, calculates the percentage of pixels whose optical flow direction deviates from the conveyor belt direction within the foreign object region, and infers the confidence level of the foreign object's stationary state based on this percentage. The calculation formula is as follows: ,in, This represents the confidence level that the foreign object in the current frame is in a state of accumulation, and the range is [value missing]. ; The proportion of pixels whose optical flow vector direction is inconsistent with the conveyor belt direction within the foreign object region; The threshold ratio for determining the accumulation state set by the system; This indicates taking the smaller of the two values ​​in parentheses, ensuring the confidence level does not exceed 1; pixel ratio with inconsistent optical flow direction. The expression is: ,in, This represents the total number of pixels within the foreign object accumulation area. For the first The cosine value of the consistency between the optical flow direction of each pixel and the direction of the conveyor belt; The set directional consistency threshold; The function is an indicative function, which takes the value of 1 when the condition in parentheses is met, and 0 otherwise. The above sub-units can provide the system with three types of clear and quantifiable detection indicators by fusing the deviation azimuth angle obtained by the edge offset difference, the directly output foreign object coordinates, and the static state confidence level calculated based on the optical flow consistency ratio. This enables the conveyor belt anomaly detection to have a clear directionality, positioning and confidence judgment basis, and significantly improves the accuracy and interpretability of anomaly identification.

[0106] Decision output module: Used to receive deviation azimuth angle, foreign object coordinates and confidence level, and generate deviation level, foreign object type and alarm command according to preset threshold;

[0107] The decision output module includes a deviation determination subunit, a foreign object classification subunit, and an alarm generation subunit; wherein:

[0108] Deviation determination subunit: Used to receive the deviation azimuth angle and compare it with the set deviation angle level threshold set, and output the corresponding deviation level according to the interval of the absolute value of the angle, including four categories: normal, slight deviation, moderate deviation and severe deviation.

[0109] Foreign object classification subunit: Used to receive the coordinates and confidence level of foreign objects, and classify them according to the confidence level and a set threshold. The comparison is performed. If the confidence level is higher than the threshold, it is determined to be a real foreign object accumulation; otherwise, it is determined to be a moving interference object, and the category of the foreign object region is determined according to the image location of the foreign object coordinates.

[0110] The process of determining the category of foreign object region is based on the system's preset image space division rules and the coordinates of the foreign object's center. Determine the functional region to which a location belongs in an image, using the following steps:

[0111] Step 1: Divide the horizontal dimension of the input image into three regions according to the conveyor belt structure:

[0112] Left side area ;

[0113] Central region ;

[0114] right side area ;in, The total width of the image; Let the boundary point of the region be set as follows: and It can be adjusted according to the actual width of the conveyor belt or the working conditions;

[0115] Step 2: The center coordinates extracted by the foreign object positioning unit are... ,in It indicates the horizontal position in the image and is the main basis for region identification;

[0116] Step 3: According to To determine the category of a region, the specific region classification rules are as follows:

[0117] like Foreign object area on the left;

[0118] like Central foreign object area;

[0119] like Foreign object area on the right;

[0120] Finally, the results are output as structured region labels.

[0121] Alarm generation subunit: Used to comprehensively judge the deviation level and foreign object type, and generate corresponding alarm commands according to preset response rules; when the deviation level is moderate or severe, or the foreign object type is actual accumulation, it outputs an alarm command data packet containing alarm type, alarm area and handling suggestions; by combining deviation angle, foreign object confidence and spatial position parameters with preset thresholds and rules for judgment, and outputting structured alarm information, it can realize multi-dimensional real-time response to abnormal conditions of the conveyor belt, which helps the system achieve efficient and accurate linkage control and maintenance decision-making.

[0122] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0123] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A video detection system for conveyor belt misalignment and foreign object accumulation, characterized in that, It includes a video acquisition module, an adaptive illumination compensation module, a multi-scale feature fusion module, a spatiotemporal consistency analysis module, and a decision output module; among which: Video acquisition module: used to acquire the raw video stream of the conveyor belt running area and output a time-series image sequence; Adaptive illumination compensation module: used to receive time-series image sequences and generate illumination-equalized images through dynamic background separation and adaptive gamma correction; Multi-scale feature fusion module: used to receive the illumination equalization image, extract and generate edge gradient feature map, texture structure feature map and motion optical flow feature map; Spatiotemporal consistency analysis module: Constructs a dynamic baseline based on edge gradient feature map and calculates the conveyor belt edge offset; identifies foreign object accumulation areas based on texture structure feature map and locates their coordinates; verifies the consistency between foreign object displacement and conveyor belt running direction by combining motion optical flow feature map; and outputs deviation azimuth angle, foreign object coordinates and confidence level. Decision output module: Used to receive deviation azimuth angle, foreign object coordinates and confidence level, and generate deviation level, foreign object type and alarm command according to preset threshold.

2. The video detection system for conveyor belt misalignment and foreign object accumulation according to claim 1, characterized in that, The video acquisition module includes a lens control unit, an image sensing unit, and a synchronization control unit; wherein: Lens control unit: used to adjust the shooting angle and zoom level, so that the camera equipment can cover the entire conveyor belt running area; Image sensing unit: integrates an image sensor component to receive visual information captured by the lens and convert it into a series of frame images; Synchronization control unit: Used to set and synchronize the time interval for image acquisition, ensuring a fixed time interval between adjacent frames, thereby forming a continuous time sequence of images.

3. The video detection system for conveyor belt misalignment and foreign object accumulation according to claim 1, characterized in that, The adaptive illumination compensation module includes a background modeling unit, a foreground extraction unit, a gamma estimation unit, and an image reconstruction unit; wherein: Background modeling unit: used to perform sliding window averaging on the received time-series image sequence, calculate the background brightness reference value of each pixel position based on the past N frames of images, and establish a static background model; The foreground extraction unit is used to perform pixel-by-pixel difference between the current frame image and the background model, and to mark the pixel regions with differences exceeding a set threshold as foreground regions, thereby achieving dynamic background separation. Gamma estimation unit: used to calculate the optimal gamma value based on the global luminance histogram distribution of the current frame image. This makes the image grayscale range within the range Internal equilibrium distribution; Image reconstruction unit: used to perform optimal gamma-value-based processing on the foreground region. The nonlinear gamma transform is used to keep the background region unchanged, ultimately generating an image with equal illumination.

4. The video detection system for conveyor belt misalignment and foreign object accumulation according to claim 1, characterized in that, The multi-scale feature fusion module includes an edge feature extraction unit, a texture feature extraction unit, an optical flow feature extraction unit, and a feature fusion unit; wherein: Edge feature extraction unit: used to perform multi-scale edge detection on the image with equal illumination. By setting multiple Sobel operators with different convolution kernel sizes, it extracts the horizontal and vertical grayscale changes of the image respectively, highlighting the position and intensity of the structural edges, and forming an edge gradient feature map. Texture feature extraction unit: Based on the local binary mode method, extract the gray-level structure of each pixel neighborhood in the illumination-equalized image, generate binary code by comparing the gray-level relationship between the center pixel and the surrounding pixels, divide the image region, and statistically analyze the LBP histogram at the region level to form a texture structure feature map. Optical flow feature extraction unit: used to perform optical flow intensity calculation on two adjacent frames of equally lit images, obtain the motion vector of each pixel in the image between two consecutive frames, thereby characterizing the motion trend of local areas of the image and forming a motion optical flow feature map; Feature fusion unit: It is used to align edge gradient feature maps, texture structure feature maps and motion optical flow feature maps according to their spatial positions, fuse them into a unified multi-scale feature tensor, and perform normalization processing after fusion to ensure that different feature scales are consistent.

5. The video detection system for conveyor belt misalignment and foreign object accumulation according to claim 1, characterized in that, The spatiotemporal consistency analysis module includes an edge offset analysis unit, a foreign object localization unit, an optical flow consistency verification unit, and an index generation unit; wherein: Edge offset analysis unit: used to receive edge gradient feature map, extract left and right edge points of conveyor belt, construct dynamic baseline of current frame, compare with set reference line, and calculate offset of both sides of conveyor belt; Foreign object positioning unit: used to receive texture structure feature map, extract the area that is inconsistent with the normal surface texture of the conveyor belt as the foreign object accumulation area, and determine the center coordinates of the foreign object accumulation area; Optical flow consistency verification unit: used to receive the motion optical flow feature map, analyze whether the optical flow vector direction in the foreign object area is consistent with the conveyor belt movement direction, and determine whether the foreign object is in a static accumulation state; The index generation unit is used to generate the deviation azimuth angle based on the results of the edge offset analysis unit, output the foreign object coordinates based on the results of the foreign object positioning unit, and calculate the confidence level based on the results of the optical flow consistency verification unit as the final detection output.

6. The video detection system for conveyor belt misalignment and foreign object accumulation according to claim 5, characterized in that, The edge offset analysis unit includes: Edge point extraction subunit: used to scan the edge gradient feature map column by column in the vertical direction, and extract the leftmost and rightmost pixel coordinates of each column whose gradient magnitude is greater than a set threshold, which are respectively used as the left edge point and right edge point of the conveyor belt in the current frame; Baseline construction sub-unit: used to fit all left edge points to the left dynamic edge curve, fit all right edge points to the right dynamic edge curve, and use the midline of the two edge curves as the dynamic baseline of the current frame. Offset calculation subunit: Used to compare the dynamic baseline of the current frame with the preset ideal conveyor belt centerline point by point in the corresponding column position, calculate the lateral offset value of each column, and take the average value as the overall lateral offset.

7. The video detection system for conveyor belt misalignment and foreign object accumulation according to claim 5, characterized in that, The foreign object positioning unit includes: Texture anomaly detection subunit: It is used to divide the received texture structure feature map into local regions, calculate the local binary pattern histogram of each region, and compare it with the preset normal texture template of the conveyor belt. When the similarity is lower than the set threshold, the region is determined to be a texture anomaly region. Region segmentation subunit: used to perform connectivity analysis on all adjacent abnormal pixel regions, merge them to form continuous foreign object accumulation candidate regions, remove noise blocks with an area lower than the minimum effective region threshold, and extract the bounding box of the main foreign object region. Center coordinate extraction sub-unit: used to calculate the center point of the bounding rectangle of the main foreign object region, as the positioning coordinates of the main foreign object region.

8. The video detection system for conveyor belt misalignment and foreign object accumulation according to claim 5, characterized in that, The optical flow consistency verification unit includes: Optical flow vector extraction subunit: used to extract two-dimensional optical flow vectors of all pixel positions inside the foreign object accumulation region from the motion optical flow feature map, forming a set of local motion vectors of the foreign object accumulation region; Directional similarity calculation subunit: used to calculate the directional cosine of each optical flow vector and the unit vector of the conveyor belt running direction to obtain the directional similarity of each pixel; The static judgment subunit is used to count the proportion of pixels in the foreign object area that meet the orientation consistency threshold condition. If the proportion exceeds the set threshold, the foreign object is judged to be in a static accumulation state; otherwise, it is judged to be a moving interference object.

9. A video detection system for conveyor belt misalignment and foreign object accumulation according to claim 5, characterized in that, The indicator generation unit includes: Misalignment azimuth calculation subunit: Used to receive the left and right edge offset values ​​of the conveyor belt output by the edge offset analysis unit, and calculate the misalignment azimuth of the conveyor belt based on the offset difference between the left and right sides. ; Foreign object coordinate output subunit: Used to receive the center position coordinates of the foreign object output by the foreign object positioning unit and output them as the spatial position parameters of the foreign object in the structured detection results; Confidence assessment subunit: Receives the output from the optical flow consistency verification unit, calculates the percentage of pixels whose optical flow direction deviates from the conveyor belt direction within the foreign object region, and infers the confidence level of the foreign object's stationary state based on this percentage. .

10. A video detection system for conveyor belt misalignment and foreign object accumulation according to claim 1, characterized in that, The decision output module includes a deviation determination subunit, a foreign object classification subunit, and an alarm generation subunit; wherein: Deviation determination subunit: Used to receive the deviation azimuth angle and compare it with the set deviation angle level threshold set, and output the corresponding deviation level according to the interval of the absolute value of the angle, including four categories: normal, slight deviation, moderate deviation and severe deviation. Foreign object classification subunit: used to receive the coordinates and confidence level of the foreign object, and classify it according to the confidence level and a set threshold. The comparison is performed. If the confidence level is higher than the threshold, it is determined to be a real foreign object accumulation; otherwise, it is determined to be a moving interference object, and the category of the foreign object region is determined according to the image location of the foreign object coordinates. Alarm generation subunit: Used to combine the deviation level and foreign object type judgment results, and generate corresponding alarm instructions according to preset response rules; when the deviation level is moderate or severe, or the foreign object type is actual accumulation, it outputs an alarm instruction data packet containing alarm type, alarm area and handling suggestions.

Images