Video detection system for conveying belt deviation and foreign matter accumulation

Abstract

The invention relates to the technical field of machine vision detection, in particular to a video detection system for conveyor belt deviation and foreign matter accumulation, which comprises a video acquisition module, a self-adaptive illumination compensation module, a multi-scale feature fusion module, a space-time consistency analysis module and a decision output module. Wherein the video acquisition module is used for acquiring an original video stream of an operation area of a conveying belt; the self-adaptive illumination compensation module is used for generating an illumination equalization image through dynamic background separation and self-adaptive gamma correction; the multi-scale feature fusion module is used for extracting and generating a plurality of feature maps; the space-time consistency analysis module is used for outputting a deviation azimuth angle, a foreign matter coordinate and a confidence coefficient; and the decision output module is used for generating a deviation grade, a foreign matter type and an alarm instruction according to a preset threshold value. According to the invention, high-precision identification and classification judgment of the deviation of the conveying belt and the accumulation state of the foreign matters are realized through cooperation of multiple modules, and the intelligent level and the practical response capability of the detection system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision detection, and in particular to a video detection system for conveyor belt deviation and foreign matter accumulation. Background Art

[0002] With the continuous improvement of the degree of industrial automation, conveyor belts, as important equipment for material transportation on production lines, have a direct impact on overall production efficiency and safety due to their operational stability. During actual operation, conveyor belts are prone to abnormal conditions such as deviation and accumulation of foreign matter. If these conditions are not detected and handled in a timely manner, they will not only cause equipment wear and material loss, but may even lead to safety accidents. Traditional conveyor belt anomaly detection mostly relies on mechanical limit devices or single-point sensors, which have low detection accuracy, delayed response, and difficulty in accurately judging the deviation direction, foreign matter location, and severity, and cannot meet the needs of modern high-precision and intelligent detection.

[0003] While existing video surveillance methods can capture image information, they often rely on manual observation or simple analysis of static images. They lack effective image enhancement, feature fusion, and spatiotemporal information analysis mechanisms, making it difficult to accurately identify conveyor belt deviation trends and foreign object accumulation under varying lighting conditions, background interference, or dynamic object interference. Therefore, a video detection system for conveyor belt deviation and foreign object accumulation is urgently needed to address these issues. Summary of the Invention

[0004] Based on the above objectives, the present invention provides a video detection system for conveyor belt deviation and foreign matter accumulation.

[0005] A video detection system for conveyor belt deviation and foreign matter accumulation includes a video acquisition module, an adaptive illumination compensation module, a multi-scale feature fusion module, a spatiotemporal consistency analysis module, and a decision output module; wherein:

[0006] Video acquisition module: used to obtain the original video stream of the conveyor belt running area and output the time-series image sequence;

[0007] Adaptive illumination compensation module: used to receive time-sequential image sequences and generate illumination-balanced images through dynamic background separation and adaptive gamma correction;

[0008] Multi-scale feature fusion module: used to receive the illumination-balanced image, extract and generate edge gradient feature maps, texture structure feature maps and motion optical flow feature maps;

[0009] Spatiotemporal consistency analysis module: This module constructs a dynamic baseline based on the edge gradient feature map and calculates the conveyor belt edge offset. It also identifies foreign object accumulation areas and locates their coordinates based on the texture structure feature map. It then uses the motion optical flow feature map to verify the consistency of foreign object displacement with the conveyor belt's running direction, and then outputs the deviation azimuth, foreign object coordinates, and confidence level.

[0010] The decision output module is configured to receive the deviation azimuth, the foreign matter coordinates and the confidence, and generate a deviation level, a foreign matter type and an alarm instruction according to a preset threshold.

[0011] Optionally, the video acquisition module comprises a lens control unit, an image sensing unit and a synchronization control unit, wherein:

[0012] The lens control unit is configured to adjust the acquisition angle and the zoom ratio, so that the camera device can cover the entire running area of the conveying belt.

[0013] The image sensing unit comprises an integrated image sensor component, configured to receive the visual information collected by the lens and convert it into continuous frame images.

[0014] The synchronization control unit is configured to set and synchronize the time interval of image acquisition, so as to ensure a fixed time interval between adjacent frames and form a continuous time sequence image sequence.

[0015] Optionally, the adaptive illumination compensation module comprises a background modeling unit, a foreground extraction unit, a gamma estimation unit and an image reconstruction unit, wherein:

[0016] The background modeling unit is configured to perform a sliding window average method on the received time sequence image sequence, calculate the background brightness reference value of each pixel position based on the past N frame images, and establish a static background model.

[0017] The foreground extraction unit is configured to perform pixel-by-pixel difference between the current frame image and the background model, mark the pixel regions with a difference value exceeding a set threshold as foreground regions, thereby realizing dynamic background separation.

[0018] The gamma estimation unit is configured to calculate the optimal gamma value based on the global brightness histogram distribution of the current frame image. so that the image gray scale range is evenly distributed in the interval .

[0019] The image reconstruction unit is configured to perform nonlinear gamma transformation on the foreground region based on the optimal gamma value , and keep the background region unchanged, thereby finally generating an illumination balanced image.

[0020] Optionally, the multi-scale feature fusion module comprises an edge feature extraction unit, a texture feature extraction unit, an optical flow feature extraction unit and a feature fusion unit, wherein:

[0021] The edge feature extraction unit is configured to perform multi-scale edge detection on the illumination balanced image, extract the horizontal and vertical gray scale changes of the image by setting a plurality of Sobel operators with different kernel sizes, highlight the position and intensity of the structural edges, and form an edge gradient feature map.

[0022] texture feature extraction unit: based on the local binary pattern method, the gray structure of the pixel neighborhood in the light balanced image is extracted, the binary code is generated by comparing the gray size relationship of the center pixel and the surrounding pixels, and then the image region is divided, the LBP histogram of the region level is counted, and the texture structure feature map is formed;

[0023] optical flow feature extraction unit: used for performing optical flow intensity calculation on adjacent two frames of light balanced images, obtaining the motion vector of each pixel point in the image between two consecutive frames, thereby representing the motion trend of the local region of the image, and constituting the motion optical flow feature map;

[0024] feature fusion unit: used for aligning the edge gradient feature map, the texture structure feature map and the motion optical flow feature map according to the spatial position, fusing them into a unified multi-scale feature tensor, and performing normalization processing after fusion to ensure that different feature levels are consistent.

[0025] Optionally, the spatiotemporal consistency analysis module includes an edge offset analysis unit, a foreign matter positioning unit, an optical flow consistency verification unit and an index generation unit; wherein:

[0026] edge offset analysis unit: used for receiving the edge gradient feature map, extracting the left and right edge points of the conveying belt, constructing the dynamic reference line of the current frame, and comparing with the set reference line to calculate the offset of the two sides of the conveying belt;

[0027] foreign matter positioning unit: used for receiving the texture structure feature map, extracting the region inconsistent with the normal surface texture of the conveying belt as the foreign matter accumulation region, and determining the center coordinates of the foreign matter accumulation region;

[0028] optical flow consistency verification unit: used for receiving the motion optical flow feature map, analyzing whether the optical flow vector direction in the foreign matter region is consistent with the motion direction of the conveying belt, and judging whether the foreign matter is in a static accumulation state;

[0029] index generation unit: used for generating the running deviation azimuth angle according to the result of the edge offset analysis unit, outputting the foreign matter coordinates according to the result of the foreign matter positioning unit, and calculating the confidence degree based on the result of the optical flow consistency verification unit as the final detection output.

[0030] Optionally, the edge offset analysis unit includes:

[0031] edge point extraction subunit: used for vertically scanning each column of the edge gradient feature map, extracting the leftmost and rightmost pixel coordinates in each column with a gradient amplitude greater than a set threshold, and taking them as the left edge point and the right edge point of the conveying belt of the current frame, respectively;

[0032] The baseline construction subunit is configured to fit all left edge points as a left dynamic edge curve, fit all right edge points as a right dynamic edge curve, and take the central axis of the two edge curves as a dynamic baseline of the current frame;

[0033] The offset calculation subunit is configured to compare the dynamic baseline of the current frame with a preset ideal center line of the conveying belt point by point in a corresponding column position, calculate a transverse offset value of each column, and take an average value as an overall transverse offset.

[0034] Optionally, the foreign matter positioning unit comprises:

[0035] The texture anomaly detection subunit is configured to divide a received texture structure feature map into local regions, calculate a local binary pattern histogram of each divided region, and compare the local binary pattern histogram with a preset normal texture template of the conveying belt in similarity, and determine a region as a texture anomaly region when the similarity is lower than a set threshold value.

[0036] The region segmentation subunit is configured to perform connectivity analysis on all adjacent abnormal pixel regions, merge to form a continuous foreign matter accumulation candidate region, and remove a noise block with an area lower than a minimum effective region threshold value, and extract a bounding box of the main foreign matter region.

[0037] The center coordinate extraction subunit is configured to calculate a center point of an external rectangle of the main foreign matter region as a positioning coordinate of the main foreign matter region.

[0038] Optionally, the optical flow consistency verification unit comprises:

[0039] The optical flow vector extraction subunit is configured to extract two-dimensional optical flow vectors of all pixel positions in the foreign matter accumulation region from a motion optical flow feature map to form a local motion vector set of the foreign matter accumulation region.

[0040] The direction similarity calculation subunit is configured to calculate a direction cosine between each optical flow vector and a unit vector of a running direction of the conveying belt to obtain a direction similarity of each pixel.

[0041] The static judgment subunit is configured to count a proportion of pixels in the foreign matter region that satisfy a direction consistency threshold value, and if the proportion exceeds a set threshold value, it is judged that the foreign matter is in a static accumulation state, and otherwise, it is judged as a belt motion interference object.

[0042] Optionally, the index generation unit comprises:

[0043] The runout azimuth angle calculation subunit is configured to receive left and right edge offset values output by the edge offset analysis unit, and calculate a runout azimuth angle of the conveying belt according to the offset difference between the left and right sides. ;

[0044] Foreign matter coordinate output subunit: for receiving the foreign matter center position coordinates output by the foreign matter positioning unit, and outputting the same as the foreign matter spatial position parameters in the structured detection result;

[0045] Confidence evaluation subunit: for receiving the output result of the optical flow consistency verification unit, calculating the pixel proportion of the foreign matter region in which the optical flow direction deviates from the conveying belt direction, and inversely deducing the confidence of the foreign matter static state based on the pixel proportion .

[0046] Optionally, the decision output module comprises a deviation determination subunit, a foreign matter classification subunit and an alarm generation subunit; wherein:

[0047] Deviation determination subunit: for receiving the deviation azimuth angle and comparing the same with a set of deviation angle level threshold values, outputting the corresponding deviation level according to the interval in which the angle absolute value is located, including four categories of normal, mild deviation, moderate deviation and severe deviation;

[0048] Foreign matter classification subunit: for receiving the foreign matter coordinates and confidence, comparing the confidence with a set threshold value , if the confidence is higher than the threshold value, it is determined as a real accumulated foreign matter; otherwise, it is determined as a disturbance moving with the belt, and the region category to which the foreign matter belongs is determined according to the image position of the foreign matter coordinates;

[0049] Alarm generation subunit: for comprehensively judging the deviation level and the foreign matter type, generating the corresponding alarm instruction according to the preset response rule; when the deviation level is moderate or severe, or the foreign matter type is real accumulation, an alarm instruction data packet containing the alarm type, alarm region and processing suggestion is output.

[0050] The beneficial effects of the present application are:

[0051] The present application realizes the complete process of image enhancement, key feature extraction and abnormal state recognition of the conveying belt running area by constructing a modularized video detection system including video acquisition, adaptive illumination compensation, multi-scale feature fusion, spatio-temporal consistency analysis and decision output; the image sequence and feature map are sequentially transmitted between the modules, forming a closed loop structure from video input to quantitative detection output, which has stable data link and processing logic, and effectively improves the adaptability and robustness in complex industrial field environment.

[0052] The present application realizes the collaborative analysis and structured output of multiple indexes such as deviation azimuth angle, foreign matter coordinates and confidence by introducing dynamic reference line construction, texture anomaly recognition, optical flow direction verification and threshold decision mechanism, the system can accurately determine the deviation level, foreign matter type and its spatial position, and generate alarm instructions in parallel, which significantly improves the intelligent level and response efficiency of the conveying belt abnormal detection. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the present application or the prior art, the drawings required to be used in the following embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only a part of the present application, and other drawings can also be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0054] Fig. 1 A schematic diagram of a video detection system according to an embodiment of the present application;

[0055] Fig. 2 A schematic diagram of a spatio-temporal consistency analysis module according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] The present application will be described in detail below with reference to the drawings and specific embodiments. It should be noted that in order to make the embodiments more detailed, the following embodiments are the best, preferred embodiments, and other alternative embodiments can also be implemented by those skilled in the art without creative effort; and the drawings are only used to more specifically describe the embodiments, and are not intended to specifically limit the present application.

[0057] It should be noted that in the specification, "one embodiment", "embodiment", "exemplary embodiment", "some embodiments" and the like indicate that the described embodiments can include a specific feature, structure or property, but not necessarily every embodiment includes the specific feature, structure or property. In addition, when a specific feature, structure or property is described in combination with an embodiment, it should be within the knowledge of those skilled in the art to realize this feature, structure or property in combination with other embodiments (whether or not explicitly described).

[0058] Generally, the terms can be understood at least in part from the context of their use. For example, depending on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular or can be used to describe combinations of features, structures, or characteristics, whether large or small, whether related or unrelated to each other. In addition, the term "based on" can be understood as not necessarily intending to convey a set of exclusive factors, but can instead, depending at least in part on the context, allow the existence of other factors not necessarily explicitly described.

[0059] As shown in Figs. 1-2 A video detection system for belt deviation and foreign object accumulation, comprising a video acquisition module, an adaptive illumination compensation module, a multi-scale feature fusion module, a spatio-temporal consistency analysis module and a decision output module; wherein:

[0060] Video acquisition module: used for acquiring the original video stream of the running area of the belt, and outputting a time sequence image sequence;

[0061] The video acquisition module includes a lens control unit, an image sensing unit, and a synchronization control unit; wherein:

[0062] Lens control unit: used to adjust the acquisition angle of view and zoom ratio so that the camera can cover the entire conveyor belt operation area;

[0063] Image perception unit: an integrated image sensor component used to receive visual information collected by the lens and convert it into continuous frame images;

[0064] Synchronization control unit: used to set and synchronize the time interval for image acquisition, ensuring that there is a fixed time interval between adjacent frames, thereby forming a continuous time-series image sequence; by setting the lens control, image perception and synchronization control sub-units, the video acquisition module can stably obtain continuous video data of the conveyor belt operation area under different working conditions, ensuring that the subsequent processing module is based on a clear and time-consistent image sequence for analysis, which is conducive to improving the stability and robustness of the overall detection system.

[0065] Adaptive illumination compensation module: used to receive time-sequential image sequences and generate illumination-balanced images through dynamic background separation and adaptive gamma correction;

[0066] The adaptive illumination compensation module includes a background modeling unit, a foreground extraction unit, a gamma estimation unit, and an image reconstruction unit; wherein:

[0067] Background modeling unit: used to perform sliding window averaging on the received time-series image sequence, calculate the background brightness reference value of each pixel position based on the past N frames of image, and establish a static background model;

[0068] The calculation formula for the background brightness reference value is: ,in, Indicates the background model at pixel position The brightness value at Indicates the Frame image in pixels Gray value at ; Indicates the number of frames involved in the calculation in the sliding window;

[0069] The foreground extraction unit is used to perform pixel-by-pixel difference between the current frame image and the background model, and mark the pixel area where the difference exceeds the set threshold as the foreground area, thereby achieving dynamic background separation;

[0070] The above difference calculation and the generation formula of the binary foreground mask are:

[0071] ;

[0072] in, represents the value of the foreground mask at pixel , 1 for foreground and 0 for background; represents the gray value of the current frame image at pixel ; represents the set luminance difference threshold;

[0073] Gamma estimation unit: for calculating the optimal gamma value according to the global luminance histogram distribution of the current frame image , so that the image gray range is evenly distributed in the interval , wherein the optimal gamma value is calculated by the following formula: , wherein, represents the average gray value of the current frame image; represents the logarithmic operation with natural logarithm as base; the value 0.5 represents the gray point mapping value in the target; the constant 255 represents the maximum gray level of an 8-bit image;

[0074] Image reconstruction unit: for performing nonlinear gamma transformation on the foreground area based on the optimal gamma value , keeping the background area unchanged, and finally generating an illumination balanced image to improve the visual quality and feature stability of the foreground area; the above unit realizes robust background separation by introducing a sliding average model to construct a background brightness reference and cooperating with threshold difference extraction of dynamic foreground; further combining image brightness distribution to adaptively adjust the gamma parameter, the image still has stable and balanced visual features under complex lighting environment, effectively improving the subsequent feature extraction and detection accuracy.

[0075] Multi-scale feature fusion module: for receiving the illumination balanced image, extracting and generating edge gradient feature map, texture structure feature map and motion optical flow feature map;

[0076] The multi-scale feature fusion module includes an edge feature extraction unit, a texture feature extraction unit, an optical flow feature extraction unit and a feature fusion unit; wherein:

[0077] Edge feature extraction unit: for performing multi-scale edge detection on the illumination balanced image, and extracting the horizontal and vertical gray changes of the image by setting a plurality of Sobel operators with different kernel sizes, highlighting the position and intensity of the structural edge to form an edge gradient feature map; the calculation formula of the edge gradient amplitude is as follows: , wherein, is the gradient amplitude of pixel in the edge gradient map; is the horizontal gradient component at pixel , obtained by horizontal convolution of the Sobel operator; is the vertical gradient component at pixel The vertical gradient component of the Sobel operator is obtained by vertical convolution; denotes a square root operation;

[0078] The texture feature extraction unit: based on the local binary pattern method, the gray structure of the neighborhood of each pixel in the light balanced image is extracted, a binary code is generated by comparing the gray size relationship between the center pixel and the surrounding pixels, and then the image region is divided, the LBP histogram of the region level is counted, and a texture structure feature map is formed;

[0079] The optical flow feature extraction unit: used for performing optical flow intensity calculation on adjacent two frames of light balanced images, obtaining the motion vector of each pixel point in the image between two consecutive frames, thereby representing the motion trend of the local region of the image, and constituting a motion optical flow feature map. The calculation formula of the optical flow intensity is as follows:

[0080] , wherein, is the optical flow intensity of the pixel ; is the horizontal flow displacement component of the pixel ; is the vertical flow displacement component of the pixel ;

[0081] The feature fusion unit: used for aligning the edge gradient feature map, the texture structure feature map and the motion optical flow feature map according to the spatial position, fusing them into a unified multi-scale feature tensor, and performing normalization processing after fusion to ensure that the different feature levels are consistent, thereby providing an input tensor with good compatibility for the subsequent module. Through the edge feature extraction unit, the image boundary change is strengthened, the texture feature extraction unit captures the regional gray structure, and the optical flow feature extraction unit reveals the pixel-level dynamic displacement trend. The three types of features are uniformly fused to form a high-dimensional multi-scale information tensor, thereby enhancing the multi-angle expression ability of the system to the off-track boundary and the shape change of the foreign matter, and improving the accuracy and stability of the detection and discrimination.

[0082] The spatiotemporal consistency analysis module: based on the edge gradient feature map, a dynamic reference line is constructed and the edge offset of the conveying belt is calculated, based on the texture structure feature map, the foreign matter accumulation area is identified and its coordinates are located, combined with the motion optical flow feature map, the consistency of the foreign matter displacement and the running direction of the conveying belt is verified, and then the off-track azimuth angle, the foreign matter coordinates and the confidence are outputted;

[0083] The spatiotemporal consistency analysis module includes an edge offset analysis unit, a foreign matter positioning unit, an optical flow consistency verification unit and an index generation unit; wherein:

[0084] Edge offset analysis unit: used for receiving edge gradient feature map, constructing dynamic reference line of current frame by extracting left and right edge points of conveyor belt, and comparing with set reference line to calculate offset of both sides of conveyor belt;

[0085] Foreign matter positioning unit: used for receiving texture structure feature map, extracting area inconsistent with normal surface texture of conveyor belt as foreign matter accumulation area, and determining center coordinates of foreign matter accumulation area;

[0086] Optical flow consistency verification unit: used for receiving motion optical flow feature map, analyzing whether optical flow vector direction in foreign matter area is consistent with motion direction of conveyor belt to judge whether foreign matter is in static accumulation state;

[0087] Index generation unit: used for generating deviation azimuth angle according to result of edge offset analysis unit, outputting foreign matter coordinates according to result of foreign matter positioning unit, and calculating confidence degree based on result of optical flow consistency verification unit as final detection output; the above units realize accurate extraction of conveyor belt deviation and foreign matter accumulation state through joint analysis of edge, texture and optical flow features, and generate structured detection index to provide reliable basis for subsequent decision and alarm of system.

[0088] The edge offset analysis unit comprises:

[0089] Edge point extraction sub-unit: used for performing vertical direction column-by-column scanning on edge gradient feature map, extracting leftmost and rightmost pixel coordinates with gradient amplitude greater than set threshold in each column as left edge point and right edge point of conveyor belt of current frame respectively;

[0090] Reference line construction sub-unit: used for fitting all left edge points into left dynamic edge curve and fitting all right edge points into right dynamic edge curve, and taking middle axis line of two edge curves as dynamic reference line of current frame;

[0091] Offset calculation sub-unit: used for performing point-by-point comparison of corresponding column positions between dynamic reference line of current frame and preset ideal conveyor belt center line, calculating transverse offset value of each column, and taking average value as overall transverse offset, and the calculation formula is as follows: wherein, is average transverse offset of current frame of conveyor belt; is column number of image; is center position of dynamic reference line of current frame of first column; is center position of preset ideal reference line of first column; represents center position of preset ideal reference line of first column; represents center position of preset ideal reference line of first column; represents center position of preset ideal reference line of first column; Column summation: By setting the sub-unit to realize the stable extraction of edge points, high-precision fitting of dynamic boundary and quantitative analysis of inter-frame offset, the two sides of the conveying belt can be accurately extracted under complex conditions such as local occlusion and edge blur, and the lateral offset amount index can be output, which provides a key basis for the judgment of the deviation trend.

[0092] The foreign matter positioning unit comprises:

[0093] Texture anomaly detection sub-unit: for local area division of the received texture structure feature map, calculating the local binary pattern histogram of each division area, and comparing the similarity with the preset normal texture template of the conveying belt, when the similarity is lower than the set threshold, the area is determined as a texture anomaly area;

[0094] Region segmentation sub-unit: for connectedness analysis of all adjacent abnormal pixel regions, merging to form continuous foreign matter accumulation candidate regions, and eliminating noise blocks with an area lower than the minimum effective region threshold, and extracting the bounding box of the main foreign matter region;

[0095] Center coordinate extraction sub-unit: for calculating the center point of the bounding rectangle of the main foreign matter region as the positioning coordinate of the main foreign matter region, and the calculation formula is as follows: , wherein, is the center coordinate of the foreign matter accumulation region; is the minimum and maximum values of the foreign matter region bounding box in the horizontal direction; is the minimum and maximum values of the foreign matter region bounding box in the vertical direction; all boundary values are calculated based on the pixel position extracted in the image coordinate system; the above sub-unit effectively realizes the automatic recognition of the non-normal surface structure region by similarity detection of local texture features and normal templates combined with connected region segmentation method, and at the same time, the foreign matter center position is calculated combined with the bounding box, so that the system can accurately complete the spatial positioning and region calibration of foreign matter accumulation without relying on color or shape.

[0096] Optical flow consistency verification unit comprises:

[0097] Optical flow vector extraction sub-unit: for extracting two-dimensional optical flow vectors of all pixel positions inside the foreign matter accumulation region from the motion optical flow feature map to form a local motion vector set of the foreign matter accumulation region;

[0098] Direction similarity calculation sub-unit: for calculating the direction cosine of each optical flow vector and the unit vector of the conveying belt running direction to obtain the direction similarity of each pixel, and the calculation formula is as follows:

[0099] , wherein, is the direction similarity of the i-th pixel; ​The cosine value of the consistency between the optical flow direction of the pixel point and the conveyor belt direction is in the range of ; For the Two-dimensional optical flow vector of pixels; The unit vector of the standard motion direction set for the conveyor belt; the numerator is the vector dot product, and the denominator is the vector modulus product;

[0100] Stationary judgment subunit: used to count the foreign body areas that meet the direction consistency threshold condition ( ) pixel ratio, where is the directional consistency threshold. If the ratio exceeds the set threshold, the foreign object is judged to be in a static accumulation state; otherwise, it is judged to be a moving interference object. The above sub-unit extracts the optical flow information inside the foreign object area and calculates the similarity between its direction and the running direction of the conveyor belt, and combines the overall consistency ratio for judgment, effectively distinguishing between truly accumulated static foreign objects and short-stayed dynamic interference targets, thereby improving the system's ability to accurately identify abnormal states in complex motion scenarios.

[0101] The indicator generation unit includes:

[0102] Deviation angle calculation subunit: used to receive the left and right edge offset values ​​of the conveyor belt output by the edge offset analysis unit, and calculate the deviation angle of the conveyor belt based on the offset difference between the left and right sides. , and its calculation formula is:

[0103] ,in, represent the average lateral offset of the right and left edges, respectively; is the height range of the conveyor belt's longitudinal projection in the image; Represents the inverse tangent function, which is used to calculate the angle offset corresponding to the edge offset difference;

[0104] Foreign object coordinate output subunit: used to receive the foreign object center position coordinates output by the foreign object positioning unit , and output it as the foreign body spatial position parameter in the structured detection result;

[0105] Confidence evaluation subunit: used to receive the output results of the optical flow consistency verification unit, calculate the pixel ratio of the optical flow direction deviating from the conveyor belt direction in the foreign body area, and infer the confidence of the foreign body's static state based on the pixel ratio , and its calculation formula is: ,in, Indicates the confidence level that foreign matter in the current frame is in an accumulation state, ranging from ; is the ratio of pixels in the foreign matter area where the direction of the optical flow vector is inconsistent with the direction of the conveyor belt; The accumulation state determination threshold ratio set by the system; Indicates taking the smaller of the two values ​​in the brackets to ensure that the confidence level does not exceed 1; the ratio of pixels with inconsistent optical flow directions The expression is: ,in, is the total number of pixels in the foreign matter accumulation area; For the The cosine value of the consistency between the optical flow direction of each pixel and the conveyor belt direction; is the set direction consistency threshold; is an indicator function, which takes the value of 1 when the conditions in the brackets are met, and 0 otherwise. The above subunits obtain the deviation azimuth angle obtained by fusing the edge offset difference, the directly output foreign object coordinates, and the static state confidence calculated based on the optical flow consistency ratio, which can provide the system with three clear and quantitative detection indicators, so that the conveyor belt anomaly detection has a clear direction, positioning and confidence judgment basis, which significantly improves the accuracy and interpretability of anomaly recognition.

[0106] Decision output module: used to receive the deviation azimuth, foreign object coordinates and confidence level, and generate the deviation level, foreign object type and alarm instructions based on the preset threshold;

[0107] The decision output module includes a deviation determination subunit, a foreign object classification subunit, and an alarm generation subunit; wherein:

[0108] Deviation determination subunit: This unit receives the deviation angle and compares it with the set deviation angle threshold. It then outputs the corresponding deviation level based on the range of the angle's absolute value, including normal, mild deviation, moderate deviation, and severe deviation.

[0109] Foreign body classification subunit: used to receive foreign body coordinates and confidence, and classify foreign bodies according to confidence and set thresholds After comparison, if the confidence level is higher than the threshold, it is determined to be a real accumulated foreign object; otherwise, it is determined to be an interference object with movement, and the area category to which the foreign object belongs is determined based on the image position of the foreign object coordinates;

[0110] The foreign body area classification process is determined by the system's preset image space division rules, based on the foreign body center coordinates The position in the image is used to determine the functional area to which it belongs. The specific steps are as follows:

[0111] Step 1: Divide the horizontal dimension of the input image into three regions according to the conveyor belt structure:

[0112] Left area ;

[0113] Central region ;

[0114] right region ; wherein, is the total width of the image; is the region boundary point, set as and , which can be adjusted according to the actual width of the conveying belt or the working condition;

[0115] Step 2: The center coordinates extracted by the foreign matter positioning unit are , wherein represents the position in the horizontal direction of the image, which is the main basis for region judgment;

[0116] Step 3: According to , the category of the belonging region is judged, and the specific region classification rules are as follows:

[0117] If , it is a left foreign matter region;

[0118] If , it is a middle foreign matter region;

[0119] If , it is a right foreign matter region;

[0120] Finally, the result is marked as a structured region label output.

[0121] The alarm generation subunit is used to comprehensively judge the deviation level and the foreign matter type, generate corresponding alarm instructions according to the preset response rules, output alarm instruction data packets containing the alarm type, alarm region and processing suggestions when the deviation level is moderate or severe, or the foreign matter type is real accumulation, and output structured alarm information by combining the deviation angle, foreign matter confidence and spatial position parameters with the preset threshold and rules, so that multi-dimensional real-time response to the abnormal state of the conveying belt can be realized, which is helpful for the system to realize efficient and accurate linkage control and maintenance decision.

[0122] The present application encompasses any substitutions, modifications, equivalent methods and schemes made on the essence and scope of the present application. In order for the public to have a thorough understanding of the present application, specific details are described in the following preferred embodiments of the present application, and the present application can also be fully understood without the description of these details by those skilled in the art. In addition, in order to avoid unnecessary confusion to the essence of the present application, well-known methods, processes, procedures, elements and circuits, etc. are not described in detail.

[0123] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principle of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A video detection system for conveyor belt deviation and foreign matter accumulation, characterized in that: It includes video acquisition module, adaptive illumination compensation module, multi-scale feature fusion module, spatiotemporal consistency analysis module and decision output module; among them: Video acquisition module: used to obtain the original video stream of the conveyor belt running area and output the time-series image sequence; Adaptive illumination compensation module: used to receive time-sequential image sequences and generate illumination-balanced images through dynamic background separation and adaptive gamma correction; Multi-scale feature fusion module: used to receive the illumination-balanced image, extract and generate edge gradient feature maps, texture structure feature maps and motion optical flow feature maps; Spatiotemporal consistency analysis module: This module constructs a dynamic baseline based on the edge gradient feature map and calculates the conveyor belt edge offset. It also identifies foreign object accumulation areas and locates their coordinates based on the texture structure feature map. It then uses the motion optical flow feature map to verify the consistency of foreign object displacement with the conveyor belt's running direction, and then outputs the deviation azimuth, foreign object coordinates, and confidence level. Decision output module: used to receive the deviation azimuth, foreign object coordinates and confidence level, and generate the deviation level, foreign object type and alarm instructions based on the preset threshold.

2. A video detection system for conveyor belt deviation and foreign matter accumulation according to claim 1, characterized in that: The video acquisition module includes a lens control unit, an image sensing unit, and a synchronization control unit; wherein: Lens control unit: used to adjust the acquisition angle of view and zoom ratio so that the camera can cover the entire conveyor belt operation area; Image perception unit: an integrated image sensor component used to receive visual information collected by the lens and convert it into continuous frame images; Synchronization control unit: used to set and synchronize the time interval of image acquisition to ensure that there is a fixed time interval between adjacent frames, thereby forming a continuous time-series image sequence.

3. The video detection system for conveyor belt deviation and foreign matter accumulation according to claim 1 is characterized in that: The adaptive illumination compensation module includes a background modeling unit, a foreground extraction unit, a gamma estimation unit and an image reconstruction unit; wherein: Background modeling unit: used to perform sliding window averaging on the received time-series image sequence, calculate the background brightness reference value of each pixel position based on the past N frames of image, and establish a static background model; The foreground extraction unit is used to perform pixel-by-pixel difference between the current frame image and the background model, and mark the pixel area where the difference exceeds a set threshold as the foreground area, thereby achieving dynamic background separation; Gamma estimation unit: used to calculate the optimal gamma value based on the global brightness histogram distribution of the current frame image , so that the image grayscale range is in the interval Internal balanced distribution; Image reconstruction unit: used to perform the optimal gamma value on the foreground area The nonlinear gamma transform keeps the background area unchanged and finally generates a lighting balanced image.

4. The video detection system for conveyor belt deviation and foreign matter accumulation according to claim 1 is characterized in that: The multi-scale feature fusion module includes an edge feature extraction unit, a texture feature extraction unit, an optical flow feature extraction unit and a feature fusion unit; wherein: Edge feature extraction unit: used to perform multi-scale edge detection on the illumination-balanced image. By setting multiple Sobel operators with different convolution kernel sizes, the horizontal and vertical grayscale changes of the image are extracted respectively, highlighting the position and intensity of the structural edge, and forming an edge gradient feature map. Texture feature extraction unit: This unit extracts the grayscale structure of each pixel neighborhood in the illumination-balanced image based on the local binary pattern method. It generates a binary code by comparing the grayscale size relationship between the central pixel and the surrounding pixels. It then divides the image area and calculates the LBP histogram at the regional level to form a texture structure feature map. Optical flow feature extraction unit: used to perform optical flow intensity calculation on two adjacent frames of illumination-balanced images, obtain the motion vector of each pixel in the image between two consecutive frames, thereby characterizing the motion trend of the local area of ​​the image and forming a motion optical flow feature map; Feature fusion unit: used to align the edge gradient feature map, texture structure feature map and motion optical flow feature map according to spatial position, fuse them into a unified multi-scale feature tensor, and perform normalization processing after fusion to ensure consistency of different feature magnitudes.

5. The video detection system for conveyor belt deviation and foreign matter accumulation according to claim 1 is characterized in that: The spatiotemporal consistency analysis module includes an edge offset analysis unit, a foreign body positioning unit, an optical flow consistency verification unit, and an index generation unit; wherein: Edge offset analysis unit: used to receive the edge gradient feature map, extract the left and right edge points of the conveyor belt, construct the dynamic baseline of the current frame, and compare it with the set reference line to calculate the offset on both sides of the conveyor belt; Foreign matter positioning unit: used to receive the texture structure feature map, extract the area inconsistent with the normal surface texture of the conveyor belt as the foreign matter accumulation area, and determine the center coordinates of the foreign matter accumulation area; Optical flow consistency verification unit: used to receive the motion optical flow feature map, analyze whether the optical flow vector direction in the foreign matter area is consistent with the conveyor belt movement direction, and determine whether the foreign matter is in a static accumulation state; Index generation unit: used to generate the deviation azimuth according to the results of the edge offset analysis unit, output the foreign object coordinates according to the results of the foreign object positioning unit, and calculate the confidence level based on the results of the optical flow consistency verification unit as the final detection output.

6. A video detection system for conveyor belt deviation and foreign matter accumulation according to claim 5, characterized in that: The edge offset analysis unit includes: Edge point extraction subunit: used to scan the edge gradient feature map vertically column by column, extract the leftmost and rightmost pixel coordinates in each column whose gradient amplitude is greater than the set threshold, and use them as the left edge point and right edge point of the conveyor belt in the current frame respectively; Baseline construction subunit: used to fit all left edge points into the left dynamic edge curve, all right edge points into the right dynamic edge curve, and use the central axis of the two edge curves as the dynamic baseline of the current frame; Offset calculation subunit: used to compare the dynamic baseline of the current frame with the preset ideal conveyor belt center line in the corresponding column position point by point, calculate the lateral offset value of each column, and take the average value as the overall lateral offset.

7. The video detection system for conveyor belt deviation and foreign matter accumulation according to claim 5 is characterized in that: The foreign body positioning unit includes: Texture anomaly detection subunit: used to divide the received texture structure feature map into local areas, calculate the local binary pattern histogram of each divided area, and compare the similarity with the preset normal conveyor belt texture template. When the similarity is lower than the set threshold, the area is determined to be a texture anomaly area; Region segmentation subunit: used to perform connectivity analysis on all adjacent abnormal pixel regions, merge them to form continuous candidate regions for foreign body accumulation, remove noise blocks with an area below the minimum valid area threshold, and extract the bounding box of the main foreign body region; Center coordinate extraction subunit: used to calculate the center point of the boundary circumscribed rectangle of the main foreign object area as the positioning coordinate of the main foreign object area.

8. The video detection system for conveyor belt deviation and foreign matter accumulation according to claim 5 is characterized in that: The optical flow consistency verification unit includes: Optical flow vector extraction subunit: used to extract the two-dimensional optical flow vectors of all pixel positions in the foreign matter accumulation area from the motion optical flow feature map, forming a local motion vector set of the foreign matter accumulation area; Directional similarity calculation subunit: used to calculate the direction cosine of each optical flow vector and the unit vector of the conveyor belt running direction to obtain the direction similarity of each pixel; The stationary judgment subunit is used to count the proportion of pixels in the foreign object area that meet the directional consistency threshold condition. If the proportion exceeds the set threshold, the foreign object is judged to be in a stationary accumulation state; otherwise, it is judged to be a moving interference object.

9. The video detection system for conveyor belt deviation and foreign matter accumulation according to claim 5 is characterized in that: The indicator generating unit includes: Deviation angle calculation subunit: used to receive the left and right edge offset values ​​of the conveyor belt output by the edge offset analysis unit, and calculate the deviation angle of the conveyor belt based on the offset difference between the left and right sides. ; Foreign body coordinate output subunit: used to receive the foreign body center position coordinates output by the foreign body positioning unit and output them as foreign body spatial position parameters in the structured detection results; Confidence evaluation subunit: used to receive the output results of the optical flow consistency verification unit, calculate the pixel ratio of the optical flow direction deviating from the conveyor belt direction in the foreign body area, and infer the confidence of the foreign body's static state based on the pixel ratio .

10. The video detection system for conveyor belt deviation and foreign matter accumulation according to claim 1 is characterized in that: The decision output module includes a deviation determination subunit, a foreign object classification subunit and an alarm generation subunit; wherein: Deviation determination subunit: This unit receives the deviation angle and compares it with the set deviation angle threshold. It then outputs the corresponding deviation level based on the range of the angle's absolute value, including normal, mild deviation, moderate deviation, and severe deviation. Foreign body classification subunit: used to receive the foreign body coordinates and confidence level, and classify the foreign body according to the confidence level and the set threshold After comparison, if the confidence level is higher than the threshold, it is determined to be a real accumulated foreign object; otherwise, it is determined to be an interference object with movement, and the area category to which the foreign object belongs is determined based on the image position of the foreign object coordinates; Alarm generation subunit: This subunit is used to combine the deviation level and foreign object type judgment results to generate corresponding alarm instructions based on preset response rules. When the deviation level is moderate or severe, or the foreign object type is true accumulation, an alarm instruction data packet containing the alarm type, alarm area, and processing suggestions is output.

Images