A method and system for detecting material blockage at a blanking port based on optical flow

Abstract

The present invention relates to a method and system for detecting material blockage at a feed opening based on optical flow. The method comprises: S1, continuously capturing video images of the feed opening area through a camera device to monitor material movement in the feed opening area in real time; S2, comparing consecutive image frames in the obtained video images, analyzing and extracting materials in the images through optical flow analysis; S3, analyzing the material flow area using the optical flow analysis results; S4, analyzing the material flow movement direction using the optical flow analysis results; S5, combining the analysis results of the material flow area and the material flow movement direction to determine the material flow state: if the material flow area increases to exceed a material flow area threshold and the degree of dispersion in the material flow movement direction exceeds a dispersion threshold, it is determined that the current feed opening is about to be blocked; otherwise, it is determined that the current feed opening is in normal state; S6, if it is determined that the feed opening is about to be blocked, a blockage alarm is issued. The method and system have a high degree of automation and can improve the real-time and accuracy of feed opening blockage detection.

Description

Technical Field

[0001] The present invention relates to the field of conveyor belt transportation, and in particular to a method and system for detecting material blockage at a material drop opening based on optical flow. Background Art

[0002] The material outlet plays a key role in conveyor belt transportation, and its smooth flow directly affects the normal operation and production efficiency of the production line. Material blockage refers to the inability of materials to pass through the material outlet normally due to various reasons during conveyor belt transportation. Material blockage can cause a series of serious problems such as production line stagnation and equipment damage. The primary problem caused by material blockage is production line stagnation. Once a blockage occurs, the conveyor belt will stop operating, resulting in a temporary interruption of the production line and a sharp drop in production efficiency. In addition, the extended downtime may cause equipment failure, causing more serious impact on the entire production process. Material blockage may also cause additional wear and tear on equipment, increase maintenance costs, and even cause damage to the equipment. Therefore, it is very important to monitor material blockage at the material outlet. The traditional monitoring method is manual inspection. However, manual inspection has the problems of wasting human resources and slow response speed, and it is easy to miss some subtle material blockage situations. In order to realize the automated monitoring of material blockage at the material outlet, some computer vision-based material blockage detection technologies have also emerged in the prior art. However, most of these detection technologies have poor detection effects and cannot detect the occurrence of material blockage in real time and accurately. Summary of the Invention

[0003] The object of the present invention is to provide a method and system for detecting material blockage at a blanking port based on optical flow. The method and system have a high degree of automation and can improve the real-time performance and accuracy of material blockage detection at the blanking port.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is: a method for detecting material blockage at a blanking port based on optical flow, comprising the following steps:

[0005] S1. Continuously collect video images of the blanking port area through a camera device to monitor the material movement in the blanking port area in real time;

[0006] S2. Compare the continuous image frames in the obtained video image, analyze and extract the material in the image by optical flow method;

[0007] S3. Analyze the material flow area using the optical flow analysis results;

[0008] S4. Analyze the direction of material flow using the optical flow analysis results;

[0009] S5. Based on the analysis results of the material flow area and the material flow direction, the material flow state is judged: if the material flow area increases to exceed the material flow area threshold and the dispersion degree of the material flow direction exceeds the dispersion degree threshold, it is determined that the current blanking port is about to be blocked; otherwise, it is determined that the current blanking port is in normal state;

[0010] S6. If it is determined that the blanking port is about to be blocked, a blockage alarm is issued.

[0011] Furthermore, in step S2, the specific method of analyzing and extracting the material in the image by the optical flow method is:

[0012] The global optical flow image L(x,y) is extracted from the previous and next frame images, and the pixel motion in the Cartesian coordinate system (x,y) is converted into the direction and amplitude of the motion in the polar coordinate system (σ,r), and the optical flow map L(σ,r) represented by the polar coordinate system is obtained:

[0013]

[0014] Since the material is the part with the largest motion amplitude in the image and occupies a small proportion of the image, the pixels with the smallest motion amplitude in the image are considered to be the background. The histogram of the amplitude component of the optical flow map L(σ,r) is calculated and the amplitude is divided into N intervals, which are defined as A1, A2, ..., A N ,in M is the amplitude, and the cumulative number in each interval is counted; the optical flow pixels in the first interval are eliminated because they have a small motion amplitude and a large number, that is, the ones with a motion amplitude smaller than the first interval are taken as background, and the rest are foreground materials; set the threshold

[0015]

[0016] The optical flow pixels in the remaining interval are moving materials because they have large motion amplitude and small number.

[0017] Furthermore, the number of intervals N=20.

[0018] Furthermore, in step S3, for all materials extracted from the image, all materials are merged to obtain a material flow area, and then an envelope of the material flow area is drawn, and the area of ​​the material flow envelope area is calculated, which is the material flow area A.

[0019] Furthermore, if the material flow area A is not greater than the set material flow area threshold T a , it is determined that the blanking port is in normal condition; if the material flow area A is greater than the material flow area threshold T a , that is, A>T a , it is considered that there is a risk of material blocking, but it is not directly determined as material blocking. Instead, the blocking situation is analyzed together with the direction of material flow.

[0020] Furthermore, in step S4, the standard deviation ε of the material flow movement is used to describe the dispersion degree of the material flow movement direction, specifically:

[0021]

[0022]

[0023] Among them, σ i is the pixel direction sample, Q is the number of samples, is the directional mean.

[0024] Furthermore, in step S5, based on the set material flow area threshold T a and the diffusion threshold T σ , determine whether the material flow area A>T is satisfied at the same time a And the standard deviation of material flow movement ε>T σ If yes, it is determined that the current blanking port is about to be blocked, otherwise it is determined that the current blanking port is in normal state.

[0025] The present invention also provides a blanking port blockage detection system based on optical flow, which is used to implement the above method, including:

[0026] A camera device is installed above the blanking port of the conveyor belt to collect video images of the blanking port area and send the collected video images to the host computer;

[0027] The host computer is connected to the camera device to receive the collected video images, and then analyze and extract the materials in the image through the optical flow method, and analyze the material flow area and material flow movement direction, and then combine the analysis results of the material flow area and material flow movement direction to judge the material flow state.

[0028] Furthermore, an alarm device is included. The host computer is connected to the alarm device so that when it is determined that the blanking port is about to be blocked, a blockage alarm is issued through the alarm device.

[0029] Compared with the existing technology, the present invention has the following beneficial effects: the present invention provides a method and system for detecting material blockage at the blanking port based on optical flow, which sets a camera on the conveyor belt to capture real-time video images of the blanking port area, and then uses optical flow analysis technology to analyze the optical flow state between the previous and next frame images, thereby analyzing the material flow area and the direction of material flow movement, and then combining the material flow area and the direction of material flow movement to determine the material blockage condition at the blanking port; this optical flow-based material blockage analysis method has strong real-time performance and strong adaptability to illumination changes and background interference, and can timely, reliably and accurately determine whether there is a material blockage, thereby providing a strong guarantee for the production process, which can not only greatly improve production efficiency, reduce downtime and equipment damage caused by material blockage, but also reduce the cost of manual inspections. Therefore, the present invention has strong practicality and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flowchart of a method implementation of an embodiment of the present invention;

[0031] Figure 2 is an optical flow amplitude histogram in an embodiment of the present invention;

[0032] Figure 3 1 is a diagram of the implementation process of material extraction in an embodiment of the present invention;

[0033] Figure 4 Schematic diagram of the material flow envelope area when no material blockage occurs in an embodiment of the present invention;

[0034] Figure 5 Schematic diagram of the material flow envelope area when material blockage occurs in an embodiment of the present invention;

[0035] Figure 6 Schematic diagram of the direction of material flow movement in an embodiment of the present invention;

[0036] Figure 7 It is a schematic diagram of the system implementation principle of an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0038] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0039] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0040] like Figure 1 As shown, this embodiment provides a method for detecting material blockage at a blanking port based on optical flow, and the specific implementation steps are as follows.

[0041] S1. Real-time image acquisition: The video images of the blanking port area are continuously collected by the camera equipment to monitor the material movement in the blanking port area in real time.

[0042] S2. Optical flow analysis of previous and next frames: Compare consecutive frames from the acquired video image and use optical flow to analyze and extract the objects in the image. Optical flow is a vector field that describes the movement of pixels in an image and can reflect the movement trajectory of objects in the image.

[0043] S3. Use the optical flow analysis results to analyze the material flow area. When the material movement is relatively normal, the material flow area is relatively small; when the material is about to be blocked, the material flow area will increase significantly.

[0044] S4. Use the optical flow analysis results to analyze the direction of material flow. Analyze the direction of material movement in the image. If the material flow direction is consistent, it indicates smooth material flow. Conversely, if the material flow direction is diffuse, it may indicate an impending blockage.

[0045] S5. Based on the analysis results of the material flow area and the material flow direction, the material flow state is determined. If the material flow area increases to exceed the material flow area threshold and the dispersion degree of the material flow direction exceeds the dispersion degree threshold, it is determined that the current material outlet is about to be blocked. Otherwise, the current material outlet is determined to be in normal state.

[0046] S6. If it is determined that the blanking port is about to be blocked, a blockage alarm or information feedback will be issued to carry out timely maintenance and processing.

[0047] Through the above detection method, the present invention can accurately analyze the status of the blanking port according to the real-time video, timely warn of the impending material blockage, and provide a reliable monitoring means for the production process.

[0048] In order to implement the above method, Figure 7 As shown, this embodiment also provides a blanking port blockage detection system based on optical flow, which includes a camera device and a host computer.

[0049] The camera device is arranged above the blanking port of the conveyor belt, and is used to collect video images of the blanking port area and send the collected video images to the host computer.

[0050] The host computer is connected to the camera equipment to receive the collected video images, and then analyze and extract the materials in the image through the optical flow method, and analyze the material flow area and material flow movement direction, and then combine the analysis results of the material flow area and material flow movement direction to judge the material flow state.

[0051] Preferably, the system further comprises an alarm device, and the host computer is connected to the alarm device so that when it is determined that the blanking port is about to be blocked, the alarm device issues a blockage alarm.

[0052] The relevant technical contents involved in this method are further described in detail below.

[0053] (1) Optical flow extraction and material segmentation of the front and back frames of the image

[0054] The material flow is the moving object located at the material outlet in the video. The present invention uses the optical flow method to extract the material in the image. Optical flow is the movement of the same object in the image between the previous and next frames.

[0055] The global optical flow image L(x,y) is extracted from the previous and next frame images, and the pixel motion in the Cartesian coordinate system (x,y) is converted into the direction and amplitude of the motion in the polar coordinate system (σ,r), and the optical flow map L(σ,r) represented by the polar coordinate system is obtained:

[0056]

[0057] Since the material is the part with the largest motion amplitude in the image and occupies a small proportion of the image, the pixels with the smallest motion amplitude in the image are considered to be the background. Calculate the histogram of the amplitude component of the optical flow map L(σ,r) and divide the amplitude into N intervals, which are defined as A1, A2, ..., A N ,in M is the amplitude, and the cumulative number in each interval is counted. The optical flow amplitude histogram is as follows Figure 2 In this embodiment, the number of intervals N=20. The optical flow pixels in the first interval are eliminated because they have a small motion amplitude and a large number. That is, the pixels with a motion amplitude smaller than the first interval are taken as background, and the rest are foreground materials. Set the threshold

[0058]

[0059] The optical flow pixels in the remaining interval are moving materials because they have large motion amplitude and small number.

[0060] The material extraction process is as follows Figure 3 shown. Figure 3 In the figure, from left to right are the original image, optical flow visualization image, and material mask image.

[0061] (2) Material flow area analysis

[0062] For all materials extracted from the image, all materials are merged to obtain the material flow area, and then the envelope of the material flow area is drawn, and the area of ​​the material flow envelope area is calculated, which is the material flow area.

[0063] If the material flow area A is not greater than the set material flow area threshold T a , it is determined that the blanking port is in normal condition; if the material flow area A is greater than the material flow area threshold T a , that is, A>T a , then it is considered that there is a risk of material blocking, but it cannot be directly determined as material blocking. The blocking situation needs to be analyzed together with the direction of material flow. When no material blocking occurs, the material flow envelope area is as follows Figure 4 shown. Figure 4 In the figure, the left side shows the material flow envelope area when no material blockage occurs, and the right side shows the corresponding monitoring video image. Figure 5 shown. Figure 5 In the figure, the left side shows the diagram of the material flow envelope area when blockage occurs. The material flow envelope area will increase and exceed the set threshold. The right side shows the corresponding monitoring video image.

[0064] (3) Analysis of material flow direction consistency

[0065] When the material outlet is not blocked, the material flow is smooth; if it is blocked, the material flow will move in all directions. Therefore, the present invention uses the consistency of the material flow direction as the condition for the analysis of the material outlet blockage. Figure 6 As shown in the radar chart, the left side shows the optical flow intensity and direction map without blockage, and the right side shows the optical flow intensity and direction map with blockage. In comparison, the optical flow direction of the unblocked material is more concentrated, with a higher intensity in a certain direction; while the optical flow direction of the blocked material is more dispersed.

[0066] In this embodiment, the standard deviation ε of the material flow movement is used to describe the dispersion degree of the material flow movement direction, specifically:

[0067]

[0068]

[0069] Among them, σ i is the pixel direction sample, Q is the number of samples, is the directional mean.

[0070] (4) Material flow status determination

[0071] Based on the set material flow area threshold T aand the diffusion threshold T σ , determine whether the material flow area A>T is satisfied at the same time a And the standard deviation of material flow movement ε>T σ If yes, it is determined that the current blanking port is about to be blocked, otherwise it is determined that the current blanking port is in normal state.

[0072] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above-disclosed technical content to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.

Claims

1. A method for detecting material blockage at a blanking port based on optical flow, characterized in that: The following steps are involved: S1. Continuously collect video images of the blanking port area through a camera device to monitor the material movement in the blanking port area in real time; S2. Compare the continuous image frames in the obtained video image, analyze and extract the material in the image by optical flow method; S3. Analyze the material flow area using the optical flow analysis results; S4. Analyze the direction of material flow using the optical flow analysis results; S5. Based on the analysis results of the material flow area and the material flow direction, the material flow state is judged: if the material flow area increases to exceed the material flow area threshold and the dispersion degree of the material flow direction exceeds the dispersion degree threshold, it is determined that the current blanking port is about to be blocked; otherwise, it is determined that the current blanking port is in normal state; S6. If it is determined that the blanking port is about to be blocked, a blocking alarm is issued; In step S2, the specific method of analyzing and extracting the material in the image by the optical flow method is as follows: The global optical flow image L(x,y) is extracted from the previous and next frame images, and the pixel motion in the Cartesian coordinate system (x,y) is converted into the direction and amplitude of the motion in the polar coordinate system (σ,r), and the optical flow map L(σ,r) represented by the polar coordinate system is obtained: Since the material is the part with the largest motion amplitude in the image and occupies a small proportion of the image, the pixels with the smallest motion amplitude in the image are considered to be the background. The histogram of the amplitude component of the optical flow map L(σ,r) is calculated and the amplitude is divided into N intervals, which are defined as A1, A2, ..., A N ,in M is the amplitude, and the cumulative number in each interval is counted; the optical flow pixels in the first interval are eliminated because the movement amplitude of these pixels is small and there are many of them, that is, the pixels with a movement amplitude smaller than the first interval are taken as background, and the rest are foreground materials; set the threshold The optical flow pixels in the remaining interval are moving materials because they have large motion amplitude and small number.

2. The method for detecting material blockage at a blanking port based on optical flow according to claim 1, characterized in that: The number of intervals N=20.

3. The method for detecting material blockage at a blanking port based on optical flow according to claim 1, characterized in that: In step S3, for all materials extracted from the image, all materials are merged to obtain a material flow area, and then an envelope line of the material flow area is drawn, and the area of ​​the material flow envelope area is calculated, which is the material flow area A.

4. The method for detecting material blockage at a blanking port based on optical flow according to claim 3, characterized in that: If the material flow area A is not greater than the set material flow area threshold T a , it is determined that the blanking port is in normal condition; if the material flow area A is greater than the material flow area threshold T a , that is, A>T a , it is considered that there is a risk of material blocking, but it is not directly determined as material blocking. Instead, the blocking situation is analyzed together with the direction of material flow.

5. The method for detecting material blockage at a blanking opening based on optical flow according to claim 1, characterized in that: In step S4, the standard deviation ε of the material flow movement is used to describe the dispersion degree of the material flow movement direction, specifically: Among them, σ i is the pixel direction sample, Q is the number of samples, is the directional mean.

6. The method for detecting material blockage at a blanking opening based on optical flow according to claim 1, characterized in that: In step S5, based on the set material flow area threshold T a and the diffusion threshold T σ , determine whether the material flow area A>T is satisfied at the same time a And the standard deviation of material flow movement ε>T σ If yes, it is determined that the current blanking port is about to be blocked, otherwise it is determined that the current blanking port is in normal state.

7. A material blockage detection system based on optical flow, used to implement the method according to any one of claims 1 to 6, characterized in that: include: A camera device is installed above the blanking port of the conveyor belt to collect video images of the blanking port area and send the collected video images to the host computer; The host computer is connected to the camera device to receive the collected video images, and then analyze and extract the materials in the image through the optical flow method, and analyze the material flow area and material flow movement direction, and then combine the analysis results of the material flow area and material flow movement direction to judge the material flow state.

8. The optical flow-based material blockage detection system according to claim 7, characterized in that: It also includes an alarm device, and the host computer is connected to the alarm device so that when it is determined that the blanking port is about to be blocked, the alarm device will issue a blockage alarm.

Images