Paper anomaly detection method and device based on vision

Abstract

The invention provides a vision-based paper anomaly detection method and device, and the method comprises the steps: configuring a template drawing for a detection region, determining a detected effective region mask according to the template drawing, and setting a counting standard-reaching parameter; collecting a preset number of initial environment images in the detection area for initial judgment when detection is started; after the initial judgment is completed, collecting a real-time environment image according to a preset time interval, determining an effective area and an invalid area in the real-time environment image according to the effective area mask, performing color recognition on the effective area in the real-time environment image, and performing frequency domain calculation on the invalid area; according to an effective area color identification result and an invalid area frequency domain calculation result of the real-time environment image and the counting up-to-standard parameter, whether paper breaking and shielding occur currently is judged; the method can improve the reliability of broken paper detection.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a vision-based method and apparatus for detecting paper anomalies. Background Technology

[0002] Industrial paper production mainly includes the following stages: raw material preparation, pulping, papermaking, finishing, and processing. The papermaking stage involves beating, conditioning, forming, pressing, and drying the paper. During these processes, paper breakage can occur. If breakage is not detected in time, damaged paper will detach, causing greater chaos and damage. Furthermore, there is no output during a break, wasting materials and reducing the overall efficiency of the production line.

[0003] Domestic and international research on paper break detection focuses on photoelectric sensor detection methods. For example, patent CN105784601B proposes a paper break detection device and its automatic calibration method. This paper break detection device with automatic calibration function includes a detection system and a calibration triggering system. The detection system samples the color of a predetermined detection area and compares the sampled color value with a standard color value to determine whether there is a paper break. When a paper break is found, it outputs a paper break detection signal. The calibration triggering system triggers the detection system to automatically calibrate the standard color value based on the paper break detection signal. The color sampling uses a light emitter and a light receiver. The light emitter projects light onto the paper surface, and the light receiver receives the light reflected from the paper surface. The difference between the received color and the standard color is then calculated to determine whether the paper is broken. However, this solution suffers from poor stability and limited versatility. It also fails to consider the problems of numerous impurities and paper obstruction in the actual production environment of paper mills. Furthermore, much of the pulp is placed on filter cloth, which can block the photoelectric sensor signal even if the paper breaks, preventing detection. Summary of the Invention

[0004] This invention provides a vision-based paper anomaly detection method and apparatus, which can improve the reliability and stability of paper breakage detection during the paper production process and can identify occlusion situations.

[0005] A vision-based paper anomaly detection method is applied to a paper production line, wherein the paper is mounted on a transmission mechanism or a support mechanism; the method includes: The transmission mechanism or support mechanism and its surrounding area are used as the detection area. A template diagram is configured for the detection area. The effective area mask for detection is determined according to the template diagram, and the counting target parameters are set. At the start of detection, a preset number of initial environmental images are collected within the detection area for initial setup. After the initial judgment that there are no abnormalities, real-time environmental images of the detection area are collected at preset time intervals. The effective and invalid regions in the real-time environmental images are determined according to the effective region mask. Color recognition is performed on the effective regions in the real-time environmental images and frequency domain calculation is performed on the invalid regions. Based on the color recognition results of the effective area of ​​the real-time environmental image, the frequency domain calculation results of the invalid area, and the counting target parameters, it is determined whether a paper break or occlusion has occurred.

[0006] Furthermore, the effective area is the support portion area on the transmission mechanism or support mechanism that directly contacts the paper and is used to support the paper; the template image is an image of the detection area when there is no paper. Determining the effective area mask of the roller based on the template diagram includes: Select the center point of the support area in the template diagram as the seed point, and set the threshold range; Starting from the seed point, the pixel values ​​of the pixels in the neighborhood of the seed point are compared with the threshold range, and the pixels whose pixel values ​​are within the threshold range are added to the candidate region list. Repeat the following steps until the pixel value of the selected pixel has no neighboring pixels within the threshold range: Select a target pixel from the candidate region list, calculate whether the pixel values ​​of its neighboring pixels are within the threshold range, add pixels whose pixel values ​​are within the threshold range to the candidate region list, and delete the selected target pixel from the candidate region list. When the pixel value of a selected target pixel has no neighboring pixels within the threshold range, the pixels in the obtained candidate region list constitute the effective region mask.

[0007] Furthermore, at the start of detection, a preset number of initial environmental images are acquired within the detection area for initial assessment, including: The effective and invalid regions in the initial environment image are determined based on the effective region mask. Color recognition is performed on the effective regions in the initial environment image and frequency domain calculation is performed on the invalid regions. Based on the color recognition results of the effective regions and the frequency domain calculation results of the invalid regions in the initial environment image, it is determined whether paper breakage or occlusion occurs during power-on. If no paper breakage or occlusion occurs, it is determined that the initial judgment is normal.

[0008] Further, determining the valid and invalid regions in the initial environment image based on the valid region mask, or determining the valid and invalid regions in the real-time environment image based on the valid region mask, includes: The initial environment image / real-time environment image is compared with the template image. The regions in the initial environment image / real-time environment image that are at the same pixel position as the effective region mask are determined as effective regions, and the regions other than the effective regions are determined as invalid regions.

[0009] Furthermore, color recognition is performed on the effective regions of the real-time environmental image, and frequency domain calculation is performed on the invalid regions, including: The effective region mask in the template image and the effective region in the real-time environment image are converted to Lab space. The first color difference between the effective region in the real-time environment image and the effective region mask is calculated in Lab space. The first color difference is used as the color recognition result of the effective region in the real-time environment image. The invalid regions in the real-time environment image are converted to grayscale to obtain a first grayscale image. The first grayscale image is then converted to the frequency domain. The first low-frequency energy and the total energy of the first grayscale image are calculated based on the frequency domain values. The first fluctuation judgment value is calculated based on the ratio of the first low-frequency energy to the total energy of the first grayscale image. The first fluctuation judgment value is used as the calculation result of the invalid regions in the real-time environment image.

[0010] Furthermore, the counting parameters include paper breakage count and obstruction count; Based on the color recognition results of the effective area of ​​the real-time environmental image, the frequency domain calculation results of the invalid area, and the count compliance parameters, it is determined whether a paper break or occlusion has occurred, including: The first color difference is compared with a preset color difference value, and a paper break mark is made on the real-time environmental image based on the comparison result; Based on the number of real-time environmental images with paper break marks and the paper break count, output the paper break determination result; The first fluctuation determination value is compared with the preset fluctuation value, and the real-time environmental image is marked with occlusion based on the comparison result. Based on the number of real-time environmental images with occlusion markers and the occlusion count, an occlusion determination result is output.

[0011] Furthermore, the first color difference is compared with a preset color difference value. If the first color difference is less than the preset color difference value, the real-time environmental image at the current moment is marked as a paper break. The real-time environmental images marked as paper breaks in the past are analyzed, counted, and compared with the paper break count to output a paper break determination result. If the first color difference is greater than or equal to the preset color difference value, then it is determined whether the first fluctuation judgment value is greater than the preset fluctuation value. If the first fluctuation judgment value is greater than the preset fluctuation value, then the real-time environment image at the current moment is marked as occlusion, and the real-time environment images marked as occlusion in the past are analyzed, counted, and compared with the occlusion count, and the occlusion judgment result is output.

[0012] Further, color recognition is performed on the effective regions in the initial environmental image, and frequency domain calculation is performed on the invalid regions, including: The effective region mask in the template image and the effective region in the initial environment image are converted to Lab space. The second color difference between the effective region in the initial environment image and the effective region mask is calculated in Lab space. The second color difference is used as the color recognition result of the effective region in the initial environment image. The invalid regions in the initial environmental image are converted to grayscale to obtain a second grayscale image. The second grayscale image is then converted to the frequency domain. The second low-frequency energy and the total energy of the second grayscale image are calculated based on the frequency domain values. The second fluctuation judgment value is calculated based on the ratio of the second low-frequency energy and the total energy of the second grayscale image. The second fluctuation judgment value is used as the calculation result of the invalid regions in the initial environmental image.

[0013] Furthermore, based on the color recognition results of the effective area and the frequency domain calculation results of the invalid area of ​​the initial environmental image, it is determined whether paper breakage or occlusion occurs during power-on, including: The calculated second color difference is compared with the preset color difference value. If there is a second color difference that is less than the preset color difference value, it is determined that a paper break occurred when the machine is turned on. If it is determined that no paper breakage occurred when the machine was turned on, the calculated second fluctuation judgment values ​​are compared with the preset fluctuation values. If there is a second fluctuation judgment value that is greater than the preset fluctuation value, it is determined that there is obstruction when the machine was turned on.

[0014] A vision-based paper anomaly detection device is applied to a paper production line. The paper is mounted on a transmission mechanism or support mechanism, and the transmission mechanism or support mechanism and its surrounding area are used as the detection area. The device includes: The configuration module is used to configure a template image for the detection area, determine the effective area mask for detection based on the template image, and set the counting target parameters. The initial judgment module is used to collect a preset number of initial environmental images within the detection area for initial settings when detection begins. The identification and calculation module is used to collect real-time environmental images within the detection area at preset time intervals after an initial judgment that there are no abnormalities, determine the effective and invalid areas in the real-time environmental images according to the effective area mask, perform color recognition on the effective areas in the real-time environmental images, and perform frequency domain calculation on the invalid areas. The judgment module is used to determine whether a paper break or occlusion has occurred based on the effective area color recognition result of the real-time environmental image, the invalid area frequency domain calculation result, and the count compliance parameter.

[0015] Furthermore, the effective area is the support portion area on the transmission mechanism or support mechanism that directly contacts the paper and is used to support the paper; the template image is an image of the detection area when there is no paper. The configuration module determines the effective region mask for detection based on the template image, including: Select the center point of the support area in the template diagram as the seed point, and set the threshold range; Starting from the seed point, the pixel values ​​of the pixels in the neighborhood of the seed point are compared with the threshold range, and the pixels whose pixel values ​​are within the threshold range are added to the candidate region list. Repeat the following steps until the pixel value of the selected pixel has no neighboring pixels within the threshold range: Select a target pixel from the candidate region list, calculate whether the pixel values ​​of its neighboring pixels are within the threshold range, add pixels whose pixel values ​​are within the threshold range to the candidate region list, and delete the selected target pixel from the candidate region list. The effective region mask is formed by the pixels in the candidate region list obtained when there are no pixels in the neighborhood of the selected target pixel whose pixel value is within the threshold range.

[0016] Furthermore, when the initial judgment module is powered on, it acquires a preset number of initial environmental images within the detection area for initial judgment, including: The effective and invalid regions in the initial environment image are determined based on the effective region mask. Color recognition is performed on the effective regions in the initial environment image and frequency domain calculation is performed on the invalid regions. Based on the color recognition results of the effective regions and the frequency domain calculation results of the invalid regions in the initial environment image, it is determined whether paper breakage or occlusion occurs during power-on. If no paper breakage or occlusion occurs, it is determined that the initial judgment is normal.

[0017] Further, the initial judgment module determines the valid and invalid regions in the initial environment image based on the valid region mask; or, the recognition and calculation module determines the valid and invalid regions in the real-time environment image based on the valid region mask, including: The initial environment image / real-time environment image is compared with the template image. The regions in the initial environment image / real-time environment image that are at the same pixel position as the effective region mask are determined as effective regions, and the regions other than the effective regions are determined as invalid regions.

[0018] Furthermore, the recognition and calculation module performs color recognition on the valid regions in the real-time environmental image and frequency domain calculation on the invalid regions, including: The effective region mask in the template image and the effective region in the real-time environment image are converted to Lab space. The first color difference between the effective region in the real-time environment image and the effective region mask is calculated in Lab space. The first color difference is used as the color recognition result of the effective region in the real-time environment image. The invalid regions in the real-time environment image are converted to grayscale to obtain a first grayscale image. The first grayscale image is then converted to the frequency domain. The first low-frequency energy and the total energy of the first grayscale image are calculated based on the frequency domain values. The first fluctuation judgment value is calculated based on the ratio of the first low-frequency energy to the total energy of the first grayscale image. The first fluctuation judgment value is used as the calculation result of the invalid regions in the real-time environment image.

[0019] Furthermore, the counting parameters include paper breakage count and obstruction count; The judgment module determines whether a paper break or obstruction has occurred based on the effective area color recognition result of the real-time environmental image, the invalid area frequency domain calculation result, and the count compliance parameter, including: The first color difference is compared with a preset color difference value, and a paper break mark is made on the real-time environmental image based on the comparison result; Based on the number of real-time environmental images with paper break marks and the paper break count, output the paper break determination result; The first fluctuation determination value is compared with the preset fluctuation value, and the real-time environmental image is marked with occlusion based on the comparison result. Based on the number of real-time environmental images with occlusion markers and the occlusion count, an occlusion determination result is output.

[0020] Furthermore, the judgment module is also used to: compare the first color difference with a preset color difference value; if the first color difference is less than the preset color difference value, mark the current real-time environmental image as a paper break, and analyze and count the number of real-time environmental images marked as paper breaks in the past and compare them with the paper break count, and output the paper break judgment result. If the first color difference is greater than or equal to the preset color difference value, then it is determined whether the first fluctuation judgment value is greater than the preset fluctuation value. If the first fluctuation judgment value is greater than the preset fluctuation value, then the real-time environment image at the current moment is marked as occlusion, and the real-time environment images marked as occlusion in the past are analyzed, counted, and compared with the occlusion count, and the occlusion judgment result is output.

[0021] Furthermore, the initial judgment module performs color recognition on the valid regions in the initial environmental image and frequency domain calculation on the invalid regions, including: The effective region mask in the template image and the effective region in the initial environment image are converted to Lab space. The second color difference between the effective region in the initial environment image and the effective region mask is calculated in Lab space. The second color difference is used as the color recognition result of the effective region in the initial environment image. The invalid regions in the initial environmental image are converted to grayscale to obtain a second grayscale image. The second grayscale image is then converted to the frequency domain. The second low-frequency energy and the total energy of the second grayscale image are calculated based on the frequency domain values. The second fluctuation judgment value is calculated based on the ratio of the second low-frequency energy and the total energy of the second grayscale image. The second fluctuation judgment value is used as the calculation result of the invalid regions in the initial environmental image.

[0022] Furthermore, the initial judgment module determines whether paper breakage or obstruction occurs during power-on based on the effective region color recognition result and the invalid region frequency domain calculation result of the initial environmental image, including: The calculated second color difference is compared with the preset color difference value. If there is a second color difference that is less than the preset color difference value, it is determined that a paper break occurred when the machine is turned on. If it is determined that no paper breakage occurred when the machine was turned on, the calculated second fluctuation judgment values ​​are compared with the preset fluctuation values. If there is a second fluctuation judgment value that is greater than the preset fluctuation value, it is determined that there is obstruction when the machine was turned on.

[0023] The vision-based paper anomaly detection method and apparatus provided by this invention have at least the following beneficial effects: (1) Based on visual detection of whether paper breaks occur during the paper production process, the reliability and stability of paper break detection can be improved, and possible occlusion phenomena can be judged, making it more widely applicable; (2) Color difference determination is performed in Lab space and occlusion is determined by frequency domain, which can effectively avoid interference from image noise and improve the stability of detection; (3) Combining the setting of appropriate counting parameters during the detection process can reduce the probability of false detection and improve the accuracy of detection. Attached Figure Description

[0024] Figure 1This is a flowchart of one embodiment of the vision-based paper anomaly detection method provided by the present invention.

[0025] Figure 2 This is a schematic diagram of a normal production scenario in the vision-based paper anomaly detection method provided by the present invention.

[0026] Figure 3 This is a schematic diagram of a scenario where a paper break occurs in the vision-based paper anomaly detection method provided by the present invention.

[0027] Figure 4 This is a schematic diagram of a scene where occlusion occurs in the vision-based paper anomaly detection method provided by the present invention.

[0028] Figure 5 A flowchart of another embodiment of the vision-based paper anomaly detection method provided by the present invention.

[0029] Figure 6 This is a schematic diagram of one embodiment of the vision-based paper anomaly detection device provided by the present invention. Detailed Implementation

[0030] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0031] refer to Figure 1 In some embodiments, a vision-based paper anomaly detection method is provided, applied to a paper production line, wherein the paper is disposed on a transmission mechanism or a support mechanism; the method includes: S1. Take the transmission mechanism or support mechanism and its surrounding area as the detection area, configure a template diagram for the detection area, determine the effective area mask for detection based on the template diagram, and set the counting target parameters. S2. At the start of detection, a preset number of initial environmental images are collected within the detection area for initial judgment; S3. After the initial judgment that there is no abnormality, real-time environmental images within the detection area are collected at preset time intervals. The effective and invalid regions in the real-time environmental images are determined according to the effective region mask. Color recognition is performed on the effective regions in the real-time environmental images and frequency domain calculation is performed on the invalid regions. S4. Based on the color recognition results of the effective area of ​​the real-time environmental image, the frequency domain calculation results of the invalid area, and the counting target parameters, determine whether a paper break or occlusion has occurred.

[0032] refer to Figures 2 to 4 , Figure 2 This is a normal production scenario for the rollers in the paper drying process. Figure 3 To create a scene where the paper breaks, Figure 4The scene where occlusion occurs.

[0033] Specifically, in some processes on a paper production line, the paper is either mounted on a transmission mechanism to move in a predetermined direction, or mounted on a support mechanism for processing, for example... Figures 2 to 4 In the middle section, the paper is dried by passing through a drying cylinder. At the same time, the drying cylinder rotates and conveys the paper to the next stage. The transmission mechanism or support mechanism and its surrounding area are used as the detection area. Taking the drying stage as an example, the detection area is the drying cylinder and its surrounding preset area.

[0034] Further, in step S1, when the system starts up, the accompanying software is used to set a template image for detection. For example, a template image corresponding to the actual detection area can be found from a pre-configured database. The template image is the detection area image without paper, and it matches the actual detection area. On the interface, the support area that directly contacts the paper and supports it is selected on the template image. For example, in the drying process, the drying cylinder area can be selected. This part of the area is the valid area, while the rest is set as the invalid area. Subsequently, the algorithm determines the valid area mask based on the template image. If the setting is abandoned, the detection system exits. At the same time, the counting target parameters are set. Considering that the system may experience unexpected situations, such as image frame loss, abnormal image display due to network lag, and interference from scene clutter, setting appropriate counting target parameters can significantly reduce the probability of false detection and improve system stability.

[0035] Further, in step S1, the effective area is the support portion area on the transmission mechanism or support mechanism that is in direct contact with the paper and is used to support the paper; the template image is an image of the detection area when there is no paper. Determining the effective region mask for detection based on the template image includes: Select the center point of the support area in the template diagram as the seed point, and set the threshold range; Starting from the seed point, the pixel values ​​of the pixels in the neighborhood of the seed point are compared with the threshold range, and the pixels whose pixel values ​​are within the threshold range are added to the candidate region list. Repeat the following steps until the pixel value of the selected pixel has no neighboring pixels within the threshold range: Select a target pixel from the candidate region list, calculate whether the pixel values ​​of its neighboring pixels are within the threshold range, add pixels whose pixel values ​​are within the threshold range to the candidate region list, and delete the selected target pixel from the candidate region list. When the pixel value of a selected target pixel has no neighboring pixels within the threshold range, the pixels in the obtained candidate region list constitute the effective region mask.

[0036] The invalid region mentioned in this embodiment refers to regions other than the valid region.

[0037] Further, in step S2, when detection begins, a preset number of initial environmental images are acquired within the detection area for initial judgment, including: The effective and invalid regions in the initial environment image are determined based on the effective region mask. Color recognition is performed on the effective regions in the initial environment image and frequency domain calculation is performed on the invalid regions. Based on the color recognition results of the effective regions and the frequency domain calculation results of the invalid regions in the initial environment image, it is determined whether paper breakage or occlusion occurs during power-on. If no paper breakage or occlusion occurs, it is determined that the initial judgment is normal.

[0038] Because the detection system may experience paper breaks or obstructions when it is powered on, the system first checks the initial environment image to determine if any such issues exist. If none are found, the system is considered to have completed its initial setup. If paper breaks or obstructions are found, the process ends.

[0039] Further, in step S3, determining the valid and invalid regions in the initial environment image based on the valid region mask, or in step S2, determining the valid and invalid regions in the real-time environment image based on the valid region mask, includes: The initial environment image / real-time environment image is compared with the template image. The regions in the initial environment image / real-time environment image that are at the same pixel position as the effective region mask are determined as effective regions, and the regions other than the effective regions are determined as invalid regions.

[0040] Further, in step S3, color recognition is performed on the effective regions of the real-time environmental image, and frequency domain calculation is performed on the invalid regions, including: S31. Convert the effective region mask in the template image and the effective region in the real-time environment image to Lab space, calculate the first color difference between the effective region in the real-time environment image and the effective region mask in Lab space, and use the first color difference as the color recognition result of the effective region in the real-time environment image. S32. Convert the invalid region in the real-time environment image to grayscale to obtain a first grayscale image. Convert the first grayscale image to the frequency domain. Calculate the first low-frequency energy and the total energy of the first grayscale image based on the frequency domain value. Calculate the first fluctuation judgment value based on the ratio of the first low-frequency energy to the total energy of the first grayscale image. Use the first fluctuation judgment value as the calculation result of the invalid region in the real-time environment image.

[0041] Specifically, in step S31, the camera's chromaticity coordinates are first calibrated by placing the camera in an ambient light chamber and illuminating it using a D65 standard light source system. The camera is then used to photograph a standard 24-color chart. The camera's white balance is locked, automatic gain is disabled, and digital noise reduction is turned off. The average RGB color value of each color patch on the chart is then recorded as follows: (1) Wherein, color matrix C ref Each row in the matrix represents the average value of one color. The color chart has 24 colors, corresponding to 24 rows in the matrix. Subsequently, the transformation matrix is ​​calculated based on the reference LAB values ​​L provided by the color chart. ref a ref b ref The CIE standard formula converts the average RGB color value of each color patch into an XYZ color space reference value. For each color on the color chart, its XYZ color space reference value is calculated using the following formula: (2) Among them, X n,D65 Y n,D65 Z n,D65 The XYZ values ​​of the white point under the CIE standard illuminant D65 are the absolute reference standard for the CIE XYZ color space. ref Y ref Z ref This represents the XYZ color space reference value corresponding to the RGB of the color patch. fx, fy, and fz represent intermediate variables for the conversion, and t is a general variable. Let represent the transformation function, and we have: (3) (4) Combining them into a matrix yields: (5) Among them, the XYZ color space reference value matrix T ref Each row represents the reference XYZ value for a single color; 0.2069 comes from the CIE standard. The transformation matrix between the camera's RGB matrix and the color space XYZ is then calculated using the least squares method with the calibrated colors from the color chart. (6) Calculate the XYZ color space values ​​for white under standard lighting conditions; (7) Among them, X n Y n Z nThis represents the XYZ color space value of white under standard lighting conditions, where M represents the transformation matrix and R represents the color space value of white under standard lighting conditions. W G W B W This represents the RGB value of white under standard lighting conditions.

[0042] During detection, the effective region mask in the template image and the effective region in the real-time environment image are first transformed in XYZ space: (8) Where X, Y, and Z are the converted XYZ color space values, and R, G, and B are the original RGB values.

[0043] Then convert it to LAB space: (9) (10) in, From the CIE standard, L, a, and b are the L component, a component, and b component after conversion, respectively.

[0044] The effective region mask in the template image and the effective region in the real-time environment image are converted to Lab color space. The first color difference between the effective region in the real-time environment image and the effective region mask is calculated in Lab color space. The specific formula for calculating the color difference between the pixels of the effective region in the real-time environment image and the effective region mask is as follows: (11) Where △E is the pixel color difference, L i a i b i Let L be the L component, a component, and b component of the i-th pixel in the effective region of the real-time environment image, respectively. c a c b c These are the L component, a component, and b component of the corresponding pixel in the effective region mask, respectively.

[0045] The first color difference is the average value of the color difference between the effective area in the real-time environment image and all pixels of the effective area mask.

[0046] Further, in step S32, the invalid areas in the real-time environment image are first converted to grayscale to obtain a first grayscale image. The calculation method is as follows: (12) Among them, I gay I represents the converted grayscale value. R The red channel value representing the invalid region, I GThe green channel value, I, represents the invalid region. B The blue channel value represents the invalid region.

[0047] Then convert the first grayscale image to the frequency domain: (13) Where (u, v) are the horizontal and vertical coordinates in the frequency domain, respectively, m is the pixel width of the first grayscale image, n is the pixel height of the first grayscale image, I(x, y) represents the grayscale value at the (x, y) coordinate in the first grayscale image, and F(u, v) represents the frequency domain value at the (u, v) coordinate in the frequency domain.

[0048] The frequency domain values ​​are shifted to the center of the spectrum using the following formula: (14) in, This represents the new frequency domain value at coordinates (u, v) after the movement. It is a centralization operator. When (u+v) is even, its value is 1, otherwise it is -1.

[0049] Define the central low-frequency region: (15) Where r is a custom radius, which can be 0.1 times the length or width of the first grayscale image. A collection of low-frequency regions at the center.

[0050] The energy density is calculated based on the shifted frequency domain values: (16) Where P(u,v) represents the energy density at coordinate (u,v).

[0051] Calculate the first low-frequency energy based on the energy density and the set of central low-frequency regions: (17) Among them, E low1 This indicates the first low-frequency energy.

[0052] Calculate the total energy of the first grayscale image based on the energy density: (18) Among them, E total1 This represents the total energy of the first grayscale image.

[0053] The first fluctuation determination criterion is the ratio of the first low-frequency energy to the total energy of the first grayscale image: (19) Among them, R1 is the first fluctuation judgment indicator.

[0054] Furthermore, the counting parameters include paper breakage count and obstruction count; refer to Figure 5 In step S4, based on the effective area color recognition result of the real-time environmental image, the invalid area frequency domain calculation result, and the count compliance parameter, it is determined whether a paper break or occlusion has occurred, including: S41. Compare the first color difference with the preset color difference value, and mark the paper breakage in the real-time environmental image according to the comparison result. S42. Based on the number of real-time environmental images with paper break marks and the paper break count, output the paper break determination result; S43. Compare the first fluctuation determination value with the preset fluctuation value, and mark the occlusion of the real-time environment image according to the comparison result. S44. Output the occlusion determination result based on the number of real-time environmental images with occlusion markers and the occlusion count.

[0055] Specifically, the first color difference is compared with a preset color difference value. If the first color difference is less than the preset color difference value, the real-time environmental image at the current moment is marked as a paper break. The real-time environmental images marked as paper breaks in the past are analyzed, counted, and compared with the paper break count to output a paper break determination result. If the first color difference is greater than or equal to the preset color difference value, then it is determined whether the first fluctuation judgment value is greater than the preset fluctuation value. If the first fluctuation judgment value is greater than the preset fluctuation value, then the real-time environment image at the current moment is marked as occlusion, and the real-time environment images marked as occlusion in the past are analyzed, counted, and compared with the occlusion count, and the occlusion judgment result is output.

[0056] Furthermore, the counting criteria also include normal counting.

[0057] Specifically, in steps S41 and S42, the first color difference is compared with a preset color difference value. If the first color difference is less than the preset color difference value, the real-time environmental image at the current moment is marked as a paper break. It is then determined whether the real-time environmental image at the previous moment was marked as a paper break. If it was marked as a paper break, a paper break determination result is output. If it was not marked as a paper break, it is then determined whether the number of real-time environmental images marked as paper breaks within the past preset time period has reached the paper break count. If it has reached the paper break count, a paper break determination result is output. If it has not reached the paper break count, the previous determination result is output.

[0058] In steps S43 and S44, if the first color difference is greater than or equal to the preset color difference value, it is determined whether the first fluctuation judgment value is greater than the preset fluctuation value. If it is greater than the preset fluctuation value, the real-time environment image at the current moment is marked as occlusion. It is determined whether the real-time environment image at the previous moment is marked as occlusion. If it is marked as occlusion, the occlusion judgment result is output. If it is not marked as occlusion, it is determined whether the number of real-time environment images marked as occlusion within the past preset time period has reached the occlusion count. If it has reached the occlusion count, the occlusion judgment result is output. If it has not reached the occlusion count, the previous judgment result is output. If the first fluctuation judgment value is less than or equal to the preset fluctuation value, the real-time environmental image at the current moment is marked as normal, and it is determined whether the real-time environmental image at the previous moment was marked as normal. If it is marked as normal, a normal judgment result is output. If it is not marked as normal, it is determined whether the number of real-time environmental images marked as normal within the past preset time period has reached the normal count. If it has reached the normal count, a normal judgment result is output. If it has not reached the normal count, the previous judgment result is output.

[0059] Further, in step S2, color recognition is performed on the effective regions in the initial environmental image, and frequency domain calculation is performed on the invalid regions, including: The effective region mask in the template image and the effective region in the initial environment image are converted to Lab space. The second color difference between the effective region in the initial environment image and the effective region mask is calculated in Lab space. The second color difference is used as the color recognition result of the effective region in the initial environment image. The invalid regions in the initial environmental image are converted to grayscale to obtain a second grayscale image. The second grayscale image is then converted to the frequency domain. The second low-frequency energy and the total energy of the second grayscale image are calculated based on the frequency domain values. The second fluctuation judgment value is calculated based on the ratio of the second low-frequency energy and the total energy of the second grayscale image. The second fluctuation judgment value is used as the calculation result of the invalid regions in the initial environmental image.

[0060] The calculation of the second color difference and the second fluctuation judgment value is the same as that of the first color difference and the first fluctuation judgment value, and will not be repeated here.

[0061] Furthermore, based on the color recognition results of the effective area and the frequency domain calculation results of the invalid area of ​​the initial environmental image, it is determined whether paper breakage or occlusion occurs during power-on, including: The calculated second color difference is compared with the preset color difference value. If there is a second color difference that is less than the preset color difference value, it is determined that a paper break occurred when the machine is turned on. If it is determined that no paper breakage occurred when the machine was turned on, the calculated second fluctuation judgment values ​​are compared with the preset fluctuation values. If there is a second fluctuation judgment value that is greater than the preset fluctuation value, it is determined that there is obstruction when the machine was turned on.

[0062] refer to Figure 6 In some embodiments, a vision-based paper anomaly detection device is also provided, applied to a paper production line. The paper is mounted on a transmission mechanism or support mechanism, and the transmission mechanism or support mechanism and its surrounding area are used as the detection area. The paper moves forward in a preset direction via a roller. The device includes: The configuration module 201 is used to configure a template image for the detection area, determine the effective area mask for detection based on the template image, and set the counting target parameters. The initial judgment module 202 is used to collect a preset number of initial environmental images within the detection area for initial judgment when the device is powered on. The identification and calculation module 203 is used to collect real-time environmental images within the detection area at preset time intervals after an initial judgment that there are no abnormalities, determine the effective and invalid areas in the real-time environmental images according to the effective area mask, perform color recognition on the effective areas in the real-time environmental images, and perform frequency domain calculation on the invalid areas. The judgment module 204 is used to determine whether a paper break or occlusion has occurred based on the effective area color recognition result of the real-time environmental image, the invalid area frequency domain calculation result, and the count compliance parameter.

[0063] Furthermore, the effective area is the support portion area on the transmission mechanism or support mechanism that directly contacts the paper and is used to support the paper; the template image is an image of the detection area when there is no paper. The configuration module 201 determines the effective region mask for detection based on the template image, including: Select the center point of the support area in the template diagram as the seed point, and set the threshold range; Starting from the seed point, the pixel values ​​of the pixels in the neighborhood of the seed point are compared with the threshold range, and the pixels whose pixel values ​​are within the threshold range are added to the candidate region list. Repeat the following steps until the pixel value of the selected pixel has no neighboring pixels within the threshold range: Select a target pixel from the candidate region list, calculate whether the pixel values ​​of its neighboring pixels are within the threshold range, add pixels whose pixel values ​​are within the threshold range to the candidate region list, and delete the selected target pixel from the candidate region list. The effective region mask is formed by the pixels in the candidate region list obtained when there are no pixels in the neighborhood of the selected target pixel whose pixel value is within the threshold range.

[0064] Furthermore, when the initial judgment module 202 is powered on, it acquires a preset number of initial environmental images within the detection area for initial judgment, including: The effective and invalid regions in the initial environment image are determined based on the effective region mask. Color recognition is performed on the effective regions in the initial environment image and frequency domain calculation is performed on the invalid regions. Based on the color recognition results of the effective regions and the frequency domain calculation results of the invalid regions in the initial environment image, it is determined whether paper breakage or occlusion occurs during power-on. If no paper breakage or occlusion occurs, it is determined that the initial judgment is normal.

[0065] Further, the identification and calculation module 203 determines the valid and invalid regions in the real-time environment image based on the valid region mask; or, the initial judgment module 202 determines the valid and invalid regions in the initial environment image based on the valid region mask, including: The initial environment image / real-time environment image is compared with the template image. The regions in the initial environment image / real-time environment image that are at the same pixel position as the effective region mask are determined as effective regions, and the regions other than the effective regions are determined as invalid regions.

[0066] Furthermore, the recognition and calculation module 203 performs color recognition on the effective regions in the real-time environmental image and frequency domain calculation on the invalid regions, including: The effective region mask in the template image and the effective region in the real-time environment image are converted to Lab space. The first color difference between the effective region in the real-time environment image and the effective region mask is calculated in Lab space. The first color difference is used as the color recognition result of the effective region in the real-time environment image. The invalid regions in the real-time environment image are converted to grayscale to obtain a first grayscale image. The first grayscale image is then converted to the frequency domain. The first low-frequency energy and the total energy of the first grayscale image are calculated based on the frequency domain values. The first fluctuation judgment value is calculated based on the ratio of the first low-frequency energy to the total energy of the first grayscale image. The first fluctuation judgment value is used as the calculation result of the invalid regions in the real-time environment image.

[0067] Furthermore, the counting parameters include paper breakage count and obstruction count; The judgment module 204 determines whether a paper breakage or occlusion has occurred based on the effective area color recognition result of the real-time environmental image, the invalid area frequency domain calculation result, and the count compliance parameter, including: The first color difference is compared with a preset color difference value, and a paper break mark is made on the real-time environmental image based on the comparison result; Based on the number of real-time environmental images with paper break marks and the paper break count, output the paper break determination result; The first fluctuation determination value is compared with the preset fluctuation value, and the real-time environmental image is marked with occlusion based on the comparison result. Based on the number of real-time environmental images with occlusion markers and the occlusion count, an occlusion determination result is output.

[0068] The judgment module 204 is further configured to: compare the first color difference with a preset color difference value; if the first color difference is less than the preset color difference value, mark the current real-time environmental image as a paper break, analyze and count the number of past real-time environmental images marked as paper breaks, compare them with the paper break count, and output a paper break judgment result. If the first color difference is greater than or equal to the preset color difference value, then it is determined whether the first fluctuation judgment value is greater than the preset fluctuation value. If the first fluctuation judgment value is greater than the preset fluctuation value, then the real-time environment image at the current moment is marked as occlusion, and the real-time environment images marked as occlusion in the past are analyzed, counted, and compared with the occlusion count, and the occlusion judgment result is output.

[0069] Furthermore, the counting threshold parameter also includes normal counting; the judgment module 204 is also used for: The first color difference is compared with a preset color difference value. If the first color difference is less than the preset color difference value, the real-time environmental image at the current moment is marked as a paper break. It is determined whether the real-time environmental image at the previous moment was marked as a paper break. If it was marked as a paper break, the paper break determination result is output. If it was not marked as a paper break, it is determined whether the number of real-time environmental images marked as paper breaks within the past preset time period has reached the paper break count. If it has reached the paper break count, the paper break determination result is output. If it has not reached the paper break count, the previous determination result is output. If the first color difference is greater than or equal to the preset color difference value, then determine whether the first fluctuation judgment value is greater than the preset fluctuation value. If it is greater than the preset fluctuation value, then mark the current real-time environment image as occluded. Determine whether the previous real-time environment image was marked as occluded. If it was marked as occluded, then output the occlusion judgment result. If it was not marked as occluded, then determine whether the number of real-time environment images marked as occluded within the past preset time period has reached the occlusion count. If it has reached the occlusion count, then output the occlusion judgment result. If it has not reached the occlusion count, then output the previous judgment result. If the first fluctuation judgment value is less than or equal to the preset fluctuation value, the real-time environmental image at the current moment is marked as normal, and it is determined whether the real-time environmental image at the previous moment was marked as normal. If it is marked as normal, a normal judgment result is output. If it is not marked as normal, it is determined whether the number of real-time environmental images marked as normal within the past preset time period has reached the normal count. If it has reached the normal count, a normal judgment result is output. If it has not reached the normal count, the previous judgment result is output.

[0070] Furthermore, the initial judgment module 202 performs color recognition on the valid regions in the initial environment image and frequency domain calculation on the invalid regions, including: The effective region mask in the template image and the effective region in the initial environment image are converted to Lab space. The second color difference between the effective region in the initial environment image and the effective region mask is calculated in Lab space. The second color difference is used as the color recognition result of the effective region in the initial environment image. The invalid regions in the initial environmental image are converted to grayscale to obtain a second grayscale image. The second grayscale image is then converted to the frequency domain, and the second low-frequency energy and the total energy of the second grayscale image are calculated. A second fluctuation judgment value is calculated based on the ratio of the second low-frequency energy to the total energy of the second grayscale image, and the second fluctuation judgment value is used as the calculation result of the invalid regions in the initial environmental image.

[0071] Furthermore, the initial judgment module 202 determines whether paper breakage or obstruction occurs during power-on based on the effective area color recognition result and the invalid area frequency domain calculation result of the initial environmental image, including: The calculated second color difference is compared with the preset color difference value. If there is a second color difference that is less than the preset color difference value, it is determined that a paper break occurred when the machine is turned on. If it is determined that no paper breakage occurred when the machine was turned on, the calculated second fluctuation judgment values ​​are compared with the preset fluctuation values. If there is a second fluctuation judgment value that is greater than the preset fluctuation value, it is determined that there is obstruction when the machine was turned on.

[0072] The vision-based paper anomaly detection method and apparatus provided in the above embodiments have at least the following beneficial effects: (1) Based on visual detection of whether paper breaks occur during the paper production process, the reliability and stability of paper break detection can be improved, and possible occlusion phenomena can be judged, making it more widely applicable; (2) Color difference determination is performed in Lab space and occlusion is determined by frequency domain, which can effectively avoid interference from image noise and improve the stability of detection; (3) Combining the setting of appropriate counting parameters during the detection process can reduce the probability of false detection and improve the accuracy of detection.

[0073] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A vision-based paper anomaly detection method, characterized in that, Applied to a paper production line, where paper is mounted on a transmission mechanism or a support mechanism; the method includes: The transmission mechanism or support mechanism and its surrounding area are used as the detection area. A template diagram is configured for the detection area. The effective area mask for detection is determined according to the template diagram, and the counting target parameters are set. At the start of detection, a preset number of initial environmental images are collected within the detection area for initial assessment; After an initial assessment that no abnormalities are found, real-time environmental images within the detection area are acquired at preset time intervals. The effective and invalid regions in the real-time environmental images are determined based on the effective region mask. Color recognition is performed on the effective regions in the real-time environmental images, and frequency domain calculation is performed on the invalid regions. Based on the color recognition results of the effective area of ​​the real-time environmental image, the frequency domain calculation results of the invalid area, and the counting target parameters, it is determined whether a paper break or occlusion has occurred.

2. The method according to claim 1, characterized in that, The effective area is the support portion on the transmission mechanism or support mechanism that is in direct contact with the paper and is used to support the paper. The template image is an image of the detection area when there is no paper. Determining the effective region mask for detection based on the template image includes: Select the center point of the support area in the template diagram as the seed point, and set the threshold range; Starting from the seed point, the pixel values ​​of the pixels in the neighborhood of the seed point are compared with the threshold range, and the pixels whose pixel values ​​are within the threshold range are added to the candidate region list. Repeat the following steps until the pixel value of the selected pixel has no neighboring pixels within the threshold range: Select a target pixel from the candidate region list, calculate whether the pixel values ​​of its neighboring pixels are within the threshold range, add pixels whose pixel values ​​are within the threshold range to the candidate region list, and delete the selected target pixel from the candidate region list. When the pixel value of a selected target pixel has no neighboring pixels within the threshold range, the pixels in the obtained candidate region list constitute the effective region mask.

3. The method according to claim 1, characterized in that, At the start of detection, a preset number of initial environmental images are collected within the detection area for initial assessment, including: The effective and invalid regions in the initial environment image are determined based on the effective region mask. Color recognition is performed on the effective regions in the initial environment image and frequency domain calculation is performed on the invalid regions. Based on the color recognition results of the effective regions and the frequency domain calculation results of the invalid regions in the initial environment image, it is determined whether paper breakage or occlusion occurs during power-on. If no paper breakage or occlusion occurs, it is determined that the initial judgment is normal.

4. The method according to claim 3, characterized in that, Determining the valid and invalid regions in the initial environment image based on the valid region mask, or determining the valid and invalid regions in the real-time environment image based on the valid region mask, includes: The initial environment image / real-time environment image is compared with the template image. The regions in the initial environment image / real-time environment image that have the same pixel position as the effective region mask are determined as effective regions, and the regions other than the effective regions are determined as invalid regions.

5. The method according to claim 1, characterized in that, Color recognition of valid regions and frequency domain calculation of invalid regions in the real-time environmental image include: The effective region mask in the template image and the effective region in the real-time environment image are converted to Lab space. The first color difference between the effective region in the real-time environment image and the effective region mask is calculated in Lab space. The first color difference is used as the color recognition result of the effective region in the real-time environment image. The invalid regions in the real-time environment image are converted to grayscale to obtain a first grayscale image. The first grayscale image is then converted to the frequency domain. The first low-frequency energy and the total energy of the first grayscale image are calculated based on the frequency domain values. The first fluctuation judgment value is calculated based on the ratio of the first low-frequency energy to the total energy of the first grayscale image. The first fluctuation judgment value is used as the calculation result of the invalid regions in the real-time environment image.

6. The method according to claim 5, characterized in that, The counting parameters include paper breakage count and obstruction count; Based on the color recognition results of the effective area of ​​the real-time environmental image, the frequency domain calculation results of the invalid area, and the count compliance parameters, it is determined whether a paper break or occlusion has occurred, including: The first color difference is compared with a preset color difference value, and a paper break mark is made on the real-time environmental image based on the comparison result; Based on the number of real-time environmental images with paper break marks and the paper break count, output the paper break determination result; The first fluctuation determination value is compared with the preset fluctuation value, and the real-time environmental image is marked with occlusion based on the comparison result. Based on the number of real-time environmental images with occlusion markers and the occlusion count, an occlusion determination result is output.

7. The method according to claim 6, characterized in that, The first color difference is compared with a preset color difference value. If the first color difference is less than the preset color difference value, the real-time environmental image at the current moment is marked as a paper break. The real-time environmental images marked as paper breaks in the past are analyzed, counted, and compared with the paper break count. The paper break determination result is then output. If the first color difference is greater than or equal to the preset color difference value, then it is determined whether the first fluctuation judgment value is greater than the preset fluctuation value. If the first fluctuation judgment value is greater than the preset fluctuation value, then the real-time environment image at the current moment is marked as occlusion, and the real-time environment images marked as occlusion in the past are analyzed, counted, and compared with the occlusion count, and the occlusion judgment result is output.

8. The method according to claim 3, characterized in that, The process of color recognition for valid regions and frequency domain calculation for invalid regions in the initial environmental image includes: The effective region mask in the template image and the effective region in the initial environment image are converted to Lab space. The second color difference between the effective region in the initial environment image and the effective region mask is calculated in Lab space. The second color difference is used as the color recognition result of the effective region in the initial environment image. The invalid regions in the initial environmental image are converted to grayscale to obtain a second grayscale image. The second grayscale image is then converted to the frequency domain. The second low-frequency energy and the total energy of the second grayscale image are calculated based on the frequency domain values. The second fluctuation judgment value is calculated based on the ratio of the second low-frequency energy and the total energy of the second grayscale image. The second fluctuation judgment value is used as the calculation result of the invalid regions in the initial environmental image.

9. The method according to claim 8, characterized in that, Based on the color recognition results of the effective area and the frequency domain calculation results of the invalid area in the initial environmental image, determine whether paper breaks or obstructions occur during power-on, including: The calculated second color difference is compared with the preset color difference value. If there is a second color difference that is less than the preset color difference value, it is determined that a paper break occurred when the machine is turned on. If it is determined that no paper breakage occurred when the machine was turned on, the calculated second fluctuation judgment values ​​are compared with the preset fluctuation values. If there is a second fluctuation judgment value that is greater than the preset fluctuation value, it is determined that there is obstruction when the machine was turned on.

10. A vision-based paper anomaly detection device, characterized in that, An apparatus for use in paper production lines, wherein paper is mounted on a transmission mechanism or support mechanism, and the transmission mechanism or support mechanism and its surrounding area are used as the detection area; the apparatus includes: The configuration module is used to configure a template image for the detection area, determine the effective area mask for detection based on the template image, and set the counting target parameters. The initial judgment module is used to collect a preset number of initial environmental images within the detection area for initial settings when detection begins. The identification and calculation module is used to collect real-time environmental images within the detection area at preset time intervals after an initial judgment that there are no abnormalities, determine the effective and invalid areas in the real-time environmental images according to the effective area mask, perform color recognition on the effective areas in the real-time environmental images, and perform frequency domain calculation on the invalid areas. The judgment module is used to determine whether a paper break or occlusion has occurred based on the effective area color recognition result of the real-time environmental image, the invalid area frequency domain calculation result, and the count compliance parameter.

Images