A vision-based paper abnormality detection method and device

Abstract

The application provides a visual-based paper abnormality detection method and device, the method comprising: configuring a template image for a detection area, determining an effective area mask for detection according to the template image, and setting a count threshold parameter; collecting a preset number of initial environment images in the detection area to perform initial judgment when starting detection; after completing the initial judgment without abnormality, collecting real-time environment images at a preset time interval, determining effective and ineffective areas in the real-time environment images according to the effective area mask, performing color recognition on the effective area in the real-time environment images and frequency domain calculation on the ineffective area; and judging whether paper breakage and shielding occur according to the color recognition result of the effective area in the real-time environment images, the frequency domain calculation result of the ineffective area and the count threshold parameter; the method can improve the reliability of paper breakage detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, in particular to a paper abnormality detection method and device based on vision. BACKGROUND

[0002] In the industrial paper production, the following links are mainly included: material preparation, pulping, papermaking, finishing and processing, wherein the papermaking link needs to beat the paper, adjust the material, form, press, dry. In these process flows, the paper may be broken. If the broken paper is not found in time, the damaged paper will be out, causing more chaos and damage, and there is no output during the broken paper period, not only wasting materials, but also reducing the running efficiency of the whole production line.

[0003] The research on broken paper detection at home and abroad focuses on photoelectric sensor detection method, for example, the patent text CN105784601B proposes a broken paper detection device and its automatic correction method. The broken paper detection device with automatic correction function includes a detection system and a correction triggering system, the detection system is used for color sampling on a predetermined detection area, and the sampled color value is compared with the standard color value to judge whether the paper web is broken, and outputs a broken paper detection signal when the paper web is broken, and the correction triggering system is used to trigger the detection system to automatically correct the standard color value according to the broken paper detection signal. The color sampling application is a light emitter and a light receiver, the light emitter is used to project light to the surface of the paper, and the light receiver is used to receive the light reflected from the surface of the paper; then the difference between the received color and the standard color is calculated to determine whether the paper is broken. But this scheme has the problems of poor stability and poor universality. It also does not consider the problems of many sundries in the environment and paper shielding in the actual production of paper mills. And many paper pulp is placed on filter cloth, even if the paper is broken, the filter cloth will block the signal of the photoelectric sensor, so it cannot be detected. SUMMARY

[0004] The present application provides a paper abnormality detection method and device based on vision, which can improve the reliability and stability of broken paper detection in the paper production process, and can identify the shielding condition.

[0005] A paper abnormality detection method based on vision is applied to a paper production line, and the paper is arranged on a transmission mechanism or a supporting mechanism; the method comprises:

[0006] The transmission mechanism or the supporting mechanism and the surrounding area thereof are taken as a detection area, a template graph is configured for the detection area, an effective area mask of the detection is determined according to the template graph, and a counting standard parameter is set;

[0007] When starting detection, a preset number of initial environment images in the detection area are collected for initial setting;

[0008] After initial judgment of no abnormality, real-time environment images in the detection area are collected at preset time intervals, valid areas and invalid areas in the real-time environment images are determined according to the effective area mask, color recognition is performed on the valid areas in the real-time environment images and frequency domain calculation is performed on the invalid areas;

[0009] According to the color recognition result of the valid area of the real-time environment image, the frequency domain calculation result of the invalid area and the counting standard parameter, it is judged whether paper breakage and shielding occur at present.

[0010] Further, the effective area is a support part area on a transmission mechanism or a support mechanism which directly contacts the paper and supports the paper; and the template image is an image of the detection area without the paper.

[0011] According to the template image, an effective area mask of the cylinder is determined, comprising:

[0012] A center point of the support part area in the template image is selected as a seed point, and a threshold range is set;

[0013] Taking the seed point as a starting point, pixel values of pixel points in a neighborhood of the seed point are compared with the threshold range, and pixel points with pixel values in the threshold range are added to a candidate area list;

[0014] The following steps are repeated until there is no pixel point in the neighborhood of the selected pixel point with a pixel value in the threshold range:

[0015] From the candidate area list, a target pixel point is selected to calculate pixel values of pixel points in its neighborhood, and pixel points with pixel values in the threshold range are added to the candidate area list, and the selected target pixel point is deleted from the candidate area list;

[0016] When there is no pixel point in the neighborhood of the selected target pixel point with a pixel value in the threshold range, pixel points in the obtained candidate area list constitute the effective area mask.

[0017] Further, when starting detection, a preset number of initial environment images in the detection area are collected for initial judgment, comprising:

[0018] According to the effective area mask, valid areas and invalid areas in the initial environment images are determined, color recognition is performed on the valid areas in the initial environment images and frequency domain calculation is performed on the invalid areas, whether paper breakage and shielding occur when starting is judged according to the color recognition result of the valid area of the initial environment image and the frequency domain calculation result of the invalid area, and no paper breakage and shielding are determined to be no abnormality in initial judgment.

[0019] Further, determining the valid region and invalid region in the initial environment image according to the valid region mask, or determining the valid region and invalid region in the real-time environment image according to the valid region mask, comprises:

[0020] Comparing the initial environment image / real-time environment image with the template image, determining the region in the initial environment image / real-time environment image with the same pixel position as the valid region mask as the valid region, and determining the region other than the valid region as the invalid region.

[0021] Further, performing color recognition on the valid region in the real-time environment image and frequency domain calculation on the invalid region, comprises:

[0022] Converting the valid region mask in the template image and the valid region in the real-time environment image to Lab space, calculating the first color difference between the valid region in the real-time environment image and the valid region mask in the Lab space, and taking the first color difference as the color recognition result of the valid region in the real-time environment image;

[0023] Converting the invalid region in the real-time environment image to a first gray scale image, converting the first gray scale image to the frequency domain, calculating the first low frequency energy and the total energy of the first gray scale image according to the frequency domain value, calculating the first fluctuation determination value according to the ratio of the first low frequency energy and the total energy of the first gray scale image, and taking the first fluctuation determination value as the calculation result of the invalid region in the real-time environment image.

[0024] Further, the counting threshold parameter comprises a paper break counting and an occlusion counting;

[0025] According to the color recognition result of the valid region, the frequency domain calculation result of the invalid region in the real-time environment image, and the counting threshold parameter, determining whether the paper break and occlusion occur at present, comprises:

[0026] Comparing the first color difference with a preset color difference value, and marking the real-time environment image according to the comparison result;

[0027] According to the number of the real-time environment image with the paper break mark and the paper break counting, outputting a paper break determination result;

[0028] Comparing the first fluctuation determination value with a preset fluctuation value, and marking the real-time environment image according to the comparison result;

[0029] According to the number of the real-time environment image with the occlusion mark and the occlusion counting, outputting an occlusion determination result.

[0030] Further, the first color difference is compared with a preset color difference value, if the first color difference is less than the preset color difference value, the real-time environment image at the current moment is marked as paper break, and the past real-time environment images marked as paper break are analyzed, counted and compared with the paper break count, and a paper break determination result is output;

[0031] If the first color difference is greater than or equal to the preset color difference value, it is determined whether the first fluctuation determination value is greater than a preset fluctuation value, if the first fluctuation determination value is greater than the preset fluctuation value, the real-time environment image at the current moment is marked as shielding, and the past real-time environment images marked as shielding are analyzed, counted and compared with the shielding count, and a shielding determination result is output.

[0032] Further, the effective area in the initial environment image is color-identified and the invalid area is frequency-domain calculated, including:

[0033] The effective area mask in the template image and the effective area in the initial environment image are converted to Lab space, the second color difference between the effective area in the initial environment image and the effective area mask is calculated in the Lab space, and the second color difference is taken as the effective area color identification result in the initial environment image;

[0034] The invalid area in the initial environment image is converted to gray scale to obtain a second gray scale image, the second gray scale image is converted to a frequency domain, a second low-frequency energy and a second total energy of the gray scale image are calculated according to the frequency domain value, a second fluctuation determination value is calculated according to the ratio of the second low-frequency energy and the second total energy of the gray scale image, and the second fluctuation determination value is taken as the calculation result of the invalid area in the initial environment image.

[0035] Further, whether paper break and shielding occur at startup is determined according to the effective area color identification result and the invalid area frequency domain calculation result of the initial environment image, including:

[0036] Each second color difference obtained by calculation is compared with a preset color difference value, if there is a second color difference less than the preset color difference value, it is determined that paper break occurs at startup;

[0037] If it is determined that paper break does not occur at startup, each second fluctuation determination value obtained by calculation is compared with a preset fluctuation value, if there is a second fluctuation determination value greater than the preset fluctuation value, it is determined that shielding occurs at startup.

[0038] A paper abnormality detection device based on vision is applied to a paper production line, a paper is arranged on a transmission mechanism or a supporting mechanism, and the transmission mechanism or the supporting mechanism and the surrounding area thereof are taken as a detection area; the device comprises:

[0039] A configuration module is configured to configure a template image for the detection area, determine an effective area mask of the detection according to the template image, and set a count threshold parameter;

[0040] An initial judgment module is configured to collect a preset number of initial environment images in the detection area to perform initial setting when starting detection.

[0041] An identification and calculation module is configured to collect real-time environment images in the detection area according to a preset time interval after the initial judgment module determines that there is no abnormality, determine effective areas and invalid areas in the real-time environment images according to the effective area mask, perform color identification on the effective areas in the real-time environment images, and perform frequency domain calculation on the invalid areas.

[0042] A judgment module is configured to determine whether there is paper breakage and occlusion according to the color identification result of the effective areas in the real-time environment images, the frequency domain calculation result of the invalid areas, and the count threshold parameter.

[0043] Further, the effective area is a support part area on a transmission mechanism or a support mechanism that directly contacts the paper to support the paper; and the template image is an image of the detection area without the paper.

[0044] The configuration module determines the effective area mask of the detection according to the template image, and includes:

[0045] A center point of the support part area in the template image is selected as a seed point, and a threshold range is set.

[0046] The pixel values of the pixel points in the neighborhood of the seed point are compared with the threshold range, and the pixel points with pixel values in the threshold range are added to a candidate area list.

[0047] The following steps are repeated until there is no pixel point in the neighborhood of the selected pixel point with a pixel value in the threshold range:

[0048] Whether the pixel values of the pixel points in the neighborhood of the selected target pixel point are in the threshold range is calculated, the pixel points with pixel values in the threshold range are added to the candidate area list, and the selected target pixel point is deleted from the candidate area list.

[0049] The pixel points in the candidate area list obtained when there is no pixel point in the neighborhood of the selected target pixel point with a pixel value in the threshold range constitute the effective area mask.

[0050] Further, the initial judgment module collects a preset number of initial environment images in the detection area to perform initial judgment when starting, and includes:

[0051] The effective region and the invalid region in the initial environment image are determined according to the effective region mask, color recognition is performed on the effective region in the initial environment image, and frequency domain calculation is performed on the invalid region, and whether paper breakage and occlusion occur during startup is judged according to the color recognition result of the effective region and the frequency domain calculation result of the invalid region in the initial environment image, and it is determined that the initial judgment is normal if no paper breakage and occlusion occur.

[0052] Further, the initial judgment module determines the effective region and the invalid region in the initial environment image according to the effective region mask, or the recognition and calculation module determines the effective region and the invalid region in the real-time environment image according to the effective region mask, comprising:

[0053] The initial environment image / real-time environment image is compared with the template image, the region in the initial environment image / real-time environment image which is the same as the pixel position of the effective region mask is determined as the effective region, and the region other than the effective region is determined as the invalid region.

[0054] Further, the recognition and calculation module performs color recognition on the effective region in the real-time environment image and frequency domain calculation on the invalid region, comprising:

[0055] The effective region mask in the template image and the effective region in the real-time environment image are converted to Lab space, the first color difference between the effective region in the real-time environment image and the effective region mask is calculated in the Lab space, and the first color difference is taken as the color recognition result of the effective region in the real-time environment image.

[0056] The invalid region in the real-time environment image is converted to grayscale to obtain a first grayscale image, the first grayscale image is converted to the frequency domain, the first low-frequency energy and the total energy of the first grayscale image are calculated according to the frequency domain value, the first fluctuation determination value is calculated according to the ratio of the first low-frequency energy and the total energy of the first grayscale image, and the first fluctuation determination value is taken as the calculation result of the invalid region in the real-time environment image.

[0057] Further, the count threshold parameter includes a paper breakage count and an occlusion count.

[0058] The judgment module judges whether paper breakage and occlusion occur at present according to the color recognition result of the effective region, the frequency domain calculation result of the invalid region in the real-time environment image and the count threshold parameter, comprising:

[0059] The first color difference is compared with a preset color difference value, and the real-time environment image is marked for paper breakage according to the comparison result.

[0060] A paper breakage determination result is output according to the number of real-time environment images with paper breakage marks and the paper breakage count.

[0061] comparing the first fluctuation determination value with a preset fluctuation value, and performing occlusion marking on the real-time environment image according to a comparison result;

[0062] outputting an occlusion determination result according to a number of the real-time environment images with the occlusion marking and the occlusion count.

[0063] Further, the judging module is further configured to: compare the first color difference with a preset color difference value, if the first color difference is less than the preset color difference value, mark the real-time environment image at the current time as paper break, analyze, count and compare the real-time environment images marked as paper break in the past with the paper break count, and output a paper break determination result;

[0064] if the first color difference is greater than or equal to the preset color difference value, determine whether the first fluctuation determination value is greater than a preset fluctuation value, if the first fluctuation determination value is greater than the preset fluctuation value, mark the real-time environment image at the current time as occlusion, analyze, count and compare the real-time environment images marked as occlusion in the past with the occlusion count, and output an occlusion determination result.

[0065] Further, the initial judging module performs color recognition on the effective area in the initial environment image and frequency domain calculation on the invalid area, including:

[0066] convert the effective area mask in the template image and the effective area in the initial environment image to Lab space, calculate a second color difference between the effective area in the initial environment image and the effective area mask in the Lab space, and take the second color difference as a color recognition result of the effective area in the initial environment image;

[0067] convert the invalid area in the initial environment image to a second gray scale image, convert the second gray scale image to a frequency domain, calculate a second low frequency energy and a total energy of the second gray scale image according to the frequency domain value, calculate a second fluctuation determination value according to a ratio of the second low frequency energy and the total energy of the second gray scale image, and take the second fluctuation determination value as a calculation result of the invalid area in the initial environment image.

[0068] Further, the initial judging module determines whether paper break and occlusion occur at the time of starting up according to the color recognition result of the effective area in the initial environment image and the frequency domain calculation result of the invalid area, including:

[0069] compare each of the calculated second color differences with a preset color difference value, if there is a second color difference less than the preset color difference value, determine that paper break occurs at the time of starting up;

[0070] If it is determined that no paper breakage occurs at the start-up, the obtained second fluctuation determination values are compared with preset fluctuation values, and if there is a second fluctuation determination value greater than the preset fluctuation value, it is determined that the shielding occurs at the start-up.

[0071] The visual-based paper abnormality detection method and device provided by the application have at least the following beneficial effects:

[0072] (1) The visual-based method can detect whether paper breakage occurs in the paper production process, improve the reliability and stability of the paper breakage detection, determine the possible shielding phenomenon, and has wider applicability.

[0073] (2) The color difference is determined in the Lab space, the shielding is determined in the frequency domain, the interference caused by the image noise can be effectively avoided, and the stability of the detection is improved.

[0074] (3) The appropriate counting standard parameters are set in the detection process, the false detection probability is reduced, and the accuracy of the detection is improved. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 The flowchart of one embodiment of the visual-based paper abnormality detection method provided by the application is shown.

[0076] Figure 2 The scene diagram of the normal production in the visual-based paper abnormality detection method provided by the application is shown.

[0077] Figure 3 The scene diagram of the paper breakage in the visual-based paper abnormality detection method provided by the application is shown.

[0078] Figure 4 The scene diagram of the shielding in the visual-based paper abnormality detection method provided by the application is shown.

[0079] Figure 5 The flowchart of another embodiment of the visual-based paper abnormality detection method provided by the application is shown.

[0080] Figure 6 The structural diagram of one embodiment of the visual-based paper abnormality detection device provided by the application is shown. DETAILED DESCRIPTION

[0081] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in combination with the drawings in the specification and the specific embodiments.

[0082] REFERENCE Figure 1 In some embodiments, a visual-based paper abnormality detection method is provided, which is applied to a paper production line, and paper is arranged on a transmission mechanism or a supporting mechanism. The method comprises:

[0083] S1, the transmission mechanism or support mechanism and its surrounding area as a detection area, configure the template map for the detection area, determine the effective area mask of the detection according to the template map, and set the count threshold parameter;

[0084] S2, collect a preset number of initial environment images in the detection area when starting detection for initial judgment;

[0085] S3, after initial judgment without exception, collect real-time environment images in the detection area according to the preset time interval, determine the effective area and invalid area in the real-time environment image according to the effective area mask, and perform color recognition on the effective area in the real-time environment image and frequency domain calculation on the invalid area;

[0086] S4, according to the effective area color recognition result of the real-time environment image, the invalid area frequency domain calculation result and the count threshold parameter, judge whether the current paper break and shielding occur.

[0087] Reference Figures 2 to 4 , Figure 2 For the normal production scene of the cylinder in the paper drying process link, Figure 3 For the scene of paper break, Figure 4 For the scene of shielding.

[0088] Specifically, on the paper production line, in some process links, the paper is set on the transmission mechanism to advance in a predetermined direction, or is set on the support mechanism for processing, for example Figures 2 to 4 In the drying link of the paper, the paper is dried by the drying cylinder, and at the same time the drying cylinder rotates to convey the paper to the next link. Taking the drying link as an example, the detection area is the drying cylinder and its surrounding preset area.

[0089] Further, in step S1, when the system is started, the template map for detection is set by using the matching software. For example, the template map corresponding to the actual detection area can be found from the pre-configured database. The template map is the detection area image without paper. The template map is consistent with the actual detection area. On the interface, the support part area directly contacting the paper for supporting the paper is selected, for example, in the drying link, the drying cylinder area can be selected. This part of the area is the effective area, and the remaining part is set as the invalid area. Then, the algorithm determines the effective area mask according to the template map. If the setting is abandoned, the detection system exits. At the same time, the count threshold parameter is set. Considering that the system may have sudden conditions, such as image frame loss, image picture abnormality caused by network lag, and scene debris interference, by setting appropriate count threshold parameter, the system false detection probability can be greatly reduced, and the system stability can be improved.

[0090] Further, in step S1, the effective area is a supporting part area on the transmission mechanism or the supporting mechanism which directly contacts the paper for supporting the paper; and the template image is an image of the detection area without the paper;

[0091] According to the template image, the effective area mask is determined, comprising:

[0092] A center point of the supporting part area in the template image is selected as a seed point, and a threshold range is set;

[0093] Starting from the seed point, pixel values of pixel points in a neighborhood of the seed point are compared with the threshold range, and pixel points with pixel values in the threshold range are added to a candidate region list;

[0094] The following steps are repeated until there is no pixel point in a neighborhood of a selected pixel point with a pixel value in the threshold range:

[0095] A target pixel point is selected from the candidate region list to calculate pixel values of pixel points in a neighborhood of the target pixel point, and pixel points with pixel values in the threshold range are added to the candidate region list, and the selected target pixel point is deleted from the candidate region list;

[0096] When there is no pixel point in a neighborhood of the selected target pixel point with a pixel value in the threshold range, pixel points in the obtained candidate region list constitute the effective area mask.

[0097] The invalid area in the embodiment is other areas except the effective area.

[0098] Further, in step S2, a preset number of initial environment images in a detection area are collected for initial judgment when starting detection, comprising:

[0099] According to the effective area mask, effective areas and invalid areas in the initial environment images are determined, color recognition is performed on the effective areas in the initial environment images, frequency domain calculation is performed on the invalid areas, whether paper breakage and occlusion occur when starting are determined according to color recognition results of the effective areas in the initial environment images and frequency domain calculation results of the invalid areas, and it is determined that the initial judgment is normal if no paper breakage and occlusion occur.

[0100] Since the detection system is prone to paper breakage or occlusion when starting, whether paper breakage and occlusion occur is determined according to the initial environment images, if not, it is marked that the system completes preliminary setting, and if paper breakage and occlusion occur, the process is ended.

[0101] Further, in step S3, the effective area and the ineffective area in the initial environment image are determined according to the effective area mask, or in step S2, the effective area and the ineffective area in the real-time environment image are determined according to the effective area mask, comprising:

[0102] The initial environment image / real-time environment image is compared with the template image, the area in the initial environment image / real-time environment image which is the same as the pixel position of the effective area mask is determined as the effective area, and the area other than the effective area is determined as the ineffective area.

[0103] Further, in step S3, the effective area in the real-time environment image is subjected to color recognition and the ineffective area is subjected to frequency domain calculation, comprising:

[0104] S31, the effective area mask in the template image and the effective area in the real-time environment image are converted to Lab space, the first color difference between the effective area in the real-time environment image and the effective area mask is calculated in the Lab space, and the first color difference is taken as the color recognition result of the effective area in the real-time environment image;

[0105] S32, the ineffective area in the real-time environment image is subjected to gray scale conversion to obtain a first gray scale image, the first gray scale image is converted to a frequency domain, the first low frequency energy and the total energy of the first gray scale image are calculated according to the frequency domain value, the first fluctuation determination value is calculated according to the ratio of the first low frequency energy and the total energy of the first gray scale image, and the first fluctuation determination value is taken as the calculation result of the ineffective area in the real-time environment image.

[0106] Specifically, in step S31, first, the camera is subjected to chroma coordinate calibration, the camera is placed in an extinction environment box, a D65 standard light source illumination system is used for illumination, and a standard 24-color color card is photographed by using the camera. And lock the white balance of the camera, disable the automatic gain, and turn off the digital noise reduction function. Then record the average value of the RGB color of each color block in the color card, denoted as:

[0107] (1)

[0108] Wherein, the color matrix C ref Each row is the average value of a color, and the color card has 24 colors, and the matrix corresponds to 24 rows. Then, the conversion matrix is calibrated and calculated, the reference LAB value L ref , a ref , b ref given by the color card and the CIE standard formula are used to convert the average value of the RGB color of each color block to the reference value of the XYZ color space, and for each color of the color card, the reference value of the XYZ color space is calculated by the following formula:

[0109] (2)

[0110] where X n,D65 , Y n,D65 , Z n,D65 are the white point XYZ values under CIE standard illuminant D65, which are the absolute reference basis of the CIE XYZ color space, X ref , Y ref , Z ref represent the XYZ color space reference values corresponding to the RGB of the color block, fx, fy, and fz represent intermediate variables of conversion, and t is a general variable, represents a conversion function, and has:

[0111] (3)

[0112] (4)

[0113] Combining them into a matrix can obtain:

[0114] (5)

[0115] where each row of the XYZ color space reference value matrix T ref represents the reference XYZ value of a single color, and 0.2069 comes from the CIE standard. Then, the conversion matrix of the camera RGB matrix and the color space XYZ is calculated, and the conversion matrix M is calculated by using the least square method through the calibration color of the color card:

[0116] (6)

[0117] The XYZ color space value of white under standard illumination is calculated;

[0118] (7)

[0119] where X n , Y n , Z n represent the XYZ color space value of white under standard illumination, M represents the conversion matrix, and R W , G W , B W represent the RGB value of white under standard illumination.

[0120] When detecting, the effective area mask in the template map and the effective area in the real-time environment image are first converted into XYZ space:

[0121] (8)

[0122] Wherein, X, Y, Z are the converted XYZ color space values, R, G, B are the converted RGB values.

[0123] Then convert it to LAB space:

[0124] ; (9)

[0125] ; (10)

[0126] Wherein, From the CIE standard, L, a, b are the converted L component, a component and b component respectively.

[0127] Convert the effective area mask in the template map and the effective area in the real-time environment image to Lab space, calculate the first color difference between the effective area in the real-time environment image and the effective area mask in Lab space, wherein the color difference of the pixels of the effective area in the real-time environment image and the effective area mask is calculated as follows:

[0128] ; (11)

[0129] Wherein, △E is the pixel color difference, L i , a i , b i L c , a c , b c are the L component, a component and b component of the i-th pixel in the effective area in the real-time environment image, L gay , a R , b G are the L component, a component and b component of the corresponding pixel in the effective area mask.

[0130] The first color difference is the average value of the color difference of all pixels of the effective area in the real-time environment image and the effective area mask.

[0131] Further, in step S32, first, the invalid area in the real-time environment image is converted to grayscale to obtain a first grayscale image, and the calculation method is:

[0132] ; (12)

[0133] Wherein, I gay represents the converted grayscale value, I R represents the red channel value of the invalid area, I G represents the green channel value of the invalid area, I B represents the blue channel value of the invalid area.

[0134] Then convert the first grayscale image to the frequency domain:

[0135] (13)

[0136] where (u, v) are horizontal and vertical coordinates in frequency domain respectively, m is the first gray image pixel width, n is the first gray image pixel height, I(x, y) represents the gray value at (x, y) coordinates in the first gray image, and F(u, v) represents the frequency domain value at frequency domain coordinates (u, v).

[0137] The frequency domain value is moved to the frequency spectrum center according to the following formula:

[0138] (14)

[0139] where, represents the new frequency domain value at (u, v) coordinates after moving, is a centering operator, which is 1 when (u + v) is even, and -1 otherwise.

[0140] The central low frequency region is defined as:

[0141] (15)

[0142] where r is a self-defined radius, which can be 0.1 times of the length or width of the first gray image, is the set of central low frequency regions.

[0143] The energy density is calculated according to the moved frequency domain value:

[0144] (16)

[0145] where P(u, v) represents the energy density at (u, v) coordinates.

[0146] The first low frequency energy is calculated according to the energy density and the set of central low frequency regions:

[0147] (17)

[0148] where E low1 represents the first low frequency energy.

[0149] The total energy of the first gray image is calculated according to the energy density:

[0150] (18)

[0151] where E total1 represents the total energy of the first gray image.

[0152] The first fluctuation determination index is the ratio of the first low frequency energy and the total energy of the first gray image:

[0153] (19)

[0154] wherein, R1 is a first fluctuation determination index.

[0155] Further, the count threshold parameter includes a paper break count and an occlusion count.

[0156] Referring to Figure 5 In step S4, it is determined whether a paper break and an occlusion occur according to the effective area color recognition result of the real-time environment image, the invalid area frequency domain calculation result, and the count threshold parameter, including:

[0157] S41, compare the first color difference with a preset color difference value, and mark the real-time environment image according to the comparison result.

[0158] S42, output a paper break determination result according to the number of real-time environment images with paper break marks and the paper break count.

[0159] S43, compare the first fluctuation determination value with a preset fluctuation value, and mark the real-time environment image according to the comparison result.

[0160] S44, output an occlusion determination result according to the number of real-time environment images with occlusion marks and the occlusion count.

[0161] Specifically, the first color difference is compared with a preset color difference value, if the first color difference is less than the preset color difference value, the real-time environment image at the current time is marked as a paper break, and the past real-time environment images marked as a paper break are analyzed, counted and compared with the paper break count, and a paper break determination result is output.

[0162] If the first color difference is greater than or equal to the preset color difference value, it is determined whether the first fluctuation determination value is greater than a preset fluctuation value, if the first fluctuation determination value is greater than the preset fluctuation value, the real-time environment image at the current time is marked as an occlusion, and the past real-time environment images marked as an occlusion are analyzed, counted and compared with the occlusion count, and an occlusion determination result is output.

[0163] Further, the count threshold parameter further includes a normal count.

[0164] Specifically, in step S41 and step S42, the first color difference is compared with a preset color difference value, if the first color difference is less than the preset color difference value, the real-time environment image at the current time is marked as paper break, whether the real-time environment image at the last time is marked as paper break is judged, if marked as paper break, the paper break determination result is output, if not marked as paper break, whether the number of real-time environment images marked as paper break in the past preset time reaches the paper break count is judged, if the paper break count is reached, the paper break determination result is output, if the paper break count is not reached, the last determination result is output.

[0165] In step S43 and step S44, if the first color difference is greater than or equal to the preset color difference value, whether the first fluctuation determination value is greater than a preset fluctuation value is judged, if greater than the preset fluctuation value, the real-time environment image at the current time is marked as shielding, whether the real-time environment image at the last time is marked as shielding is judged, if marked as shielding, the shielding determination result is output, if not marked as shielding, whether the number of real-time environment images marked as shielding in the past preset time reaches the shielding count is judged, if the shielding count is reached, the shielding determination result is output, if the shielding count is not reached, the last determination result is output.

[0166] If the first fluctuation determination value is less than or equal to the preset fluctuation value, the real-time environment image at the current time is marked as normal, and whether the real-time environment image at the last time is marked as normal is judged, if marked as normal, the normal determination result is output, if not marked as normal, whether the number of real-time environment images marked as normal in the past preset time reaches the normal count is judged, if the normal count is reached, the normal determination result is output, if the normal count is not reached, the last determination result is output.

[0167] Further, in step S2, color recognition is performed on the valid area in the initial environment image and frequency domain calculation is performed on the invalid area, including:

[0168] The valid area mask in the template image and the valid area in the initial environment image are converted to Lab space, the second color difference between the valid area in the initial environment image and the valid area mask is calculated in the Lab space, and the second color difference is taken as the color recognition result of the valid area in the initial environment image.

[0169] The invalid area in the initial environment image is converted to gray scale to obtain a second gray scale image, the second gray scale image is converted to a frequency domain, the second low frequency energy and the total energy of the second gray scale image are calculated according to the frequency domain value, the second fluctuation determination value is calculated according to the ratio of the second low frequency energy and the total energy of the second gray scale image, and the second fluctuation determination value is taken as the calculation result of the invalid area in the initial environment image.

[0170] The calculation of the second color difference and the second fluctuation determination value is the same as the first color difference and the first fluctuation determination value, and will not be repeated here.

[0171] Further, whether paper breakage and occlusion occur during startup is determined according to the color recognition result of the effective area of the initial environment image and the frequency domain calculation result of the invalid area, including:

[0172] The calculated second color difference is compared with a preset color difference value, and if there is a second color difference less than the preset color difference value, it is determined that paper breakage occurs during startup.

[0173] If it is determined that no paper breakage occurs during startup, the calculated second fluctuation determination value is compared with a preset fluctuation value, and if there is a second fluctuation determination value greater than the preset fluctuation value, it is determined that occlusion occurs during startup.

[0174] Reference Figure 6 In some embodiments, a visual-based paper abnormality detection device is also provided, which is applied to a paper production line, and a paper is arranged on a transmission mechanism or a support mechanism, and the transmission mechanism or the support mechanism and the surrounding area thereof are taken as a detection area; the paper advances to a preset direction through a roller; the device comprises:

[0175] A configuration module 201 is configured to configure a template image for the detection area, determine an effective area mask for detection according to the template image, and set a count threshold parameter;

[0176] An initial judgment module 202 is configured to collect a preset number of initial environment images in the detection area during startup for initial judgment;

[0177] An identification and calculation module 203 is configured to collect real-time environment images in the detection area at a preset time interval after initial judgment of no abnormality, determine effective areas and invalid areas in the real-time environment images according to the effective area mask, perform color recognition on the effective areas in the real-time environment images, and perform frequency domain calculation on the invalid areas;

[0178] A judgment module 204 is configured to determine whether paper breakage and occlusion occur currently according to the color recognition result of the effective areas in the real-time environment images, the frequency domain calculation result of the invalid areas, and the count threshold parameter.

[0179] Further, the effective area is a support part area on the transmission mechanism or the support mechanism that directly contacts the paper to support the paper; and the template image is an image of the detection area without the paper;

[0180] The configuration module 201 determines the effective area mask for detection according to the template image, including:

[0181] Select the center point of the support part region in the template image as a seed point, and set a threshold range;

[0182] With the seed point as a starting point, compare the pixel values of the pixel points in the neighborhood of the seed point with the threshold range, and add the pixel points with pixel values within the threshold range to a candidate region list;

[0183] Repeat the following steps until there is no pixel point in the neighborhood of the selected pixel point with a pixel value within the threshold range:

[0184] Select a target pixel point from the candidate region list to calculate whether the pixel values of the pixel points in its neighborhood are within the threshold range, add the pixel points with pixel values within the threshold range to the candidate region list, and delete the selected target pixel point from the candidate region list;

[0185] The pixel points in the candidate region list obtained when there is no pixel point in the neighborhood of the selected target pixel point with a pixel value within the threshold range constitute the effective region mask.

[0186] Further, the initial judgment module 202 collects a preset number of initial environment images in the detection region when starting to perform initial judgment, including:

[0187] According to the effective region mask, determine the effective region and the invalid region in the initial environment image, perform color recognition on the effective region and frequency domain calculation on the invalid region in the initial environment image, and judge whether paper breakage and occlusion occur when starting according to the color recognition result of the effective region and the frequency domain calculation result of the invalid region in the initial environment image. If no paper breakage and occlusion occur, it is determined that the initial judgment is normal.

[0188] Further, the recognition and calculation module 203 determines the effective region and the invalid region in the real-time environment image according to the effective region mask, or the initial judgment module 202 determines the effective region and the invalid region in the initial environment image according to the effective region mask, including:

[0189] Compare the initial environment image / real-time environment image with the template image, determine the region in the initial environment image / real-time environment image with the same pixel position as the effective region mask as the effective region, and determine the region other than the effective region as the invalid region.

[0190] Further, the recognition and calculation module 203 performs color recognition on the effective region and frequency domain calculation on the invalid region in the real-time environment image, including:

[0191] convert the effective area mask in the template image and the effective area in the real-time environment image to Lab space, calculate a first color difference between the effective area in the real-time environment image and the effective area mask in the Lab space, and take the first color difference as an effective area color recognition result in the real-time environment image;

[0192] gray-scale convert the ineffective area in the real-time environment image to obtain a first gray-scale image, convert the first gray-scale image to a frequency domain, calculate a first low-frequency energy and a first total energy of the gray-scale image according to frequency domain values, calculate a first fluctuation determination value according to a ratio of the first low-frequency energy to the first total energy of the gray-scale image, and take the first fluctuation determination value as an ineffective area calculation result in the real-time environment image.

[0193] Further, the counting threshold parameter includes a paper break counting and a shielding counting.

[0194] The judging module 204 judges whether a paper break and shielding occur according to the effective area color recognition result, the ineffective area frequency domain calculation result, and the counting threshold parameter of the real-time environment image, and includes:

[0195] compare the first color difference with a preset color difference value, and mark the real-time environment image according to a comparison result;

[0196] output a paper break determination result according to a number of the real-time environment images with the paper break mark and the paper break counting;

[0197] compare the first fluctuation determination value with a preset fluctuation value, and mark the real-time environment image according to a comparison result;

[0198] output a shielding determination result according to a number of the real-time environment images with the shielding mark and the shielding counting.

[0199] The judging module 204 is further configured to compare the first color difference with a preset color difference value, mark the real-time environment image at a current time as a paper break if the first color difference is less than the preset color difference value, analyze, count, and compare the real-time environment images marked as the paper break in the past with the paper break counting, and output a paper break determination result.

[0200] If the first color difference is greater than or equal to the preset color difference value, judge whether the first fluctuation determination value is greater than a preset fluctuation value, mark the real-time environment image at a current time as a shielding if the first fluctuation determination value is greater than the preset fluctuation value, analyze, count, and compare the real-time environment images marked as the shielding in the past with the shielding counting, and output a shielding determination result.

[0201] Further, the count threshold parameter further comprises a normal count; the judging module 204 is further configured to:

[0202] compare the first color difference with a preset color difference value, if the first color difference is less than the preset color difference value, mark the real-time environment image at the current moment as paper break; judge whether the real-time environment image at the last moment is marked as paper break, if marked as paper break, output the paper break determination result, if not marked as paper break, judge whether the number of real-time environment images marked as paper break in the past preset time length reaches the paper break count, if reaching the paper break count, output the paper break determination result, if not reaching the paper break count, output the last determination result;

[0203] if the first color difference is greater than or equal to the preset color difference value, judge whether the first fluctuation determination value is greater than a preset fluctuation value, if greater than the preset fluctuation value, mark the real-time environment image at the current moment as occlusion; judge whether the real-time environment image at the last moment is marked as occlusion, if marked as occlusion, output the occlusion determination result, if not marked as occlusion, judge whether the number of real-time environment images marked as occlusion in the past preset time length reaches the occlusion count, if reaching the occlusion count, output the occlusion determination result, if not reaching the occlusion count, output the last determination result;

[0204] if the first fluctuation determination value is less than or equal to the preset fluctuation value, mark the real-time environment image at the current moment as normal, and judge whether the real-time environment image at the last moment is marked as normal, if marked as normal, output the normal determination result, if not marked as normal, judge whether the number of real-time environment images marked as normal in the past preset time length reaches the normal count, if reaching the normal count, output the normal determination result, if not reaching the normal count, output the last determination result.

[0205] Further, the initial judging module 202 performs color identification on the effective area in the initial environment image and frequency domain calculation on the invalid area, comprising:

[0206] convert the effective area mask in the template image and the effective area in the initial environment image to Lab space, calculate a second color difference between the effective area in the initial environment image and the effective area mask in the Lab space, and take the second color difference as the effective area color identification result in the initial environment image;

[0207] The invalid area in the initial environment image is converted into grayscale to obtain a second grayscale image, the second grayscale image is converted into a frequency domain, a second low-frequency energy and a second total energy of the grayscale image are calculated, a second fluctuation determination value is calculated according to a ratio of the second low-frequency energy and the second total energy of the grayscale image, and the second fluctuation determination value is taken as a calculation result of the invalid area in the initial environment image.

[0208] Further, the initial judgment module 202 judges whether paper breakage and shielding occur during startup according to the color recognition result of the valid area of the initial environment image and the frequency domain calculation result of the invalid area, and the judgment includes:

[0209] The second color difference is compared with a preset color difference value, and if the second color difference is less than the preset color difference value, it is determined that paper breakage occurs during startup.

[0210] If it is determined that paper breakage does not occur during startup, the second fluctuation determination value is compared with a preset fluctuation value, and if the second fluctuation determination value is greater than the preset fluctuation value, it is determined that shielding occurs during startup.

[0211] The paper abnormality detection method and device based on vision provided by the above embodiments at least have the following beneficial effects:

[0212] (1) The method can detect whether paper breakage occurs in the paper production process based on vision, can improve the reliability and stability of paper breakage detection, can determine possible shielding, and has wider applicability.

[0213] (2) The color difference is determined in the Lab space, and shielding is determined in the frequency domain, which can effectively avoid interference caused by image noise and improve the stability of detection.

[0214] (3) The detection process can reduce the false detection probability and improve the accuracy of detection by combining with the setting of appropriate counting standard parameters.

[0215] Although the preferred embodiments of the present application have been described, those skilled in the art can make further changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including all changes and modifications falling within the scope of the present application. Obviously, those skilled in the art can make various modifications and changes to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and changes of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and changes.

Claims

1. A vision-based paper anomaly detection method, characterized in that, Applied to a paper production line, where paper is mounted on a transmission mechanism or a support mechanism; the method includes: The transmission mechanism or support mechanism and its surrounding area are used as the detection area. A template diagram is configured for the detection area. The effective area mask for detection is determined according to the template diagram, and the counting target parameters are set. At the start of detection, a preset number of initial environmental images are collected within the detection area for initial assessment; After the initial judgment that there are no abnormalities, real-time environmental images of the detection area are collected at preset time intervals. The effective and invalid regions in the real-time environmental images are determined according to the effective region mask. Color recognition is performed on the effective regions in the real-time environmental images and frequency domain calculation is performed on the invalid regions. Based on the color recognition results of the effective area of ​​the real-time environmental image, the frequency domain calculation results of the invalid area, and the counting target parameters, it is determined whether a paper break or occlusion has occurred.

2. The method according to claim 1, characterized in that, The effective area is the support portion on the transmission mechanism or support mechanism that is in direct contact with the paper and is used to support the paper. The template image is an image of the detection area when there is no paper. Determining the effective region mask for detection based on the template image includes: Select the center point of the support area in the template diagram as the seed point, and set the threshold range; Starting from the seed point, the pixel values ​​of the pixels in the neighborhood of the seed point are compared with the threshold range, and the pixels whose pixel values ​​are within the threshold range are added to the candidate region list. Repeat the following steps until the pixel value of the selected pixel has no neighboring pixels within the threshold range: Select a target pixel from the candidate region list, calculate whether the pixel values ​​of its neighboring pixels are within the threshold range, add pixels whose pixel values ​​are within the threshold range to the candidate region list, and delete the selected target pixel from the candidate region list. When the pixel value of a selected target pixel has no neighboring pixels within the threshold range, the pixels in the obtained candidate region list constitute the effective region mask.

3. The method according to claim 1, characterized in that, At the start of detection, a preset number of initial environmental images are collected within the detection area for initial assessment, including: The effective and invalid regions in the initial environment image are determined based on the effective region mask. Color recognition is performed on the effective regions in the initial environment image and frequency domain calculation is performed on the invalid regions. Based on the color recognition results of the effective regions and the frequency domain calculation results of the invalid regions in the initial environment image, it is determined whether paper breakage or occlusion occurs during power-on. If no paper breakage or occlusion occurs, it is determined that the initial judgment is normal.

4. The method according to claim 3, characterized in that, Determining the valid and invalid regions in the initial environment image based on the valid region mask, or determining the valid and invalid regions in the real-time environment image based on the valid region mask, includes: The initial environment image / real-time environment image is compared with the template image. The regions in the initial environment image / real-time environment image that have the same pixel position as the effective region mask are determined as effective regions, and the regions other than the effective regions are determined as invalid regions.

5. The method according to claim 1, characterized in that, The process of color recognition for valid regions and frequency domain calculation for invalid regions in the real-time environmental image includes: The effective region mask in the template image and the effective region in the real-time environment image are converted to Lab space. The first color difference between the effective region in the real-time environment image and the effective region mask is calculated in Lab space. The first color difference is used as the color recognition result of the effective region in the real-time environment image. The invalid regions in the real-time environment image are converted to grayscale to obtain a first grayscale image. The first grayscale image is then converted to the frequency domain. The first low-frequency energy and the total energy of the first grayscale image are calculated based on the frequency domain values. The first fluctuation judgment value is calculated based on the ratio of the first low-frequency energy to the total energy of the first grayscale image. The first fluctuation judgment value is used as the calculation result of the invalid regions in the real-time environment image.

6. The method according to claim 5, characterized in that, The counting parameters include paper breakage count and obstruction count; Based on the effective area color recognition results of the real-time environmental image, the invalid area frequency domain calculation results, and the count compliance parameters, it is determined whether a paper break or occlusion has occurred, including: The first color difference is compared with a preset color difference value, and a paper break mark is made on the real-time environmental image based on the comparison result; Based on the number of real-time environmental images with paper break marks and the paper break count, output the paper break determination result; The first fluctuation determination value is compared with the preset fluctuation value, and the real-time environmental image is marked with occlusion based on the comparison result. Based on the number of real-time environmental images with occlusion markers and the occlusion count, the occlusion determination result is output.

7. The method according to claim 6, characterized in that, The first color difference is compared with a preset color difference value. If the first color difference is less than the preset color difference value, the real-time environmental image at the current moment is marked as a paper break. The real-time environmental images marked as paper breaks in the past are analyzed, counted, and compared with the paper break count. The paper break determination result is then output. If the first color difference is greater than or equal to the preset color difference value, then it is determined whether the first fluctuation judgment value is greater than the preset fluctuation value. If the first fluctuation judgment value is greater than the preset fluctuation value, then the real-time environment image at the current moment is marked as occlusion, and the real-time environment images marked as occlusion in the past are analyzed, counted, and compared with the occlusion count, and the occlusion judgment result is output.

8. The method according to claim 3, characterized in that, The process of color recognition for valid regions and frequency domain calculation for invalid regions in the initial environmental image includes: The effective region mask in the template image and the effective region in the initial environment image are converted to Lab space. The second color difference between the effective region in the initial environment image and the effective region mask is calculated in Lab space. The second color difference is used as the color recognition result of the effective region in the initial environment image. The invalid regions in the initial environmental image are converted to grayscale to obtain a second grayscale image. The second grayscale image is then converted to the frequency domain. The second low-frequency energy and the total energy of the second grayscale image are calculated based on the frequency domain values. The second fluctuation judgment value is calculated based on the ratio of the second low-frequency energy and the total energy of the second grayscale image. The second fluctuation judgment value is used as the calculation result of the invalid regions in the initial environmental image.

9. The method according to claim 8, characterized in that, Based on the color recognition results of the effective area and the frequency domain calculation results of the invalid area in the initial environmental image, determine whether paper breaks or obstructions occur during power-on, including: The calculated second color difference is compared with the preset color difference value. If there is a second color difference that is less than the preset color difference value, it is determined that a paper break occurred when the machine is turned on. If it is determined that no paper breakage occurred when the machine was turned on, the calculated second fluctuation judgment values ​​are compared with the preset fluctuation values. If there is a second fluctuation judgment value that is greater than the preset fluctuation value, it is determined that there is obstruction when the machine was turned on.

10. A vision-based paper anomaly detection device, characterized in that, An apparatus for use in paper production lines, wherein paper is mounted on a transmission mechanism or support mechanism, and the transmission mechanism or support mechanism and its surrounding area are used as the detection area; the apparatus includes: The configuration module is used to configure a template image for the detection area, determine the effective area mask for detection based on the template image, and set the counting target parameters. The initial judgment module is used to collect a preset number of initial environmental images within the detection area for initial settings when detection begins. The identification and calculation module is used to collect real-time environmental images within the detection area at preset time intervals after an initial judgment that there are no abnormalities, determine the effective and invalid areas in the real-time environmental images according to the effective area mask, perform color recognition on the effective areas in the real-time environmental images, and perform frequency domain calculation on the invalid areas. The judgment module is used to determine whether a paper break or occlusion has occurred based on the effective area color recognition result of the real-time environmental image, the invalid area frequency domain calculation result, and the count compliance parameter.

Images