A biscuit package breakage recognition system based on image processing

Abstract

The present application relates to the technical field of packaging detection, and more particularly to a biscuit packaging damage identification system based on image processing. The system comprises a gradient extrusion module, an image acquisition module, a height difference screening module, a normal change detection module and a defect anomaly positioning module. The present application realizes multi-level identification of packaging damage state and defect traceability feedback by acquiring packaging surface images under the pressure state of biscuit packaging and combining three-dimensional space modeling, height difference comparison, normal change curve calculation and texture similarity analysis. The system can distinguish three states of complete sealing, slight damage and serious defect, and provide corresponding prompt information for different damage causes. Through this technology, the identification accuracy is improved, and the feedback regulation and control ability of the detection system is greatly enhanced, making the packaging detection more intelligent and accurate, greatly improving the automation level of the production line and the product quality control ability.

Description

Technical Field

[0001] This invention relates to the field of packaging inspection technology, and in particular to an image processing-based system for identifying damaged biscuit packaging. Background Technology

[0002] Currently, automated packaging processes for pre-packaged foods such as biscuits are becoming increasingly common. However, the automatic identification of packaging damage still relies on manual sampling or single physical sensing methods, resulting in low identification accuracy and slow response. This makes it impossible to efficiently identify non-structural defects such as minor creases and slight scratches. Furthermore, traditional systems often rely on fixed thresholds for image grayscale analysis, lacking the ability to adjust in conjunction with actual production conditions, which can easily lead to false alarms or missed detections.

[0003] Existing systems have attempted to incorporate image processing algorithms to enhance recognition capabilities, but they generally suffer from the following technical problems: First, they fail to combine the structural features and texture variation patterns of the packaging area for classification and judgment, making it difficult to distinguish between different types of defects; second, the image judgment criteria are fixed, making it impossible to dynamically adjust relevant parameters of the production process based on the detection data, resulting in a lack of adaptive capabilities. Therefore, there is an urgent need for a new type of damage recognition system with image processing and intelligent feedback adjustment capabilities, capable of distinguishing multiple types of packaging defects.

[0004] Chinese Patent Publication No. CN119313666A discloses a method for detecting the sealing of packaged can lids based on image processing. This invention relates to the field of image processing technology, specifically to a method for detecting the sealing of packaged can lids based on image processing. The method includes: acquiring a grayscale image of the surface of the aluminum foil lid on the packaged can at the end of the sealing test; dividing the grayscale image into multiple annular regions; counting the number of pixels in each annular region whose deformation degree is greater than a threshold, and determining whether the packaged can needs to be resealed based on the number; that is, the solution of this invention can accurately detect the sealing performance of packaged can lids. However, this invention also lacks a feedback adjustment mechanism and the ability to link with production data. The overall structure only includes image acquisition, region division, pixel analysis and judgment; it designs a large number of parameters such as included angle, grayscale fluctuation, texture difference, etc., with complex mathematical construction. However, most of the analysis is based on static models or single-point comparison, lacking data-driven mechanisms or multi-period sampling support, thus exhibiting poor stability and adaptability in dynamic production environments. Summary of the Invention

[0005] To address this issue, the present invention provides a biscuit packaging damage recognition system based on image processing, which overcomes the problem in the prior art where the fixed detection parameters are outdated and the accuracy of fault type recognition is poor, resulting in weak process feedback adjustment and inability to adapt to multiple production states.

[0006] To achieve the above objectives, the present invention provides an image processing-based biscuit packaging damage recognition system, comprising:

[0007] The gradient extrusion module includes a conveyor belt for conveying the biscuit packaging to be inspected, a flexible gripper, and a set of pressure rollers for applying a set gradient pressure to the biscuit packaging to be inspected.

[0008] The image acquisition module, which is connected to the gradient extrusion module, includes several image acquisition devices spaced apart on one side of the pressure roller group and several light sources arranged with the same spatial spacing. The image acquisition module is used to acquire images of the packaging surface of each biscuit package to be tested under the applied pressure.

[0009] The height difference screening module is connected to the gradient extrusion module and the image acquisition module respectively. It is used to determine the rejected defective biscuit packages and the unconfirmed damaged biscuit packages based on the peak height difference of the first packaging surface image.

[0010] The normal change detection module is connected to the gradient extrusion module, the image acquisition module and the height difference screening module respectively. It is used to obtain the set of normal changes of the center peak points on the center lines on both sides of the pressure roller group in the smooth area and the corresponding center peak point normal change curve based on the second packaging surface image when the second comparison result is obtained, so as to detect the normal change of the biscuit packaging to be confirmed as damaged, and obtain qualified biscuit packaging and slightly damaged biscuit packaging.

[0011] The defect anomaly localization module, which is connected to the gradient extrusion module, the image acquisition module, and the normal change detection module, is used to call the second packaging surface image of the slightly damaged biscuit packaging for defect identification, and determine several defect types and make corresponding adjustments based on the normal change curve of the central peak point.

[0012] Furthermore, the height difference filtering module includes a region division unit and a height difference comparison unit, wherein,

[0013] The region division unit is used to construct a three-dimensional spatial model of the packaging surface image, divide it, and obtain the division results and categories of each region of the packaging surface in a static state.

[0014] The height difference comparison unit is used to compare the peak height difference of the same area under static and pressure conditions, and compare the peak height difference with a preset standard peak height threshold.

[0015] Furthermore, the region segmentation unit includes a spatial construction subunit and an image structure recognition subunit, wherein,

[0016] The spatial construction subunit is used to construct a three-dimensional spatial model of the packaging surface based on the acquired images of the packaging surface;

[0017] The image structure recognition subunit is used to divide the packaging surface based on the lateral autocorrelation coefficient of the image sub-blocks. The division results include the packaging area, the connection area, and the smooth area.

[0018] Furthermore, the spatial construction subunit constructs a three-dimensional spatial model of the packaging surface based on the acquired images of the packaging surface, including:

[0019] The packaging surface image is divided into several image sub-blocks according to a preset block division standard, and the row and column number identifiers and corresponding position information of each image sub-block are obtained.

[0020] The geometric center points of image sub-blocks are determined as feature points and extracted. Based on the triangulation method, each feature point is converted into corresponding three-dimensional spatial coordinates and a three-dimensional spatial model is generated.

[0021] Furthermore, the image structure recognition subunit divides the packaging surface based on the lateral autocorrelation coefficient of image sub-blocks, including:

[0022] Obtain the set of horizontal autocorrelation coefficients composed of the horizontal autocorrelation coefficients of each image sub-block;

[0023] Based on the comparison results of each transverse autocorrelation coefficient with the preset transverse autocorrelation coefficient threshold, the corresponding packaging surface segmentation results are obtained, including the encapsulation area, the connection area, and the smooth area.

[0024] Furthermore, the normal change detection module includes a front and back region segmentation unit and a similarity comparison unit, wherein,

[0025] The front and rear area division unit is used to divide the smooth area into a front pressure area and a rear pressure area based on the extension direction of the center seam of the biscuit packaging bag to be tested and the real-time position of the pressure roller group.

[0026] Based on the extension direction of the center seam, a center line is constructed through the geometric center point in the smooth area. With the pressure roller group as the boundary, the area in the smooth area that first passes through the pressure roller group and applies pressure is the front pressure area, and the area that passes through the pressure roller group and applies pressure later is the back pressure area.

[0027] The similarity comparison unit is used to calculate the curvature change similarity based on the normal change curves of the left and right central peaks.

[0028] Furthermore, the similarity comparison unit includes a curvature detection subunit and a curve change trend comparison subunit, wherein,

[0029] The curvature detection subunit is used to calculate the local curvature of each central peak point on the center line on both sides of the pressure roller group and form a curvature data set;

[0030] The curve change trend comparison sub-unit is used to construct the curvature change curve of the center peak point based on the curvature data set, and to calculate the similarity between the curvature change curves of the two center peak points.

[0031] The curve similarity judgment result of the defect corresponding to the speed detection of the roller group is the second similarity judgment result.

[0032] Furthermore, the curvature detection subunit calculates the local curvature of each central peak point on the center line on both sides of the pressure roller assembly and forms a curvature data set, including:

[0033] While the pressure roller group is moving, the curvatures corresponding to the left and right center peaks in the packaging surface image are acquired and stored in the left center peak curvature set and the right center peak curvature set, respectively.

[0034] Based on the curvature set of the left central peak and the curvature set of the right central peak, respectively, fit the curvature change curve of the left central peak and the curvature change curve of the right central peak. After reversing the curvature change curve of the left central peak and aligning it with the curvature change curve of the right central peak, calculate the similarity between the two curvature change curves of the two central peaks and obtain the similarity judgment result.

[0035] Furthermore, the defect anomaly localization module includes a defect identification unit and a source tracing and adjustment unit, wherein,

[0036] The defect identification unit is used to identify defects based on the packaging surface image of the smooth area under the second pressure when the second similarity judgment result and device interference occur, and to determine the category to which the defect belongs.

[0037] The anomaly location unit is used to determine the corresponding process and its improvement measures based on the identification results.

[0038] Furthermore, the defect identification unit includes a texture identification subunit, a reflectivity change judgment subunit, and a region matching subunit, wherein,

[0039] The texture recognition subunit is used to call up the packaging surface image of the smooth area under the second applied pressure and perform texture detection to obtain the texture detection result;

[0040] The reflectance change judgment subunit is used to obtain the reflectance change value detection result under the first texture detection result;

[0041] The region matching subunit is used to obtain the region where the defect is located and extract the contour of the encapsulation region to determine the type of highly abrupt morphology.

[0042] Compared with existing technologies, the advantages of this invention lie in the fact that it applies different pressures to the biscuit packaging through a gradient extrusion module, and combines this with an image acquisition module to capture images of the packaging surface under various pressure conditions. The height difference screening module can accurately compare the peak height difference of the packaging surface under static and pressure conditions, promptly detecting any defects on the packaging surface. The normal change detection module accurately identifies minute packaging damage by analyzing changes in the normals of the packaging surface, and further, through a defect anomaly localization module combined with the packaging surface images, automatically identifies the type of defect and traces it back to the production process. This invention uses multi-level detection methods, such as texture recognition, reflectivity change analysis, and curvature change calculation, to conduct in-depth and detailed analysis of the packaging surface, thereby avoiding missed and false detections. The system has an efficient feedback and adjustment mechanism, enabling real-time adjustments to the packaging production process during detection, further optimizing the production flow, reducing the defect rate, improving production efficiency, and enhancing product quality stability while optimizing process parameters and reducing production costs, resulting in significant economic and production benefits.

[0043] Furthermore, by setting up multiple image acquisition units and light source arrays, it is ensured that images of the biscuit packaging surface can be acquired efficiently and stably under pressure. The acquired image data provides a high-resolution and highly reliable raw data source for subsequent defect detection, ensuring accurate identification of minor damage and details.

[0044] Furthermore, by comparing the peak height difference under static and pressure conditions, areas of deformation or damage to the packaging surface can be identified. By comparing this difference with a preset standard peak height threshold, defective packaging that does not meet the standards can be effectively screened out, improving the accuracy and efficiency of the inspection and avoiding missed detections or misjudgments. Attached Figure Description

[0045] Figure 1 This is a connection diagram of the image processing-based biscuit packaging damage recognition system according to an embodiment of the present invention;

[0046] Figure 2 This is a connection diagram of the height difference screening module according to an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the connection of the normal change detection module in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of the biscuit packaging damage identification device according to an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0050] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0051] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0052] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0053] Please see Figure 1 The diagram shown is a connection schematic of an image processing-based biscuit packaging damage recognition system according to an embodiment of the present invention. The present invention provides an image processing-based biscuit packaging damage recognition system, comprising:

[0054] The gradient extrusion module includes a conveyor belt for conveying the biscuit packaging to be inspected, a flexible gripper, and a set of pressure rollers for applying a set gradient pressure to the biscuit packaging to be inspected.

[0055] The image acquisition module, which is connected to the gradient extrusion module, includes several image acquisition devices spaced apart on one side of the pressure roller group and several light sources arranged with the same spatial spacing. The image acquisition module is used to acquire images of the packaging surface of each biscuit package to be tested under the applied pressure.

[0056] The height difference screening module is connected to the gradient extrusion module and the image acquisition module respectively. It is used to determine the rejected defective biscuit packages and the unconfirmed damaged biscuit packages based on the peak height difference of the first packaging surface image.

[0057] The normal change detection module is connected to the gradient extrusion module, the image acquisition module and the height difference screening module respectively. It is used to obtain the set of normal changes of the center peak points on the center lines on both sides of the pressure roller group in the smooth area and the corresponding center peak point normal change curve based on the second packaging surface image when the second comparison result is obtained, so as to detect the normal change of the biscuit packaging to be confirmed as damaged, and obtain qualified biscuit packaging and slightly damaged biscuit packaging.

[0058] The defect anomaly localization module, which is connected to the gradient extrusion module, the image acquisition module, and the normal change detection module, is used to call the second packaging surface image of the slightly damaged biscuit packaging for defect identification, and determine several defect types and make corresponding adjustments based on the normal change curve of the central peak point.

[0059] In this embodiment, small LED lights with a diameter of 10mm and a power of 0.2W are selected and set with a spatial spacing of 100mm;

[0060] The applied pressure includes a first gradient pressure and a second gradient pressure. The first gradient pressure is applied to the connecting area of ​​the biscuit packaging, and the first gradient pressure is selected as 15 N / cm².

[0061] The second gradient pressure applies pressure to the smooth area of ​​the biscuit packaging, and the second gradient pressure is selected as 5 N / cm².

[0062] The conveyor belt includes a first conveyor belt and a second conveyor belt. The first conveyor belt is a receiving area conveyor belt, and the second conveyor belt is a vision inspection area conveyor belt. The receiving area conveyor belt is used to transport the biscuit packaging to be inspected from the initial position to the vision inspection area to start vision inspection.

[0063] The image acquisition module includes an upward-looking camera and a side-looking camera, which are fixedly installed directly above and to the side of the packaging conveying path, respectively.

[0064] The top-view camera is used to acquire image data of the upper surface of the packaging;

[0065] During the packaging process, the biscuit packaging to be inspected passes through the camera area by the conveyor belt at a set speed; the system controls two cameras to simultaneously capture images at 30 frames per second when the packaging enters the shooting area through a synchronous triggering mechanism, and obtains images of the top surface and the side surface respectively;

[0066] The second conveyor belt is provided with grooves so that the stitching in the packaging bag is nested in the grooves. The grooves are 15mm wide and 5mm deep to ensure that the packaging bag can be precisely aligned and to prevent slippage.

[0067] The conveyor belt is 150mm wide and its length is customized according to the layout of the production line. The conveyor belt length of the vision inspection section of this product is 5m and the speed of the conveyor belt is 15m / minute.

[0068] The flexible gripper has a diameter of 150mm and a rotational straightening speed of 10 meters per minute.

[0069] The flexible gripper has a roller with multiple adjustable rollers. These rollers can adjust the angle between the center seam of the biscuit package and the direction of movement by friction with the surface of the packaging bag, so that the biscuit package is conveyed onto the conveyor belt and nested in the groove of the conveyor belt; in actual production, the angle between the center seam and the direction of movement is taken as 0.

[0070] When the gripper picks up the biscuit package, it picks up the biscuit package from the end of the first conveyor belt to the beginning of the second conveyor belt, and ensures that the center seam of the biscuit package is facing down and aligned with the groove of the second conveyor belt, so that the center seam of the package is embedded in the groove. During the picking and placing process, the gripper ensures that the center seam is aligned with the groove of the second conveyor belt, and the extension direction of the center seam is parallel to the running direction of the second conveyor belt.

[0071] The biscuit packaging is subjected to varying pressures by a gradient extrusion module, while an image acquisition module captures images of the packaging surface under each pressure condition. A height difference screening module accurately compares the peak height difference between the packaging surface under static and pressure conditions, promptly identifying surface defects. A normal change detection module analyzes changes in the surface normals to accurately identify minor packaging damage. Furthermore, a defect anomaly localization module, combined with the packaging surface images, automatically identifies the type of defect and traces it back to the production process. This invention employs multi-layered detection methods, such as texture recognition, reflectivity change analysis, and curvature change calculation, to conduct in-depth and detailed analysis of the packaging surface, thereby avoiding missed and false detections. The system features an efficient feedback and adjustment mechanism, enabling real-time adjustments to the packaging production process during inspection. This further optimizes the production flow, reduces defect rates, improves production efficiency, enhances product quality stability, optimizes process parameters, and reduces production costs, resulting in significant economic and production benefits.

[0072] See Figure 2 As shown, it is a connection diagram of the height difference screening module in an embodiment of the present invention;

[0073] Specifically, the height difference filtering module includes a region division unit and a height difference comparison unit, wherein,

[0074] The region division unit is used to construct a three-dimensional spatial model of the packaging surface image, divide it, and obtain the division results and categories of each region of the packaging surface in a static state.

[0075] The height difference comparison unit is used to compare the peak height difference of the same area under static and pressure conditions, and compare the peak height difference with a preset standard peak height threshold.

[0076] In this embodiment, the peak height difference is the difference between the pressure peak height and the static peak height. The pressure peak height is the highest height under the first gradient pressure applied to the connecting area by the pressure roller group, and the static peak height is the highest height of the packaging surface in the static state.

[0077] A preset standard peak height threshold of 0.3mm is set, and the peak height difference is compared with the preset standard peak height threshold.

[0078] If the peak height difference is less than or equal to the preset standard peak height threshold, the first comparison result is obtained, indicating that the rebound height of the biscuit packaging cannot reach the rebound height when it is completely sealed, and the biscuit packaging is leaking air. The leaking biscuit packaging is then separated from the leaking packaging diversion line.

[0079] If the peak height difference is greater than the preset standard peak height threshold, it means that the rebound height of the biscuit packaging has reached the rebound height when it is completely sealed. The second comparison result is obtained, and the surface image of the second packaging under the second gradient pressure is analyzed.

[0080] By setting up multiple image acquisition units and light source arrays, the system ensures efficient and stable acquisition of images of the biscuit packaging surface under pressure. The acquired image data provides a high-resolution and highly reliable raw data source for subsequent defect detection, guaranteeing accurate identification of minor damage and details.

[0081] Specifically, the region division unit includes a spatial construction subunit and an image structure recognition subunit, wherein,

[0082] The spatial construction subunit is used to construct a three-dimensional spatial model of the packaging surface based on the acquired images of the packaging surface;

[0083] The image structure recognition subunit is used to divide the packaging surface based on the lateral autocorrelation coefficient of the image sub-blocks. The division results include the packaging area, the connection area, and the smooth area.

[0084] By comparing the peak height difference under static and pressure conditions, areas of deformation or damage to the packaging surface can be identified. By comparing this difference with a preset standard peak height threshold, defective packaging that does not meet the standards can be effectively screened out, improving the accuracy and efficiency of detection and avoiding missed detections or false positives.

[0085] Specifically, the spatial construction subunit constructs a three-dimensional spatial model of the packaging surface based on the acquired images of the packaging surface, including:

[0086] The packaging surface image is divided into several image sub-blocks according to a preset block division standard, and the row and column number identifiers and corresponding position information of each image sub-block are obtained.

[0087] The geometric center points of image sub-blocks are determined as feature points and extracted. Based on the triangulation method, each feature point is converted into corresponding three-dimensional spatial coordinates and a three-dimensional spatial model is generated.

[0088] In this embodiment, the image is converted to grayscale.

[0089] The image is divided into blocks using a 5mm×5mm preset block division standard, and the row and column numbers of each sub-block are obtained. Sub-block (1,1) is located in the upper left corner of the image, and sub-block (1,2) is sorted to its right.

[0090] For each image sub-block, the geometric center point is extracted as the feature point of that sub-block. If the top left corner coordinates of a sub-block are (10,20) and its size is 5×5 pixels, then its geometric center point is located at (12.5,22.5).

[0091] Based on the triangulation method, triangulation calculations are performed on the same feature point from two perspectives;

[0092] One camera is located on the length and width plane, and the obtained coordinate content is P1(x1,y1,z1). Another camera is located on the height plane, and the obtained coordinate content is P2(x2,y2,z2).

[0093] Feature points are extracted. The coordinates of the feature points on the length and width planes are (x1, y1), and the coordinates on the length and height planes are (x2, y2). The coordinates of the feature points in three-dimensional space are calculated using triangulation.

[0094] A three-dimensional spatial model is constructed using two known viewpoints and their corresponding image coordinates;

[0095]

[0096] Where P1 and P2 are the projection matrices of the camera on the long and wide planes and the long and tall planes, respectively;

[0097] Interconnect coordinates to construct a three-dimensional spatial model.

[0098] By dividing the image into sub-blocks and extracting the geometric center point of each sub-block as feature points, image data can be processed efficiently, providing accurate spatial coordinate information for subsequent 3D reconstruction. Two-dimensional coordinates of the feature points are obtained from the perspectives of two cameras, and their three-dimensional coordinates are calculated using triangulation, thereby constructing a 3D model of the packaging surface. This effectively improves reconstruction accuracy while accurately reflecting the true shape of the packaging.

[0099] Specifically, the image structure recognition subunit divides the packaging surface based on the lateral autocorrelation coefficient of image sub-blocks, including:

[0100] Obtain the set of horizontal autocorrelation coefficients composed of the horizontal autocorrelation coefficients of each image sub-block;

[0101] Based on the comparison results of each horizontal autocorrelation coefficient with the preset horizontal autocorrelation coefficient threshold, the corresponding packaging surface division results are obtained, including the encapsulation area, the connection area and the smooth area.

[0102] In this embodiment, the preset horizontal autocorrelation coefficient threshold is set to 0.7;

[0103] Each horizontal autocorrelation coefficient is compared with a preset horizontal autocorrelation coefficient threshold.

[0104] Image sub-blocks with horizontal autocorrelation coefficients greater than a preset horizontal autocorrelation coefficient threshold are obtained and marked as encapsulated regions;

[0105] Image sub-blocks with a horizontal autocorrelation coefficient less than or equal to a preset horizontal autocorrelation coefficient threshold are obtained and marked as unencapsulated regions, and further subdivided.

[0106] Obtain the spatial coordinates of the cookie, and update the area in the non-encapsulated area where the length and width of the cookie correspond to the cookie packaging in the vertical and horizontal directions as a smooth area, and update the remaining area as a connected area;

[0107] The formula for calculating the horizontal autocorrelation coefficient is as follows:

[0108]

[0109] in, This represents the grayscale value of the pixel in the sub-block;

[0110] μ is the average gray value of the sub-block;

[0111] j represents the horizontal displacement;

[0112] The calculated horizontal autocorrelation coefficient is used to assess whether the sub-block belongs to the encapsulation region.

[0113] By segmenting image sub-blocks based on the lateral autocorrelation coefficient, accurate classification of packaging surface areas is achieved. The calculation of the lateral autocorrelation coefficient effectively extracts linear features from the packaging image, distinguishing between sealed and unsealed areas. Comparison with a set threshold enables rapid identification of heat-sealed areas, improving classification accuracy. Dividing unsealed areas into smooth and connected regions ensures accurate positioning of each area on the packaging surface, optimizing the efficiency and accuracy of subsequent defect identification and detection.

[0114] See Figure 3As shown, it is a connection diagram of the normal change detection module in an embodiment of the present invention;

[0115] Specifically, the normal change detection module includes a front and back region segmentation unit and a similarity comparison unit, wherein,

[0116] The front and rear area division unit is used to divide the smooth area into a front pressure area and a rear pressure area based on the extension direction of the center seam of the biscuit packaging bag to be tested and the real-time position of the pressure roller group.

[0117] Based on the extension direction of the center seam, a center line is constructed through the geometric center point in the smooth area. With the pressure roller group as the boundary, the area in the smooth area that first passes through the pressure roller group and applies pressure is the front pressure area, and the area that passes through the pressure roller group and applies pressure later is the back pressure area.

[0118] The similarity comparison unit is used to calculate the curvature change similarity based on the normal change curves of the left and right central peaks.

[0119] In this embodiment, the position of the center seam is determined from the biscuit packaging to be inspected, and the direction of the center seam extension is determined, which is used as the reference baseline for dividing the smooth area.

[0120] Obtain the real-time position of the pressure roller assembly;

[0121] Based on the direction of the central suture, a geometric center point is selected within the smooth area, and a centerline is constructed along the direction of the central suture.

[0122] The smooth area is divided into two parts, with the pressure roller assembly as the boundary.

[0123] The front pressure zone is the area that passes before the pressure roller group and is pressured by the pressure roller group first, while the rear pressure zone is the area that passes after the pressure roller group and is pressured by the pressure roller group last.

[0124] The division into front and rear pressure zones ensures that regional differences during the pressure application process can be accurately captured. Based on the extension direction of the center seam and the real-time position of the pressure roller group, the two zones provide a clear regional division for subsequent normal change detection. At the same time, according to the order in which the pressure roller group applies pressure, it is possible to effectively determine the different states of pressure on the packaging surface, providing judgment data for subsequent defect location and judgment.

[0125] Specifically, the similarity comparison unit includes a curvature detection subunit and a curve change trend comparison subunit, wherein,

[0126] The curvature detection subunit is used to calculate the local curvature of each central peak point on the center line on both sides of the pressure roller group and form a curvature data set;

[0127] The curve change trend comparison sub-unit is used to construct the curvature change curve of the center peak point based on the curvature data set, and to calculate the similarity between the curvature change curves of the two center peak points.

[0128] The curve similarity judgment result of the defect corresponding to the speed detection of the roller group is the second similarity judgment result.

[0129] Specifically, the curvature detection subunit calculates the local curvature of each central peak point on the center line on both sides of the pressure roller assembly and forms a curvature data set, including:

[0130] While the pressure roller group is moving, the curvatures corresponding to the left and right center peaks in the packaging surface image are acquired and stored in the left center peak curvature set and the right center peak curvature set, respectively.

[0131] Based on the curvature set of the left central peak and the curvature set of the right central peak, respectively, fit the curvature change curve of the left central peak and the curvature change curve of the right central peak. After reversing the curvature change curve of the left central peak and aligning it with the curvature change curve of the right central peak, calculate the similarity between the two curvature change curves of the two central peaks and obtain the similarity judgment result.

[0132] In this embodiment, the positions of the left and right center peaks in the image are extracted using an edge detection algorithm;

[0133] The local curvature of the left central peak, assuming the point's location is (xL, yL), is calculated using the following formula:

[0134]

[0135] The curvature KR of the right central peak point was calculated using the same method.

[0136] The calculated local curvature values ​​are stored in the curvature set of the left central peak and the curvature set of the right central peak;

[0137] Cubic spline interpolation is used to fit the left and right curvature sets to generate smooth curvature change curves.

[0138] The number of interpolation points is set to 100;

[0139] After the fitting is completed, the curvature change curve of the left central peak point is reversed.

[0140] The inversion operation mirrors the horizontal axis of the curve, aligning the curvature curves on the left and right sides for similarity comparison.

[0141] The Pearson correlation coefficient was calculated to assess the similarity between the curvature change curves of the left and right central peaks.

[0142] The formula for calculating the Pearson correlation coefficient is as follows:

[0143]

[0144] in, The point is on the left-hand curvature change curve. The point is on the right-hand curvature change curve;

[0145] This represents the mean of the curvature variation curve on the left. This represents the mean of the curvature variation curve on the right.

[0146] The calculated Pearson correlation coefficient is compared with the preset Pearson correlation coefficient threshold. In this embodiment, the preset Pearson correlation coefficient threshold is set to 0.85.

[0147] If the Pearson correlation coefficient is greater than the preset Pearson correlation coefficient threshold, the two curves are similar, and the first similarity judgment result is obtained.

[0148] If the Pearson correlation coefficient is less than or equal to the preset Pearson correlation coefficient threshold, the two curves are not similar, and the second similarity judgment result is obtained.

[0149] By accurately calculating the local curvature of the left and right center peaks and fitting it into a smooth curvature change curve, and then inverting and comparing the similarity of the two curves, it is possible to effectively assess whether there are uneven pressure or deformation defects on the packaging surface. Using the Pearson correlation coefficient to evaluate the similarity of the two curves can accurately determine the pressure effect of the pressure roller group, ensuring the consistency and quality of the packaging surface, facilitating timely adjustment of production parameters, optimizing production processes, and improving product quality control.

[0150] Specifically, the defect anomaly localization module includes a defect identification unit and a source tracing and adjustment unit, wherein,

[0151] The defect identification unit is used to identify defects based on the packaging surface image of the smooth area under the second pressure when the second similarity judgment result and device interference occur, and to determine the category to which the defect belongs.

[0152] The anomaly location unit is used to determine the corresponding process and its improvement measures based on the identification results.

[0153] In this embodiment, the defect identification unit includes texture identification, region matching judgment, uniformity judgment, and reflectivity change judgment;

[0154] The steps of texture recognition are as follows:

[0155] The image is divided into several sub-regions of fixed size of 32×32 pixels, and Gabor kernels are generated using five frequencies of 0.1, 0.2, 0.3, 0.4 and 0.5 and four directions of 0°, 45°, 90° and 135°.

[0156] Gabor filtering is performed on each sub-region to obtain the energy response map;

[0157] Calculate the mean pixel energy and standard deviation of energy response for all directions in each sub-region;

[0158] The formula for calculating the average pixel energy is as follows:

[0159]

[0160] The formula for calculating the standard deviation is:

[0161]

[0162] The standard deviation of the energy response of each sub-region is compared with the standard energy response standard deviation threshold.

[0163] If the standard deviation of the energy response is greater than the standard standard deviation threshold of the energy response, the first texture recognition result is obtained and recorded as a wrinkle before further region matching judgment is performed.

[0164] The steps for region matching are as follows:

[0165] Determine whether a sub-region whose energy response standard deviation is greater than the standard energy response standard deviation threshold is within the encapsulation region;

[0166] The energy response standard deviation is used to measure the degree of texture energy change in the region. The larger the energy response standard deviation, the more drastic the texture change in the region, and the presence of wrinkles. wrinkle The obvious structural undulations are contrasted with the smooth texture areas;

[0167] The two encapsulation areas are rectangular frames. and Determine whether the center point of the sub-region whose standard deviation of energy response is greater than the standard standard deviation threshold is within any rectangle, i.e., whether it is within the encapsulation area;

[0168] If the standard deviation of the energy response is greater than the standard standard deviation threshold of the energy response, and the center point of the sub-region is within any of the rectangles, i.e. within the encapsulation area, then a uniformity judgment is further performed.

[0169] If the center point of the sub-region is not within the two rectangles (i.e., not within the encapsulation area), and the standard deviation of the energy response is greater than the standard deviation threshold of the energy response, then it is a defect.

[0170] The steps for determining uniformity are as follows:

[0171] Image edge detection is used to obtain the edges of the indentation, and the contour lines of the main indentation are extracted.

[0172] Fit the indentation edge or centerline to a discrete point sequence in image space. ;

[0173] For each intermediate point i, calculate using the central difference. The first and second derivatives;

[0174]

[0175]

[0176]

[0177]

[0178] Calculate curvature,

[0179]

[0180] Output sequence { , ,..., } represents a curvature sequence;

[0181] Calculate the curvature variation coefficient, which is the ratio of the standard deviation of the curvature sequence to the mean curvature. The formula for calculating the mean curvature is:

[0182]

[0183] Where n is the number of sub-blocks;

[0184] The formula for calculating the standard deviation of a curvature sequence is:

[0185]

[0186] The curvature variation coefficient threshold is set to 0.25, and the curvature variation coefficient is compared with the curvature variation coefficient threshold.

[0187] If the curvature change coefficient is greater than the curvature change coefficient threshold, the overall texture of the seal is uniform, which is classified as a full seal defect.

[0188] If the curvature change coefficient is less than or equal to the curvature change coefficient threshold, it is classified as a local sealing defect.

[0189] The steps for determining reflectivity changes are as follows:

[0190] Obtain the reflection intensity within each sub-block and calculate the average reflectance value of each sub-region. ;

[0191] Calculate the reflectance variation coefficient, which is the ratio of the standard deviation of reflectance to the mean reflectance. The formula for calculating the mean reflectance is:

[0192]

[0193] Where n is the number of sub-blocks;

[0194] The formula for calculating the standard deviation of reflectance is:

[0195]

[0196] The reflectance variation threshold was selected as 0.23 based on historical production samples;

[0197] The reflectivity change coefficient is compared with the reflectivity change threshold.

[0198] If the reflectivity change coefficient is greater than the reflectivity change threshold, the reflection is abnormal, which is classified as a defect in a smooth connection region.

[0199] If the reflectance change coefficient is less than or equal to the reflectance change threshold, there is a false detection, and the package is re-identified;

[0200] The anomaly localization unit is used to determine the corresponding process and its improvement measures based on the identification results.

[0201] The defect identification results include full-closure defect categories, partial-closure defect categories, and smooth connection area defect categories;

[0202] For the full sealing defect category, it manifests as the failure to form an effective heat-sealing structure in the entire sealing area. Insufficient sealing pressure leads to inadequate adhesion of the packaging material, which in turn triggers the sealing pressure adjustment prompt to ensure that sufficient pressure is generated during the pressurization process so that the sealing interface can achieve reliable fusion, thereby forming a continuous and stable sealing strip.

[0203] For the local sealing defect category, it manifests as a local area in the sealing region that is not fused or not bonded enough, triggering a sealing temperature adjustment prompt to ensure that the fusion temperature of the sealing interface meets the sealing heat fusion requirements;

[0204] For defects in smooth connection areas, the defects are not caused by the sealing process itself, but are related to external interference during the operation or transfer of the packaging on the conveyor belt. This triggers packaging transfer protection prompts to avoid damage to the packaging caused by external interference.

[0205] Accurate defect identification and anomaly localization improve the accuracy and efficiency of the packaging inspection process. Under the second pressure application, images of smooth areas are used for defect identification, effectively filtering out erroneous identifications caused by device interference, thus ensuring the reliability of defect identification results. Through detailed analysis of the packaging surface, the defect identification unit can accurately determine the category of the defect and promptly feed it back to the system for source tracing analysis. The anomaly localization unit identifies the corresponding process for the defect, provides improvement measures, and optimizes the production process to reduce the defect rate. This module not only improves detection accuracy but also enables adaptive adjustments to the production line through an intelligent feedback mechanism, ensuring the stability and consistency of product quality.

[0206] Specifically, the defect identification unit includes a texture identification subunit, a reflectivity change judgment subunit, and a region matching subunit, wherein,

[0207] The texture recognition subunit is used to call up the packaging surface image of the smooth area under the second applied pressure and perform texture detection to obtain the texture detection result;

[0208] The reflectance change judgment subunit is used to obtain the reflectance change value detection result under the first texture detection result;

[0209] The region matching subunit is used to obtain the region where the defect is located and extract the contour of the encapsulation region to determine the type of highly abrupt morphology.

[0210] The texture detection and reflectivity change judgment subunit further analyzes the texture detection results by performing texture detection on the smooth area image and uses the reflectivity change to detect possible packaging defects; the region matching subunit extracts the contour of the packaging area, judges the abrupt changes in the area, accurately determines the defect location and classifies it; based on the detection results, the defect identification unit can accurately distinguish various types of defects, ensuring the product quality stability of the production line while improving the efficiency and reliability of the detection process.

[0211] See Figure 4 The diagram shows a schematic representation of the biscuit packaging damage identification device according to an embodiment of the present invention. The present invention also provides a biscuit packaging damage identification device applied to the biscuit packaging damage identification system of this embodiment, comprising: a gradient extrusion mechanism 1, a conveyor belt 101, a flexible gripper 102, a pressure roller group 103, an image acquisition mechanism 2, an image acquisition device 201, several light sources 202, a height difference screening mechanism 3, a normal change detection mechanism 4, a defect / abnormality location mechanism 5, and a control mechanism (not shown in the diagram).

[0212] The gradient extrusion mechanism 1 includes a conveyor belt 101 for conveying the biscuit packaging to be inspected, a flexible gripper 102 for adjusting the posture of the biscuit packaging to be inspected, and a pressure roller group 103 for applying a set gradient pressure to the biscuit packaging to be inspected. The gradient extrusion mechanism is used to apply controllable extrusion conditions to induce observable deformation.

[0213] Image acquisition mechanism 2, which is connected to the gradient extrusion mechanism 1, includes a number of image acquisition units 201 spaced apart on one side of the pressure roller group and a number of light sources 202 arranged at the same spatial spacing, for acquiring images of the packaging surface of each biscuit package to be tested under pressure.

[0214] A control mechanism, connected to the gradient extrusion mechanism 1 and the image acquisition mechanism 2 respectively, includes a height difference screening unit, a normal change detection unit, and a defect anomaly localization unit. It controls each unit to perform packaging image acquisition, defect judgment, and result classification according to a set procedure.

[0215] The height difference screening unit is used to extract the peak height difference based on the first packaging surface image and compare it with the standard peak height threshold. Based on the comparison result, it outputs the judgment result of rejecting defective packaging and packaging to be confirmed as damaged.

[0216] The normal change detection unit is used to extract the set of normal changes and the normal change curve of the center peak point on both sides of the center line of the pressure roller in the smooth area based on the second packaging surface image after obtaining the second comparison result, and to identify the normal change of the damaged packaging to be confirmed, and output the classification result, including qualified packaging and slightly damaged packaging.

[0217] The defect anomaly localization unit is used to call the second packaging surface image of the slightly damaged biscuit packaging to perform defect identification when the second similarity judgment result is true, and to determine several defect types and causes based on the normal change curve of the center peak point, while outputting parameter information that can be adjusted for subsequent feedback.

[0218] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0219] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A biscuit packaging damage recognition system based on image processing, characterized in that, include, The gradient extrusion module includes a conveyor belt for conveying the biscuit packaging to be inspected, a flexible gripper, and a set of pressure rollers for applying a set gradient pressure to the biscuit packaging to be inspected. The image acquisition module, which is connected to the gradient extrusion module, includes several image acquisition devices spaced apart on one side of the pressure roller group and several light sources arranged with the same spatial spacing. The image acquisition module is used to acquire images of the packaging surface of each biscuit package to be tested under the applied pressure. The height difference screening module is connected to the gradient extrusion module and the image acquisition module respectively. It is used to determine the rejected defective biscuit packages and the unconfirmed damaged biscuit packages based on the peak height difference of the first packaging surface image. The normal change detection module is connected to the gradient extrusion module, the image acquisition module, and the height difference screening module, respectively. It is used to obtain the set of normal changes of the center peak points on the center lines of the pressure roller group on both sides of the smooth area and the corresponding center peak point normal change curve based on the second packaging surface image when the peak height difference is greater than the preset standard peak height threshold. This is used to detect the normal change of the biscuit packaging to be confirmed as damaged, and to obtain qualified biscuit packaging and slightly damaged biscuit packaging. The defect anomaly localization module is connected to the gradient extrusion module, the image acquisition module, and the normal change detection module. It is used to call the second packaging surface image of the slightly damaged biscuit packaging to identify defects, and to determine several defect types and make corresponding adjustments based on the normal change curve of the center peak point. The normal change detection module includes a front and back region segmentation unit and a similarity comparison unit, wherein... The front and rear area division unit is used to divide the smooth area into a front pressure area and a rear pressure area based on the extension direction of the center seam of the detected biscuit packaging bag and the real-time position of the pressure roller group. Based on the extension direction of the center seam, a center line is constructed through the geometric center point in the smooth area. With the pressure roller group as the boundary, the area in the smooth area that first passes through the pressure roller group and applies pressure is the front pressure area, and the area that passes through the pressure roller group and applies pressure later is the back pressure area. The similarity comparison unit is used to calculate the curvature change similarity based on the normal change curves of the left and right central peaks.

2. The image processing-based biscuit packaging damage recognition system according to claim 1, characterized in that, The height difference filtering module includes a region division unit and a height difference comparison unit, wherein, The region division unit is used to construct a three-dimensional spatial model of the packaging surface image, divide it, and obtain the division results and categories of each region of the packaging surface in a static state. The height difference comparison unit is used to compare the peak height difference of the same area under static and pressure conditions, and compare the peak height difference with a preset standard peak height threshold.

3. The image processing-based biscuit packaging damage recognition system according to claim 2, characterized in that, The region division unit includes a spatial construction subunit and an image structure recognition subunit, wherein, The spatial construction subunit is used to construct a three-dimensional spatial model of the packaging surface based on the acquired images of the packaging surface; The image structure recognition subunit is used to divide the packaging surface based on the lateral autocorrelation coefficient of the image sub-blocks. The division results include the packaging area, the connection area, and the smooth area.

4. The image processing-based biscuit packaging damage recognition system according to claim 3, characterized in that, The spatial construction subunit constructs a three-dimensional spatial model of the packaging surface based on the acquired images of the packaging surface, including: The packaging surface image is divided into several image sub-blocks according to a preset block division standard, and the row and column number identifiers and corresponding position information of each image sub-block are obtained. The geometric center points of image sub-blocks are determined as feature points and extracted. Based on the triangulation method, each feature point is converted into corresponding three-dimensional spatial coordinates and a three-dimensional spatial model is generated.

5. The image processing-based biscuit packaging damage recognition system according to claim 4, characterized in that, The image structure recognition subunit divides the packaging surface based on the lateral autocorrelation coefficient of image sub-blocks, including: Obtain the set of horizontal autocorrelation coefficients composed of the horizontal autocorrelation coefficients of each image sub-block; Based on the comparison results of each horizontal autocorrelation coefficient with the preset horizontal autocorrelation coefficient threshold, the corresponding packaging surface division results are obtained, including the encapsulation area, the connection area and the smooth area.

6. The image processing-based biscuit packaging damage recognition system according to claim 5, characterized in that, The similarity comparison unit includes a curvature detection subunit and a curve change trend comparison subunit, wherein... The curvature detection subunit is used to calculate the local curvature of each central peak point on the center line on both sides of the pressure roller group and form a curvature data set; The curve change trend comparison sub-unit is used to construct the curvature change curve of the center peak point based on the curvature data set, and to calculate the similarity between the curvature change curves of the two center peak points.

7. The image processing-based biscuit packaging damage recognition system according to claim 6, characterized in that, The curvature detection subunit calculates the local curvature of each central peak point on the center line on both sides of the pressure roller assembly and forms a curvature data set, including... While the pressure roller group is moving, the curvatures corresponding to the left and right center peaks in the packaging surface image are acquired and stored in the left center peak curvature set and the right center peak curvature set, respectively. Based on the curvature set of the left central peak and the curvature set of the right central peak, respectively, fit the curvature change curve of the left central peak and the curvature change curve of the right central peak. After reversing the curvature change curve of the left central peak and aligning it with the curvature change curve of the right central peak, calculate the similarity between the two curvature change curves of the two central peaks and obtain the similarity judgment result.

8. The image processing-based biscuit packaging damage recognition system according to claim 1, characterized in that, The defect anomaly localization module includes a defect identification unit and a source tracing and adjustment unit, wherein... The defect identification unit is used to identify defects based on the packaging surface image of the smooth area under the second pressure when the similarity judgment result is the second similarity and the device is interfering, and to determine the category to which the defect belongs. The traceability adjustment unit is used to determine the corresponding process and its improvement measures based on the identification results.

9. The image processing-based biscuit packaging damage recognition system according to claim 8, characterized in that, The defect identification unit includes a texture identification subunit, a reflectivity change judgment subunit, and a region matching subunit, wherein... The texture recognition subunit is used to call up the packaging surface image of the smooth area under the second applied pressure and perform texture detection to obtain the texture detection result; The reflectance change judgment subunit is used to obtain the reflectance change value detection result under the first texture detection result; The region matching subunit is used to obtain the region where the defect is located and extract the contour of the encapsulation region to determine the type of highly abrupt morphology.

Images