An industrial vision inspection system and method for food packaging

Abstract

The present application relates to the field of machine vision detection, and discloses an industrial vision detection system and method for food packaging. The present application mainly solves the problem of high defect mis-detection rate of flexible food packaging caused by high surface reflection and random physical wrinkles in the prior art. The system irradiates from multiple angles through a reflection correction module, generates a basic image by using non-saturation zone characteristics to compensate for saturated reflection zones; a three-dimensional recovery module reconstructs three-dimensional curved surface data representing packaging deformation based on gray scale changes; a curved surface unfolding module maps the basic image to a three-dimensional curved surface and reversely flattens it to generate a two-dimensional image without wrinkle distortion, taking the identified wrinkle area as a constraint; a defect detection module extracts the structural features of the two-dimensional image for difference comparison and marks suspected defect areas; and a verification control module confirms defects through cross-frame stable tracking and generates a rejection instruction in combination with the packaging conveying state.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection, and discloses an industrial vision inspection system and method for food packaging. Background Technology

[0002] Although machine vision has been widely used in food packaging defect detection, it still faces significant bottlenecks in image quality and deformation recognition in practical applications. On the one hand, highly reflective packaging materials are prone to localized highlight saturation, and conventional processing methods can lead to the loss of details in dark areas or misjudging reflective edges as scratches. On the other hand, the random three-dimensional physical wrinkles generated by flexible packaging during high-speed transmission will appear as complex shadows and pattern distortions in two-dimensional images. Traditional two-dimensional models cannot effectively distinguish between normal physical wrinkles and real surface defects, resulting in a high false detection rate and extremely poor model generalization ability.

[0003] Furthermore, existing detection solutions have significant dynamic deficiencies in handling complex on-site environments and the coordination of physical actuators. The traditional "single image, single judgment" mechanism is highly susceptible to interference from mechanical vibrations and airborne dust, resulting in false defect reports in a single frame, and lacks verification of the continuous temporal trajectory of the target. At the same time, existing technologies fail to accurately bind the defect coordinates output by the algorithm to the physical operating cycle of the high-speed conveyor belt in time and space, leading to delay errors in the final mechanical removal action, which cannot meet the stringent requirements of high-speed automated production lines for removal accuracy. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: an industrial visual inspection system for food packaging, comprising: The reflection correction module is used to illuminate food packaging in the conveying state from multiple angles to obtain a continuous image sequence; it identifies saturated areas based on the changes in reflection intensity under different illumination angles, and uses the spatial continuity features of unsaturated areas for compensation to generate a base image. The three-dimensional restoration module is used to obtain the normal information of the packaging surface based on the grayscale changes of different illumination directions in the continuous image sequence, and to perform surface reconstruction to generate three-dimensional surface data characterizing the deformation of the packaging surface. The surface unfolding module is used to identify wrinkled regions based on the three-dimensional surface data, and use the wrinkled regions as constraints to map the base image onto the three-dimensional surface data and unfold it to generate a two-dimensional image with wrinkle distortion eliminated. The defect detection module is used to extract the structural features of the two-dimensional image and compare them with preset standard packaging features. When the difference exceeds a threshold, the suspected defect area is marked. The verification control module is used to continuously track the suspected defective area frame by frame. When the position change meets the stability condition, the defect is confirmed, and a rejection instruction is generated according to the packaging and conveying status.

[0005] Preferably, when the reflection correction module identifies saturated regions based on changes in reflection intensity at different illumination angles and compensates for these regions using the spatial continuity features of unsaturated regions to generate the base image, it is specifically used for: Extract the extreme values ​​of reflectance grayscale of each pixel in the continuous image sequence; when the extreme values ​​of reflectance grayscale reach a preset overexposure limit, the corresponding set of pixels is determined as a saturated region, and the set of pixels that do not reach the preset overexposure limit is determined as a non-saturated region. Extract the boundary pixels adjacent to the saturated region in the unsaturated region, and calculate the gray-level gradient distribution characteristics of the boundary pixels as the spatial continuity features; Based on the gray-level gradient distribution characteristics, bidirectional tensor interpolation is performed into the saturated region, and the interpolation compensation result is replaced in the region corresponding to the continuous image sequence to output the base image.

[0006] Preferably, when the three-dimensional restoration module obtains the normal information of the packaging surface based on the grayscale changes of different illumination directions in the continuous image sequence, it is specifically used for: Retrieve the spatial light source direction coordinates corresponding to different illumination angles; Calculate the rate of change of reflected grayscale of the target pixel in the continuous image sequence under illumination by different spatial light source directional coordinates; Establish a set of reflectance equations relating the rate of change of reflectance grayscale to the directional coordinates of the spatial light source, and solve the set of reflectance equations to obtain the surface gradient vector of the target pixel. The surface gradient vector is spatially normalized to output the normal information of the packaging surface.

[0007] Preferably, when the three-dimensional restoration module performs surface reconstruction and generates three-dimensional surface data characterizing the deformation of the packaging surface, it is specifically used for: Construct a partial differential equation based on the divergence distribution of the normal information of the packaging surface; Pre-defined boundary smoothing constraints are introduced to correct the partial differential equation; The height and depth distribution characteristics in three-dimensional space are obtained by iteratively solving the modified partial differential equation using the least squares method. The height and depth distribution features are topologically stitched together with the corresponding spatial coordinate grid to generate the three-dimensional surface data.

[0008] Preferably, when identifying wrinkle regions based on the three-dimensional surface data, the surface unfolding module is specifically used for: Calculate the principal curvature extrema of each coordinate node on the three-dimensional surface data; Calculate the gradient of principal curvature extrema between adjacent coordinate nodes; When the gradient of the extreme value of the principal curvature exceeds the preset deformation tolerance limit, it is determined that the coordinate nodes in the corresponding connected range have undergone abrupt folding, and the clustering range of the coordinate nodes that have undergone abrupt folding is marked as the folded region.

[0009] Preferably, when the surface unfolding module maps the base image onto the three-dimensional surface data using the wrinkled region as a constraint to generate a two-dimensional image with wrinkle distortion eliminated, it is specifically used for: The base image is used as a texture map to cover the three-dimensional surface data, and a mapping relationship between two-dimensional texture coordinates and three-dimensional spatial coordinates is established. Under the constraint of locking the boundary position of the folded region, the optimal geodesic distance of the three-dimensional surface data projected onto the target plane is calculated based on the energy minimization model; The two-dimensional texture coordinates are inversely stretched and compensated based on the optimal geodesic distance to output a flattened two-dimensional image.

[0010] Preferably, when the defect detection module extracts the structural features of the two-dimensional image and compares them with preset standard packaging features, and marks suspected defect areas when the difference exceeds a threshold, it is specifically used for: Edge response extraction and feature corner detection are performed on the two-dimensional image to generate local topological contour vectors as structural features; Obtain the standard feature manifold space generated based on defect-free sample training as a preset standard packaging feature; Calculate the geodesic distance between the local topological contour vector and the standard feature manifold space in the corresponding coordinate dimensions; When the feature geodesic distance is greater than the set deviation limit, the corresponding image pixel set is closed and connected, and marked as a suspected defect area.

[0011] Preferably, when the verification control module performs continuous frame tracking on the suspected defect area and confirms the defect when the position change meets the stability condition, it is specifically used for: Extract the centroid coordinates and area contour features of the suspected defect region; Between consecutively acquired adjacent image frames, the suspected defective areas belonging to the same package are matched and tracked based on the area contour features. Calculate the variance of the centroid position coordinates of the tracked suspected defect area over a preset number of consecutive frames; When the fluctuation variance is within the set stationary error range, it is determined that the stability condition is met, and the suspected defect area is confirmed to be a physical defect.

[0012] Preferably, when the verification control module generates a rejection instruction based on the packaging conveying status, it is specifically used for: The conveyor belt running cycle parameters fed back by external sensors are read in real time, and the running cycle parameters are used as the packaging conveying status. Obtain the spatial physical coordinates of the current location of the identified physical defect, and calculate the running path difference between the spatial physical coordinates and the hardware coordinates of the workstation that performs the removal action; The trigger delay time for the elimination action is calculated based on the difference in the running path and the running beat parameters. The trigger delay time is written into the control protocol frame for hardware timing binding, and then synthesized into the culling instruction sent to the external hardware system.

[0013] An industrial visual inspection method for food packaging includes: S1: Illuminate the food packaging under transport conditions from multiple angles to obtain a continuous image sequence; identify saturated areas based on the changes in reflection intensity under different illumination angles, and use the spatial continuity features of unsaturated areas for compensation to generate a basic image; S2: Obtain the normal information of the packaging surface based on the grayscale changes of different illumination directions in the continuous image sequence, and perform surface reconstruction to generate three-dimensional surface data characterizing the deformation of the packaging surface; S3: Identify the wrinkled region based on the three-dimensional surface data, and use the wrinkled region as a constraint to map the base image onto the three-dimensional surface data and then unfold it to generate a two-dimensional image with wrinkle distortion eliminated. S4: Extract the structural features of the two-dimensional image and compare them with preset standard packaging features. When the difference exceeds a threshold, mark the suspected defect area. S5: Perform continuous frame tracking on the suspected defective area. When the position change meets the stability condition, confirm the defect and generate a rejection instruction based on the packaging and conveying status.

[0014] Compared with the prior art, the present invention has the following beneficial effects: This invention utilizes a reflection correction module for multi-angle, time-division illumination and precisely locates saturated areas based on changes in reflection intensity. By extracting the spatial continuity features of unsaturated areas and performing bidirectional tensor interpolation compensation, overexposed areas are successfully repaired without losing the original texture of the packaging surface. This mechanism eliminates the interference of highly reflective materials on visual algorithms at the physical source, providing a clean, information-free foundation image for subsequent accurate detection.

[0015] This invention overcomes the limitations of traditional two-dimensional detection. The three-dimensional restoration module reconstructs the three-dimensional surface data of the packaging surface using grayscale changes from different illumination directions. The surface unfolding module further uses the wrinkled areas on the surface as constraint boundaries and, based on principles such as equidistant mapping, inversely maps and "physically flattens" the base image. This feature cleverly transforms uncontrollable three-dimensional physical wrinkles into standard, distortion-free two-dimensional planar images, so that normal surface deformation no longer interferes with the visual pattern. This fundamentally eliminates false detections due to feature variations caused by packaging bending and squeezing, and significantly reduces the dependence on the large sample library of deep learning.

[0016] The defect detection module of this invention abandons pixel-level template matching, which is easily affected by lighting and subtle color differences. Instead, it extracts the structural features of two-dimensional images and calculates their topological distance to a preset standard packaging feature space. This mathematical comparison method based on structural tensors and manifold spaces has extremely high tolerance for conventional printing drift on the packaging surface, while also being able to keenly detect substantial topological abrupt changes such as damage and foreign objects, significantly improving the accuracy and robustness of detection.

[0017] The verification control module of this invention introduces a timing state machine logic. By continuously tracking the variance of the centroid coordinate fluctuation of suspected defects over multiple frames, it effectively filters out transient noise in a single frame caused by mechanical vibration or dust. More importantly, this invention directly reads the conveyor belt's operating cycle parameters, converts the calculated physical path difference into an accurate trigger delay time, and writes it into the control protocol. This achieves millisecond-level precise alignment between the spatial coordinates of the visual algorithm and the physical time of the on-site actuator, ensuring a high success rate for rejection even under high-speed operation. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of an industrial vision inspection system for food packaging according to the present invention. Figure 2 This is a schematic diagram of an industrial visual inspection method for food packaging according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example

[0020] Please see Figure 1 As shown, this embodiment provides an industrial vision inspection system for food packaging, specifically including the following steps: This embodiment details the specific internal operating mechanism and algorithm execution steps of the reflection correction module. In the production process of food packaging materials such as flexible plastic films or metallic aluminum foil composites, strong highlights are easily generated in localized areas. These highlights can completely obscure the true texture of the packaging surface, causing conventional visual algorithms to misjudge the highlight edges as material scratches or pattern damage. To eliminate this interference from both physical and mathematical perspectives, the reflection correction module in this invention is configured to execute a high-precision image stripping and tensor repair algorithm.

[0021] The reflection correction module continuously receives a sequence of images from a front-end industrial camera. These image sequences are acquired by time-division stroboscopic illumination of the same food packaging within a preset conveyor belt operating cycle. For multiple frames of images in the buffer matrix, the processor initiates a traversal addressing procedure. Using the absolute spatial two-dimensional coordinates of the image as a reference, the processor extracts the extreme values ​​of reflection grayscale at each absolute coordinate pixel in the temporal dimension of the continuous image sequence. Under the conventional 8-bit image depth standard, the effective range of pixel grayscale values ​​is 0 to 255. The system internally sets a strict overexposure limit, which is usually set between 240 and 253, and specifically 245 in this preferred embodiment. When the extreme value of reflection grayscale at a certain pixel across time reaches or exceeds the preset overexposure limit of 245, it indicates that photon saturation overflow has occurred at that physical location when illuminated by light at a specific angle, resulting in the loss of color details carried by diffuse reflection. The processor then extracts these pixel coordinates and determines their set as a saturation region. Conversely, for pixels whose reflectance grayscale values ​​remain consistently between 0 and 244 regardless of the light source, the processor classifies them as unsaturated regions. The reason for setting the range of 240 to 253 is that if the threshold is below 240, it is easy to misjudge normal pure white printed patterns on food packaging as reflective blind spots; if the threshold is above 253, it is impossible to completely remove the blurred highlight halos that are in a critical saturation state, thus leaving characteristic noise.

[0022] To perform physical-level seamless restoration of saturated regions with lost information, the module needs to perform mathematical deduction using the surrounding real physical texture trends. The processor performs morphological dilation calculations on the topological boundary zone of the two regions defined above, thereby extracting the boundary pixels directly adjacent to the saturated region in the unsaturated region. To ensure that the extracted features have sufficient statistical significance and do not introduce interference from irrelevant backgrounds that are too far away, the boundary pixels extracted by the processor form a ring-shaped transition zone with a width ranging from 3 to 9 pixels in space, with a preferred sampling width of 5 pixels. Next, the module performs depth feature extraction on these boundary pixels located within the ring-shaped transition zone, that is, calculates the gray-level gradient distribution features of the boundary pixels as spatially continuous features. The processor uses a differential convolution calculation window with a size between 3x3 pixels and 7x7 pixels to traverse and calculate the spatial partial derivatives of each boundary pixel in the horizontal and vertical directions. The absolute magnitude of the gray-level change of the pixel and the steepest direction angle of the change are calculated from the partial derivatives, and the numerical range of this direction angle is mapped to a distribution between 0 degrees and 360 degrees. The amplitude represents the intensity of color or texture change, while the direction angle represents the actual extension trajectory of lines or color blocks on the packaging surface. These comprehensive multi-dimensional vector data, including gradient amplitude and direction angle, constitute the grayscale gradient distribution characteristics, which perfectly describe how the physical information of the unsaturated region extends continuously and smoothly into the saturated region.

[0023] After establishing the spatial continuity features, the reflection correction module activates its core bidirectional tensor interpolation calculation engine. Based on the grayscale gradient distribution features, the processor performs bidirectional tensor interpolation calculations within the saturation region. The processor constructs a multidimensional feature tensor space and mathematically approximates the central void region from the two relative boundaries of the saturation region. For any missing pixel to be repaired within the saturation region, the processor emits virtual addressing rays outward along the horizontal and vertical axes in its two-dimensional coordinate system, reaching the known pixels within the extracted annular transition zone. At this point, the algorithm accurately calculates the spatial Euclidean geometric distance from the missing pixel to each intersecting boundary pixel. In actual industrial food packaging inspection, this geometric distance typically ranges from 1 to 100 pixels, depending on the physical area of ​​the light spot formed by the light source on the film surface. The processor uses the reciprocal of this Euclidean geometric distance as the core weight attenuation coefficient; the closer the physical distance between boundary points, the greater their contribution to the internal repair result. To ensure the convergence stability of the computational system, the effective mapping range of the weight decay coefficient is strictly normalized and limited to between 0.01 and 1.0. Based on these normalized weight decay coefficients and the boundary feature tensor, the processor deduces the original true grayscale value of the missing pixel through high-order matrix multiplication. If the boundary feature shows a black printed barcode extending into the highlight area at a 45-degree angle, the interpolation algorithm will strictly follow this angle and the surrounding grayscale gradient to redraw the missing part of the barcode within the saturation region.

[0024] After all pixels within the saturated region have had their pixel values ​​calculated using the tensor derivation described above, the processor generates an interpolation compensation matrix that perfectly matches the physical shape and spatial coordinates of the original saturated region. To meet the extremely high inspection rate of 300 to 600 pieces per minute in modern high-speed industrial production lines, the system time for each feature extraction and tensor interpolation matrix calculation process is strictly limited to within 2 to 8 milliseconds by the underlying hardware-level concurrent program. After the calculation is completed, the reflection correction module performs a memory-level data overwrite operation, precisely replacing the interpolation compensation result with the original coordinate region corresponding to the continuous image sequence. After the data replacement, the originally glaring and large-area specular highlight white spots are completely eliminated, replaced by a digitally restored image that seamlessly blends with the surrounding real texture. Finally, the reflection correction module independently outputs this fully physically and logically restored image as the base image.

[0025] It should be noted that the reflection correction module in this embodiment is not only used to improve the display effect of two-dimensional images, but also to provide effective optical information after physical reflection separation for the subsequent three-dimensional reconstruction process. During multi-angle illumination, the specular reflection component of the food packaging surface will cause abnormal grayscale changes. Therefore, the three-dimensional reconstruction module does not directly use the original continuous image sequence, but extracts surface optical features based on the corresponding area of ​​the base image output by the reflection correction module. Specifically, the reflection correction module first identifies the saturated reflection area caused by direct light source according to the pixel grayscale change relationship under different illumination directions, and restores the missing information by utilizing the texture continuity of adjacent unsaturated areas. Subsequently, the three-dimensional reconstruction module only calls the corrected effective grayscale change area as the basis for surface morphology calculation, thereby avoiding the offset of surface normal calculation caused by specular highlights. Through the above processing, the reflection correction result is correlated with the three-dimensional reconstruction process, which not only eliminates the interference of highly reflective materials on grayscale measurement, but also ensures the consistency of input data in the subsequent surface reconstruction process.

[0026] After completing the initial image reflection correction and acquiring the basic image free from strong light interference, the system enters the core physical morphology reconstruction stage. This embodiment details the specific operating mechanism of the 3D reconstruction module. Flexible food packaging often exhibits highly random compression deformation and surface undulations on the production line. Relying solely on the grayscale features of a two-dimensional plane cannot accurately distinguish the color boundaries of printed patterns from the actual physical wrinkles and shadows. Therefore, the 3D reconstruction module is configured to use photometric stereo algorithm logic to inversely analyze the two-dimensional grayscale changes into three-dimensional surface normal vectors, thereby reconstructing a realistic three-dimensional curved surface.

[0027] To acquire the surface normal information of the packaging, the 3D reconstruction module needs to establish a calculation benchmark for spatial optical geometry. Therefore, the processor retrieves the spatial light source direction coordinates corresponding to different illumination angles. Externally to the industrial camera, a multi-directional light source array with the production line inspection center as the origin is pre-installed at the system hardware level. After the light source array is installed, spatial orientation calibration of the light sources is required. Specifically, a standard reflective calibration piece with known spatial dimensions is placed at the center of the food packaging inspection area, placing it within the field of view of the industrial camera. Each light source is individually illuminated in a preset sequence. The industrial camera acquires images of the corresponding reflective areas and determines the spatial orientation information of each light source relative to the camera coordinate system based on the positional relationship between the surface reflection position of the calibration piece and the camera imaging coordinates. The spatial orientation information includes the direction of the light source and the incident direction of the light, used to establish the correspondence between subsequent grayscale changes at different illumination angles and the surface normal of the packaging. The calibrated spatial light source orientation information is stored internally in the controller and directly retrieved during actual inspection.

[0028] The processor calculates the rate of change of reflected grayscale of a target pixel in a continuous image sequence under different spatial light source directional coordinates. The system approximates the diffuse reflective surface of food packaging as a perfect Lambertian reflector. Since the pre-processed reflection correction module has eliminated the interference of specular reflection saturated pixels, the grayscale variation of any target pixel in the image is now entirely determined by the angle between the microscopic physical tilt surface of that point and the angle of the incident light. The processor in the 3D reconstruction module extracts the absolute grayscale value of a target pixel in four consecutive time slice images. Subsequently, the processor calculates the dynamic difference of the grayscale value of that point under different illumination excitations, and performs a derivative operation on the ratio of this dynamic difference to the known angle difference of the light source switching, thereby obtaining the specific rate of change of reflected grayscale for each microscopic surface. This rate of change essentially reflects the sensitivity of a specific physical surface to the energy absorption and reflection of incident light from different directions.

[0029] After obtaining the grayscale variation pattern, the 3D reconstruction module enters the equation construction and solution stage. The processor establishes a set of reflectivity equations relating the rate of change of reflected grayscale to the spatial light source direction coordinates. Specifically, the processor sets the unknown surface normal vector of the target pixel as the unknown solution set of the multivariate equations, uses the known spatial light source direction coordinate matrix as the coefficient matrix of the equations, and uses the extracted rate of change of reflected grayscale as the constant term vector of the equations. Based on Lambert's cosine theorem, the processor describes this complex optical mapping relationship by constructing a set of overdetermined linear algebraic equations. Subsequently, the processor calls algebraic algorithms such as singular value decomposition to solve the reflectivity equations through the pseudo-inverse operation of the matrix to obtain the surface gradient vector of the target pixel. This surface gradient vector contains the initial physical slope information of the pixel in the horizontal and vertical axes, and mathematically accurately represents the three-dimensional tilt attitude of the food packaging surface in each minimal local region.

[0030] To unify the computational dimensions of subsequent large-scale surface integral reconstruction, the processor performs spatial normalization on the surface gradient vectors. Since the initial vectors obtained from solving the equations often contain inherent reflectivity and albedo information from the printed material on the packaging surface, their vector lengths are not standard unit lengths. The processor independently calculates the Euclidean magnitude of each initial gradient vector in three-dimensional space and divides each directional component of the original vector by this magnitude, thus completely filtering out the albedo interference caused by varying shades of color in the food packaging surface printing. After rigorous normalization, all surface vectors are forcibly converted into unit direction vectors with a constant spatial length of 1, which the system outputs as the final packaging surface normal information. This extremely densely distributed normal information precisely guides the true physical orientation of every pixel on the packaging surface.

[0031] After acquiring the globally dense normal information of the packaging surface, the system automatically triggers a deeper surface reconstruction logic. The 3D recovery module then initiates the surface reconstruction program, aiming to restore the microscopic normal orientation integral to the macroscopic deformation characteristics of the packaging surface. Due to the unavoidable dust noise and abrupt folds and wrinkles at the edges of flexible packaging in industrial environments, traditional simple path integral algorithms are prone to the unlimited accumulation and expansion of calculation errors, ultimately resulting in a severely distorted reconstructed surface. To overcome this industry technical bottleneck and significantly enhance the remarkable progress of this invention, the system creatively introduces a partial differential equation reconstruction algorithm with boundary adaptive smoothing constraints.

[0032] In the initial stage of surface reconstruction, the processor constructs a partial differential equation based on the divergence distribution of the surface normal information. In continuous calculus geometry, the divergence of a two-dimensional plane characterizes the source-sink divergence and convergence of the vector field at a certain point. The processor extracts the relative gradient of the normal information of each pixel along the horizontal axis and the relative gradient along the vertical axis, and adds them together to obtain the global divergence matrix of the normal vector field. Under an ideal mathematical model, the Laplacian operator of the height field of the target 3D surface should be strictly equivalent to the divergence of this normal vector field. Therefore, the processor constructs the initial basic Poisson partial differential equation based on the core logic that the sum of the second partial derivatives of the height field is equivalent to the normal divergence.

[0033] Food packaging on the production line has clearly defined physical edge cut lines, and its surface exhibits abrupt physical wrinkles caused by vacuuming or compression of the contents. In these topological extreme regions, the normal information exhibits discontinuous, abrupt jumps. To prevent the integral surface from tearing infinitely at these jump points, the 3D recovery module introduces preset boundary smoothing constraints to correct the partial differential equation. Specifically, the system adds an energy penalty constraint term driven by physical curvature to the end of the original Poisson equation. When the underlying algorithm detects a drastic abrupt change in the local normal, this constraint automatically increases the smoothing calculation weight for that region, forcing the integral surface to maintain its macroscopic morphological gradient while effectively suppressing local abnormal peaks and computational divergence.

[0034] During the surface reconstruction process, the processor first determines the 3D recovery range based on the edge contour of the packaging area and uses the packaging edge nodes as fixed boundary nodes to avoid the background area from participating in the surface calculation. For internal region nodes, a discrete mesh structure is established based on the spatial connection relationship between adjacent nodes, and the node height values ​​are updated synchronously in each iteration. When the change in the height of all nodes is within the preset allowable range in two consecutive iterations, or when the overall error meets the preset convergence condition, the iterative calculation is stopped, and the current height distribution is used as the final 3D surface data.

[0035] To ensure the innovative correction process possesses extreme numerical stability and physical interpretability, the processor employs the following core iterative correction partial differential formula for accurate derivation of the three-dimensional height features: in, The absolute height-depth distribution characteristic value of the food packaging surface to be solved in three-dimensional space is represented by the physical dimension of length unit, which characterizes the vertical physical height of each pixel on the packaging surface relative to the conveyor belt plane; the signs of the first and second terms on the left side of the formula are the height-depth distribution characteristic value on the horizontal axis of the two-dimensional coordinate system. Direction and vertical axis The continuous second-order partial derivatives in the direction, the sum of which constitute the Laplace operator of the height field core, describe the average degree of divergence of the local surface relative to its surrounding neighborhood. Represents the surface normal information of food packaging on the horizontal axis Initial gradient projection components in the direction, Represents the normal information of the food packaging surface on the vertical axis The sum of the first-order partial derivatives of the initial gradient projection components of the direction is the macroscopic divergence distribution of the entire normal vector field. The smoothing constraint weight coefficient represents the pre-calibrated value of the system. This parameter is a dimensionless constant, and its value is pre-set by the system according to the physical elastic modulus of the packaging material. It directly controls the physical filtering and ironing intensity of the algorithm on the tiny discrete wrinkles on the surface of the packaging bag. Representing the local average physical curvature of the spatial neighborhood of the current operation pixel, this parameter serves as an adaptive geometric adjustment factor. By calculating the geometric curvature of the current neighborhood nodes in real time, it dynamically guides the mathematical convergence direction in the surface reconstruction process, effectively limiting the abnormal mathematical divergence of height values ​​at the edge fault of the packaging.

[0036] After establishing the constrained improved formula, the processor iteratively solves the modified partial differential equation using the least squares method. Faced with a massive and highly sparse matrix equation system composed of millions of pixel nodes, the processor employs high-order discretization numerical analysis methods such as conjugate gradient descent for large-scale parallel and rapid solution. In each microsecond-level iteration, the system dynamically corrects the absolute value of the height prediction for each pixel based on the Gaussian error residual of the current grid, until the global energy calculation error of the entire height matrix converges below the set convergence threshold. At this point, the value locked at each node in the matrix network represents the calculated height-depth distribution characteristics in three-dimensional space, accurately reflecting the absolute physical depth of the packaging surface relative to the bottom surface of the reference conveyor belt.

[0037] The 3D reconstruction module executes a spatial 3D modeling protocol, topologically stitching the height and depth distribution features with the corresponding spatial coordinate mesh. The processor generates a 2D Euclidean mesh base in explicit memory that absolutely corresponds to the physical pixel resolution of the front-end industrial camera. Subsequently, the processor maps and assigns the height and depth distribution feature values ​​of each local pixel calculated in the previous steps, as the absolute axial coordinates of the vertical physical depth, to the corresponding intersection nodes on this 2D mesh base. The originally flat 2D planar coordinate mesh is instantly endowed with 3D digital elevation undulations, thus successfully completing the dimensional leap from planar data to a 3D physical model, ultimately generating and outputting 3D curved surface data.

[0038] After the 3D restoration module successfully reconstructs the 3D surface data representing the absolute physical depth of the food packaging surface undulations, the system faces the core task of geometric coordinate system transformation. Although the 3D surface data perfectly captures all the undulations of the packaging bag, downstream defect detection algorithms, such as template contour matching and printed character recognition, are based on an absolutely flat 2D pixel grid. If an image with 3D wrinkles and undulations is directly fed into the defect detection module, the originally standard printed fonts will undergo severe topological distortion due to the curvature of the physical surface, resulting in a very high false positive rate. Therefore, this embodiment elaborates on the specific operating mechanism of the surface unfolding module, which aims to virtually flatten the three-dimensional digital elevation undulations like an iron through rigorous differential geometric operations, restoring the original flat topological shape of the packaging surface.

[0039] For the specific execution steps of the surface unfolding module to identify and delineate wrinkled areas on the packaging surface, the system must be able to accurately locate which areas are smooth, natural transitions and which areas are severe folds caused by vacuuming or physical compression from a continuous three-dimensional height field. To this end, the processor deeply analyzes the input three-dimensional surface data and constructs a local calculus geometric field. The processor extracts the three-dimensional absolute coordinates of each coordinate node on the three-dimensional surface data surface and uses the difference of Gaussians operator combined with Hessian matrix analysis to calculate the principal curvature extrema of each coordinate node on the three-dimensional surface data surface. In differential geometry, any point on a spatial surface has two orthogonal principal curvatures, representing the two tangential directions with the maximum and minimum curvature at that point. The processor calculates the absolute values ​​of the maximum and minimum principal curvatures for each coordinate node; these two values ​​constitute the principal curvature extrema of that node, which physically quantify the local sharpness and unevenness of the packaging surface at any point.

[0040] Relying solely on the principal curvature extrema of a single node is insufficient to completely distinguish between the cylindrical natural bulges of a packaging bag and true abrupt, precipitous creases. For example, a fully inflated packaging bag may have a high principal curvature overall, but this does not fall under the category of creases requiring special treatment. Therefore, the processor further calculates the gradient of principal curvature extrema changes between adjacent coordinate nodes. The system establishes a 3x3 or 5x5 local topological neighborhood centered on the current coordinate node in the 3D mesh space, extracts the difference in principal curvature extrema between the central node and all its neighboring nodes, and divides this difference by the spatial physical distance between the nodes to obtain the rate of change of the principal curvature extrema in the spatial dimension, i.e., the gradient. This gradient data filters out the overall macroscopic natural curvature of the packaging bag, acting like a digital scalpel to specifically highlight those microscopic regions where drastic bending and morphological abrupt changes occur within extremely short physical distances.

[0041] The system retrieves material physical deformation parameters pre-stored in the controller and sets a deformation tolerance limit specifically for the current flexible food packaging material. When the processor compares and finds that the gradient of the principal curvature extreme value change in a local neighborhood exceeds the preset deformation tolerance limit, the system determines at the algorithm level that the coordinate nodes within the corresponding connected range have undergone unnatural abrupt folding. At this point, simply marking isolated aberrant nodes is insufficient; the processor initiates a region growing clustering algorithm based on multi-source convergence. Using these abruptly folded nodes that exceed the limit as seed points, the system performs a grid-based search to spread to the surrounding transition regions where gradient values ​​gradually decay, merging all spatially adjacent nodes with aberrant gradients into a complete digital connected domain. Finally, the system precisely labels the clustering range of coordinate nodes with abrupt folding as the wrinkled region. This labeling process not only depicts the core skeleton of the wrinkles but also completely encapsulates the deformation transition zone at the edge of the wrinkles, providing extremely precise geometric constraint boundaries for subsequent reverse stretching.

[0042] After accurately locating the spatial positions of all folded areas, the system automatically triggers deeper geometric dimensionality reduction and texture compensation logic. The surface unfolding module then starts the two-dimensional image generation program, aiming to use the folded areas as the core constraint to perfectly map the basic image, which has been freed from strong light interference, onto the three-dimensional surface data and perform physical flattening.

[0043] Before surface unfolding, the surface unfolding module first performs mesh discretization on the 3D surface data. Specifically, triangular mesh elements are created based on the positional relationships between adjacent coordinate nodes in the 3D surface data, with each mesh node corresponding to a texture sampling position in the base image. When a wrinkled region is detected, the boundary nodes of the wrinkled region are used as fixed constraint nodes during the unfolding process to maintain their spatial topological relationships. For nodes located inside the wrinkles and in the surrounding transition regions, adjustments are made step-by-step based on the connection relationships between adjacent meshes. During the adjustment process, the system calculates the unfolded 2D position based on the changes in distance between nodes, ensuring that the adjacency relationships in the 3D surface remain continuous in the 2D plane. After all mesh nodes have completed 2D position mapping, the corresponding texture pixels are resampled based on the 2D node coordinates to generate the flattened 2D image.

[0044] In the initial stage of reconstruction, the processor performs a fusion mapping operation between texture mapping and spatial binding. The processor extracts the base image output from the preceding reflection correction module and overlays it as a high-resolution texture map onto the 3D surface data. Through a homogeneous coordinate transformation matrix, the system establishes a rigorous and one-to-one mapping relationship between the 2D planar pixel coordinate system of the base image and the 3D Cartesian coordinate system of the 3D surface data. At this point, each geometric node on the 3D surface mesh not only possesses an absolute 3D spatial position and curvature characteristics but is also endowed with the true color and grayscale attributes of the corresponding pixel in the base image. This operation of binding visual texture with physical topology depth is a prerequisite for achieving lossless flattening.

[0045] To non-destructively project and flatten this complex, textured 3D surface onto a 2D target plane, the system creatively introduces an energy minimization model based on the principles of elasticity. If a simple perpendicular parallel projection algorithm is used, the printed patterns on the wrinkled sidewalls will be severely compressed and overlapped, resulting in permanent information loss. True physical flattening requires simulating the deformation recovery process of a flexible film under uniform four-dimensional stretching. Therefore, under the constraint of locking the boundary positions of the wrinkled regions, the processor calculates the optimal geodesic distance from the 3D surface data to the target plane based on the energy minimization model. The geodesic represents the true physical shortest crawling path between two points on the 3D curved surface.

[0046] To ensure the geodesic derivation and surface flattening process possess ultimate numerical stability and physical interpretability, and to effectively avoid mesh tearing during the flattening process, the processor employs the following core energy functional calculus formula for accurate derivation of the optimal geodesic distance: ,in, This represents the total global elastic deformation energy generated by the entire coordinate grid system during the virtual flattening process. The ultimate computational goal of the system is to minimize this total energy value through iterative optimization. The horizontal spatial coordinates of the target projection plane; lowercase letters in the formula The vertical spatial coordinate variable representing the target projection plane. The intrinsic tensile elastic modulus of food packaging materials is a constant that directly determines the mathematical stiffness of the mesh in resisting planar tensile deformation. This represents the difference between the optimal geodesic distance that the processor is currently solving and the initial spatial Euclidean distance between the corresponding two points. The square of this difference directly represents the energy penalty for pure stretching deformation caused by the flattening operation. The intrinsic bending stiffness coefficient represents the material's ability to resist folding. This represents the gradient of principal curvature extrema between adjacent coordinate nodes obtained in the preceding step; The first spatial derivative of the optimal geodesic distance to be solved in the local coordinate grid is represented by the first spatial derivative. It, together with the principal curvature gradient, constrains the grid flow rate of the folded region during flattening. This represents the system's preset anchoring weight for the folded region boundary. It is a very large penalty coefficient used to limit the unwarranted deformation of non-folded regions. The region identifier characteristic function represents the coordinate node. When the coordinate node currently performing the double integration operation is located inside the folded region defined in the previous step, the value of this characteristic function is one, while when the node is located in a flat region outside the folded region, the value of this characteristic function is zero.

[0047] After establishing the energy minimization formula with regional constraints, the processor derived the corresponding Euler-Lagrange partial differential equations using the variational method and performed large-scale numerical solutions on a finite element mesh using a semi-implicit Euler discretization method. Within an extremely short computation cycle, the system found a set of two-dimensional planar coordinate distributions that minimized the global deformation energy E. The optimal geodesic distance calculated at this point perfectly simulated the actual surface dimensions of the packaging bag after it was uniformly flattened in the physical world, completely eliminating the spatial perspective shortening effect caused by three-dimensional folding.

[0048] The surface unfolding module enters the pixel reconstruction and image output stage. Based on the calculated optimal geodesic distance, the processor performs inverse stretching compensation on the 2D texture coordinates bound to the 3D mesh. The system establishes a completely new 2D target canvas. For pixels severely compressed within wrinkled areas, the processor uses bicubic spline interpolation to fill in missing grayscale information between pixels according to the expansion ratio indicated by the optimal geodesic distance, precisely physically expanding the originally stacked printed pattern. For pixels slightly stretched at the edge transition zones, appropriate shrinkage calculations are performed. After global inverse stretching compensation, the originally distorted barcode geometric lines and the expiration date characters swallowed by wrinkles are all restored to their original flat printing state. Finally, the system outputs a 2D image in an absolutely flat state. This 2D image completely eliminates the dual interference of 3D physical deformation and strong light reflection, like a perfectly flat original printed film laid out on the inspection platform. This provides an impeccable ultimate visual analysis benchmark for the system to finally execute sub-pixel-level comparison of minute defects and precise defect rejection instructions.

[0049] After the surface unfolding module successfully outputs a flattened 2D image that is completely free from the dual interference of 3D physical deformation and strong light reflection, the system then enters the most crucial quality assessment stage. This embodiment will explain in detail the specific operating mechanism of the defect detection module.

[0050] This module undertakes the demanding task of accurately identifying minute defects such as scratches, missing prints, ink droplet contamination, and residual fine wrinkles from complex food packaging printing backgrounds. To balance detection sensitivity and robustness at extremely high production line speeds, the system abandons the traditional global pixel subtraction method and instead adopts a topological feature comparison architecture based on a high-dimensional manifold space.

[0051] The defect detection module requires in-depth geometric feature analysis of the input 2D image. After receiving the flattened 2D image, the processor activates a high-pass filtering engine to perform edge response extraction and feature corner detection. In the edge response extraction stage, the processor calls a preset multi-scale directional gradient convolution kernel to perform sliding convolution scanning on the pixel matrix along the horizontal, vertical, and two diagonal directions. This operation can accurately capture abrupt grayscale changes in areas such as barcode edges, trademark outlines, and text strokes, thereby generating high-precision gradient magnitude and direction matrices. Simultaneously, in the feature corner detection stage, the processor constructs the autocorrelation matrix of local pixels to find geometric intersections with significant grayscale gradients in multiple directions. These corners typically correspond to key nodes in packaging printing patterns or turning points in font strokes, possessing extremely high spatial localization value. Subsequently, the system performs tensor dimensionality reduction fusion with the extracted edge gradient information and corner spatial distribution features to generate a high-dimensional local topological contour vector for each local sliding window region in the image. This vector not only encapsulates the local contour shape and texture frequency, but also contains the geometric topological relationship of the pixel spatial arrangement, thus serving as the core structural feature representing the current detailed state of the packaging surface.

[0052] After acquiring the structural features of real-time images, the system requires an absolutely accurate reference benchmark that accommodates minor, reasonable fluctuations. To this end, the defect detection module acquires a standard feature manifold space generated based on defect-free samples and uses it as the preset standard packaging feature. During the offline calibration phase before the system's official launch, engineers input manually confirmed defect-free food packaging samples into the system. The system performs statistical analysis on the local topological contour vectors of multiple samples to establish a standard feature space reflecting the range of normal packaging structural variations. The system extracts the local topological contour vectors of these perfect samples and projects them into a high-dimensional mathematical coordinate system. Because even after flattening, flexible food packaging still exhibits slight legitimate randomness due to the microscopic stretching of printed patterns and the natural blurring of ink, the high-dimensional vectors of these perfect samples do not cluster into an absolutely isolated point. Instead, they are distributed in the high-dimensional space to form a continuously curved mathematical surface with a certain thickness; this surface is called the standard feature manifold space.

[0053] During the real-time detection phase, the defect detection module undergoes a rigorous difference comparison process. The processor projects the locally extracted topological contour vector into the constructed high-dimensional coordinate system and calculates the geodesic distance between this local topological contour vector and the standard feature manifold space in the corresponding coordinate dimension. Here, the system innovatively uses geodesic distance instead of the traditional Euclidean straight-line distance. When processing complex topological features, simply calculating the shortest Euclidean perpendicular distance from the target point to the manifold surface can easily misjudge legitimate deformations that reasonably extend along the manifold surface as abnormal defects. Geodesic distance, however, forces the algorithm to find a path within the manifold surface that best fits the sample distribution trend. By evaluating the path length within the manifold surface between the real-time vector and the nearest legitimate node, the true topological difference between the two is quantified. This distance metric perfectly filters out false differences caused by the natural deformation of flexible materials, ensuring the system's focus is solely on genuine defects that deviate from the regularities of the manifold surface.

[0054] The system enters a closed loop of visualization and calibration of the defect area. The processor retrieves a pre-set deviation limit parameter, which represents the maximum degree of topological variation the system can tolerate. When the processor finds that the feature geodesic distance calculated for a certain local sliding window area is greater than the set deviation limit, the system determines at the algorithm level that an abnormal variation has occurred in that local area, most likely due to ink loss or foreign object obstruction causing pattern destruction. Due to the ultra-high resolution of industrial cameras, these initially identified anomalies often appear as discrete, fragmented pixel arrays. To generate complete connected components that are easy for subsequent rejection mechanisms to identify, the defect detection module activates the mathematical morphology processing engine to perform a closed-loop connectivity operation on the corresponding image pixel set. The processor first uses preset structuring elements to perform dilation operations on these discrete abnormal pixels, filling the tiny gaps between them and bridging them into a complete whole. Then, it performs an erosion operation of the same scale to smooth the boundaries and remove isolated noise points. After this rigorous morphological adjustment, the originally fragmented abnormal pixels are seamlessly stitched together. The system officially marks them as suspected defect areas and packages the key geometric parameters such as the centroid coordinates, area contour, etc. of the area, and sends them to the downstream verification and control module to provide a solid visual basis for the final rejection instruction generation.

[0055] This embodiment will explain in detail how the verification control module elevates isolated static spatial features to a dynamic spatiotemporal linkage system.

[0056] In real industrial production line environments, numerous random physical interferences exist that are difficult to completely shield. For example, tiny dust particles suspended in the air can momentarily obscure a camera lens; transient voltage surges from stroboscopic light sources can cause a sharp and abnormal increase in localized reflections on packaging; or momentary ghosting can be caused by vibrations in the mechanical production line. These accidental interferences can easily be disguised as genuine surface damage or missing printing ink within a single image frame. If the final decision is made based solely on the static characteristics of a single frame, the system will inevitably generate numerous false alarms, leading to the incorrect rejection of a large number of perfectly qualified food packages. To fundamentally overcome this highly challenging industry pain point, the verification control module is specifically configured to establish a time-series mechanism that integrates multi-frame continuous tracking verification with an absolute mapping to physical space.

[0057] For continuous frame tracking and stability identification of suspected defect areas, the processor of the verification control module first extracts the centroid coordinates and area contour features of the suspected defect areas. Specifically, the processor performs high-order image geometric moment operations on the irregular closed connected pixel set output by the preceding defect detection module. By calculating the zeroth-order spatial integral moment of this closed region, the processor obtains its precise absolute area enclosed by pixels; by calculating the first-order spatial integral moment and dividing it by the zeroth-order spatial moment, the system accurately locates the mass center position, i.e., the centroid coordinates, of the abnormal region at the sub-pixel level. At the same time, the processor extracts a series of curvature extrema points and edge gradient direction distributions along the pixel boundary of the abnormal region to construct a unique area contour feature. This feature combination is like a digital topological fingerprint specific to the suspected defect, completely covering its macroscopic spatial scale and microscopic edge morphology.

[0058] Between consecutively acquired image frames, the processor tracks suspected defective areas belonging to the same packaging based on extracted area contour features. Because the industrial camera is set to continuously trigger shooting at an extremely high frame rate, as the bottom conveyor belt advances at a constant speed, the same defective food packaging will leave multiple consecutive snapshots with physical displacement differences within the camera's field of view. The processor uses a dynamic sliding window block matching algorithm and topological feature similarity cross-validation technology to search for anomalous patches in the predicted region of the subsequent frame that highly match the area contour features of the previous frame. If the anomaly is caused by actual physical packaging damage or ink stains, it will inevitably adhere firmly to the surface of the packaging bag, undergoing rigid translational movement along with the bag itself. Its contour features will maintain extremely high topological inheritance and morphological consistency between adjacent frames. The system thus establishes a robust digital tracking anchor chain on the timeline, tightly linking and locking the same suspected target in different frames.

[0059] To accurately quantify the stability of this spatiotemporal tracking at the underlying mathematical logic and effectively filter environmental noise, the system creatively introduces a dynamic variance evaluation model with spatiotemporal motion compensation. The processor calculates the fluctuation variance of the centroid position coordinates of the tracked suspected defect area over a preset series of consecutive frames. Traditional simple variance calculations cannot separate the forward displacement of the conveyor belt itself, easily misjudging normal physical displacement as coordinate fluctuation. Therefore, this system has specifically designed the following proprietary dynamic fluctuation variance derivation formula: ,in, The comprehensive evaluation value of the dynamic fluctuation variance of the suspected defect in multiple consecutive frames of images is represented by the final calculation of the system. This represents the total number of frames that the system has pre-set for continuous image tracking and verification. This value directly determines the length of the time window for the system to observe and confirm defects. This represents the index number of the specific image frame sequence currently undergoing iterative calculation; Representing the current number The confidence weight coefficient for spatial centroid feature matching in frame images is dynamically assigned by the system based on the overall clarity and edge sharpness of the current frame. This represents the absolute coordinate residual displacement of the suspected defect centroid in the horizontal axis of the two-dimensional pixel coordinate system after the macroscopic motion displacement compensation and reverse subtraction of the conveyor belt. This represents the absolute coordinate residual displacement of the suspected defect centroid in the vertical direction of the two-dimensional pixel coordinate system after the macroscopic motion displacement compensation and reverse subtraction of the conveyor belt. This represents the system's preset proportional penalty adjustment constant for the variation range of the defect contour area, specifically used to balance the penalty weight of spatial position runout and morphological area runout in the total variance result; This represents the difference in the absolute area enclosed by the suspected defect contour between two adjacent image frames.

[0060] By executing this highly precise mathematical formula, the verification control module can accurately identify the true physical properties of the target. False defects caused by airborne dust or light reflections will still exhibit drastic and irregular spatial jumps in their centroid coordinates even after eliminating the overall rigid displacement of the packaging bag. Their area may even flicker or change abruptly between adjacent frames, causing a sharp increase in the dynamic fluctuation variance comprehensive evaluation value calculated using the formula. Conversely, when the calculated fluctuation variance is within the set stable error range (i.e., the calculated value is at an extremely low level), it indicates that the abnormal area is not only highly stable across frames in terms of shape and area, but its spatial micro-motion trajectory is also perfectly synchronized with the macroscopic physical displacement of the food packaging body at the pixel level. At this point, the processor determines that the tracking link meets the stability conditions at the underlying algorithm level and confirms that the suspected defect area is a real physical defect existing on the packaging surface, completely eliminating the interference of occasional environmental noise.

[0061] The processor must monitor the real-time dynamic operating status of the conveyor system and read the conveyor belt cycle parameters fed back from external sensors. Modern food processing lines typically have high-resolution photoelectric rotary encoders connected to the drive rollers. As the conveyor belt moves forward, the encoder sends dense pulse signals to the system. The underlying hardware interrupt service routine captures and accumulates these pulses at high frequency. By analyzing the pulse reception frequency and total number, it calculates the current instantaneous linear velocity and cumulative physical distance of the conveyor belt in real time. The system uses these operating parameters as an absolute benchmark for the packaging and conveying status, thereby completely eliminating time and space synchronization errors caused by fluctuations in traction motor speed or belt slippage.

[0062] The processor acquires the current spatial coordinates of the physical defect. Using the camera's intrinsic and extrinsic parameter matrix model, the system projects the defect's two-dimensional pixel coordinates into absolute physical millimeter-level coordinates in the real three-dimensional world through perspective transformation. Subsequently, the processor calculates the path difference between these coordinates and the hardware coordinates of the rejection station. A fixed mechanical distance is inevitably reserved between the visual inspection station and the physical rejection station. The processor extracts the absolute installation origin coordinates of the rejection station's actuator and performs a spatial vector subtraction operation with these coordinates to obtain the precise absolute length difference of the physical path that the defective package must traverse before reaching the rejection point.

[0063] After determining the remaining path length and real-time beat rate, the processor calculates the trigger delay time for the rejection action based on these two parameters. The processor proportionally converts the path difference into the target encoder pulse requirement and, combined with the real-time pulse receiving frequency, deduces the absolute time span required for the defective package to align with the rejection spray area. Because hardware beat tracking is used instead of simple software timing, the calculation accuracy of the trigger delay time can consistently reach sub-millisecond levels, ensuring that the actuator perfectly hits the high-speed moving defective target the moment it strikes, without affecting adjacent qualified packages.

[0064] The processor writes the calculated trigger delay time into the control protocol frame for hardware timing binding, synthesizes and sends the rejection command. The processor constructs control data packets according to the industrial Ethernet standard, encapsulating the delay time parameter and a specific hardware action identifier code in the core load area. Once the synthesized rejection command is sent to the external motion controller, the hardware system pushes it into a high-priority hardware interrupt timer to wait. When the countdown precisely reaches zero, the external hardware immediately executes the action, accurately removing the defective package from the logistics line, achieving seamless closed-loop control from visual perception to physical isolation.

[0065] Example 2:

[0066] Please see Figure 2 As shown, parts not described in detail in this embodiment are described in Embodiment 1. An industrial visual inspection method for food packaging is provided, including:

[0067] The system performs multi-angle photometric acquisition and imaging, along with high reflectivity suppression: It drives multiple high-brightness flash source arrays to sequentially illuminate the rapidly conveyed flexible food packaging from multiple angles, simultaneously triggering a high-speed industrial camera to capture a continuous image sequence. The processor compares the dynamic changes in reflectivity at different illumination angles pixel by pixel, identifying highlight areas where grayscale saturation is caused by specular reflection on the material surface. Subsequently, the system extracts the spatial continuity features of the surrounding unsaturated areas and, through texture extension, interpolation, and structural tensor guidance, performs pixel-level compensation for highlight blind spots, outputting a base image with uniform illumination, filtered reflectivity, and preserved underlying ink details.

[0068] The process involves performing surface normal derivation and 3D surface reconstruction based on photometric stereo vision: The processor analyzes the microscopic grayscale gradation patterns caused by the switching of light source directions in a continuous image sequence. Based on a diffuse reflection illumination model, it solves for the normal vector information of each pixel node on the packaging surface in 3D space. Next, the system performs a globally optimal 2D surface integral operation on the discrete normal vector field to reconstruct a 3D geometric space with continuous depth step changes, thereby generating 3D surface data representing the true bending, protrusion, and concave deformation of the packaging surface.

[0069] The system performs surface topology mapping and inverse physical flattening under deformation constraints: It analyzes the local principal curvature extrema of the 3D surface data, and by setting a curvature gradient change threshold, it peels away the macroscopic natural curvature, accurately identifying folds and wrinkles caused by mechanical compression. Using the wrinkle boundaries as geometric topological constraints, the system maps the base image as a 2D texture map onto the 3D surface. Based on the principles of geodesic distance energy conservation and deformation energy minimization, it performs inverse flattening simulation calculations on the 3D mesh model, causing adaptive stretching and contraction compensation of pixel coordinates, ultimately outputting a 2D image that eliminates wrinkle distortion and completely restores the pattern to a flat state.

[0070] The system performs high-dimensional manifold feature comparison and precise location of suspected defects: It extracts multi-scale edge gradient responses and detects key spatial corners on the flattened 2D image, reducing and fusing geometric micro-topological information to generate structural feature vectors representing local texture and structural density. Simultaneously, the system retrieves a standard feature manifold space trained offline on defect-free samples and calculates the geodesic distance between the structural feature vectors and the standard space. This non-linear comparison method accommodates legitimate physical tolerances. When the comparison difference exceeds a set deviation limit, a mathematical morphological closed-loop connectivity operation is performed to cluster and merge discrete abnormal pixels, formally marking them as suspected defect regions.

[0071] The system executes a spatiotemporal dynamic tracking-based defect verification and precise physical removal control process: To filter out false alarms caused by dust or vibration in a single frame, the system performs continuous frame tracking verification on suspected defect areas. The centroid coordinates and area contours of the abnormal area are extracted and dynamically matched between adjacent image frames. The system reads the production line's cycle time parameters, uses a spatiotemporal motion compensation algorithm to subtract the overall displacement of the conveyor belt, and analyzes the variance of the defect's microscopic coordinates. When the variance is extremely small and meets the stability condition, it is confirmed as a real physical defect. Finally, combining the defect's physical coordinates, production line speed, and removal station distance, a sub-millisecond trigger delay time is calculated, generating a removal command to drive the execution mechanism for precise removal.

[0072] It should be noted that all formulas in this manual are calculated by removing dimensions and taking their numerical values. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0073] The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for users of ordinary technical skills, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An industrial vision inspection system for food packaging, characterized in that, include: The reflection correction module is used to illuminate food packaging in the conveying state from multiple angles to obtain a continuous image sequence; it identifies saturated areas based on the changes in reflection intensity under different illumination angles, and uses the spatial continuity features of unsaturated areas for compensation to generate a base image. The three-dimensional restoration module is used to obtain the normal information of the packaging surface based on the grayscale changes of different illumination directions in the continuous image sequence, and to perform surface reconstruction to generate three-dimensional surface data characterizing the deformation of the packaging surface. The surface unfolding module is used to identify wrinkled regions based on the three-dimensional surface data, and use the wrinkled regions as constraints to map the base image onto the three-dimensional surface data and unfold it to generate a two-dimensional image with wrinkle distortion eliminated. The defect detection module is used to extract the structural features of the two-dimensional image and compare them with preset standard packaging features. When the difference exceeds a threshold, the suspected defect area is marked. The verification control module is used to continuously track the suspected defective area frame by frame. When the position change meets the stability condition, the defect is confirmed, and a rejection instruction is generated according to the packaging and conveying status.

2. The industrial visual inspection system for food packaging according to claim 1, characterized in that, The reflection correction module, when identifying saturated regions based on changes in reflection intensity at different illumination angles and compensating for unsaturated regions using spatial continuity features to generate a base image, specifically serves to: Extract the extreme values ​​of reflectance grayscale of each pixel in the continuous image sequence; when the extreme values ​​of reflectance grayscale reach a preset overexposure limit, the corresponding set of pixels is determined as a saturated region, and the set of pixels that do not reach the preset overexposure limit is determined as a non-saturated region. Extract the boundary pixels adjacent to the saturated region in the unsaturated region, and calculate the gray-level gradient distribution characteristics of the boundary pixels as the spatial continuity features; Based on the gray-level gradient distribution characteristics, bidirectional tensor interpolation is performed into the saturated region, and the interpolation compensation result is replaced in the region corresponding to the continuous image sequence to output the base image.

3. The industrial visual inspection system for food packaging according to claim 1, characterized in that, When the three-dimensional restoration module obtains the normal information of the packaging surface based on the grayscale changes of different illumination directions in the continuous image sequence, it is specifically used for: Retrieve the spatial light source direction coordinates corresponding to different illumination angles; Calculate the rate of change of reflected grayscale of the target pixel in the continuous image sequence under illumination by different spatial light source directional coordinates; Establish a set of reflectance equations relating the rate of change of reflectance grayscale to the directional coordinates of the spatial light source, and solve the set of reflectance equations to obtain the surface gradient vector of the target pixel. The surface gradient vector is spatially normalized to output the normal information of the packaging surface.

4. The industrial visual inspection system for food packaging according to claim 1, characterized in that, When performing surface reconstruction and generating three-dimensional surface data characterizing the deformation of the packaging surface, the three-dimensional restoration module is specifically used for: Construct a partial differential equation based on the divergence distribution of the normal information of the packaging surface; Pre-defined boundary smoothing constraints are introduced to correct the partial differential equation; The height and depth distribution characteristics in three-dimensional space are obtained by iteratively solving the modified partial differential equation using the least squares method. The height and depth distribution features are topologically stitched together with the corresponding spatial coordinate grid to generate the three-dimensional surface data.

5. The industrial visual inspection system for food packaging according to claim 1, characterized in that, When identifying wrinkle regions based on the three-dimensional surface data, the surface unfolding module is specifically used for: Calculate the principal curvature extrema of each coordinate node on the three-dimensional surface data; Calculate the gradient of principal curvature extrema between adjacent coordinate nodes; When the gradient of the extreme value of the principal curvature exceeds the preset deformation tolerance limit, it is determined that the coordinate nodes in the corresponding connected range have undergone abrupt folding, and the clustering range of the coordinate nodes that have undergone abrupt folding is marked as the folded region.

6. The industrial visual inspection system for food packaging according to claim 1, characterized in that, When the surface unfolding module maps the base image to the three-dimensional surface data using the wrinkled region as a constraint to generate a two-dimensional image with wrinkle distortion eliminated, it is specifically used for: The base image is used as a texture map to cover the three-dimensional surface data, and a mapping relationship between two-dimensional texture coordinates and three-dimensional spatial coordinates is established. Under the constraint of locking the boundary position of the folded region, the optimal geodesic distance of the three-dimensional surface data projected onto the target plane is calculated based on the energy minimization model; The two-dimensional texture coordinates are inversely stretched and compensated based on the optimal geodesic distance to output a flattened two-dimensional image.

7. The industrial visual inspection system for food packaging according to claim 1, characterized in that, The defect detection module, when extracting structural features from the two-dimensional image and comparing them with preset standard packaging features, marks suspected defect areas when the difference exceeds a threshold, specifically uses the following methods: Edge response extraction and feature corner detection are performed on the two-dimensional image to generate local topological contour vectors as structural features; Obtain the standard feature manifold space generated based on defect-free sample training as a preset standard packaging feature; Calculate the geodesic distance between the local topological contour vector and the standard feature manifold space in the corresponding coordinate dimensions; When the feature geodesic distance is greater than the set deviation limit, the corresponding image pixel set is closed and connected, and marked as a suspected defect area.

8. The industrial visual inspection system for food packaging according to claim 1, characterized in that, When the verification control module performs continuous frame tracking on the suspected defect area and confirms the defect when the position change meets the stability condition, it is specifically used for: Extract the centroid coordinates and area contour features of the suspected defect region; Between consecutively acquired adjacent image frames, the suspected defective areas belonging to the same package are matched and tracked based on the area contour features. Calculate the variance of the centroid position coordinates of the tracked suspected defect area within a preset series of consecutive frames; When the fluctuation variance is within the set stationary error range, it is determined that the stability condition is met, and the suspected defect area is confirmed to be a physical defect.

9. An industrial visual inspection system for food packaging according to claim 1, characterized in that, When the verification control module generates a rejection instruction based on the packaging conveying status, it is specifically used for: The conveyor belt running cycle parameters fed back by external sensors are read in real time, and the running cycle parameters are used as the packaging conveying status. Obtain the spatial physical coordinates of the current location of the identified physical defect, and calculate the difference in the running path between the spatial physical coordinates and the hardware coordinates of the workstation that performs the removal action; The trigger delay time for the elimination action is calculated based on the difference in the running path and the running beat parameters. The trigger delay time is written into the control protocol frame for hardware timing binding, and then synthesized into the culling instruction sent to the external hardware system.

10. An industrial visual inspection method for food packaging, implemented by an industrial visual inspection system for food packaging according to any one of claims 1 to 9, characterized in that, include: S1: Irradiate the food packaging in the conveying state from multiple angles to obtain a continuous image sequence; identify saturated areas based on the changes in reflection intensity under different irradiation angles, and use the spatial continuity features of unsaturated areas for compensation to generate a basic image; S2: Obtain the normal information of the packaging surface based on the grayscale changes of different illumination directions in the continuous image sequence, and perform surface reconstruction to generate three-dimensional surface data characterizing the deformation of the packaging surface; S3: Identify the wrinkled areas based on the three-dimensional surface data, and use the wrinkled areas as constraints to map the base image onto the three-dimensional surface data and then unfold it to generate a two-dimensional image with wrinkle distortion eliminated. S4: Extract the structural features of the two-dimensional image and compare them with preset standard packaging features. When the difference exceeds a threshold, mark the suspected defect area. S5: Perform continuous frame tracking on the suspected defective area. When the position change meets the stability condition, confirm the defect and generate a rejection instruction based on the packaging and conveying status.

Images