A method and system for visual inspection of food packaging

Abstract

The application provides a food packaging visual detection method and system, relates to the field of vision, and solves the technical problem of high false alarm rate of packaging defects caused by mirror glare interference in the existing food packaging scene. The method comprises the following steps: acquiring an orthogonal polarization image sequence and performing preprocessing to obtain a glare-removed image sequence; based on a physical optical reflection model and the glare-removed image sequence, reconstructing the three-dimensional topography of the surface of the food packaging to be detected and calculating the local curvature field of the surface of the food packaging to be detected; inputting the local curvature field and the glare-removed image into a multi-scale feature pyramid network respectively, identifying a suspected defect area, and based on a Bayesian inference framework and a pre-constructed prior defect distribution model, calculating the posterior defect probability of the suspected defect area; and judging the defect condition of the food packaging to be detected based on the posterior defect probability. The application is used in the process of food packaging defects.

Description

Technical Field

[0001] This application relates to the field of vision, and more particularly to a visual inspection method and system for food packaging. Background Technology

[0002] In automated food packaging production, vision inspection systems are commonly used to identify surface defects such as pinholes, scratches, or bulges. Existing methods mostly rely on texture analysis based on RGB images or combine structured light and lasers to obtain 3D information. However, when the food packaging being inspected contains highly reflective materials, such as aluminized film or transparent plastic film, the strong specular reflection (glare) generated on the surface can severely contaminate the image observation data. This causes discrimination models based on brightness or texture to misclassify normal reflective areas as defects, or fail to effectively extract the geometric and appearance features of true defects. Especially in high-speed online inspection scenarios, the lack of physical modeling and suppression mechanisms for glare interference results in existing systems generally exhibiting high false alarm rates and poor robustness under mixed material inspection conditions, making it difficult to meet the high-precision and high-reliability quality control requirements of food packaging.

[0003] Therefore, there is an urgent need for a visual inspection method that can synergistically integrate geometric shape and surface texture information and effectively suppress glare interference, so as to achieve high-precision, low-false-alarm identification of minute defects on the surface of food packaging of various materials. Summary of the Invention

[0004] This application provides a visual inspection method and system for food packaging, which solves the technical problem of high false alarm rate of packaging defects caused by mirror glare interference in existing food packaging scenarios.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, a visual inspection method for food packaging is provided, comprising: A multispectral ring light source is used to illuminate the food packaging under test in a time-division manner with multiple preset bands and multiple illumination angles. For each combination of band and illumination angle, two images with mutually orthogonal polarization directions are acquired to obtain an orthogonal polarization image sequence composed of multiple orthogonal polarization image pairs. The orthogonal polarization image sequence is subjected to adaptive polarization filtering to obtain a deglare image sequence; Based on the physical optics reflection model and the deglare image sequence, the three-dimensional morphology of the food packaging surface under test is reconstructed; based on the three-dimensional morphology, the local curvature field of the food packaging surface under test is calculated. The local curvature field and the deglare image are respectively input into a multi-scale feature pyramid network to identify suspected defect areas; The features of each suspected defect region are obtained, and the posterior defect probability of the suspected defect region is calculated based on the Bayesian inference framework and the pre-built prior defect distribution model. The defect status of the food packaging under test is judged based on the posterior defect probability.

[0006] Based on the above technical solutions, in the visual inspection method for food packaging provided in this application, the strong specular glare generated by the surface of highly reflective materials (such as aluminized film and transparent plastic) seriously interferes with traditional 2D image analysis, causing minor defects (such as pinholes and scratches) to be masked or misjudged. Methods relying solely on texture or geometric information are insufficient to balance accuracy and robustness in multi-material mixed inspection scenarios. Therefore, this solution proposes a detection framework that integrates physical and data-driven approaches: structured observation data is acquired through multispectral and multi-angle polarization imaging, and high-quality deglare images are generated based on material-adaptive polarization filtering; high-fidelity three-dimensional topography is reconstructed using a physical optics model, and the local curvature field sensitive to defects is extracted; the curvature field and the deglare image are input into a multi-scale network in a dual-modal manner to achieve joint perception of geometrical abrupt changes and texture distortion; a Bayesian inference mechanism is introduced to integrate historical defect priors with current observation features, probabilistically determining the authenticity of defects. This solution addresses the core challenges of high false alarm rates and weak generalization ability in defect detection under highly reflective packaging by suppressing glare at the source, fusing multi-dimensional features in the intermediate layer, and introducing interpretable reasoning in the decision layer. It significantly improves detection accuracy, robustness, and industrial applicability.

[0007] In conjunction with the first aspect above, in one possible implementation, the method for obtaining the orthogonal polarization image sequence includes: The multispectral ring light source is controlled to illuminate the food packaging under test at different times with multiple different wavelengths and multiple asymmetrically distributed illumination angles. For each combination of wavelength and illumination angle, during the duration of a single illumination, the polarization direction in the imaging optical path is switched by an electronically controlled polarization device, and two original images with mutually orthogonal polarization directions are acquired sequentially. All acquired raw images are time-aligned and parameter-correlated according to their corresponding bands, illumination angles, and polarization directions to form an orthogonal polarization image sequence composed of multiple orthogonal polarization image pairs.

[0008] In conjunction with the first aspect above, in one possible implementation, the method for acquiring the deglare image sequence includes: Identify the material type of the food packaging to be tested; obtain the corresponding glare suppression parameters based on the material type, and calculate the weighted difference coefficients of a pair of orthogonal polarization images; wherein, the pair of orthogonal polarization images includes a first orthogonal polarization image and a second orthogonal polarization image, the polarization axis of the first orthogonal polarization image is aligned with the principal direction of specular reflection to capture the maximum glare component, and the polarization axis of the second orthogonal polarization image is orthogonal to the principal direction of specular reflection to obtain diffuse reflection details; The weighted difference coefficients are used to linearly fuse the pair of orthogonal polarization images to generate a deglare image under the current lighting conditions; The anti-glare images generated under all lighting conditions are combined according to the acquisition time sequence to obtain the anti-glare image sequence.

[0009] In conjunction with the first aspect above, in one possible implementation, the method for identifying the material type of the food packaging to be tested includes: Physical features are extracted from the orthogonal polarization image sequence. These physical features include color and printing texture features in the visible light band, the transmission-to-reflection intensity ratio in the near-infrared band, and the response curves of polarization degree as a function of illumination angle at different incident angles. Based on the physical characteristics, a fusion representation vector is reconstructed and input into a lightweight classification model that embeds optical prior knowledge. The classification model identifies the material type of the food packaging to be tested. The reconstruction of the fusion representation vector involves normalizing, aligning, and splicing or weighting the multimodal physical features extracted from the orthogonal polarization image sequence to form a fixed-dimensional numerical vector. The material type includes one of the following: metal coating, transparent polymer film, or matte paper.

[0010] In conjunction with the first aspect above, in one possible implementation, calculating the weighted difference coefficients of the pair of orthogonal polarization images includes: Based on the identified material type, the corresponding polarization characteristic parameters are retrieved from the preset material-polarization response model library; Based on the incident angle and wavelength of the current illumination source, the energy proportion of the specular reflection component in the orthogonal polarization image pair is calculated using the polarization characteristic parameters; based on the energy proportion, a weighted difference coefficient is generated through a preset nonlinear mapping function.

[0011] In conjunction with the first aspect above, in one possible implementation, calculating the local curvature field of the surface of the food packaging to be tested includes: The three-dimensional morphology of the surface of the food packaging under test is obtained based on the physical optics reflection model and the reconstruction of the deglare image sequence. The three-dimensional morphology is represented in the form of a height map. The height map is subjected to noise suppression and edge-preserving smoothing processing. By performing second-order spatial differentiation on each pixel position, its local curvature properties are calculated. The local curvature properties include principal curvature direction and curvature intensity. Based on the local curvature properties, a curvature field representing the local geometry of the surface is generated.

[0012] In conjunction with the first aspect above, in one possible implementation, the method for reconstructing the three-dimensional morphology of the surface of the food packaging to be tested includes: The illumination angle and wavelength of the light source corresponding to each image in the anti-glare image sequence are obtained, and a physical mapping relationship between the pixel brightness at the same pixel coordinate position in each image and the normal vector of the packaging surface corresponding to that position is established by combining the physical optical reflection model. Based on the physical mapping relationship, the optimal surface normal vector corresponding to each pixel coordinate position is solved; The normal vector field formed by the optimal surface normal vectors at all pixel coordinate positions is globally integrated, and the continuity constraint of the packaging surface is introduced to eliminate local ambiguity, thus obtaining the three-dimensional morphology of the surface of the food packaging to be tested.

[0013] In conjunction with the first aspect above, in one possible implementation, the method for identifying the suspected defective region includes: The local curvature field and the deglare image are respectively input into different branches of a multi-scale feature pyramid network; wherein, the branches include a first branch and a second branch, the first branch is used to extract features representing apparent geometric anomalies, and the second branch is used to extract features representing apparent texture anomalies. At each level of the multi-scale feature pyramid network, cross-modal correlation between geometric features and texture features is calculated through a fusion attention mechanism, and the two types of features are weighted to enhance or suppress them. Based on the weighted multi-scale features, a spatial attention heatmap is generated, and connected regions whose response values ​​exceed a preset threshold are marked as suspected defect regions; wherein, the response value of the attention heatmap represents the possibility of a joint geometric and texture anomaly at the corresponding location.

[0014] In conjunction with the first aspect above, in one possible implementation, calculating the posterior defect probability of the suspected defect region includes: The geometric and textural observation features of each suspected defect region are obtained. Based on a pre-constructed prior defect distribution model and a Bayesian inference framework, the posterior probability that each suspected defect region is a real defect is calculated. The output of the prior defect distribution model represents the frequency of occurrence of different defect types in historical production data and their typical feature distribution. If the posterior probability is greater than a preset threshold, the suspected defect region is a real defect and a graded alarm is triggered. If the posterior probability is less than or equal to the preset threshold, the suspected defect region is a false defect.

[0015] Secondly, this application provides a visual inspection system for food packaging, comprising: a defect identification module, and an acquisition module and an analysis module connected thereto; wherein, the acquisition module is used to illuminate the food packaging under test with a multispectral ring light source at multiple preset bands and multiple illumination angles in a time-division manner, and for each combination of band and illumination angle, acquire two images with mutually orthogonal polarization directions to obtain an orthogonal polarization image sequence composed of multiple orthogonal polarization image pairs; the analysis module is used to perform adaptive polarization filtering on the orthogonal polarization image sequence to obtain a deglare image sequence; reconstruct the three-dimensional morphology of the surface of the food packaging under test based on the physical optics reflection model and the deglare image sequence; calculate the local curvature field of the surface of the food packaging under test based on the three-dimensional morphology; the defect identification module is used to input the local curvature field and the deglare image into a multi-scale feature pyramid network to identify suspected defect areas; acquire the features of each suspected defect area, and calculate the posterior defect probability of the suspected defect area based on a Bayesian inference framework and a pre-constructed prior defect distribution model; and determine the defect status of the food packaging under test based on the posterior defect probability.

[0016] This application provides a visual inspection method and system for food packaging, enabling high-precision, low-false-alarm, and interpretable intelligent identification of minute defects in complex industrial scenarios involving high reflectivity and mixed material inspection. The method utilizes a multispectral ring light source and polarization imaging to acquire a structured orthogonal polarization image sequence. It then adaptively performs deglare processing based on material type, effectively suppressing specular reflection interference caused by metal coatings or transparent films. Furthermore, it reconstructs a high-fidelity three-dimensional morphology using a physical optics reflection model and calculates a local curvature field sensitive to geometric anomalies, thereby capturing three-dimensional defects without significant texture changes, such as pinholes, scratches, and bulges. The curvature field and deglare-free images are further input into a multi-scale feature pyramid network, fusing geometric and texture features through a cross-modal attention mechanism to accurately locate suspected defect areas. Finally, a Bayesian inference framework is introduced, fusing prior knowledge of historical defect distributions with current observation features to probabilistically determine defect authenticity, avoiding misclassification of normal creases, printing gradients, and other process features as defects. The entire solution connects the entire chain of "physical perception - geometric modeling - intelligent discrimination", combining the advantages of physical interpretability and data-driven approach. It not only significantly improves detection accuracy and robustness, but also supports defect classification alarms and quality traceability, meeting the stringent requirements of high-speed food packaging production lines for high reliability, strong generalization ability and intelligent decision-making.

[0017] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0018] Figure 1 A system architecture diagram of a visual inspection system for food packaging provided in this application embodiment; Figure 2 A schematic flowchart illustrating a visual inspection method for food packaging provided in an embodiment of this application; Figure 3 A schematic flowchart of an image deglare removal light processing method provided in an embodiment of this application; Figure 4This is a flowchart illustrating a method for obtaining the local curvature field of a food packaging surface, as provided in an embodiment of this application. Detailed Implementation

[0019] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The visual inspection method for food packaging provided in this application embodiment can be applied to a visual inspection system for food packaging, such as... Figure 1 As shown, the system includes: an acquisition module, a defect identification module, and an analysis module; The acquisition module is used to illuminate the food packaging under test with a multispectral ring light source at multiple preset bands and multiple illumination angles in a time-division manner. For each combination of band and illumination angle, two images with mutually orthogonal polarization directions are acquired to obtain an orthogonal polarization image sequence composed of multiple orthogonal polarization image pairs. The analysis module is used to perform adaptive polarization filtering on the orthogonal polarization image sequence to obtain a deglare image sequence; based on the physical optics reflection model and the deglare image sequence, the three-dimensional morphology of the surface of the food packaging under test is reconstructed; based on the three-dimensional morphology, the local curvature field of the surface of the food packaging under test is calculated. The defect identification module is used to input the local curvature field and the deglare image into the multi-scale feature pyramid network to identify suspected defect areas; obtain the features of each suspected defect area, and calculate the posterior defect probability of the suspected defect area based on the Bayesian inference framework and the pre-built prior defect distribution model; and judge the defect status of the food packaging under test based on the posterior defect probability.

[0021] To address the high false alarm rate of packaging defects caused by specular glare interference in existing food packaging scenarios, this application provides a visual inspection method for food packaging. The method includes: using a multispectral ring light source to illuminate the food packaging under test at multiple preset wavelengths and angles; acquiring two mutually orthogonal polarization images for each combination of wavelength and angle to obtain an orthogonal polarization image sequence composed of multiple orthogonal polarization image pairs; performing adaptive polarization filtering on the orthogonal polarization image sequence to obtain a deglare image sequence; reconstructing the three-dimensional morphology of the food packaging surface based on a physical optics reflection model and the deglare image sequence; calculating the local curvature field of the food packaging surface based on the three-dimensional morphology; and inputting the local curvature field and the deglare image into a multi-scale feature pyramid network to identify suspected defect areas. The application acquires features of each suspected defect region and calculates the posterior defect probability of the suspected defect region based on a Bayesian inference framework and a pre-built prior defect distribution model. Based on the posterior defect probability, the application determines the defect status of the food packaging under test. To this end, the application obtains structured observation data through multispectral and multi-angle polarization imaging, and effectively suppresses high reflectivity interference by combining material-adaptive polarization filtering, providing high-quality input for subsequent processing. A high-fidelity three-dimensional morphology is reconstructed using a physical optics model, and a local curvature field sensitive to minute defects is extracted, compensating for the blind spot of traditional 2D detection for textureless abnormal defects. Furthermore, the multi-scale features of the curvature field and the deglare image are integrated to achieve collaborative recognition of geometric and texture anomalies. Finally, a Bayesian inference mechanism is introduced to combine historical defect statistical priors with current observation features, quantifying the authenticity of defects in an interpretable probabilistic form. This technical approach overcomes the bottlenecks of existing methods, such as high false alarm rate and weak generalization ability in high reflectivity and multi-material scenarios. It not only improves the detection accuracy of small defects such as pinholes, scratches, and bulges, but also supports hierarchical alarms and quality traceability through posterior probability, significantly enhancing the reliability, robustness and intelligence of the system in high-speed industrial production lines.

[0022] like Figure 2 As shown in the embodiment of this application, a visual inspection method for food packaging includes: S201. A multispectral ring light source is used to illuminate the food packaging under test in multiple preset bands and multiple illumination angles in a time-division manner. For each combination of band and illumination angle, two images with mutually orthogonal polarization directions are acquired to obtain an orthogonal polarization image sequence composed of multiple orthogonal polarization image pairs.

[0023] Among them, an orthogonal polarization image pair refers to two images acquired by an electronically controlled polarizer at the front end of an imaging system under the same illumination conditions, i.e., the same light source band and the same illumination angle, with their polarization directions differing by 90°. For example, the polarization axis of one image is set to 0° and the other is set to 90°, so as to capture the dominant component of specular reflection and the dominant component of diffuse reflection, respectively.

[0024] It should be noted that the orthogonal polarization image sequence is not the final image used for defect detection, but rather serves as raw observation data for subsequent generation of deglare images, estimation of surface polarization characteristics, or identification of packaging material type. Its core value lies in preserving complete information on the interaction between light and packaging surface under different polarization states, providing a physical basis for suppressing specular glare caused by metal coatings or plastic films.

[0025] S202. Perform adaptive polarization filtering on the orthogonal polarization image sequence to obtain a deglare image sequence.

[0026] The adaptive polarization filtering process includes: for each orthogonal polarization image pair, dynamically determining a fusion strategy to suppress specular reflection based on the current illumination band, illumination angle, and material type of the food packaging to be tested.

[0027] S203. Based on the physical optics reflection model and the deglare image sequence, reconstruct the three-dimensional morphology of the surface of the food packaging to be tested; based on the three-dimensional morphology, calculate the local curvature field of the surface of the food packaging to be tested.

[0028] The local curvature field is obtained by performing a second-order spatial differential operation on the height map. It is used to quantify the local curvature of the surface geometry. Significant abrupt changes in curvature values ​​correspond to potential pinholes, scratches, or bulges.

[0029] It should be noted that glare-reducing image sequences are a key prerequisite for achieving high-precision 3D reconstruction. If the original image containing specular reflection is used directly, its brightness will deviate significantly from the assumptions of the physical optics model, leading to errors in the normal vector calculation. This solution effectively suppresses glare interference caused by metal coatings or plastic films through pre-processing adaptive polarization filtering, enabling the reconstruction results to truly reflect the micron-level geometry of the packaging surface, thus providing a reliable basis for subsequent defect detection based on curvature fields.

[0030] S204. Input the local curvature field and the deglare image into the multi-scale feature pyramid network to identify suspected defect areas.

[0031] The aforementioned defective areas will be analyzed and compensated for. The multi-scale feature pyramid network comprises two parallel branches: the first branch receives a local curvature field as input to extract features characterizing surface geometric anomalies, including curvature extrema, edge discontinuities, and local convex-concave variations; the second branch receives a deglare image as input to extract features characterizing apparent texture anomalies, including local contrast abrupt changes, frequency domain energy disturbances, and printing pattern distortion. At each pyramid level, a fusion attention mechanism is used to perform cross-modal interaction on the features from the two branches, dynamically enhancing the response of regions consistent with geometric and texture anomalies and suppressing interference from single-modal anomalies.

[0032] It should be noted that using deglare images alone can easily misjudge normal printing textures or lighting gradients as defects, while relying solely on local curvature fields may miss stains or color difference defects without significant geometric deformation. This solution significantly improves the detection rate and discrimination accuracy of complex micro-defects by jointly analyzing the abnormal consistency between geometric shape and apparent texture, such as pinholes accompanied by ink diffusion and micro-scratches accompanied by reflective changes. It effectively solves the technical problem of high false alarm rate and high missed detection rate of existing visual inspection methods in high reflectivity and multi-material food packaging scenarios.

[0033] S205. Obtain the features of each suspected defect area, and calculate the posterior defect probability of the suspected defect area based on the Bayesian inference framework and the pre-built prior defect distribution model; judge the defect status of the food packaging to be tested based on the posterior defect probability.

[0034] Among them, the features of suspected defect areas include geometric features and texture features. Geometric features are derived from the local curvature field, including curvature extrema, principal direction change rate and surface continuity index. Texture features are derived from the deglare image, including local contrast, gray-level co-occurrence matrix energy and frequency domain wavelet coefficients. The prior defect distribution model is a probabilistic model constructed based on historical production data statistics, which characterizes the frequency of occurrence and typical feature distribution of different defect types (such as pinholes, scratches and printing misalignments) on various packaging materials.

[0035] It should be noted that traditional detection methods typically rely on fixed thresholds or single features for binary discrimination, making it difficult to distinguish between genuine defects and normal process variations, such as packaging creases or ink gradations. In contrast, this solution introduces a Bayesian posterior probability mechanism, organically combining historical statistical priors with current multidimensional observation features to achieve a probabilistic and reliable assessment of the authenticity of defects. This not only significantly reduces the false alarm rate but also enables the quantitative grading of defect severity based on posterior probability values, providing a more reliable and interpretable decision-making basis for food packaging quality control.

[0036] Based on the above technical solutions, this application provides a visual inspection method for food packaging. In the visual inspection of food packaging, the specular glare on the surface of highly reflective materials can easily mask real defects or cause false alarms. Existing methods often rely only on texture or a single geometric modality, making it difficult to balance accuracy and stability in multi-material mixed inspection scenarios. To address this issue, this solution constructs a complete technology chain from perception to decision-making: it acquires raw data rich in physical information through multispectral and multi-angle polarization imaging, and adaptively filters out glare based on material properties to obtain high-quality deglare images; on this basis, it reconstructs accurate three-dimensional morphology by combining a physical optical reflection model and calculates the local curvature field sensitive to minute defects; further, it inputs the curvature field and the deglare image into a multi-scale feature pyramid network to achieve collaborative recognition of geometric anomalies and texture anomalies; finally, it introduces a Bayesian inference framework to fuse prior defect distribution with current observation features, quantifying the authenticity of defects in a probabilistic manner. This solution effectively overcomes the core bottleneck of low reliability in defect detection under high reflectivity by suppressing interference through physical modeling, deep fusion of multimodal features, and improving the reliability of discrimination through statistical inference. It significantly enhances the accuracy, robustness and generalization ability of the system in complex industrial scenarios.

[0037] In one possible implementation of this application embodiment, the above-mentioned S201 can be specifically implemented by the following S301, S302 and S303, which are described in detail below: S301. Control the multispectral ring light source to illuminate the food packaging under test at different times with multiple different wavelengths and multiple asymmetrically distributed illumination angles.

[0038] S302. For each combination of wavelength and illumination angle, during the duration of a single illumination, the polarization direction in the imaging optical path is switched by an electronically controlled polarization device, and two original images with mutually orthogonal polarization directions are acquired sequentially.

[0039] In some implementations, the spectral ring light source includes the ultraviolet band (365nm), the visible light band (450-650nm), and the near-infrared band (850nm); the illumination angle is switched sequentially at 15°, 25°, 35°, 45°, 55°, and 65°; for each (band, angle) combination, the industrial camera synchronously triggers two exposures, and the polarization direction is quickly switched with the help of the liquid crystal polarizer, with the polarization directions differing by 90°, to complete the acquisition of a pair of orthogonal polarized images.

[0040] S303. All acquired raw images are time-series aligned and parameter-correlated according to their corresponding bands, illumination angles, and polarization directions to form an orthogonal polarization image sequence composed of multiple orthogonal polarization image pairs.

[0041] Among them, the multispectral bands include the visible light band and the near-infrared band, which are used to capture the reflective texture information and penetrating optical response of the packaging surface, respectively. The asymmetrically distributed illumination angle is used to enhance the sensitivity to geometric changes of the packaging surface and avoid concentrated interference from the specular reflection direction.

[0042] In some implementations, timing alignment is achieved through hardware synchronization signals: when switching bands or illumination angles, the light source controller sends trigger pulses to the camera and electronically controlled polarization device to ensure that the image acquisition of each illumination-polarization combination is strictly performed according to the preset timing sequence; parameter association is accomplished by embedding the corresponding band identifier, light source angle code, and polarization direction label in the image metadata, so that the subsequent processing module can accurately identify the acquisition conditions of each image; furthermore, the system automatically pairs two orthogonally polarized images acquired under the same band and illumination angle, and organizes them into a structured orthogonally polarized image sequence according to the brightness order, for use by the glare removal processing, material recognition, or 3D reconstruction modules.

[0043] Based on the above technical solutions, in the visual inspection of food packaging, the packaging materials are diverse, and their surface optical properties vary significantly. Traditional imaging methods struggle to simultaneously adapt to areas with high reflectivity and weak texture, leading to unreliable defect feature extraction. Especially in high-speed production line environments, if polarization image acquisition lacks a structured design, subsequent glare removal and 3D reconstruction can fail due to lighting redundancy, temporal misalignment, or information mismatch. To address this, this solution proposes a structured polarization imaging method oriented towards physical perception: multispectral and asymmetric multi-angle illumination is used to excite differentiated optical responses in different materials; the polarization direction is rapidly switched within a single illumination cycle to ensure strict spatiotemporal alignment of orthogonal images, avoiding motion blur and illumination fluctuation interference; and a highly ordered orthogonal polarization image sequence is constructed through precise correlation of bands, angles, and polarization parameters. This design not only provides high-quality input for subsequent adaptive glare removal but also lays the data foundation for physical model-based 3D reconstruction and material recognition. Its core advantage lies in the deep integration of lighting strategy, polarization control and timing synchronization, realizing the transformation from "passive acquisition" to "active sensing". While ensuring industrial real-time performance, it significantly improves the reliability and generalization ability of high reflectivity and multi-material food packaging detection.

[0044] In one possible implementation of the embodiments of this application, such as Figure 3 As shown, the above S202 can be specifically implemented through the following S401, S402 and S403, which are explained in detail below: S401. Identify the material type of the food packaging to be tested; Specifically, S4011, extract physical features from the orthogonal polarization image sequence; wherein, the physical features include color and printing texture features in the visible light band, the transmission and reflection intensity ratio in the near-infrared band, and the response curve of polarization degree as a function of illumination angle under different incident angles.

[0045] S4012. Based on physical features, reconstruct and fuse the representation vector, and input it into a lightweight classification model that embeds optical prior knowledge. The classification model identifies the material type of the food packaging to be tested. The material type includes one of the following: metal coating, transparent polymer film, or matte paper.

[0046] Specifically, the reconstruction and fusion representation vector involves: statistically analyzing the channel mean and variance of color features in the visible light band; extracting energy and contrast indices from printed texture features using local binary mode or gray-level co-occurrence matrix; calculating the average transmission response of the near-infrared band at multiple illumination angles based on the transmission-reflection intensity ratio; extracting slope, extreme points, or curvature parameters from the response curves of polarization degree changing with illumination angle at different incident angles through linear or piecewise fitting; normalizing each of the above sub-features and concatenating them into a single high-dimensional vector in a preset order, which is the fusion representation vector.

[0047] For example, color features are represented by a 3D mean vector (R, G, B), texture features by a 4D gray-level co-occurrence matrix statistic, near-infrared transmittance-reflectance ratio by a scalar, and polarization degree-angle response curve by two fitted parameters (such as slope and intercept) before being concatenated to obtain a 10-dimensional fused representation vector. This vector comprehensively reflects the optical properties of the packaging material and serves as the data foundation for classification decisions.

[0048] Prior knowledge of optics includes: metal-coated materials exhibit significantly increased polarization and strong near-infrared reflection upon grazing incidence; transparent polymer films exhibit high transmission and low scattering characteristics in the near-infrared; and matte paper materials exhibit low polarization and diffuse reflection across all wavelengths.

[0049] Based on the above technical solutions, in visual inspection of food packaging, different materials exhibit significant differences in light reflection, transmission, and polarization responses. Traditional material identification methods often rely on single color or texture features, making it difficult to distinguish between materials that appear similar but have different physical properties (such as silver printed paper and aluminum foil), leading to the failure of subsequent glare removal, 3D reconstruction, and defect discrimination strategies. To address this, this solution proposes a material identification method based on multimodal physical feature fusion: Visible light color and texture, near-infrared transmittance and reflectance ratio, and polarization degree-angle response curves are systematically extracted from orthogonal polarization image sequences. A fusion representation vector is constructed using a structured approach, and each sub-feature is statistically quantified, normalized, and sequentially concatenated to form a highly discriminative input. This vector is then input into a lightweight classification model embedded with optical prior knowledge, ensuring that the decision logic conforms to the inherent optical laws of materials, such as the increased polarization degree of metals under grazing incidence and the high transmittance of transparent films in the near-infrared region. This approach abandons the reliance of black-box deep learning on large amounts of labeled data, and instead achieves accurate material classification in a physically interpretable, computationally efficient, and highly generalizable manner. This provides key priors for subsequent adaptive visual processing and fundamentally improves the robustness and intelligence of the entire detection system in multi-material mixed detection scenarios.

[0050] S402. Obtain the corresponding glare suppression parameters based on the material type, and calculate the weighted difference coefficients of a pair of orthogonal polarization images.

[0051] The pair of orthogonal polarization images includes a first orthogonal polarization image and a second orthogonal polarization image. The polarization axis of the first orthogonal polarization image is aligned with the principal direction of specular reflection to capture the maximum glare component, and the polarization axis of the second orthogonal polarization image is orthogonal to the principal direction of specular reflection to obtain diffuse reflection details.

[0052] Specifically, S4021, based on the identified material type, retrieve the corresponding polarization characteristic parameters from the preset material-polarization response model library; Among them, the polarization characteristic parameters include the Fresnel reflectance range and the sensitivity of the main polarization direction of the material under the current illumination band.

[0053] The material-polarization response model library stores the typical polarization response laws of different packaging materials under multispectral and multi-angle illumination conditions. Each type of material corresponds to a set of polarization characteristic parameters, including the Fresnel reflectance range of the material in the visible light band, the transmission-scattering ratio in the near-infrared band, and the sensitivity curve of polarization degree as a function of incident angle. When the material identification module outputs that the packaging to be tested belongs to one of the following types: metal coating, transparent polymer film, or matte paper, the system retrieves and loads the polarization characteristic parameters that match that type from the model library for subsequent adaptive polarization filtering or glare suppression processing.

[0054] It should be noted that the material-polarization response model library is not a general optical database, but a set of dedicated models calibrated based on a large number of food packaging samples under actual production line lighting conditions. Its parameters have been simplified and quantified for high-speed visual inspection scenarios, which can significantly reduce real-time computing overhead while ensuring accuracy. By dynamically associating the material recognition results with the pre-stored physical response models, the system can avoid repeatedly performing complex polarization physical inversions on each frame of image, thereby meeting the real-time requirements of industrial online inspection while maintaining high deglare performance.

[0055] S4022. Based on the incident angle and wavelength of the current illumination source, estimate the energy proportion of the specular reflection component in the orthogonal polarization image pair using polarization characteristic parameters; based on the energy proportion, generate weighted difference coefficients through a preset nonlinear mapping function, such that when the energy proportion is lower than the first threshold, the coefficients approach zero, and when the energy proportion is higher than the second threshold, the coefficients approach saturation.

[0056] Among them, the energy ratio refers to the proportion of specular reflection component in the total reflected light. Its calculation process combines the incident angle and wavelength of the current illumination with polarization characteristic parameters (including Fresnel reflectivity and polarization sensitivity) retrieved from the material-polarization response model library. The theoretical brightness ratio of the two images in the orthogonal polarization image pair is calculated through the physical optics model, and the relative intensity of specular reflection energy is inferred from this. The nonlinear mapping function is designed as an S-shaped or piecewise smooth function to dynamically convert the energy ratio into weighted difference coefficients to achieve a linear fusion strategy of "low glare weak suppression and high glare strong suppression".

[0057] It should be noted that if a fixed difference coefficient is used directly, it will lead to excessive suppression in low reflective areas (such as matte paper) and loss of printing texture, while insufficient suppression in high reflective areas (such as aluminum foil) will result in residual glare. However, this solution uses an energy-ratio driven nonlinear mapping mechanism to accurately match the weighted difference coefficient with the actual glare intensity, which not only preserves surface details but also effectively eliminates specular interference, significantly improving the quality of the deglare image and the reliability of subsequent defect detection.

[0058] In some implementations, the nonlinear mapping function is represented as: ; in, These are the weighted difference coefficients. For the estimated energy percentage, This is the saturation value. Midpoint threshold, Gain parameters used to control the steepness of the curve; the first threshold and the second threshold correspond to... and The system automatically adjusts based on the type of packaging material. and The value can be set to a higher value, for example, for metal-plated materials. For matte paper products, a lower setting is used. .

[0059] Based on the above technical solutions, in visual inspection of food packaging, different materials exhibit significant differences in their polarization response characteristics to light. Traditional glare removal methods typically employ fixed or empirical differential coefficients, failing to dynamically adjust the suppression intensity according to the actual material and lighting conditions. This results in residual glare in high-reflectivity areas and overly smoothed areas in low-reflectivity areas, severely interfering with subsequent defect identification. To address this issue, this solution introduces a physical prior-based adaptive weighting mechanism: based on the identified material type, corresponding polarization characteristic parameters are retrieved from a pre-defined material-polarization response model library; combined with the incident angle and wavelength of the current illumination, these parameters are used to quantitatively estimate the energy proportion of specular reflection in orthogonal polarization image pairs; finally, through a pre-defined nonlinear mapping function, this energy proportion is converted into weighted differential coefficients. When glare is weak, the coefficients approach zero to preserve texture; when glare is strong, the coefficients saturate to fully suppress it. This method deeply integrates material optical priors, real-time lighting conditions, and physical modeling, giving the glare removal process a clear physical basis and adaptive capability, avoiding the limitations of a "one-size-fits-all" strategy. The resulting differential coefficients can accurately match the actual reflection behavior of different packaging surfaces, effectively eliminating glare while preserving the true surface details to the maximum extent. This lays a reliable foundation for high-precision 3D reconstruction and defect detection, and significantly improves the robustness and practicality of the system in industrial scenarios with multiple materials and working conditions.

[0060] S403. Linearly fuse a pair of orthogonal polarization images using weighted difference coefficients to generate a deglare image under the current lighting conditions; combine the deglare images generated under all lighting conditions according to the acquisition time sequence to obtain a deglare image sequence.

[0061] Among them, linear fusion is to use weighted difference coefficients to linearly fuse a pair of orthogonally transformed polarization images from the deglare image under the current lighting conditions to generate a deglare image under the current lighting conditions; the process of identifying and adaptively analyzing the deglare image sequence is carried out through the acquisition of deglare images. Linear fusion refers to the fusion of a pair of orthogonally polarized images acquired under the same lighting conditions. and Based on the aforementioned weighted difference coefficients The deglare image is calculated using a linear combination method. Or in an equivalent form, the specular reflection component is effectively suppressed, while the surface texture dominated by diffuse reflection is preserved; the system arranges the deglare images corresponding to each illumination condition (i.e. each combination of band and illumination angle) in chronological order of acquisition time to form a structured deglare image sequence for subsequent three-dimensional morphology reconstruction or defect identification.

[0062] In some implementations, linear fusion employs a pixel-level weighted differential strategy: radiometric consistency correction is performed on orthogonal polarization image pairs to eliminate brightness deviations caused by camera response nonlinearity; then, the local material properties are dynamically adjusted. Values, such as using high values ​​in metallic areas. Use low density in printed graphic areas The output glare-free image not only has no global glare, but also has high fidelity in local details, making it particularly suitable for visual inspection of micron-level defects.

[0063] It should be noted that traditional polarization deglare removal methods typically use a fixed difference coefficient or a simple selection of... As an output, it cannot meet the suppression intensity requirements under different materials and lighting conditions, which can easily lead to residual glare in high reflective areas or texture distortion in low reflective areas. However, this solution achieves an adaptive effect of "on-demand suppression" by driving the fusion process with weighted difference coefficients based on physical estimation, which significantly improves the versatility and robustness of deglare images in multi-material mixed inspection scenarios.

[0064] Based on the above technical solutions, in visual inspection of food packaging, specular glare from highly reflective materials can severely contaminate image information, making it difficult for traditional glare removal methods to take into account the optical characteristics of different materials: fixed difference coefficients tend to over-suppress texture in low-reflectivity areas and under-suppress in high-reflectivity areas, resulting in missed defects or false alarms. To address this, this solution proposes a material-driven adaptive polarization glare removal mechanism: identifying the packaging material type, retrieving matching glare suppression parameters, and dynamically calculating weighted difference coefficients; these coefficients accurately reflect the energy proportion of specular reflection under specific lighting conditions and are used to linearly fuse a pair of orthogonally polarized images (capturing maximum glare and diffuse reflection details respectively) to achieve "on-demand suppression." Since the polarization axis of the first image is aligned with the main specular reflection direction to capture strong glare, and the second image is orthogonal to preserve the true surface texture, their fusion effectively separates and weakens glare components while maximizing the preservation of printing details and geometric features. By combining the glare removal results under all lighting conditions in a temporal sequence, a high-quality, structured glare removal image sequence is formed, providing reliable input for subsequent 3D reconstruction and defect identification.

[0065] In one possible implementation of the embodiments of this application, such as Figure 4As shown, the above S203 can be implemented through the following S501, S502 and S503, which are explained in detail below: S501. Obtain the three-dimensional morphology of the food packaging surface under test, which is reconstructed based on the physical optical reflection model and the deglare image sequence. The three-dimensional morphology is represented in the form of a height map. Specifically, S5011, obtain the light source illumination angle and band corresponding to each image in the anti-glare image sequence, and combine the physical optical reflection model to establish the physical mapping relationship between the pixel brightness at the same pixel coordinate position in each image and the packaging surface normal vector corresponding to that position.

[0066] The physical mapping relationship refers to the fact that for any fixed pixel coordinate position in an image, the brightness value presented under different lighting conditions (i.e., different light source illumination angles and bands) is jointly determined by the normal vector of the micro-surface element of the packaging surface corresponding to that position, the direction of the light source, and the optical reflection characteristics of the material. By introducing the Lambert-non-Lambert hybrid reflection model as the physical optical reflection model, the observed brightness is modeled as a function of the surface normal vector, thereby establishing the reversible mapping foundation of "multi-view brightness response → surface orientation", providing a theoretical basis for subsequent normal vector calculation.

[0067] In some implementations, the physical optical reflection model dynamically selects a sub-model based on the currently identified packaging material type: when the material is a metal coating, the weight of the non-Lambertian components is increased to reflect its strong directional reflection; when the material is a matte paper, the Lambertian model is mainly used; when the material is a transparent polymer film, a transmission-scattering coupling term is introduced to compensate for the subsurface scattering effect. Through this material-adaptive model configuration, the physical mapping relationship is made to better fit the optical behavior of the actual packaging surface, further improving the robustness of 3D reconstruction.

[0068] It should be noted that if the original image containing specular reflection is used directly to construct the mapping relationship, the brightness of the highlight area deviates significantly from the diffuse reflection assumption, which will lead to a systematic bias in the normal vector estimation. However, this scheme is based on the modeling of the deglare image sequence after removing glare interference, ensuring that the brightness response at each viewpoint is mainly dominated by diffuse reflection, which significantly improves the accuracy of the physical mapping relationship and the reliability of the reconstruction results. This is a key prerequisite for achieving high-precision three-dimensional topography restoration.

[0069] S5012. Based on the physical mapping relationship, solve for the optimal surface normal vector corresponding to each pixel coordinate position, so that the residual between the brightness predicted by the normal vector and the actual observed brightness is minimized under all lighting conditions.

[0070] The solution process is performed independently for each fixed pixel coordinate position in the image: taking all observed brightness values ​​corresponding to that position in the deglare image sequence as input, and combining the known light source direction and band information under various lighting conditions, an optimization problem about the surface normal vector is constructed using physical mapping relationships; through iterative search or analytical methods, the normal vector that minimizes the overall difference between the predicted brightness and the actual observed brightness is found on a unit sphere, and this normal vector is the optimal surface normal vector at that pixel position, representing the local orientation of the packaging surface at that point.

[0071] In some implementations, regularization constraints are introduced into the solution process to improve robustness: for example, a neighborhood smoothing term is added to the optimization objective to encourage the normal vector directions of adjacent pixels to be similar, thereby suppressing local oscillations caused by noise; at the same time, the optimization weights are dynamically adjusted according to the currently identified material type, such as giving higher weight to large-angle lighting data for metal-coated materials and relying more on small-angle data for paper materials; the output surface normal vector field is both faithful to the observation data and meets the physical continuity requirements of the packaging surface, laying the foundation for subsequent global integration to generate three-dimensional morphology.

[0072] It should be noted that since a single frame image cannot uniquely determine the surface normal vector, a reliable solution must be achieved by relying on brightness changes under multi-view illumination. However, this scheme provides rich observation constraints through multi-angle and multi-band deglare image sequences, which significantly enhances the stability and accuracy of the normal vector solution. This is especially true for highly reflective or weakly textured areas, such as aluminum foil sealing edges and transparent films.

[0073] S5013. Perform global integration on the normal vector field formed by the optimal surface normal vectors at all pixel coordinate positions, and introduce the packaging surface continuity constraint to eliminate local ambiguity, thereby obtaining the three-dimensional morphology of the food packaging surface to be tested.

[0074] Global integration refers to the process of converting a discrete surface normal field into a continuous height map. Since each normal vector only reflects the local surface orientation, direct integration may result in inconsistent heights due to noise or solution errors. Therefore, a global optimization method based on energy minimization is adopted to solve for height values ​​simultaneously across the entire image domain, so that the gradient of the generated height map matches the input normal field as closely as possible, thereby reconstructing a smooth and geometrically consistent 3D shape.

[0075] In some implementations, continuity constraints are achieved by constructing a sparse linear equation system: the image is meshed into pixel nodes, the height difference relationship between adjacent nodes is derived based on the normal vector, and a Laplacian smoothing term is added to penalize drastic non-physical changes; for known packaging structure priors, such as the sealing area should be planar and the printing area should not have abrupt changes, hard or soft boundary conditions are further applied; finally, a globally consistent height map is obtained by solving this equation system, which is the three-dimensional topography, with an accuracy of up to the micrometer level, which is sufficient to characterize the geometric features of typical defects such as pinholes, scratches, or microbulges.

[0076] It should be noted that food packaging surfaces typically possess physical characteristics of overall continuity with localized minor variations, such as smooth film surfaces, regular creases, or tiny bulges. If only local integration is performed or surface continuity is ignored, false undulations or breakage artifacts are easily generated in areas with weak texture or high reflectivity. This solution, by explicitly introducing constraints on the continuity of the packaging surface, such as high smoothness, differentiability, or boundary consistency, effectively suppresses local ambiguity and noise propagation in the normal vector field, ensuring that the reconstruction results truly reflect the actual geometry of the packaging and providing a reliable foundation for subsequent curvature-based defect detection.

[0077] Based on the above technical solutions, in visual inspection of food packaging, specular glare from highly reflective material surfaces severely interferes with traditional 2D image analysis, making it difficult to reliably detect minute defects. If the original image is used directly for 3D reconstruction, the brightness anomalies caused by glare violate the assumptions of the physical optics model, resulting in incorrect normal vector estimation and thus distorting the 3D shape. To address this, this solution obtains high-quality input through polarization deglare removal, and then establishes an invertible mapping relationship between pixel brightness and surface normal vectors based on a physical optics reflection model under multi-angle illumination. The optimal normal vector field is solved through global optimization, and the continuity constraint of the packaging surface is introduced for integration, effectively suppressing noise and local ambiguity. This technical approach deeply integrates physical priors, multi-view observation, and geometric constraints, not only overcoming the industry challenge of 3D reconstruction failure in highly reflective scenarios but also achieving high-fidelity restoration of micron-level geometric shapes. This provides a solid foundation for subsequent curvature-based defect identification, significantly improving detection accuracy and robustness.

[0078] S502. Perform noise suppression and edge-preserving smoothing on the height map to eliminate high-frequency artifacts introduced during reconstruction and preserve the geometric abrupt features of the defect area. On the smoothed height map, noise suppression and edge-preserving smoothing are performed. The principal curvature direction and curvature intensity are calculated by performing second-order spatial differentiation on each pixel position.

[0079] The second-order spatial differentiation operation is used to obtain the second-order partial derivative information of the height map in the neighborhood of each pixel, including the curvature changes along the x-direction, y-direction and the mixed direction. Based on these second-order derivatives, a local Hessian matrix is ​​constructed, and its eigenvectors and eigenvalues ​​are solved by eigenvalue decomposition. The eigenvectors correspond to the principal curvature directions, and the eigenvalues ​​correspond to the curvature intensity, thus fully characterizing the local bending characteristics of the surface at that point.

[0080] It should be noted that the height map obtained from 3D reconstruction inevitably contains high-frequency artifacts caused by lighting noise, normal vector solution errors, or integral accumulation. If curvature calculation is performed directly, these artifacts will be misjudged as high-curvature defects. While ordinary smoothing methods (such as Gaussian filtering) can reduce noise, they will blur the geometric abrupt edges corresponding to real defects such as pinholes and scratches. This solution adopts an edge-preserving smoothing strategy (such as bilateral filtering or guided filtering) to effectively suppress noise while sharpening or preserving curvature abrupt regions. This allows subsequent curvature calculations to reflect the geometric anomalies of real defects while avoiding false responses, significantly improving the signal-to-noise ratio and reliability of defect detection.

[0081] In some implementations, the second-order spatial differentiation operation is achieved through predefined convolution kernels, such as using 3×3 or 5×5 Sobel, Scharr, or Gaussian second-derivative kernels to estimate the derivatives. , and Subsequently, a 2×2 Hessian matrix is ​​constructed and its characteristic system is solved. The final output principal curvature direction is represented in the form of an angle map, and the curvature intensity is stored in the form of a scalar field. The two together constitute a multi-channel local curvature attribute map, which is used for the subsequent location and classification of defect areas.

[0082] S503. Based on the principal curvature direction and curvature intensity, generate a curvature field that characterizes the local geometry of the surface.

[0083] In some implementations, the curvature field is represented as a multi-channel image, including at least one of the following: maximum principal curvature map, minimum principal curvature map, Gaussian curvature map, and average curvature map. The system normalizes each curvature map and dynamically sets anomaly thresholds based on preset material types. For example, for metal-coated packaging, pinholes typically exhibit localized high negative Gaussian curvature and strong radial curvature concentration, while bulges show positive Gaussian curvature and bidirectional protrusion features. For transparent polymer films, scratches often appear as unidirectional high curvature linear structures. By analyzing the shape, scale, and curvature sign combination of connected regions exceeding the threshold in the curvature field, the defect type can be preliminarily distinguished and its spatial location can be located, providing key geometric priors for subsequent Bayesian probability discrimination.

[0084] Based on the above technical solutions, in visual inspection of food packaging, tiny 3D defects often do not cause obvious color or texture changes, but only manifest as local geometric deformations, which are difficult to effectively identify using traditional 2D image analysis methods. Furthermore, the 3D morphology reconstructed directly from the original image is often affected by reconstruction noise, integration errors, and high-frequency artifacts. If used for curvature calculation without processing, noise will be misjudged as high-curvature defects, leading to a large number of false alarms. To address this, this solution proposes a geometric feature extraction process for defect perception: a high-fidelity height map is reconstructed based on a physical optical reflection model and a high-quality deglare image sequence; noise suppression and edge smoothing are applied to this map, filtering out high-frequency artifacts while preserving the geometric abrupt boundaries of the defect area; second-order spatial differentiation is performed on each pixel to accurately calculate the principal curvature direction and curvature intensity, constructing local curvature attributes that truly reflect the surface bending characteristics; and a structured curvature field is generated as the core representation of the defect's geometric features. This solution achieves high sensitivity and low false alarm detection of micron-level 3D defects through physical modeling to ensure reconstruction quality, edge smoothing to improve the signal-to-noise ratio, and differential geometry to quantify local deformation.

[0085] In one possible implementation of this application embodiment, the above-mentioned S204 can be specifically implemented by the following S601, S602, S603 and S604, which are described in detail below: S601. Input the local curvature field and the deglare image into different branches of the multi-scale feature pyramid network, respectively.

[0086] S602. At each level of the multi-scale feature pyramid network, cross-modal correlation between geometric features and texture features is calculated by fusing attention mechanisms, and the two types of features are weighted to enhance or suppress them.

[0087] In some implementations, the multi-scale feature pyramid network includes two parallel encoding branches: the first branch receives the local curvature field as input and uses a lightweight convolutional module to extract multi-scale features representing surface geometric anomalies, including curvature extreme response, edge discontinuity, and local convex-concave patterns; the second branch receives the deglare image as input and extracts multi-scale visual features representing printing texture, color anomalies, and frequency domain disorder through standard convolutional layers; both branches output feature maps of the same resolution at each pyramid level, and feature alignment and weighted interaction are performed through a cross-modal attention mechanism in the subsequent fusion stage, enabling the network to focus on regions where geometric abrupt changes and texture distortion coexist, thereby effectively suppressing false detections caused by a single modality and improving the ability to identify complex micro-defects, such as pinholes accompanied by ink diffusion and micro-scratches accompanied by reflective changes.

[0088] S603. Based on the weighted multi-scale features, generate a spatial attention heatmap and mark connected regions whose response values ​​exceed a preset threshold as suspected defect regions.

[0089] The spatial attention heatmap is a two-dimensional probability response map of the same size as the input image. The response value of each pixel represents the combined confidence level of the presence of both geometric and texture anomalies at that location. A preset threshold is used to distinguish between valid defect signals and background noise. Regions with response values ​​higher than the threshold are considered to have a higher probability of being defective. Adjacent high-response pixels are aggregated into independent regions through connected component analysis and marked as suspected defect regions.

[0090] In some implementations, spatial attention heatmaps are generated by applying a 1×1 convolution and a sigmoid activation function to the top feature map of a multi-scale feature pyramid, and then upsampling and refining the low-level high-resolution features to improve boundary localization accuracy. The preset threshold can be a fixed value or dynamically adjusted according to the current packaging material type or production line conditions. For example, a higher threshold is used for highly reflective metallic-coated packaging to suppress interference from normal creases, while a lower threshold is used for matte paper packaging to avoid missing weak stains. The final output of suspected defect areas is passed to the backend Bayesian inference module in the form of a binary mask or bounding box to calculate the posterior defect probability.

[0091] S604. Obtain the observed geometric and textural features of each suspected defect region, and calculate the posterior probability that each suspected defect region is a real defect based on the pre-constructed prior defect distribution model and Bayesian inference framework. The output of the prior defect distribution model represents the frequency of occurrence of different defect types in historical production data and their typical feature distribution.

[0092] The geometric observation features include principal curvature intensity, Gaussian curvature sign, edge curvature gradient, and surface continuity index extracted from the local curvature field. The texture observation features include local contrast, gray-level co-occurrence matrix energy, frequency domain wavelet coefficients, and printing pattern consistency extracted from the deglare image. The prior defect distribution model is a probabilistic model constructed based on historical production data. Its output represents the frequency of occurrence of different defect types on various food packaging materials and their typical geometric-texture joint distribution. Under the Bayesian inference framework, the current observation features are used as the likelihood term and combined with the prior defect distribution model. The confidence that the region is a real defect is calculated using the posterior probability formula.

[0093] In some implementations, the prior defect distribution model is grouped and modeled according to the packaging material category: the metal coating model focuses on modeling the pinhole distribution in high curvature negative Gaussian regions, the transparent film model focuses on the joint probability of linear high curvature and light transmission anomalies, and the paper model focuses on the correlation statistics between texture disorder and low curvature abrupt changes. In the inference stage, the system automatically loads the corresponding sub-model according to the material type identified in the previous step, and calculates the posterior probability in combination with the current observation features. If the posterior probability exceeds the dynamic threshold, it is judged as a valid defect and an alarm is triggered; otherwise, it is classified as interference noise and filtered out.

[0094] It should be noted that traditional detection methods usually rely on fixed thresholds or single features for binary discrimination, making it difficult to distinguish between real defects and normal process variations (such as packaging creases, ink gradations, or material seams). In contrast, this solution uses a Bayesian posterior probability mechanism to organically combine historical statistical priors with current multidimensional observation features, enabling interpretable and quantifiable assessment of defect authenticity. This not only significantly reduces the false alarm rate but also supports defect severity grading based on posterior probability values, thereby improving the level of intelligence in quality decision-making.

[0095] If the posterior probability is greater than a preset threshold, the suspected defect area is a real defect and a graded alarm is triggered; if the posterior probability is less than or equal to the preset threshold, the suspected defect area is a false defect.

[0096] Based on the above technical solutions, in visual inspection of food packaging, relying solely on texture or geometric features can easily lead to misjudgments: normal creases in highly reflective areas may be mistaken for scratches, while tiny pinholes or bulges without obvious color changes are easily missed. While existing deep learning methods can integrate multi-source information, they are mostly black-box models, lacking an understanding of the physical nature of defects and struggling to distinguish between real defects and process interference. Therefore, this solution constructs a physically guided, interpretable dual-modal collaborative detection mechanism: the local curvature field (representing geometric anomalies) and the deglare image (representing texture anomalies) are input into two dedicated branches of a multi-scale feature pyramid network, respectively. At each level, a fusion attention mechanism dynamically evaluates the cross-modal consistency of geometric and texture features, enhancing the response only when both are simultaneously abnormal; otherwise, noise is suppressed. The resulting spatial attention heatmap can accurately locate complex suspected defect areas. Furthermore, by combining a priori defect distribution model constructed from historical production data, a Bayesian inference framework is used to calculate the posterior probability that each suspected area is a real defect, achieving a leap from "detection" to "credible discrimination."

[0097] Some of the data in the above formula are calculated by removing dimensions and taking their numerical values. The formula is the closest to the real situation obtained by software simulation of a large amount of collected data. The preset parameters and preset thresholds in the formula are set by those skilled in the art according to the actual situation or obtained through simulation of a large amount of data.

Claims

1. A visual inspection method for food packaging, characterized in that, include: A multispectral ring light source is used to illuminate the food packaging under test in a time-division manner with multiple preset bands and multiple illumination angles. For each combination of band and illumination angle, two images with mutually orthogonal polarization directions are acquired to obtain an orthogonal polarization image sequence composed of multiple orthogonal polarization image pairs. The orthogonal polarization image sequence is subjected to adaptive polarization filtering to obtain a deglare image sequence; Based on the physical optics reflection model and the deglare image sequence, the three-dimensional morphology of the food packaging surface under test is reconstructed; based on the three-dimensional morphology, the local curvature field of the food packaging surface under test is calculated. The local curvature field and the deglare image are respectively input into a multi-scale feature pyramid network to identify suspected defect areas; The features of each suspected defect region are obtained, and the posterior defect probability of the suspected defect region is calculated based on the Bayesian inference framework and the pre-built prior defect distribution model. The defect status of the food packaging under test is judged based on the posterior defect probability.

2. The visual inspection method for food packaging according to claim 1, characterized in that, The method for obtaining the orthogonal polarization image sequence includes: The multispectral ring light source is controlled to illuminate the food packaging under test at different times with multiple different wavelengths and multiple asymmetrically distributed illumination angles. For each combination of wavelength and illumination angle, during the duration of a single illumination, the polarization direction in the imaging optical path is switched by an electronically controlled polarization device, and two original images with mutually orthogonal polarization directions are acquired sequentially. All acquired raw images are time-aligned and parameter-correlated according to their corresponding bands, illumination angles, and polarization directions to form an orthogonal polarization image sequence composed of multiple orthogonal polarization image pairs.

3. The visual inspection method for food packaging according to claim 1, characterized in that, The method for obtaining the deglare image sequence includes: Identify the material type of the food packaging to be tested; obtain the corresponding glare suppression parameters based on the material type, and calculate the weighted difference coefficients of a pair of orthogonal polarization images; wherein, the pair of orthogonal polarization images includes a first orthogonal polarization image and a second orthogonal polarization image, the polarization axis of the first orthogonal polarization image is aligned with the principal direction of specular reflection to capture the maximum glare component, and the polarization axis of the second orthogonal polarization image is orthogonal to the principal direction of specular reflection to obtain diffuse reflection details; The weighted difference coefficients are used to linearly fuse the pair of orthogonal polarization images to generate a deglare image under the current lighting conditions; The anti-glare images generated under all lighting conditions are combined according to the acquisition time sequence to obtain the anti-glare image sequence.

4. The visual inspection method for food packaging according to claim 3, characterized in that, The method for identifying the material type of the food packaging to be tested includes: Physical features are extracted from the orthogonal polarization image sequence. These physical features include color and printing texture features in the visible light band, the transmission-to-reflection intensity ratio in the near-infrared band, and the response curves of polarization degree as a function of illumination angle at different incident angles. Based on the physical characteristics, a fusion representation vector is reconstructed and input into a lightweight classification model that embeds optical prior knowledge. The classification model identifies the material type of the food packaging to be tested. The reconstruction of the fusion representation vector involves normalizing, aligning, and splicing or weighting the multimodal physical features extracted from the orthogonal polarization image sequence to form a fixed-dimensional numerical vector. The material type includes one of the following: metal coating, transparent polymer film, or matte paper.

5. The visual inspection method for food packaging according to claim 3, characterized in that, The calculation of the weighted difference coefficients of the pair of orthogonal polarization images includes: Based on the identified material type, the corresponding polarization characteristic parameters are retrieved from the preset material-polarization response model library; Based on the incident angle and wavelength of the current illumination source, the energy proportion of the specular reflection component in the orthogonal polarization image pair is calculated using the polarization characteristic parameters; based on the energy proportion, a weighted difference coefficient is generated through a preset nonlinear mapping function.

6. The visual inspection method for food packaging according to claim 1, characterized in that, The calculation of the local curvature field of the surface of the food packaging to be tested includes: The three-dimensional morphology of the surface of the food packaging under test is obtained based on the physical optics reflection model and the reconstruction of the deglare image sequence. The three-dimensional morphology is represented in the form of a height map. The height map is subjected to noise suppression and edge-preserving smoothing processing. By performing second-order spatial differentiation on each pixel position, its local curvature properties are calculated. The local curvature properties include principal curvature direction and curvature intensity. Based on the local curvature properties, a curvature field representing the local geometry of the surface is generated.

7. The visual inspection method for food packaging according to claim 6, characterized in that, The method for reconstructing the three-dimensional morphology of the surface of the food packaging to be tested includes: The illumination angle and wavelength of the light source corresponding to each image in the anti-glare image sequence are obtained, and a physical mapping relationship between the pixel brightness at the same pixel coordinate position in each image and the normal vector of the packaging surface corresponding to that position is established by combining the physical optical reflection model. Based on the physical mapping relationship, the optimal surface normal vector corresponding to each pixel coordinate position is solved; The normal vector field formed by the optimal surface normal vectors at all pixel coordinate positions is globally integrated, and the continuity constraint of the packaging surface is introduced to eliminate local ambiguity, thus obtaining the three-dimensional morphology of the surface of the food packaging to be tested.

8. The visual inspection method for food packaging according to claim 1, characterized in that, The method for identifying the suspected defective region includes: The local curvature field and the deglare image are respectively input into different branches of a multi-scale feature pyramid network; wherein, the branches include a first branch and a second branch, the first branch is used to extract features representing apparent geometric anomalies, and the second branch is used to extract features representing apparent texture anomalies. At each level of the multi-scale feature pyramid network, cross-modal correlation between geometric features and texture features is calculated through a fusion attention mechanism, and the two types of features are weighted to enhance or suppress them. Based on the weighted multi-scale features, a spatial attention heatmap is generated, and connected regions whose response values ​​exceed a preset threshold are marked as suspected defect regions.

9. A visual inspection method for food packaging according to claim 8, characterized in that, The calculation of the posterior defect probability of the suspected defect region includes: The geometric and textural observation features of each suspected defect region are obtained. Based on a pre-constructed prior defect distribution model and a Bayesian inference framework, the posterior probability that each suspected defect region is a real defect is calculated. The output of the prior defect distribution model represents the frequency of occurrence of different defect types in historical production data and their typical feature distribution. If the posterior probability is greater than a preset threshold, the suspected defect region is a real defect and a graded alarm is triggered. If the posterior probability is less than or equal to the preset threshold, the suspected defect region is a false defect.

10. A visual inspection system for food packaging, operating based on the visual inspection method for food packaging according to any one of claims 1-9, characterized in that, It includes a defect identification module, as well as an acquisition module and an analysis module connected to it; The acquisition module is used to illuminate the food packaging under test with a multispectral ring light source at multiple preset bands and multiple illumination angles in a time-division manner. For each combination of band and illumination angle, two images with mutually orthogonal polarization directions are acquired to obtain an orthogonal polarization image sequence composed of multiple orthogonal polarization image pairs. The analysis module is used to perform adaptive polarization filtering on the orthogonal polarization image sequence to obtain a deglare image sequence; Based on the physical optics reflection model and the deglare image sequence, the three-dimensional morphology of the food packaging surface under test is reconstructed; based on the three-dimensional morphology, the local curvature field of the food packaging surface under test is calculated. The defect identification module is used to input the local curvature field and the deglare image into a multi-scale feature pyramid network to identify suspected defect areas. The features of each suspected defect region are obtained, and the posterior defect probability of the suspected defect region is calculated based on the Bayesian inference framework and the pre-built prior defect distribution model. The defect status of the food packaging under test is judged based on the posterior defect probability.

Images