Printed packaging visual inspection method based on dual detection of frequency domain and spatial features

Abstract

This invention relates to the fields of machine vision and image recognition technology, and discloses a visual inspection method for printed packaging based on dual detection of frequency domain and spatial features. The method acquires an image of the printed packaging surface, performs two-dimensional cross-scanning using a selective state-space model to obtain a deep spatial feature map; performs a two-dimensional fast Fourier transform on the surface image, generates a dynamic mask based on the amplitude response at each frequency position in the amplitude spectrum, and obtains a frequency domain suppressed feature map through complex spectrum modulation, inverse Fourier transform, and convolution mapping; aligns and concatenates the two types of features and inputs them into a multi-head self-attention global inference module, adjusts the aggregation weights based on the matching relationship between frequency domain anomalies and spatial image layout, boundary contours, and periodic textures, and uses three loss terms for joint training to output the defect location, category, and confidence level. This invention can suppress periodic texture responses according to the current image, reduce false alarms of normal high-frequency edges, and retain random, elongated, or weak defect responses.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and image recognition technology, specifically to a visual inspection method for printed packaging products based on dual detection of frequency domain and spatial features. Background Technology

[0002] Corrugated cardboard boxes, holographic film packaging, premium gift boxes, labels, and other printed packaging products are prone to surface defects such as scratches, ink smudges, smudges, missing prints, and dents during the production process. These types of packaging products also feature graphic layouts, boundary outlines, periodic halftone patterns, and holographic or brushed textures, requiring the identification of random, subtle, or elongated defects within complex, normal structures.

[0003] Current visual inspection typically employs spatial domain image processing techniques such as template matching and edge extraction, or deep learning techniques such as convolutional neural networks and object detection networks. For periodic background textures, high-energy frequency locations can be located using Fourier transform and suppressed using fixed bandstop, Gaussian, or other preset filters; other methods concatenate or fuse spatial and frequency domain features before the inspection head outputs the defect result.

[0004] However, fixed or unrelated frequency domain filtering is difficult to adjust the suppression level according to the packaging material and graphic content, and may weaken the high-frequency edges of defects at the same time; when relying solely on the spatial domain model, it is insufficient to extract long-distance correlations of continuous defects such as long scratches; when directly splicing frequency domain and spatial domain features, the high-frequency response generated by normal graphic boundaries may be misjudged as a defect; if the frequency domain mask lacks training constraints corresponding to the periodic peak suppression, identity mapping or under-filtering may also occur.

[0005] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a visual inspection method for printed packaging based on dual detection of frequency domain and spatial features, in order to solve the problems in the prior art, such as the difficulty in adjusting the suppression degree of frequency domain filtering according to the current image, the inadequacy of spatial domain model in extracting long-distance correlations of slender defects, the easy misjudgment of normal high-frequency edges as defects by direct splicing of frequency and spatial features, and the easy occurrence of identity mapping or under-filtering in frequency domain mask training.

[0007] The technical solution of this invention is as follows: a visual inspection method for printed packaging based on dual detection of frequency domain and spatial features, comprising: acquiring a surface image of the printed packaging; inputting the surface image into a dual-branch feature extraction network, and performing two-dimensional cross-scanning using a selective state-space model through the spatial domain branch to obtain a deep spatial domain feature map; performing a two-dimensional fast Fourier transform on the surface image through the frequency domain branch, inputting the amplitude spectrum into a frequency domain gating network, and generating a dynamic mask according to the relationship between isolated high-energy frequency positions corresponding to periodic textures and wide-band distribution positions corresponding to random defects; using the dynamic mask to modulate the complex spectrum and perform inverse Fourier transform and convolution mapping to obtain a frequency domain suppressed feature map; and performing dimensional alignment and... Channel concatenation yields a joint feature sequence. This joint feature sequence is then input into a global inference module with a multi-head self-attention mechanism. The correlation between the two types of features at their corresponding spatial locations is calculated, and the aggregation weights are adjusted according to the matching results between the frequency domain anomaly response and the spatial map layout, boundary contour, or periodic texture. The weights are reduced when at least one match is determined, increased when none of the three matches are determined, and the current weights are maintained when the aforementioned matching results cannot be determined, resulting in a global fusion feature map. During training, a defect classification loss, a bounding box regression loss, and a frequency domain mask regularization loss based on the global variance of the modulated amplitude spectrum are constructed, and the three are weighted and fused to update the network parameters. During detection, the defect location, category, and confidence level are output based on the global fusion feature map.

[0008] Optionally, the step of obtaining a deep spatial feature map by performing two-dimensional cross-scanning using a selective state-space model through spatial branching includes: dividing the surface image into non-overlapping image blocks and flattening them into a one-dimensional sequence; inputting the one-dimensional sequence into a cascaded selective state-space model, cross-scanning along the row and column directions of the image, and downsampling between adjacent models to obtain the deep spatial feature map containing long-distance continuous defect information.

[0009] Optionally, the step of performing a two-dimensional fast Fourier transform on the surface image through a frequency domain branch, inputting the amplitude spectrum into a frequency domain gating network, generating a dynamic mask according to the relationship between isolated high-energy frequency positions corresponding to periodic textures and wide-band distribution positions corresponding to random defects, and using the dynamic mask to modulate the complex spectrum and perform inverse Fourier transform and convolution mapping to obtain a frequency domain suppression feature map includes: performing a two-dimensional fast Fourier transform on the surface image to obtain the complex spectrum containing the amplitude spectrum and phase spectrum; inputting the amplitude spectrum into the frequency domain gating network, which outputs the dynamic mask with the same size as the complex spectrum according to the amplitude response at each frequency position; multiplying the dynamic mask element-wise with the complex spectrum, and performing inverse Fourier transform and convolution mapping on the modulated complex spectrum to obtain the frequency domain suppression feature map.

[0010] Optionally, the step of inputting the amplitude spectrum into the frequency domain gating network, and having the frequency domain gating network output the dynamic mask with the same size as the complex spectrum based on the amplitude response at each frequency position, includes: extracting the amplitude response at each frequency position in the amplitude spectrum; generating corresponding weights based on the amplitude response by the frequency domain gating network, reducing the weights of frequency positions that are isolated with high amplitude distribution and correspond to periodic textures, and increasing the weights of frequency positions that are wide-band distributed and correspond to random defects, thereby forming the dynamic mask so that the frequency domain suppression feature map weakens the periodic texture and retains the defect response.

[0011] Optionally, the step of dimensional alignment and channel concatenation of the deep spatial feature map and the frequency domain suppression feature map to obtain a joint feature sequence includes: adjusting the frequency domain suppression feature map according to the spatial size of the deep spatial feature map, and using convolutional mapping to make the channel dimensions of the two types of features meet the concatenation condition; concatenating the two types of features after dimension alignment along the channel direction and unfolding them according to their spatial position to obtain the joint feature sequence containing spatial context information and frequency domain anomaly response.

[0012] Optionally, the step of inputting the joint feature sequence into a global inference module with a multi-head self-attention mechanism, calculating the correlation between the two types of features at corresponding spatial locations, and adjusting the aggregation weights according to the matching results of the frequency domain anomaly response and the spatial map layout, boundary contour, or periodic texture; reducing the weight when it can be determined that it matches at least one of them, increasing the weight when it can be determined that it does not match any of the three, and maintaining the current weight when the aforementioned matching results cannot be determined, to obtain a global fused feature map, includes: inputting the joint feature sequence into the global inference module, calculating the correlation between the deep spatial feature map and the frequency domain suppression feature map at corresponding spatial locations; and weighting and aggregating the two types of features according to the correlation, so that the frequency domain anomaly response and the spatial map layout, boundary contour, and periodic texture form a global dependency relationship.

[0013] Optionally, the step of weighted aggregation of the two types of features based on the correlation to establish a global dependency between the frequency domain anomaly response and the spatial domain graphic layout, boundary contour, and periodic texture includes: determining the matching relationship between the frequency domain anomaly response and the spatial domain graphic layout, boundary contour, and periodic texture based on the spatial location corresponding to the frequency domain anomaly response; reducing the aggregation weight of the frequency domain anomaly response when it can be determined that the frequency domain anomaly response matches at least one of the spatial domain graphic layout, boundary contour, or periodic texture; increasing the aggregation weight of the frequency domain anomaly response when it can be determined that the frequency domain anomaly response does not match any of the three; and maintaining the current aggregation weight of the frequency domain anomaly response and generating the global fusion feature map when the aforementioned matching result cannot be determined.

[0014] Optionally, during the detection process, the defect location, category, and confidence level are output based on the global fusion feature map, including: inputting the global fusion feature map into the detection head, and having the detection head generate a defect category prediction value, a defect bounding box prediction value, and a defect confidence level respectively; when the defect confidence level is greater than a preset confidence threshold, outputting defect location information and defect classification information corresponding to the defect confidence level.

[0015] Optionally, during training, a defect classification loss, a bounding box regression loss, and a frequency domain mask regularization loss based on the global variance of the modulated amplitude spectrum are constructed, including: constructing a focal classification loss based on the difference between the defect category prediction value and the defect category label, so as to reduce the influence of normal background samples on parameter updates; and constructing a complete intersection-union regression loss based on the overlap, center point distance, and aspect ratio difference between the defect bounding box prediction value and the defect bounding box label, so as to constrain the bounding box localization of slender defects.

[0016] Optionally, the step of constructing the frequency domain mask regularization loss and weightedly fusing the three to form a joint loss function further includes: obtaining the amplitude spectrum after modulation by the dynamic mask, calculating the global variance of the amplitude at each frequency position in the amplitude spectrum, and using the global variance as the frequency domain mask regularization loss; weightedly fusing the frequency domain mask regularization loss, the focus classification loss, and the complete intersection-union regression loss to obtain the joint loss function; and backpropagating according to the joint loss function to update the parameters of the frequency domain gating network, the global inference module, and the detection head until a preset convergence condition is met.

[0017] The beneficial effects of this invention are as follows:

[0018] This invention inputs the surface image of the same printed packaging product into spatial and frequency domain branches, utilizes a selective state-space model to form a long-range spatial context, and drives a frequency-domain gating network to generate a dynamic mask based on the current image amplitude spectrum. This reduces the weight of isolated high-energy frequency positions corresponding to periodic textures and increases the weight of broadband distribution positions corresponding to random defects. By aligning and stitching the frequency domain suppression feature map with the deep spatial feature map and calculating the correlation of corresponding positions, the aggregation weight is adjusted according to the matching relationship between frequency domain anomalies and image layout, boundary contours, and periodic textures, thus distinguishing normal high-frequency edges from real defects. By jointly training the global variance of the modulated amplitude spectrum with the classification loss and bounding box regression loss, the frequency domain gating is simultaneously constrained by the detection target and the global variance of the modulated amplitude spectrum. This helps reduce isolated high-energy periodic peak responses that are not properly attenuated and suppresses periodic texture responses with the current image, reducing false alarms of normal high-frequency edges and preserving responses to random, elongated, or weak defects. Attached Figure Description

[0019] Figure 1This is a schematic diagram of the overall process of a visual inspection method for printed packaging based on dual detection of frequency domain and spatial features provided in one embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram of spatial branch feature extraction and frequency-space feature alignment provided in an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of frequency domain dynamic mask generation and spectrum modulation provided in one embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of the correlation between the corresponding positions of the frequency-space features and the adjustment of the aggregation weights provided in one embodiment of the present invention;

[0023] Figure 5 This is a schematic diagram of the detection output and joint loss training feedback provided in one embodiment of the present invention;

[0024] Figure 6 This is a schematic diagram of the overall network architecture of a visual inspection method for printed packaging based on dual detection of frequency domain and spatial features provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described embodiments are merely some embodiments of the invention, and not all embodiments. The components of the embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0026] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0027] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0028] As mentioned earlier, the surfaces of corrugated boxes, holographic film packaging, holographic paper, brushed silver, luxury gift boxes, labels, and other printed packaging products often simultaneously exhibit graphic layouts, boundary contours, high-frequency repetitive printing patterns, periodic halftone patterns, holographic textures, or brushed textures. Defects such as scratches, ink smears, smudges, missing prints, and indentations can easily be confused with the aforementioned normal textures and patterns in the spatial domain. Fixed or image-independent frequency domain filtering is difficult to adjust the suppression level according to the packaging material and graphic content, thus easily weakening the high-frequency edges of defects such as minor scratches and extremely faint smudges. When using only spatial domain models to process high-resolution images, it is also difficult to continuously characterize long-distance defects such as long scratches and continuous die-cutting indentations.

[0029] To address this, the present invention provides a visual inspection method for printed packaging based on dual detection of frequency and spatial features. This method integrates frequency domain transformation, frequency domain adaptive suppression, spatial domain feature extraction, and cross-domain feature fusion within the same end-to-end deep neural network architecture. The method uses an image of the same printed packaging surface as the common input to both the spatial and frequency domain branches. It extracts long-range spatial context through a selective state-space model, drives a frequency-domain gating network to generate a dynamic mask using the current amplitude spectrum, and then uses spatial image layout, boundary contours, and periodic textures to determine the corresponding location of frequency-domain anomalies. Finally, it updates the network parameters in reverse using a joint loss to output the defect location, category, and confidence level.

[0030] The following is combined Figures 1 to 6 This invention is described in detail.

[0031] Example 1:

[0032] like Figure 1 As shown in the embodiment of the present invention, the visual inspection method for printed packaging based on dual detection of frequency domain and spatial features can be executed by a computing device capable of performing image tensor operations and neural network training or inference. The method mainly includes the following steps:

[0033] S100. Obtain a surface image of the printed packaging, input the surface image into a dual-branch feature extraction network, and perform two-dimensional cross-scanning using a selective state-space model through a spatial branch to obtain a deep spatial feature map.

[0034] S200. Perform a two-dimensional fast Fourier transform on the surface image through frequency domain branching, input the amplitude spectrum into the frequency domain gated network, and generate a dynamic mask according to the relationship between isolated high-energy frequency positions corresponding to periodic textures and wide-frequency distribution positions corresponding to random defects.

[0035] S300. Modulate the complex spectrum using the dynamic mask and perform inverse Fourier transform and convolution mapping to obtain a frequency domain suppression feature map;

[0036] S400. Perform dimension alignment and channel concatenation on the deep spatial domain feature map and the frequency domain suppression feature map to obtain a joint feature sequence;

[0037] S500: Input the joint feature sequence into the global inference module with a multi-head self-attention mechanism, calculate the correlation between the two types of features at their corresponding spatial locations, and adjust the aggregation weights according to the matching results of the frequency domain anomaly response and the spatial domain image layout, boundary contour, or periodic texture; reduce the weights when it can be determined that there is a match with at least one of them, increase the weights when it can be determined that there is no match with any of the three, and maintain the current weights when the aforementioned matching results cannot be determined, thereby obtaining a global fusion feature map;

[0038] S600 During training, defect classification loss, bounding box regression loss, and frequency domain mask regularization loss based on the global variance of the modulated amplitude spectrum are constructed respectively. The three are weighted and fused to update the network parameters. During detection, the defect location, category, and confidence level are output according to the global fused feature map.

[0039] Based on the above steps, the present invention forms deep spatial domain features and frequency domain suppression features through the same surface image, drives a dynamic mask to differentially modulate the frequency response of periodic textures and random defects through the current amplitude spectrum, judges the corresponding position of frequency domain anomalies through spatial image layout and boundary contour, and forms training feedback through joint loss, thereby constituting a continuous processing process from surface image input to defect localization, classification and confidence output.

[0040] Example 2:

[0041] To provide a more detailed explanation of the technical solutions offered in the above embodiments, the present invention also provides Embodiment Two. For example... Figure 6 As shown, the overall network architecture of this embodiment two sequentially includes image acquisition and preprocessing, dual-branch feature extraction, cross-domain feature fusion, and end-to-end defect prediction; among which, Figures 2 to 5The methods described in detail are spatial branch feature extraction and frequency-spatial feature alignment, frequency domain dynamic mask generation and spectrum modulation, correlation of corresponding positions of frequency-spatial features and adjustment of aggregation weights, and training feedback of detection output and joint loss. In this embodiment, the method is still executed by a computing device capable of performing image tensor operations and neural network training or inference.

[0042] An industrial camera acquires a raw, high-resolution image of the surface of printed packaging, and records this raw, high-resolution image as... Image size and number of channels are derived from the industrial camera's acquisition configuration and network input interface (e.g., image...). The width is 1024 pixels, the height is 1024 pixels, and the number of channels is 3.

[0043] In the image Before inputting the image into the dual-branch feature extraction network, preprocessing is performed. Preprocessing includes image decoding, grayscale or color channel consistency processing, denoising, normalization, and resizing. The denoising window is derived from the preprocessing configuration (e.g., a window size of 3 pixels by 3 pixels), the normalization range is derived from the network input configuration (e.g., normalizing pixel values ​​to a dimensionless range of 0 to 1), and the resized width and height are consistent with the aforementioned network input dimensions. For laser film packaging or color-printed graphics that need to retain color information, three color channels are preserved; for network configurations using a single-channel input, grayscale processing is performed according to the same training configuration. The preprocessed image still maintains its positional correspondence with the original printed packaging surface area and simultaneously enters both the spatial and frequency domain branches.

[0044] like Figure 2 As shown, in another embodiment of the present invention, in step S100, the process of the spatial branch performing a two-dimensional cross-scan using a selective state-space model may include:

[0045] S110. Divide the surface image into non-overlapping image blocks and flatten them into a one-dimensional sequence;

[0046] S120. Input the one-dimensional sequence into a cascaded selective state space model, cross-scan along the row and column directions of the image, and downsample between adjacent models to obtain the deep spatial feature map containing long-distance continuous defect information.

[0047] For example, steps S110 to S120 may further include the following steps:

[0048] First, the image The image is divided into non-overlapping image blocks. The side length of each image block is derived from the model initialization configuration (e.g., the side length of each image block is 16 pixels), thus dividing the 1024-pixel by 1024-pixel image into 64-row by 64-column image blocks. Each image block is flattened and arranged sequentially according to its row and column order in the original image, resulting in a one-dimensional sequence, while preserving the correspondence between each sequence position and the original image region.

[0049] Secondly, the one-dimensional sequence is input into a selective state-space model composed of multiple cascaded MambaBlocks. The sequence length is denoted as... It is determined by the number of image patches (e.g., (4096 sequence positions). The selective state-space model uses... The linear time complexity performs two-dimensional cross-scanning along the row and column directions of the image, enabling continuous transfer of features within the same row, column, and across multiple image blocks. The number of cascaded layers and the number of inter-layer downsampling times in the selective state-space model are derived from the network structure configuration (e.g., setting 4 layers of MambaBlock and performing 3 times of double downsampling between adjacent models), thereby changing the spatial grid from 64 rows by 64 columns to 32 rows by 32 columns, 16 rows by 16 columns, and 8 rows by 8 columns.

[0050] Then, the number of final-level feature channels is derived from the network structure configuration (e.g., set to 128 feature channels), thereby obtaining a deep spatial feature map with a size of 8 rows by 8 columns and 128 feature channels. This deep spatial feature map contains not only packaging layout, boundary contours, and periodic textures, but also cross-regional continuous defect information such as long scratches, ink smears, and continuous die-cutting indentations, and serves as the spatial input for subsequent cross-domain feature alignment.

[0051] While completing the spatial domain branching process, the frequency domain branch also processes the same image. Process it. For example... Figure 3 As shown, in step S200, the two-dimensional fast Fourier transform and dynamic mask generation process may include:

[0052] S210. Perform a two-dimensional fast Fourier transform on the surface image to obtain the complex spectrum containing the amplitude spectrum and the phase spectrum;

[0053] S220. The amplitude spectrum is input into the frequency domain gating network, and the frequency domain gating network outputs the dynamic mask with the same size as the complex spectrum according to the amplitude response at each frequency position.

[0054] In step S220, the process of forming the dynamic mask may further include:

[0055] S221. Extract the amplitude response at each frequency position in the amplitude spectrum;

[0056] S222. The frequency domain gating network generates corresponding weights based on the amplitude response, reduces the weights of frequency positions that are isolated with high amplitude distribution and correspond to periodic textures, and increases the weights of frequency positions that are wide-band distribution and correspond to random defects, forming the dynamic mask so that the frequency domain suppression feature map weakens the periodic textures and retains the defect response.

[0057] For example, steps S210 to S222 may further include the following steps:

[0058] First, the differential frequency domain transform layer utilizes a two-dimensional fast Fourier transform that supports automatic differentiation to transform the current image. Perform a frequency domain transformation channel by channel to obtain a complex spectral representation. This includes the amplitude spectrum and the phase spectrum. A standard tensor implementation of the two-dimensional fast Fourier transform can receive input images or shallow feature maps; in the main path used in this embodiment, the input is fixed as a surface image. The amplitude spectrum is used to represent the amplitude response at each frequency position, while the phase spectrum is used to preserve the positional relationships of the image structure. and This is a dimensionless frequency position index.

[0059] Next, the amplitude spectrum is input into a lightweight frequency-domain gating network. Based on the amplitude distribution of each frequency position and its neighborhood in the current image amplitude spectrum, the frequency-domain gating network outputs a dynamic attention weight mask with a size consistent with the complex spectrum. The output range of the dynamic attention weights is derived from the gating network output mapping configuration (e.g., dimensionless continuous weights between 0 and 1). This mask is directly driven by the amplitude spectrum of the current image, without using fixed band-stop filters, pre-built frequency tables, or globally shared masks independent of the current image.

[0060] Then, for frequency locations exhibiting isolated high-amplitude distributions that can be explained by periodic printing patterns, the frequency-domain gating network assigns lower weights; for frequency locations exhibiting broad-band distributions and corresponding to random scratches, blemishes, omissions, or indentations, the frequency-domain gating network assigns higher weights. For example, isolated high-amplitude frequency locations within the same image are assigned lower weights by the gating network output (e.g., (0.25, dimensionless), the broadband frequency distribution location is given a higher weight by the gating network output (e.g., (0.80, dimensionless), thus enabling the frequency amplitude distribution corresponding to different packaging materials and graphic content to participate in the spectrum modulation of the current image.

[0061] After obtaining the dynamic mask, in step S300, the complex spectrum modulation, inverse Fourier transform, and convolution mapping process may include:

[0062] S310. Multiply the dynamic mask element-wise with the complex spectrum, and perform inverse Fourier transform and convolution mapping on the modulated complex spectrum to obtain the frequency domain suppression feature map.

[0063] For example, step S310 may further include the following steps:

[0064] First, apply the dynamic mask. With complex spectrum Multiply element-wise at the same frequency positions. Continuing with the previous example, if the complex spectrum value at an isolated high-energy frequency position is... Given that the unit is dimensionless relative amplitude and the mask weight at this location is 0.25, the modulated complex value is: Its amplitude decreases from 20 to 5; if the complex spectrum value of the broadband distribution location is Given that the unit is dimensionless relative amplitude and the mask weight at this location is 0.80, the modulated complex value is: Its amplitude was retained from 10 to 8.

[0065] Secondly, a two-dimensional inverse fast Fourier transform is performed on the modulated complex spectrum to restore its spatial features, which are then processed through convolutional layers for feature mapping. The number of convolutional mapping layers depends on the network structure configuration (e.g., setting two convolutional mapping layers), and the output is a frequency-domain suppressed feature map that filters out some periodic texture responses and retains random defect responses. This frequency-domain suppressed feature map is denoted as the clean defect feature map. In this application, it is uniformly referred to as a frequency domain suppressed feature map. For example, after convolution mapping, it forms an 8-row by 8-column feature map with 64 feature channels. Then proceed to the subsequent dimensional alignment process.

[0066] After obtaining the deep spatial domain feature map and the frequency domain suppression feature map, such as Figure 2 As shown, in step S400, the dimension alignment and channel stitching process may include:

[0067] S410. Adjust the frequency domain suppression feature map according to the spatial size of the deep spatial domain feature map, and make the channel dimensions of the two types of features meet the splicing conditions through convolution mapping.

[0068] S420. The two types of features after dimension alignment are spliced ​​along the channel direction and expanded according to spatial position to obtain the joint feature sequence containing spatial context information and frequency domain anomaly response.

[0069] For example, steps S410 to S420 may further include the following steps:

[0070] First, using deep spatial domain feature maps Using the spatial dimensions as an alignment reference, the frequency domain suppression feature map Adjust the scale to the same 8 rows by 8 columns, and maintain 64 feature channels through convolutional mapping. The scaling method is derived from the feature alignment configuration consistent with the training network, and can use interpolation, pooling, or convolutional mapping, but the correspondence between the two types of features and the spatial regions of the original image should be maintained.

[0071] Secondly, the 8 rows by 8 columns, with 128 feature channels With 8 rows by 8 columns and 64 feature channels The features are stitched together along the channel direction to form a joint feature map with 8 rows by 8 columns and 192 feature channels. Then, the joint feature map is unfolded into a joint feature sequence with 64 spatial locations and 192 feature values ​​for each location, in the same order from left to right and from top to bottom. A position code corresponding to its two-dimensional spatial location is added to each sequence location so that subsequent multi-head self-attention can establish a positional relationship between frequency domain anomaly response and spatial layout.

[0072] After forming the joint feature sequence, such as Figure 4 As shown, in step S500, the corresponding spatial location correlation calculation and weighted aggregation process may include:

[0073] S510. Input the joint feature sequence into the global inference module to calculate the correlation between the deep spatial domain feature map and the frequency domain suppression feature map at the corresponding spatial locations.

[0074] S520. Based on the correlation, the two types of features are weighted and aggregated to form a global dependency between the frequency domain anomaly response and the spatial domain graphic layout, boundary contour and periodic texture.

[0075] In step S520, the matching relationship determination and aggregation weight adjustment process may further include:

[0076] S521. Based on the spatial location corresponding to the frequency domain anomaly response, determine the matching relationship between the frequency domain anomaly response and the spatial domain graphic layout, boundary contour, and periodic texture;

[0077] S522. When it can be determined that the frequency domain anomaly response matches at least one of the spatial domain graphic layout, boundary contour, or periodic texture, the aggregation weight of the frequency domain anomaly response is reduced; when it can be determined that the frequency domain anomaly response does not match any of the three, the aggregation weight of the frequency domain anomaly response is increased; when the aforementioned matching result cannot be determined, the current aggregation weight of the frequency domain anomaly response is maintained, and the global fusion feature map is generated.

[0078] For example, steps S510 to S522 may further include the following steps:

[0079] First, the position-encoded joint feature sequence is fed into the Transformer global inference module. The number of Transformer layers is derived from the single-layer aggregation structure (e.g., setting 1 TransformerEncoder layer), and the number of multi-head self-attention heads is derived from the joint feature dimension and the network initialization configuration (e.g., setting 8 attention heads). With a joint feature dimension of 192, each attention head processes 24 feature dimensions.

[0080] Secondly, the multi-head self-attention mechanism maps the joint feature sequence to queries, keys, and values, and calculates the correlation between the deep spatial domain feature map and the frequency domain suppressed feature map in their corresponding spatial locations and global contexts. Instead of extracting features from a single image domain, this Transformer module utilizes the packaging layout, boundary contours, and periodic textures formed by the Mamba branch to determine whether the highlight anomaly response formed by the frequency domain branch is located at a normal edge of the printed image or a true defect in a solid color area or a non-periodic region.

[0081] Then, the aggregation weights are determined jointly by the multi-head self-attention calculation and the matching results, and are dimensionless. The current aggregation weight is derived from the current network forward output (e.g., the current aggregation weight at a certain frequency domain anomaly location is 0.68). When the location is located at a repeating graph-text boundary, and it can be determined that the frequency domain anomaly response matches at least one of the spatial graph-text layout, boundary contour, or periodic texture, its aggregation weight is reduced (e.g., from 0.68 to 0.30); when the frequency domain response at another location has a wide frequency distribution, and it can be determined that the frequency domain anomaly response does not match the spatial graph-text layout, boundary contour, or periodic texture, its aggregation weight is increased (e.g., from 0.68 to 0.82). When the aforementioned matching results cannot be determined, the current aggregation weight is maintained, and no forced suppression or enhancement is performed.

[0082] Finally, the aggregation results of each attention head are mapped to form a high-level global fusion feature map. This globally fused feature map simultaneously preserves long-range spatial context, frequency domain anomaly responses suppressed by periodic texture, and cross-domain judgment results at corresponding locations, and then incorporates them into the detection head or training loss calculation process.

[0083] like Figure 5 As shown, after obtaining the global fusion feature map, step S600 executes either detection output or joint training according to the current operating mode. Firstly, in the detection mode, the process of outputting the defect location, category, and confidence level may include:

[0084] S610. Input the global fusion feature map into the detection head, and the detection head generates the defect category prediction value, the defect bounding box prediction value, and the defect confidence value respectively.

[0085] S620. When the defect confidence level is greater than a preset confidence threshold, output the defect location information and defect classification information corresponding to the defect confidence level.

[0086] For example, steps S610 to S620 may further include the following steps:

[0087] First, the global fused feature map Input detection head. The detection head may sequentially include convolutional layers, a Feature Pyramid Network (FPN), and a prediction layer. The prediction layer generates defect category predictions, defect bounding box predictions, and defect confidence scores, respectively. The specific number of channels in each layer of the detection head depends on the network structure configuration and is not the only limitation of this method.

[0088] Secondly, the preset confidence threshold is derived from validation data statistics and detection output configuration (e.g., set to 0.50, dimensionless). Using the aforementioned surface image, the detection head outputs the "scratch" category for a long, narrow candidate region, along with the pixel coordinates of the upper left and lower right corners of the bounding box. The pixel and defect confidence score are 0.76, which is dimensionless. Since 0.76 is greater than 0.50, the defect location information and defect classification information of the candidate region are output; when the defect confidence score is not greater than 0.50, the low confidence state is retained and no formal defect conclusion is formed.

[0089] On the other hand, in the training mode, the construction process of the focal classification loss, complete intersection-union regression loss, frequency domain mask regularization loss, and joint loss may include:

[0090] S630. Construct a focus classification loss based on the difference between the defect category prediction value and the defect category label to reduce the impact of normal background samples on parameter updates;

[0091] S640. Construct a complete intersection-union regression loss based on the overlap between the predicted value of the defect bounding box and the label of the defect bounding box, the distance between the center points and the aspect ratio difference, so as to constrain the location of the bounding box of the slender defect.

[0092] After obtaining the focus classification loss and the complete intersection-union regression loss, the following steps are performed:

[0093] S650. Obtain the amplitude spectrum after modulation by the dynamic mask, calculate the global variance of the amplitude at each frequency position in the amplitude spectrum, and use the global variance as the frequency domain mask regularization loss.

[0094] S660. The frequency domain mask regularization loss, the focal classification loss, and the complete intersection-union regression loss are weighted and fused to obtain the joint loss function;

[0095] S670. Backpropagate according to the joint loss function to update the parameters of the frequency domain gated network, the global inference module and the detection head until the preset convergence condition is met.

[0096] For example, steps S630 to S670 may further include the following steps:

[0097] First, the training data includes images of printed packaging surfaces, defect category annotations, and defect bounding box annotations. The ratio of the training set to the validation set is determined by the training data management configuration (e.g., an 8:2 split, with 8 parts used for training and 2 parts for validation). The loss weights for the three loss functions are derived from the training initialization configuration and validation set adjustments (e.g., ...). For 1.0, For 1.0, (The value is 0.5, and all three are dimensionless). Joint loss function. According to formula (1):

[0098] (1)

[0099] In formula (1), Indicates joint loss, in dimensionless units; This represents the defect classification loss, in dimensionless units. This represents the regression loss of the defect bounding box, in dimensionless form. This represents the frequency domain mask regularization loss, in dimensionless units; , and The loss weights are dimensionless. Formula (1) unifies the three parallel losses into a single backpropagation input.

[0100] Then, defect classification loss Focus loss is employed. The class balance coefficient and focus index are derived from the positive and negative sample distributions and validation results (e.g., 0.25 Both are dimensionless (2.0), and the predicted probability of the true class of the sample is output by the detection head (e.g., The value is 0.80 (dimensionless). The focus classification loss is calculated according to formula (2):

[0101] (2)

[0102] In formula (2), This represents the predicted probability of the true class of the sample, in dimensionless form. This represents the category balance coefficient, and its unit is dimensionless. This indicates a focus index, with dimensionless units. This is used to reduce the gradient contribution of a large number of simple background samples, allowing the network to focus more attention on complex background patterns and difficult negative samples that are easily misclassified as defects. Substituting the aforementioned example values ​​into formula (2), we obtain the focus classification loss. It is approximately 0.002231, dimensionless.

[0103] Subsequently, the defect bounding box regression loss A complete IoU loss is employed. For defects with extreme aspect ratios, such as extremely fine and long scratches and ink smears, traditional IoU loss or... The loss function is insufficient in constraining center offset and aspect ratio differences. The complete intersection-union (CIU) loss introduces center point distance penalties and aspect ratio consistency penalties on the basis of the overlap between the predicted and ground truth boxes. The CIU regression loss is calculated according to formula (3):

[0104] (3)

[0105] In formula (3), This represents the intersection-union ratio (IUU) of the predicted bounding box and the labeled bounding box, in dimensionless units. and These represent the center of the predicted bounding box and the center of the labeled bounding box, respectively, with the coordinate unit being pixels; This represents the Euclidean distance between the two centers, in pixels. This represents the diagonal length of the smallest bounding box that simultaneously encloses the prediction box and the label box, in pixels; This indicates the aspect ratio consistency term, with dimensionless units. This represents the adaptive weight of the aspect ratio consistency term, in dimensionless form.

[0106] Using the aforementioned prediction frame Pixels, bounding box annotations are derived from training sample annotations (e.g., the annotation box is...). (pixels). The intersection-union ratio (IU) calculated from the two bounding boxes is approximately 0.828947, dimensionless; the squared center distance is 25 square pixels; the squared diagonal of the minimum bounding box is 325924 square pixels; the normalized center distance term is approximately 0.000076705, dimensionless; aspect ratio consistency term... Approximately 0.000007410, dimensionless, adaptive weights It is approximately 0.000043320, dimensionless. Substituting into formula (3), we obtain the complete intersection-union regression loss. It is approximately 0.171129, dimensionless.

[0107] Then, the amplitude spectrum modulated by the dynamic mask in step S310 is obtained. To fully preserve the algebraic expression, the original complex spectrum is denoted as... It is similar to the aforementioned Refers to the complex spectrum of the same current image; This indicates element-wise multiplication. The filtered amplitude spectrum. Calculate according to formula (4):

[0108] (4)

[0109] In formula (4), Indicates the current image's frequency position. The complex spectrum of , in units of dimensionless relative amplitude; Represents the dynamic attention weight mask at the same location, in dimensionless units; This represents element-wise multiplication at corresponding frequency positions; This represents the amplitude spectrum of the modulated complex spectrum, expressed in dimensionless relative amplitude. Using the two frequency positions mentioned earlier, the modulated amplitude at isolated high-energy positions is 5, and the modulated amplitude at wideband distribution positions is 8. This result is used both in the inverse Fourier transform of the modulated complex spectrum and in the calculation of frequency domain mask regularization loss.

[0110] Finally, frequency domain masking regularization loss Calculate according to formula (5):

[0111] (5)

[0112] In formula (5), This represents the calculation of the global variance of the amplitude across all frequency positions of the current modulated amplitude spectrum. For the output of formula (4), The frequency domain mask regularization loss is calculated based on normalized relative amplitudes, with the loss unit being dimensionless. The calculation object and normalization method of the global variance are derived from the current spectral tensor and training configuration, and the same normalization configuration is used in the same training process (for example, using the modulated amplitude spectrum corresponding to the aforementioned 1024 pixel multiplied by 1024 pixel image, the global variance is calculated to be 0.04, dimensionless). When isolated high-amplitude periodic peaks are not properly attenuated, their residual amplitude increases the global variance and increases the frequency domain mask regularization loss, thereby forming a training constraint on the identity mapping or underfiltered state of the dynamic mask.

[0113] After obtaining the above three losses, return to formula (1). Continuing with the aforementioned... The value is 0.002231. It is 0.171129. 0.04 and For 1.0, For 1.0, The value is 0.5, and the joint loss is calculated. The value is 0.193360, which is dimensionless. Backpropagation is then performed based on this joint loss to update the parameters of the frequency-gated network, the global inference module, and the detection head. The preset convergence condition is derived from the training process configuration and changes in the validation loss (e.g., stopping parameter updates in the current round if the validation loss does not decrease for five consecutive training rounds). Once the convergence condition is met, the most recently validated valid network parameters are saved, and the detection mode calls these valid network parameters without performing backpropagation.

[0114] Through the above training, the frequency domain mask generation serves the final defect detection target, preserving the high-frequency edges required for defect discrimination while suppressing periodic background textures. Multi-layer Mamba... The linear complexity of the model extracts long-range features in the spatial domain. After the feature map is fully downsampled, the single-layer Transformer performs cross-domain aggregation. Combined with the tensor fast Fourier transform concurrent operation in the deep learning framework, the frequency-space dual-domain model can be implemented on industrial vision computing devices, while taking into account the detection process and computational resource consumption of high-resolution printed packaging images.

[0115] To maintain the continuity of the above processing under abnormal input or abnormal training conditions, this embodiment also handles common anomalies.

[0116] If the input image cannot be decoded, its size or channels are inconsistent with the network configuration, or there are non-finite pixel values, the current image will not be fed into the dual-branch feature extraction network. If the spatial size, sequence length, or position index of the two types of features are inconsistent, scale adjustment, channel mapping, and position index generation will be re-executed. If they still cannot be consistent, the processing of the current image will be terminated. If the dynamic mask size is inconsistent with the complex spectrum or has a non-finite value, the current spectrum modulation will be stopped, and the most recently validated valid parameters will be called to recalculate the forward calculation; if the anomaly still exists, the defective result of the current image will not be output. If the matching result between the frequency domain anomaly response and the spatial domain image layout, boundary contour, or periodic texture cannot be determined, the current aggregation weights will be maintained. If the training samples lack valid class labels or bounding box labels, the corresponding samples will not participate in the corresponding loss calculation; if the joint loss has a non-finite value, the validation loss continues to deteriorate, or the parameter saving fails, the candidate parameter update will be stopped, and the candidate parameters will not be used to overwrite the most recently valid parameters. If the predicted box exceeds the image range, it will be cropped to the image boundary, and the predicted box with no effective area after cropping will be discarded.

[0117] For example, when the deep spatial domain feature map forms an 8x8 spatial grid, and the frequency domain suppression feature map forms a 7x8 spatial grid after the first scale adjustment, channel stitching of the two types of features is not performed. Instead, scale adjustment and position index generation are re-executed. If 64 corresponding spatial positions still cannot be formed after this reprocessing, the processing of the current surface image is terminated, and the method is re-entered after the next surface image passes the validity check. The network parameters that have recently been validated remain unchanged throughout this process.

[0118] Accordingly, this invention acquires and preprocesses surface images of printed packaging products using an industrial camera, forms a deep spatial feature map containing long-distance continuous defect information through a multi-layer MambaBlock and selective state-space model, forms a frequency domain suppression feature map through a differentiable two-dimensional fast Fourier transform, a dynamic mask driven by the current amplitude spectrum, an inverse Fourier transform, and convolutional mapping, and implements conditional weighting and scaling on the frequency domain anomaly response through position encoding and multi-head self-attention of a single-layer Transformer, and outputs the defect location, category, and confidence level through the detection head, and forms an end-to-end training feedback with focus classification loss, complete intersection-union regression loss, and global variance of the modulated amplitude spectrum, so that spatial feature extraction, frequency domain suppression, cross-domain judgment, defect output, and network update are continuously carried out according to the same technical main line.

[0119] The above description is merely a preferred embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A visual inspection method for printed packaging products based on dual detection of frequency domain and spatial features, characterized in that, include: A surface image of the printed packaging is obtained, and the surface image is input into a dual-branch feature extraction network. The network then performs a two-dimensional cross-scan using a selective state-space model through a spatial branch to obtain a deep spatial feature map. The surface image is subjected to a two-dimensional fast Fourier transform by frequency domain branching, and the amplitude spectrum is input into the frequency domain gated network to generate a dynamic mask according to the relationship between isolated high-energy frequency positions corresponding to periodic textures and wide-frequency distribution positions corresponding to random defects. The complex spectrum is modulated using the dynamic mask, and inverse Fourier transform and convolution mapping are performed to obtain a frequency domain suppression feature map; The deep spatial domain feature map and the frequency domain suppressed feature map are dimensionally aligned and channel-concatenated to obtain a joint feature sequence. The joint feature sequence is input into a global inference module with a multi-head self-attention mechanism. The correlation between the two types of features at their corresponding spatial locations is calculated. The aggregation weight is adjusted according to the matching results of the frequency domain anomaly response and the spatial domain image layout, boundary contour, or periodic texture. The weight is reduced when it can be determined that it matches at least one of them, increased when it can be determined that it does not match any of the three, and the current weight is maintained when the matching results cannot be determined. A global fusion feature map is obtained. During training, a defect classification loss, a bounding box regression loss, and a frequency domain mask regularization loss based on the global variance of the modulated amplitude spectrum are constructed separately. The three are then weighted and fused to update the network parameters. During detection, the defect location, category, and confidence level are output based on the global fused feature map.

2. The visual inspection method for printed packaging products according to claim 1, characterized in that, The process of obtaining a deep spatial feature map by performing a two-dimensional cross-scan using a selective state-space model through spatial branching includes: The surface image is divided into non-overlapping image blocks and flattened into a one-dimensional sequence; The one-dimensional sequence is input into a cascaded selective state-space model, which is cross-scanned along the row and column directions of the image and downsampled between adjacent models to obtain the deep spatial feature map containing long-distance continuous defect information.

3. The visual inspection method for printed packaging products according to claim 1, characterized in that, The process involves performing a two-dimensional fast Fourier transform on the surface image through a frequency domain branch, inputting the amplitude spectrum into a frequency domain gating network, generating a dynamic mask according to the relationship between isolated high-energy frequency locations corresponding to periodic textures and wide-band distribution locations corresponding to random defects, and using the dynamic mask to modulate the complex spectrum and perform inverse Fourier transform and convolution mapping to obtain a frequency domain suppressed feature map, including: A two-dimensional fast Fourier transform is performed on the surface image to obtain the complex spectrum containing the amplitude spectrum and the phase spectrum; The amplitude spectrum is input into the frequency domain gated network, and the frequency domain gated network outputs the dynamic mask with the same size as the complex spectrum based on the amplitude response at each frequency position. The dynamic mask is multiplied element-wise with the complex spectrum, and the modulated complex spectrum is subjected to inverse Fourier transform and convolution mapping to obtain the frequency domain suppression feature map.

4. The visual inspection method for printed packaging products according to claim 3, characterized in that, The step of inputting the amplitude spectrum into the frequency domain gating network, and having the frequency domain gating network output the dynamic mask with the same size as the complex spectrum based on the amplitude response at each frequency position, includes: Extract the amplitude response at each frequency position in the amplitude spectrum; The frequency-domain gating network generates corresponding weights based on the amplitude response, reduces the weights of frequency positions that are isolated with high amplitude distribution and correspond to periodic textures, and increases the weights of frequency positions that are wide-band distribution and correspond to random defects, forming the dynamic mask so that the frequency-domain suppression feature map weakens periodic textures and retains defect responses.

5. The visual inspection method for printed packaging products according to claim 1, characterized in that, The step of dimensional alignment and channel concatenation between the deep spatial domain feature map and the frequency domain suppressed feature map to obtain a joint feature sequence includes: The frequency domain suppression feature map is adjusted according to the spatial dimensions of the deep spatial domain feature map, and the channel dimensions of the two types of features are made to meet the splicing conditions through convolution mapping. The two types of features, after dimensional alignment, are concatenated along the channel direction and expanded according to their spatial positions to obtain the joint feature sequence containing spatial context information and frequency domain anomaly response.

6. The visual inspection method for printed packaging products according to claim 1, characterized in that, The joint feature sequence is input into a global inference module with a multi-head self-attention mechanism to calculate the correlation between the two types of features at their corresponding spatial locations. The aggregation weights are adjusted according to the matching results between the frequency domain anomaly response and the spatial domain image layout, boundary contour, or periodic texture. The weights are reduced when a match with at least one of the features is determined, increased when a match with all three is determined, and the current weights are maintained when the matching results cannot be determined. This process yields a global fused feature map, including: The joint feature sequence is input into the global inference module to calculate the correlation between the deep spatial domain feature map and the frequency domain suppression feature map at the corresponding spatial locations; Based on the correlation, the two types of features are weighted and aggregated to form a global dependency between the frequency domain anomaly response and the spatial domain graphic layout, boundary contour, and periodic texture.

7. The visual inspection method for printed packaging products according to claim 6, characterized in that, The step of weighted aggregation of the two types of features based on the correlation, so that the frequency domain anomaly response forms a global dependency with the spatial domain image layout, boundary contour, and periodic texture, includes: Based on the spatial location corresponding to the frequency domain anomaly response, determine the matching relationship between the frequency domain anomaly response and the spatial domain graphic layout, boundary contour, and periodic texture; When it can be determined that the frequency domain anomaly response matches at least one of the spatial domain graphic layout, boundary contour, or periodic texture, the aggregation weight of the frequency domain anomaly response is reduced; when it can be determined that the frequency domain anomaly response does not match any of the three, the aggregation weight of the frequency domain anomaly response is increased; when the aforementioned matching result cannot be determined, the current aggregation weight of the frequency domain anomaly response is maintained, and the global fusion feature map is generated.

8. The visual inspection method for printed packaging products according to claim 1, characterized in that, During the detection process, the defect location, category, and confidence level are output based on the global fusion feature map, including: The global fusion feature map is input into the detection head, which generates defect category prediction, defect bounding box prediction, and defect confidence respectively. When the defect confidence level is greater than a preset confidence threshold, the defect location information and defect classification information corresponding to the defect confidence level are output.

9. The visual inspection method for printed packaging products according to claim 1, characterized in that, During training, defect classification loss, bounding box regression loss, and frequency domain mask regularization loss based on the global variance of the modulated amplitude spectrum are constructed, including: A focus classification loss is constructed based on the difference between the defect category prediction value and the defect category label to reduce the impact of normal background samples on parameter updates; A complete intersection-union regression loss is constructed based on the overlap between the predicted defect bounding box and the defect bounding box annotation, the center point distance, and the aspect ratio difference, in order to constrain the location of the bounding box of slender defects.

10. The visual inspection method for printed packaging products according to claim 9, characterized in that, The construction of frequency domain mask regularization loss, and the weighted fusion of the three to form a joint loss function, also includes: Obtain the amplitude spectrum modulated by the dynamic mask, calculate the global variance of the amplitude at each frequency position in the amplitude spectrum, and use the global variance as the frequency domain mask regularization loss; The joint loss function is obtained by weighting and fusing the frequency domain mask regularization loss, the focal point classification loss, and the complete intersection-union regression loss. Backpropagation is performed based on the joint loss function to update the parameters of the frequency-domain gated network, the global inference module, and the detection head until a preset convergence condition is met.

Images