An image-based intelligent analysis method for biscuit processing quality

Abstract

The application discloses a kind of based on image biscuit processing quality intelligent analysis method, comprising the following steps: S1, based on each mask image training semantic segmentation model, extract main body segmentation dataset;S2, carry out pixel-by-pixel annotation, obtain crack segmentation dataset;S3, construct biscuit segmentation model, output final segmentation mask result;S4, based on SegFormer's biscuit crack segmentation model, output crack segmentation result and generate crack mask;S5, calculate each biscuit parameter.The application carries out accurate segmentation to biscuit main body area, automatically extracts the length, width, area, perimeter, color feature and texture parameter of biscuit;Also introduce semantic segmentation model to identify surface crack, and quantify the uniformity of crack;The method can complete objective detection of large quantities of samples in a short time, improve analysis efficiency and result consistency, provide reliable technical support for biscuit quality evaluation, and have good application prospect.

Description

Technical Field

[0001] This invention relates to the field of intelligent food analysis technology, specifically to an image-based intelligent analysis method for biscuit processing quality. Background Technology

[0002] Biscuits, as a common baked food, are widely used in various scenarios such as the snack food market, infant nutritional supplements, and military rations. Their production quality directly affects the product's taste, packaging suitability, and consumer acceptance. In industrial production, the appearance of biscuits (such as size, outline integrity, and color uniformity) not only reflects the stability of the shaping and baking processes but is also a key indicator for companies to control product quality and standardize grading. Meanwhile, the crack characteristics on the biscuit surface are also significant: on the one hand, moderate cracks help improve taste and water absorption; on the other hand, irregular or excessive cracks often indicate quality defects caused by production anomalies (such as unbalanced raw material ratios, abnormal temperature control, and uneven stress). Furthermore, the uniformity of crack distribution can serve as an important reference for assessing the internal structure and baking uniformity of biscuits. Therefore, accurately identifying and quantifying the macroscopic phenotype and microscopic crack structure of biscuits is crucial for improving product consistency and intelligent manufacturing levels.

[0003] In most companies, phenotypic assessment and crack analysis of biscuits still rely on manual visual inspection or traditional image processing methods. Manual inspection is inefficient, highly subjective, and difficult to adapt to the needs of high-volume, high-speed production. It also struggles to accurately locate fine-grained features (such as micro-cracks and edge defects). Traditional image processing methods are highly dependent on lighting conditions, shooting angle, and hardware configuration. They often exhibit poor stability and weak generalization ability under conditions of background interference, uneven color, and complex textures, making it difficult to meet the requirements of high-precision, automated quality inspection. Traditional image processing methods, taking a sandwich cookie defect detection device and method as an example, include a base plate, a sandwich cookie, a light source, a polarizer mounted on a first support between the light source and the sandwich cookie, a color cookie blank detection camera including a main body and a front polarizer of the main body lens, a filling detection camera, and a beam splitter mounted on a second support to split the reflected light from the sandwich cookie into two beams. The cookie blank detection camera acquires the first beam of reflected light passing through the beam splitter to obtain a cookie blank image, and the filling detection camera acquires the second beam of reflected light split by the beam splitter to obtain a filling image. The cookie blank detection camera and the filling detection camera send the cookie blank image and the filling image to a processing device to determine whether the sandwich cookie has a defect. However, this traditional image processing method has high requirements for the layout and selection of devices; even slight deviations can have a significant impact on the results. Furthermore, it exhibits poor stability and weak generalization ability when facing background interference, uneven color, and complex textures, making it difficult to meet the requirements of high-precision and automated quality inspection. Summary of the Invention

[0004] To address the aforementioned problems, the purpose of this invention is to propose an image-based intelligent analysis method for biscuit processing quality. First, the main biscuit region is accurately segmented using a U-Net network. Then, contour extraction and morphological analysis are employed to automatically extract the biscuit's length, width, area, perimeter, color features, and texture parameters. Subsequently, a SegFormer model based on the Transformer architecture is introduced to perform high-resolution identification of surface cracks in the biscuit. Combined with skeleton extraction and distribution analysis, the uniformity and spatial characteristics of the cracks are quantified. This method possesses advantages such as no human intervention, strong adaptability, fast detection speed, and strong crack detail recovery capability, providing efficient, objective, and quantifiable technical support for biscuit quality assessment, and has promising prospects for industrial application.

[0005] This was achieved through the following technical solutions:

[0006] A method for intelligent analysis of biscuit processing quality based on images includes the following steps: S1. Based on multiple original images, each biscuit in each original image is instantiated and annotated using an image annotation tool to generate a mask image containing multiple foreground regions; a semantic segmentation model is trained based on each mask image to extract the complete biscuit region and use it as the main body segmentation dataset; wherein, each original image is taken under the same background and lighting conditions, and a ruler is set in the background as a reference; S2. Based on the main body segmentation dataset in step S1, each biscuit region in each original image is cropped to obtain the first cropped data; the crack region in the first cropped data is annotated pixel by pixel using an image annotation tool to obtain the crack segmentation dataset; S3. The main body segmentation dataset in step S1 and the crack segmentation dataset in step S2 are used as input data to construct a biscuit segmentation model based on U-Net. The construction process includes: The input data is processed sequentially through normalization, tensor form conversion, first multi-scale feature extraction, and outputting the final segmentation mask result; S4, the corresponding region of each mask image in each original image is cropped to obtain the second cropped data; the second cropped data is input into the biscuit crack segmentation model based on SegFormer, and further preprocessing, second multi-scale feature extraction, layer fusion, crack probability prediction, outputting the crack probability map and generating the crack mask is performed; S5, based on the scale in step S1, the conversion relationship between pixels in each original image and the actual physical size is determined, and then the parameters of each biscuit are calculated. The calculation process includes: based on the conversion relationship, obtaining the morphological features of each biscuit in each biscuit region; then converting the color of each biscuit region from RGB to CIELAB color space to obtain the color mean parameter; based on the gray-level co-occurrence matrix, extracting the texture parameters from the complete biscuit region in step S1.

[0007] Preferably, each original image in step S1 contains multiple cookies, and each cookie in each mask image is recorded as 1 and the background is recorded as 0.

[0008] Preferably, the main body segmentation dataset in step S1 and the crack segmentation dataset in step S2 are both divided into training set, validation set and test set in a ratio of 8:1:1.

[0009] Preferably, when performing pixel-by-pixel annotation in step S2, a single-channel mask is used to label each surface crack region as a foreground, while each other region outside each foreground is labeled as a background. The pixel size of each foreground is controlled to be the same, and the set of each foreground is used as the crack segmentation dataset.

[0010] Preferably, in step S3, when constructing the U-Net-based cookie segmentation model, each image in the input data is first scaled by pixel values ​​to the [0, 1] range and its size is uniformly adjusted; then, each image after uniform size adjustment is converted into tensor form and input into the cookie segmentation model; then, in the encoder of the cookie segmentation model, downsampling operations are performed sequentially through four downsampling modules. Each downsampling module consists of two consecutive 3×3 convolutional layers, and a ReLU activation function is used for nonlinear transformation after each convolution; each feature map corresponding to each image is obtained using the four sampling modules. Each downsampling module also performs spatial downsampling on each feature map through a 2×2 window max pooling operation to halve the size of each feature map and double the number of channels, thereby obtaining each multi-scale feature map. Figure 1 Next, at the bottleneck layer of the cookie segmentation model, each multi-scale feature is further compressed. Figure 1 The context information is fused using 3×3 convolution, and each fused result is transmitted to the decoder of the cookie segmentation model for decoding to obtain the final segmentation mask result.

[0011] Preferably, decoder one includes multiple decoding modules; when decoder one performs decoding, it first uses each decoding module to perform transposed convolution to complete the upsampling operation, and then each result of the upsampling operation is concatenated with each feature map in the corresponding encoder one through skip connections to obtain each concatenated image; then each concatenated image is compressed to 1 channel number through a 1×1 convolution to generate a single-channel pixel-level probability map, and each pixel-level probability map is mapped to the [0,1] interval through the Sigmoid activation function and binarized with a set threshold to obtain the final segmentation mask result.

[0012] Preferably, in step S4, after inputting the second cropped data into the SegFormer-based biscuit crack segmentation model, normalization and size adjustment preprocessing are performed first. Each preprocessed image is then transmitted to encoder two in the biscuit crack segmentation model for hybrid encoding operation, outputting each multi-level feature map to decoder two in the biscuit crack segmentation model. Decoder two uses a multilayer perceptron to upsample each multi-level feature map to the same size and perform layer fusion. The layer fusion result is used to predict the crack probability of each pixel through pointwise convolution, and then the corresponding crack probability map is output. The hybrid encoding operation includes: first, dividing each preprocessed image into 7×7 local blocks through overlapping block embedding, with each local block linearly projected into an embedding vector; then, inputting each embedding vector into each hybrid architecture encoding block in encoder two through layered design, and generating each multi-scale feature through four stages of progressive downsampling. Figure 2 And preserve the global context relationship, for each multi-scale feature Figure 2 The data is transmitted to decoder 2; each hybrid architecture coding block is equipped with a self-attention mechanism and a multilayer perceptron.

[0013] Preferably, each multi-scale feature Figure 2 Both include shallow and deep features; before predicting the crack probability, decoder 2, based on a self-attention mechanism and a multilayer perceptron, performs multi-scale feature analysis on each feature. Figure 2 Shallow features capture texture differences for each multi-scale feature. Figure 2 The deep features establish long-distance dependencies, which are used to distinguish between split patterns and textures.

[0014] Preferably, in step S5, when confirming the conversion relationship, the actual length corresponding to each pixel of each original image is recorded as real_pixel in centimeters. , This indicates the actual width of the scale, expressed in centimeters. The scale represents the pixel scale in each original image; the morphological features of each cookie include area A, perimeter P, roundness C, length L1 of the minimum bounding rectangle, and width W1 of the minimum bounding rectangle. N is the total number of pixels for each cookie region; , The pixel perimeter is obtained based on the 8-connected boundary tracking algorithm; 0 <C≤1; , and These represent the maximum and minimum coordinates of the smallest bounding rectangle in the horizontal direction, respectively. , and These represent the maximum and minimum coordinates of the smallest bounding rectangle in the vertical direction, respectively; when calculating the color mean parameter... , Let L2 represent the color mean parameter, A2 represent the red-green axis, and B2 represent the yellow-blue axis. The overline on L2, A2, and B2 all represent the mean. N represents the total number of pixels in the corresponding cookie region, where i = 1, 2, 3, ..., N. When extracting texture parameters, a sliding window with a size of 5 and a step size of 4 is set. The sliding window is used to extract each crack mask from each original image, and each sliding window with a valid center pixel is retained. Then, the skimage.feature.graycomatrix function in Python is used to select neighboring pixel pairs with a distance of 1 to calculate the co-occurrence relationship. The calculation is also set in four directions: 0°, 45°, 90°, and 135° to obtain the crack parameters. The crack parameters include contrast, dissimilarity, homogeneity, energy, correlation, entropy, mean, and variance.

[0015] Preferably, each cookie parameter in step S5 further includes a crack proportion parameter and a crack uniformity parameter; when calculating the crack proportion parameter, the number of pixels with a value of 1 in any crack region of the crack mask is denoted as N. crack Let N be the number of pixels with a value of 1 within the main body region of the main body segmentation dataset corresponding to any crack region. cookie Let R be the parameter representing the crack ratio. When calculating the crack uniformity parameter, a mesh partitioning method is used, dividing each biscuit region into 10×10 equal-area sub-mesh. Within each equal-area sub-mesh, the ratio of crack pixels to the total number of biscuit pixels is calculated and used as the corresponding crack proportion value. Specifically, within any biscuit region, the crack proportion value of the i-th equal-area sub-mesh is denoted as... Let the total number of equal-area sub-mesh be n=100, the crack proportion of equal-area sub-mesh be μ, the standard deviation of the crack proportion be σ, and the crack uniformity parameter be CV.

[0016] .

[0017] The beneficial effects of this invention compared to the prior art are:

[0018] The technical solution of this invention firstly performs precise segmentation of the main body region of the biscuit based on the U-Net network, and then automatically extracts the length, width, area, perimeter, color features and texture parameters of the biscuit using contour extraction and morphological analysis. Subsequently, a SegFormer model based on the Transformer architecture is introduced to identify cracks on the biscuit surface at high resolution, and the uniformity and spatial features of the cracks are quantified by combining skeleton extraction and distribution analysis. This method has advantages such as no human intervention, strong adaptability, fast detection speed and strong crack detail recovery ability, providing efficient, objective and quantifiable technical support for biscuit quality assessment, and has good prospects for industrial application. Attached Figure Description

[0019] Figure 1 A flowchart of an image-based intelligent analysis method for biscuit processing quality;

[0020] Figure 2 This is a schematic diagram of an original image;

[0021] Figure 3 This is a schematic diagram illustrating the morphological characteristics of a type of biscuit.

[0022] Figure 4 This is a schematic diagram illustrating color transitions in a cookie area.

[0023] Figure 5 Schematic diagram of cracks and their uniformity in different types of biscuits;

[0024] Figure 6 Texture feature maps for different types of cookies;

[0025] Figure 7 This is a schematic diagram showing the cracks and their uniformity in different types of cookies. Detailed Implementation

[0026] The following will be combined with the present invention Figures 1 to 7 The technical solutions in the embodiments of the present invention will be described in detail below.

[0027] like Figure 1 The diagram shows a flowchart of an image-based intelligent analysis method for biscuit processing quality. It uses a precise segmentation method to distinguish the biscuit area from the background, and then combines it with a biscuit crack segmentation model to achieve high-precision identification of cracks on the biscuit surface. At the same time, it uses multiple calculation methods to calculate the parameters of each biscuit, which can reflect the overall quality of the biscuit in a comprehensive, intuitive and reliable manner.

[0028] An image-based intelligent analysis method for biscuit processing quality specifically includes the following steps:

[0029] S1. Based on multiple original images, use an image annotation tool (such as Labelme tool) to instantiate and annotate each cookie in each original image to generate each mask image containing multiple foreground regions. At the same time, record each cookie in each mask image as 1 and the background as 0 for differentiation.

[0030] This example uses image data of biscuits made from 80 wheat varieties, totaling 80 original RGB images, such as... Figure 2 The image shown is a schematic diagram of a set of original images, each containing six cookies. Each original image was captured under consistent shooting conditions, using a standard background and uniform lighting, and placed within the shooting scene. Figure 2 The scale shown serves as a size reference for subsequent pixel-to-actual-size conversions. Each original image has a resolution of 3800×3000 pixels.

[0031] Based on each mask image, a semantic segmentation model is trained to extract the image data of the complete region of each cookie and aggregate them as the main body segmentation dataset. The image data in the main body segmentation dataset is divided into training, validation, and test sets in an 8:1:1 ratio to ensure that the trained semantic segmentation model has good generalization ability.

[0032] S2. Based on the main body segmentation dataset from step S1, each cookie region in each original image is cropped to obtain the first cropped data, totaling 480 cropped images. The crack regions in the first cropped data are then labeled pixel-by-pixel using an image annotation tool to obtain a set of crack images, which is the crack segmentation dataset. During pixel-by-pixel annotation, a single-channel mask is used, labeling only each surface crack region as a foreground, while labeling all other regions outside the foreground as background. The pixel size of each foreground is kept constant, and each foreground is used as a crack image. Furthermore, each crack image is uniformly 1024×1024 pixels to preserve sufficient structural details.

[0033] In addition, in the crack segmentation dataset, each crack image is divided into a training set, a validation set, and a test set in an 8:1:1 ratio.

[0034] S3. Using the main body segmentation dataset from step S1 and the crack segmentation dataset from step S2 as input data, construct a cookie segmentation model based on U-Net. The construction process includes: normalizing the input data in sequence, tensor form transformation, first multi-scale feature extraction, and outputting the final segmentation mask result.

[0035] In this embodiment, when constructing the U-Net-based cookie segmentation model, the pixel values ​​of each image in the input data are first scaled to the [0, 1] range and their size is uniformly adjusted. Then, each image after uniform size adjustment is converted into tensor form and input into the cookie segmentation model. Next, in the encoder of the cookie segmentation model, downsampling operations are performed sequentially through four downsampling modules. Each downsampling module consists of two consecutive 3×3 convolutional layers, and a ReLU activation function is used for nonlinear transformation after each convolution.

[0036] Then, four sampling modules are used to obtain each feature map corresponding to each image. Each downsampling module also performs spatial downsampling on each feature map through a 2×2 window max pooling operation, which halves the size of each feature map and doubles the number of channels, thereby realizing multi-scale feature extraction from low-level details to high-level semantics, and thus obtaining each multi-scale feature. Figure 1 At the bottleneck layer of the cookie segmentation model, each multi-scale feature is further compressed. Figure 1 The context information is fused using 3×3 convolution to improve the ability to perceive the global structure; then each fused result is transmitted to the decoder of the cookie segmentation model for decoding to obtain the final segmentation mask result.

[0037] Decoder 1 comprises multiple decoding modules. During decoding, each module first undergoes transposed convolution to perform upsampling, improving resolution. Then, each upsampling result is concatenated with a corresponding feature map from encoder 1 via skip connections, simultaneously utilizing deep semantic information and shallow boundary details to obtain each stitched image. Next, each stitched image is compressed to 1 channel using a 1×1 convolution to generate a single-channel pixel-level probability map, representing the probability distribution of each pixel belonging to the "cookie" category. Each pixel-level probability map is then mapped to the [0,1] interval using a Sigmoid activation function and binarized with a set threshold to obtain the final segmentation mask result. The threshold can be set by the user according to actual needs, and will not be elaborated here.

[0038] S4. The corresponding region of each mask image from step S1 in each original image is further cropped to obtain second cropped data. This second cropped data is then input into the SegFormer-based cookie crack segmentation model to output crack segmentation results and generate a crack mask. This process includes preprocessing, a second multi-scale feature extraction, layer fusion, crack probability prediction, outputting a crack probability map, and generating the crack mask. SegFormer, or segmentation architecture, is a lightweight and efficient segmentation model.

[0039] After the second cropped data is input into the SegFormer-based biscuit crack segmentation model, it is first preprocessed by normalization and size adjustment. Each preprocessed image is then transmitted to encoder 2 in the biscuit crack segmentation model for hybrid encoding. Each multi-level feature map is output to decoder 2 in the biscuit crack segmentation model. Decoder 2 uses a multilayer perceptron to upsample each multi-level feature map to the same size and perform layer fusion. The layer fusion result is used to predict the crack probability of each pixel through pointwise convolution, and then the corresponding crack probability map is output.

[0040] The hybrid coding operation includes: 1) First, each preprocessed image is divided into 7×7 local blocks through overlapping block embedding. Each local block is linearly projected into an embedding vector to reduce the computational cost while preserving the continuity between adjacent blocks; 2) Each embedding vector is input into each hybrid architecture coding block in the hierarchical design of encoder 2, and each multi-scale feature is generated through progressive downsampling in 4 stages. Figure 2 And preserve the global context relationship, for each multi-scale feature Figure 2 The data is transmitted to decoder 2; each hybrid architecture coding block is equipped with a self-attention mechanism and a multilayer perceptron.

[0041] S5. Based on the scale in step S1, determine the conversion relationship between pixels in each original image and the actual physical size, and then calculate the parameters of each cookie, including: based on the conversion relationship, calculate the morphological features of each cookie in each cookie region; then convert the color of each cookie region from RGB (red, green, blue) to CIELAB (luminance-red-green-yellow-blue color space), and calculate the color mean parameter; based on the gray-level co-occurrence matrix, extract the texture parameters from the complete cookie region in step S1.

[0042] When confirming the conversion relationship, the actual length corresponding to each pixel of each original image is denoted as real_pixel in centimeters. , This indicates the actual width of the scale, expressed in centimeters. This indicates the pixel scale of the scale in each original image.

[0043] like Figure 3 The diagram shows the shape characteristics of a cookie. Green represents the perimeter, red borders represent the length or width, and pink areas represent the area. Each cookie's shape characteristics include area A, perimeter P, roundness C, length L1 of the smallest bounding rectangle, and width W1 of the smallest bounding rectangle. N is the total number of pixels for each cookie region; , The pixel perimeter is obtained based on the 8-connected boundary tracking algorithm; 0 <C≤1; , and These represent the maximum and minimum coordinates of the smallest bounding rectangle in the horizontal direction, respectively. , and These represent the maximum and minimum coordinates of the smallest bounding rectangle in the vertical direction, respectively.

[0044] like Figure 4 The image shows a schematic diagram of color transformation in a cookie area, illustrating the color comparison of a cookie area under the original RGB, L channel (luminance channel), A channel (red-green channel), and B channel (yellow-blue channel). Specifically, when calculating the color mean parameter, , L2 represents the color mean parameter, A2 represents the red-green axis, B2 represents the yellow-blue axis, the overline of L2, A2 and B2 all represent the mean, N represents the total number of pixels in the corresponding cookie region, i=1, 2, 3, ..., N.

[0045] Extracting texture parameters is crucial for further analysis of the fineness and texture variations of the cookie surface. Specifically, a sliding window with a size of 5 and a step size of 4 can be set. This window extracts each image patch and crack mask from each original image, while retaining the effective center pixel of each sliding window. Image patches, also known as patch regions, decompose each relatively large original image into numerous local units, facilitating crack identification. Then, the `skimage.feature.graycomatrix` function in Python (suitable for calculating the gray-level co-occurrence matrix) is used to calculate co-occurrence relationships between neighboring pixel pairs with a distance of 1. Calculations are performed in four directions: 0°, 45°, 90°, and 135°, comprehensively capturing pixel co-occurrence patterns in the horizontal, upper right to lower left, vertical, and upper left to lower right directions, effectively obtaining the corresponding crack parameters. These crack parameters include contrast, dissimilarity, homogeneity, energy, correlation, entropy, mean, and variance. Furthermore, the energy parameter can be represented by the square of the energy value.

[0046] It should be noted that contrast ratio measures the intensity of gray-level differences between pixels; a higher value indicates a more pronounced contrast in the image. Dissimilarity is similar to contrast ratio, but measures the degree of gray-level change in absolute difference. Homogeneity measures the concentration of elements near the main diagonal; a higher value indicates a smoother gray-level change. Energy represents the stability and repeatability of image texture. The square of energy represents the regularity of the image's gray-level distribution. Correlation measures the degree of linear correlation between gray-level values ​​in an image. Entropy measures the complexity and information content of the texture; a higher value indicates a more chaotic gray-level distribution. Mean reflects the overall brightness level of the image's gray-level values. Variance reflects the overall dispersion of the image's gray-level values. Specifically, the calculation formulas for various crack parameters such as contrast ratio are already well-known and will not be elaborated upon here.

[0047] like Figure 5 The diagram shown illustrates different types of cracks in cookies and their uniformity. Figure 6 The image shown is a texture feature map of different types of cookies; as shown... Figure 7 The diagram shown illustrates the crack patterns and uniformity of different types of cookies; combined with... Figure 5 , Figure 6 and Figure 7 As shown, each multi-scale feature Figure 2 Both include shallow and deep features; before predicting the crack probability, decoder 2, based on a self-attention mechanism and a multilayer perceptron, performs multi-scale feature analysis on each feature. Figure 2 Shallow features capture texture differences for each multi-scale feature. Figure 2 The deep features are used to establish long-range dependencies to distinguish cracks and texture features, which is consistent with the characteristic that cracks are usually long, thin, and discontinuous. Each cookie parameter also includes a crack proportion parameter and a crack uniformity parameter; when calculating the crack proportion parameter, the number of pixels with a value of 1 in any crack region in the crack mask is denoted as N. crack Let N be the number of pixels with a value of 1 within the main body region of the main body segmentation dataset corresponding to any crack region. cookie Let R be the parameter representing the crack ratio. .

[0048] When calculating the crack uniformity parameter, a mesh partitioning method is used, dividing each biscuit region into 10×10 equal-area sub-mesh. Within each equal-area sub-mesh, the ratio of crack pixels to the total number of pixels in the biscuit region is calculated and used as the corresponding crack proportion value. Specifically, within any biscuit region, the crack proportion value of the i-th equal-area sub-mesh is denoted as... Let the total number of equal-area sub-mesh be n=100, the crack proportion of equal-area sub-mesh be μ, the standard deviation of the crack proportion be σ, and the crack uniformity parameter be CV. .

[0049] In summary, this invention uses the U-Net network to accurately segment the main body of the biscuit, and further utilizes contour extraction and morphological analysis to automatically extract the length, width, area, perimeter, color features, and texture parameters of the biscuit. Subsequently, a SegFormer model based on the Transformer architecture is introduced to identify cracks on the biscuit surface at high resolution, and combined with skeleton extraction and distribution analysis, the uniformity and spatial characteristics of the cracks are quantified. This method has advantages such as no human intervention, strong adaptability, fast detection speed, and strong crack detail recovery capability, providing efficient, objective, and quantifiable technical support for biscuit quality assessment, with good prospects for industrial application and significant progress.

[0050] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. An image-based intelligent analysis method for biscuit processing quality, characterized in that, Includes the following steps: S1. Based on multiple original images, each cookie in each original image is instantiated and labeled using an image annotation tool to generate each mask image containing multiple foreground regions; a semantic segmentation model is trained based on each mask image to extract the complete cookie region and use it as the main body segmentation dataset; each original image is taken under the same background and lighting conditions, and a ruler is set in the background as a reference. S2. Based on the main body segmentation dataset in step S1, each cookie region in each original image is cropped to obtain the first cropped data; the crack region in the first cropped data is labeled pixel by pixel using an image annotation tool to obtain the crack segmentation dataset. S3. Using the main body segmentation dataset from step S1 and the crack segmentation dataset from step S2 as input data, construct a cookie segmentation model based on U-Net. The construction process includes: normalizing the input data in sequence, tensor form transformation, first multi-scale feature extraction, and outputting the final segmentation mask result. In constructing the U-Net-based cookie segmentation model, each image in the input data is first scaled to the [0, 1] range and its size is uniformly adjusted. Then, each image after uniform size adjustment is converted into a tensor and input into the cookie segmentation model. Next, in the encoder of the cookie segmentation model, downsampling operations are performed sequentially through four downsampling modules. Each downsampling module consists of two consecutive 3×3 convolutional layers, and a ReLU activation function is used for non-linear transformation after each convolution. Each feature map corresponding to each image is obtained using the four sampling modules. Each downsampling module also performs spatial downsampling on each feature map using a 2×2 window max pooling operation, halving the size of each feature map and doubling the number of channels to obtain each multi-scale feature map. Then, in the bottleneck layer of the cookie segmentation model, each multi-scale feature map is further compressed and context information is fused using 3×3 convolution. Each fused result is transmitted to the decoder of the cookie segmentation model for decoding to obtain the final segmentation mask result. S4. Cropping the corresponding region of each mask image in each original image in step S1 to obtain the second cropping data; inputting the second cropping data into the biscuit crack segmentation model based on SegFormer, and continuing to perform preprocessing, second multi-scale feature extraction, layer fusion, predicting crack probability, outputting crack probability map and generating crack mask. In this process, the second cropped data is input into the SegFormer-based biscuit crack segmentation model. Normalization and resizing preprocessing are performed on each image. Each preprocessed image is then transmitted to encoder 2 in the biscuit crack segmentation model for hybrid encoding. Each multi-level feature map is output to decoder 2 in the same model. Decoder 2 uses a multilayer perceptron to upsample each multi-level feature map to the same size and performs layer fusion. The fusion result is used to predict the crack probability of each pixel through pointwise convolution, and then the corresponding crack probability map is output. The hybrid encoding operation includes: first, dividing each preprocessed image into 7×7 local blocks using overlapping block embedding; each local block is linearly projected into an embedding vector; then, each embedding vector is input into each layered hybrid architecture encoding block in encoder 2, and each multi-scale feature map 2 is generated through four stages of progressive downsampling while preserving global context; each multi-scale feature map 2 is then transmitted to decoder 2. Each hybrid architecture encoding block incorporates a self-attention mechanism and a multilayer perceptron. S5. Based on the scale in step S1, determine the conversion relationship between pixels in each original image and actual physical size, and then calculate the parameters of each cookie. The calculation process includes: based on the conversion relationship, obtaining the morphological features of each cookie in each cookie region; converting the color of each cookie region from RGB to CIELAB color space and obtaining the color mean parameter; and extracting the texture parameter from the complete cookie region in step S1 based on the gray-level co-occurrence matrix.

2. The image-based intelligent analysis method for biscuit processing quality according to claim 1, characterized in that, Each original image in step S1 contains multiple cookies, and each cookie in each mask image is recorded as 1 and the background is recorded as 0.

3. The image-based intelligent analysis method for cookie processing quality according to claim 1, characterized in that, The main body segmentation dataset in step S1 and the crack segmentation dataset in step S2 are both divided into training set, validation set and test set in a ratio of 8:1:

1.

4. The image-based intelligent analysis method for biscuit processing quality according to claim 1, characterized in that, In step S2, when performing pixel-by-pixel annotation, a single-channel mask is used to label each surface crack region as a foreground, while each other region outside each foreground is labeled as a background. The pixel size of each foreground is controlled to be the same, and the set of each foreground is used as the crack segmentation dataset.

5. The image-based intelligent analysis method for biscuit processing quality according to claim 1, characterized in that, Decoder 1 includes multiple decoding modules. When decoding, Decoder 1 first performs transposed convolution on each decoding module to complete the upsampling operation. Then, each result of the upsampling operation is concatenated with each feature map in the corresponding encoder 1 through skip connections to obtain each concatenated image. Each concatenated image is then compressed to 1 channel by a 1×1 convolution to generate a single-channel pixel-level probability map. Each pixel-level probability map is mapped to the [0,1] interval by the Sigmoid activation function and binarized with a set threshold to obtain the final segmentation mask result.

6. The image-based intelligent analysis method for biscuit processing quality according to claim 1, characterized in that, Each multi-scale feature map 2 includes shallow features and deep features. Before predicting the crack probability, the decoder 2 uses a self-attention mechanism and a multilayer perceptron to capture texture differences in the shallow features of each multi-scale feature map 2 and establish long-distance dependencies in the deep features of each multi-scale feature map 2 to distinguish between cracks and textures.

7. The image-based intelligent analysis method for biscuit processing quality according to claim 1, characterized in that, In step S5, when confirming the conversion relationship, the actual length corresponding to each pixel of each original image is recorded as real_pixel in centimeters. , This indicates the actual width of the scale, expressed in centimeters. This indicates the pixel scale of the scale in each original image; The morphological characteristics of each cookie include area A, perimeter P, roundness C, length L1 of the smallest bounding rectangle, and width W1 of the smallest bounding rectangle. N is the total number of pixels for each cookie region; , The pixel perimeter is obtained based on the 8-connected boundary tracking algorithm; 0 <C≤1; , and These represent the maximum and minimum coordinates of the smallest bounding rectangle in the horizontal direction, respectively. , and These represent the maximum and minimum coordinates of the minimum bounding rectangle in the vertical direction, respectively. When calculating the color mean parameter, , L2 represents the color mean parameter, A2 represents the red-green axis, B2 represents the yellow-blue axis, the overline of L2, A2 and B2 all represent the mean, N represents the total number of pixels in the corresponding cookie region, i=1, 2, 3, ..., N; When extracting texture parameters, a sliding window with a size of 5 and a step size of 4 is set. The sliding window is used to extract the corresponding crack mask from each original image, and each sliding window with a valid center pixel is retained. Then, the skimage.feature.graycomatrix function in Python is used to select neighboring pixel pairs with a distance of 1 to calculate the co-occurrence relationship. The calculation is also set in four directions: 0°, 45°, 90° and 135° to obtain the crack parameters. The crack parameters include contrast, dissimilarity, homogeneity, energy, correlation, entropy, mean and variance.

8. The image-based intelligent analysis method for biscuit processing quality according to claim 1, characterized in that, Each cookie parameter in step S5 also includes a crack percentage parameter and a crack uniformity parameter; When calculating the crack proportion parameter, the number of pixels with a value of 1 in any crack region of the crack mask is denoted as N. crack Let N be the number of pixels with a value of 1 within the main body region of the main body segmentation dataset corresponding to any crack region. cookie Let R be the parameter representing the crack ratio. ; When calculating the crack uniformity parameter, a mesh partitioning method is used, dividing each biscuit region into 10×10 equal-area sub-mesh. Within each equal-area sub-mesh, the ratio of crack pixels to the total number of pixels in the biscuit region is calculated and used as the corresponding crack proportion value. Specifically, within any biscuit region, the crack proportion value of the i-th equal-area sub-mesh is denoted as... Let the total number of equal-area sub-mesh be n=100, the crack proportion of equal-area sub-mesh be μ, the standard deviation of the crack proportion be σ, and the crack uniformity parameter be CV. .

Images