Dough impurity detection system and method based on image recognition

Abstract

The invention discloses a dough impurity detection system and method based on image recognition, and relates to the technical field of food detection, and the method comprises the steps: fusing a color difference feature and a brightness gradient feature, and generating a first fusion feature vector; determining a background region according to the first fusion feature vector, generating a second fusion feature vector of the background region through sparse representation training, and performing background differential correction on the original dough image according to the second fusion feature vector to obtain a dough image after background correction; performing multi-scale image enhancement processing on the corrected dough image to obtain an enhanced dough image; performing threshold segmentation on the enhanced dough image, extracting a candidate impurity region, and inputting morphological characteristics of the candidate impurity region into a convolutional neural network model to obtain a target impurity region; according to the method, through feature fusion, background difference correction and multi-scale enhancement, the accuracy, robustness and automation degree of dough surface impurity detection are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of food testing technology, specifically to a dough impurity detection system and method based on image recognition. Background Technology

[0002] In the food industry, especially on automated dough production lines, strict control of dough quality is crucial. Impurities in the dough (such as discolored particles, non-food materials, burnt or contaminated debris) not only affect the safety and quality of the final food product but can also damage equipment and, in severe cases, lead to food safety incidents. Therefore, the rapid and accurate detection and location of impurities on the dough surface is a significant technical challenge in the food industry.

[0003] Traditional impurity detection methods typically rely on manual visual inspection or simple machine vision techniques. However, manual visual inspection is limited by the fatigue and subjectivity of the inspectors, resulting in significant instability and the risk of false positives and false negatives. Conventional machine vision methods usually identify impurity areas through simple threshold segmentation, background subtraction, or color space conversion. However, due to the uneven color distribution on the dough surface and the small difference in features between impurities and background areas, these traditional methods often struggle to effectively distinguish between impurity and background areas, leading to insufficient detection accuracy.

[0004] Specifically, the color and texture variations on the surface of dough are subtle, especially under uneven lighting conditions in actual industrial environments. These subtle changes are difficult to capture effectively using existing simple image processing methods. Meanwhile, background modeling and correction techniques are typically based on simple subtraction using static or single-color features, which is ill-suited to the complex and ever-changing industrial landscape. Furthermore, most existing multi-scale image enhancement methods employ only single-scale or simple filtering techniques, offering limited enhancement of the subtle texture features on the dough surface, further impacting the accuracy of impurity region extraction.

[0005] Therefore, there is still an urgent need for a more effective, stable and accurate method for detecting impurities on the surface of dough, which can more sensitively capture the subtle differences in features between impurities and background areas, and improve the accuracy and robustness of impurity area detection. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for detecting dough impurities based on image recognition, so as to solve the problems in the background art mentioned above.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] In a first aspect, the present invention provides a dough impurity detection method based on image recognition, comprising:

[0009] Acquire an original image of the dough surface to be tested, and divide the original dough image into multiple local regions;

[0010] Calculate the covariance matrix of different color channels in each local region, and perform eigenvalue decomposition on the covariance matrix to obtain color difference features that characterize the color differences in each local region.

[0011] Obtain the brightness gradient features of each local region, and fuse the color difference features and brightness gradient features of each local region to generate the first fused feature vector;

[0012] The background region is determined based on the first fusion feature vector, and a second fusion feature vector of the background region is generated through sparse representation training. The original dough image is then subjected to background difference correction based on the second fusion feature vector of the background region to obtain a background-corrected dough image.

[0013] The background-corrected dough image is subjected to multi-scale image enhancement processing to obtain an enhanced dough image;

[0014] Thresholding segmentation is performed on the enhanced dough image to extract candidate impurity regions. The morphological features of the candidate impurity regions are then input into a pre-trained convolutional neural network model to obtain the target impurity regions.

[0015] Secondly, the present invention provides a dough impurity detection system based on image recognition, implemented based on the aforementioned dough impurity detection method based on image recognition, comprising:

[0016] The acquisition module is used to acquire an original dough image of the surface of the dough to be tested, and to divide the original dough image into multiple local regions;

[0017] The decomposition module is used to calculate the covariance matrix of different color channels in each local region, and to perform eigenvalue decomposition on the covariance matrix to obtain color difference features that characterize the color differences in each local region.

[0018] The fusion module is used to obtain the brightness gradient features of each local region, and fuse the color difference features and brightness gradient features of each local region to generate the first fused feature vector;

[0019] The correction module is used to determine the background region based on the first fusion feature vector, and generate a second fusion feature vector of the background region through sparse representation training. The original dough image is then subjected to background difference correction based on the second fusion feature vector of the background region to obtain a background-corrected dough image.

[0020] The enhancement module is used to perform multi-scale image enhancement processing on the background-corrected dough image to obtain an enhanced dough image;

[0021] The detection module is used to perform threshold segmentation on the enhanced dough image, extract candidate impurity regions, and input the morphological features of the candidate impurity regions into a pre-trained convolutional neural network model to obtain the target impurity regions.

[0022] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0023] This invention addresses the problem that existing technologies struggle to effectively distinguish between impurities and background areas on the surface of dough, resulting in insufficient feature representation of impurity areas and low detection accuracy. It proposes a method that obtains color difference features by nonlinearly adjusting color channel values, calculating the covariance matrix and its eigenvalue decomposition, and combining this with brightness gradient direction entropy. This significantly improves the effectiveness of feature differentiation between impurity and non-impurity areas, enhancing detection sensitivity.

[0024] To further address the problem that impurity regions in existing technologies are severely affected by background interference and are difficult to distinguish accurately, this paper proposes an automatic determination of background regions based on a fusion feature modulus threshold. The baseline features of the background regions are obtained by using sparse representation and dictionary learning, which are then used for differential correction of the original image. This effectively suppresses background interference, achieves accurate extraction of impurity region features, and significantly improves the stability and reliability of dough impurity detection.

[0025] Furthermore, this invention solves the problem of weak impurity features being easily ignored and insufficient detection accuracy in the prior art by designing a multi-scale Laplacian pyramid nonlinear enhancement method, achieving full enhancement of texture and edge details of impurity regions at different scales; and combines convolutional neural networks to automatically classify and identify the morphological features of candidate impurity regions, effectively improving the automation level and industrial applicability of impurity region detection, and meeting the actual production needs for accuracy and efficiency in dough impurity detection. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0027] Figure 1 This is a flowchart of an image recognition-based dough impurity detection method according to the present invention;

[0028] Figure 2 This is a framework diagram of an image recognition-based dough impurity detection system according to the present invention. Detailed Implementation

[0029] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided to make the description of this application more complete and comprehensive, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The drawings are merely illustrative illustrations of this application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0030] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more exemplary embodiments. Numerous specific details are provided in the following description to give a full understanding of the exemplary embodiments disclosed in this application. However, those skilled in the art will recognize that the technical solutions disclosed in this application can be practiced with one or more specific details omitted, or other methods, components, steps, etc., can be employed. In other instances, well-known structures, methods, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the disclosure of this application.

[0031] Example 1

[0032] like Figure 1 As shown, this embodiment discloses a dough impurity detection method based on image recognition, including:

[0033] S101: Obtain the original dough image of the surface of the dough to be tested, and divide the original dough image into multiple local regions;

[0034] It should be noted that the original dough image on the surface of the dough to be tested refers to the digital image data directly acquired by an industrial camera without any processing. For example, an RGB channel color digital image obtained by taking a picture with an industrial-grade RGB camera with a resolution of 2048×1536 pixels.

[0035] It should also be noted that the method of dividing the original dough image data into multiple local regions is the conventional rectangular grid division method, that is, dividing the image data into equal horizontal and vertical dimensions to obtain multiple non-overlapping rectangular regions that completely cover the entire original dough image; for example, for the original dough image data with a size of 2048×1536 pixels, it is divided into 16 regions horizontally and 12 regions vertically, resulting in 192 rectangular local regions with completely equal areas;

[0036] It should be understood that multiple local areas are determined in advance by human intervention based on the actual needs of impurity detection in the dough to be tested. The more areas there are and the smaller the area scale, the higher the corresponding detection accuracy. For example, in actual industrial applications, the area size is usually determined by a combination of factors such as dough conveying speed, impurity size requirements, or industrial camera resolution to ensure that the detection accuracy meets the actual production needs.

[0037] S102: Calculate the covariance matrix of different color channels in each local region, and perform eigenvalue decomposition on the covariance matrix to obtain the color difference features that characterize the color differences in each local region; this can more sensitively and stably characterize the difference intensity between different color channels, which is beneficial to make the color difference features between the dough impurity region and the non-impurity region more significant.

[0038] In implementation, calculating the covariance matrix of different color channels within each local region includes:

[0039] Calculate the degree of deviation of the pixel value of each color channel within each local region from the mean of that color channel;

[0040] d i,c =|x i,c -μ c |;

[0041] In the formula: d i,c The degree of deviation;

[0042] It should be noted that the deviation of color channel values ​​refers to the absolute difference between the value of a single color channel and the mean value of that color channel for each pixel in a local area.

[0043] The non-linear mapping intensity parameter of the color channel corresponding to each local region is determined based on the degree of deviation of pixel values;

[0044] Specifically, determining the nonlinear mapping intensity parameter for the color channel corresponding to each local region includes:

[0045] Calculate the standard deviation of the pixel values ​​for each color channel within each local region;

[0046] For example, for color channel c, the standard deviation σ c The calculation formula is clearly defined as follows:

[0047]

[0048] In the formula: c∈{R,G,B} represents the red, green, and blue channels; x i,c Let be the pixel value of the i-th pixel in color channel c; N is the total number of pixels in the local region;

[0049] The standard deviation is proportional to the mean of the color channel to obtain the ratio coefficient between the standard deviation and the mean.

[0050] The formula for proportional calculation is as follows:

[0051]

[0052] In the formula: k c ε is the proportionality constant; ε is a very small positive number (e.g., 10). -6 This is used to avoid the denominator being zero;

[0053] Based on the scaling factor, the nonlinear mapping intensity parameter of the color channel is determined;

[0054] It is worth noting that the nonlinear mapping intensity parameter is used for nonlinear adjustment of subsequent color channel values;

[0055] For example, the formula for calculating the nonlinear mapping intensity parameter of the i-th pixel on color channel c is:

[0056] γ i,c =1+k c ;

[0057] In the formula: γ i,c For nonlinear mapping intensity parameters;

[0058] The color channel values ​​of each pixel are non-linearly adjusted using a determined non-linear mapping intensity parameter to obtain the non-linearly adjusted color channel values.

[0059] It should be noted that adjusting the color channel values ​​through the nonlinear mapping intensity parameter aims to further highlight the differences within the color channels, which is helpful for the subsequent extraction of impurity region features.

[0060] For example, the formula for calculating the adjusted color channel value of the i-th pixel on color channel c is:

[0061]

[0062] In the formula: y i,c For color channel values, the above formula uses normalized pixel values, x i,c The value range is [0,1];

[0063] Understandably, in the above calculation, the pixel color value x i,c It has been explicitly normalized to the interval [0,1] to ensure the effectiveness of nonlinear mapping operations; for example, if the color channel value is 128 (original value range 0-255), the normalized color channel value is 128 / 255≈0.502;

[0064] The covariance matrix is ​​calculated using the non-linearly adjusted color channel values.

[0065] It should be noted that this step is based on the non-linearly adjusted color channel values ​​to calculate the covariance between each color channel, in order to reflect the more prominent differences between the color channels after non-linear adjustment.

[0066] For example, the covariance matrix C of the non-linearly adjusted color channel values ​​is defined as a 3x3 matrix, and the calculation formula is as follows:

[0067]

[0068] in:

[0069]

[0070] In the formula: m,n∈{R,G,B} represent the red, green, and blue channels; y i,m and y i,n Let m and n represent the non-linearly adjusted color channel values ​​of the i-th pixel, respectively. and Let m and n represent the mean values ​​of all pixels in color channel m after non-linear adjustment, respectively.

[0071] For example, the The calculation formula is: same The computational logic, The logic for obtaining it is the same, and will not be repeated here;

[0072] In implementation, the eigenvalue decomposition of the covariance matrix includes:

[0073] The covariance matrix of each local region is decomposed into eigenvalues ​​to obtain three eigenvalues ​​λ1, λ2 and λ3.

[0074] It should be noted that covariance matrix eigenvalue decomposition refers to performing mathematical decomposition on each 3×3 covariance matrix to obtain eigenvalues ​​that explicitly represent the characteristics of the matrix.

[0075] For example, the eigenvalue decomposition of a single covariance matrix C is explicitly performed, satisfying the following mathematical formula:

[0076] C×V=V×Λ;

[0077] In the formula: V is the eigenvector matrix; Λ is the diagonal matrix composed of eigenvalues; where, The eigenvalues ​​satisfy the descending order relation λ max ≥λ mid ≥λ min ;λ max λ is the largest eigenvalue. mid λ is an intermediate eigenvalue. min It is the smallest eigenvalue;

[0078] Sort the three eigenvalues ​​in descending order to obtain the sorted eigenvalues ​​λ. max , λ mid and λ min And calculate λ max With λ mid and λ min The ratio of the sum of these values ​​serves as the color difference characteristic of each local region.

[0079] It is worth noting that this step explicitly uses the ratio of the largest eigenvalue to the sum of the other two eigenvalues ​​as the color difference feature of each local region to highlight the significance of color channel differences.

[0080] The formula for calculating the proportion is as follows:

[0081]

[0082] In the formula; F color ε is a proportional value, representing the color difference characteristic; ε is a very small positive number (e.g., 10). -6 This is used to avoid the denominator being zero;

[0083] Understandably, the larger the calculated color difference feature value, the more significant the difference between color channels in a local area, which plays a positive role in accurately distinguishing impurity areas from non-impurity areas.

[0084] S103: Obtain the brightness gradient features of each local region, and fuse the color difference features and brightness gradient features of each local region to generate the first fused feature vector;

[0085] In implementation, obtaining the brightness gradient features of each local region includes:

[0086] The original dough image is converted to grayscale to obtain a grayscale image;

[0087] Specifically, the grayscale image conversion formula is:

[0088] I gray (x,y)=0.299×I R (x,y)+0.587×I G (x,y)+0.114×I B (x,y);

[0089] In the formula: I gray (x,y) represents the grayscale value, I R (x,y), I G (x,y) and I B (x, y) represent the pixel values ​​of the red, green, and blue channels, respectively;

[0090] The Sobel operator is used to calculate the gradient magnitude and gradient direction of each pixel in the grayscale image;

[0091] Specifically, the gradient magnitude and gradient direction are:

[0092]

[0093] In the formula: G(x,y) is the gradient magnitude, θ(x,y) is the gradient direction, and G... x (x,y) represents the gradient intensity of the pixel at position (x,y) in the horizontal (X-axis) direction, G y (x,y) represents the gradient intensity of the pixel at position (x,y) in the vertical (Y-axis) direction;

[0094] Constructing the gradient direction histogram for each local region specifically includes:

[0095] range of gradient direction Divide into R equal sub-intervals;

[0096] Count the number of pixels whose gradient direction falls into each sub-interval within each local region to obtain a gradient direction histogram;

[0097] For example, the gradient direction histogram is as follows:

[0098]

[0099] Where: n i The number of pixels falling into the i-th sub-interval, where N is the total number of pixels in the local region;

[0100] The nonlinear local gradient direction entropy is calculated based on the gradient direction histogram, and this local gradient direction entropy is used as a feature of the brightness gradient. The specific calculation formula is as follows:

[0101]

[0102] In the formula: E gard Here, ε is the local gradient direction entropy value, and ε is a minimum positive number (e.g., 10). -6 ), where log is the logarithmic function;

[0103] It should be noted that the first fused feature vector is obtained by directly concatenating and fusing the color difference feature and the brightness gradient feature;

[0104] For example, for the k-th local region, the fused feature vector is:

[0105]

[0106] In the formula: Indicates color difference characteristics. This represents the brightness gradient feature.

[0107] S104: Determine the background region based on the first fusion feature vector, and generate the second fusion feature vector of the background region through sparse representation training. Perform background difference correction on the original dough image based on the second fusion feature vector of the background region to obtain the background-corrected dough image.

[0108] It should be noted that this step explicitly uses the first fusion feature vector magnitude to automatically determine the background region, so as to ensure that the determined background region accurately reflects the features of the background part in the dough image;

[0109] In implementation, determining the background area includes:

[0110] Calculate the magnitude of the fused feature vector for each local region to obtain the set of magnitudes for all local regions;

[0111] The formula for calculating the modulus length is as follows:

[0112]

[0113] In the formula: L K f is the modulus length. j,k This represents the specific value of the j-th feature component in the fused feature vector of the k-th local region;

[0114] For example, for the k-th local region, its first fused feature vector is: The formula for calculating the modulus is as follows: Among them, f 1,k For the color difference feature of the k-th local region, f 2,k The brightness gradient feature of the k-th local region;

[0115] Based on the set of modulus lengths, calculate the average and standard deviation of the fused feature vector modulus lengths, and then set the average μ... L and standard deviation σ L The difference is used as the modulus threshold T = μ L -σ L ;

[0116] The formula for the average value of the modulus is as follows:

[0117]

[0118] The formula for the standard deviation of the modulus is as follows:

[0119]

[0120] Local regions whose fused feature vector magnitude is less than the magnitude threshold are defined as background regions;

[0121] In implementation, the step of generating the second fused feature vector of the background region through sparse representation training includes:

[0122] Multiple first fusion feature vectors are extracted from the defined background region to form a background feature sample set;

[0123] Based on the aforementioned background feature sample set, a background feature dictionary is trained using a sparse representation dictionary learning algorithm.

[0124] It should be noted that the sparse representation dictionary learning algorithm adopts the K-SVD algorithm. The K-SVD algorithm is a classic sparse dictionary learning algorithm. Its essence is to continuously update the dictionary matrix and sparse coefficients through an alternating iterative optimization method, so that the training data samples can be accurately represented by the atoms in the dictionary in the sparsest way possible.

[0125] Specifically, the K-SVD algorithm includes the following steps: First, randomly initialize the dictionary D or randomly select samples from the training data as initial dictionary atoms; Second, fix the dictionary D and use the Orthogonal Matching Pursuit (OMP) algorithm to calculate the sparse representation coefficients A for each sample; Third, fix the sparse coefficient matrix A and update the dictionary atoms one by one: update only one atom d at a time. j The method is to explicitly update the dictionary atoms using singular value decomposition (SVD). The specific update method is as follows: locate all training samples that use this atom representation; calculate the representation error; perform SVD on the representation error matrix to update the dictionary atoms and their corresponding sparse coefficients; in the fourth step, repeat steps 2 and 3 until the convergence condition is met (the error is small enough or the iteration limit is reached).

[0126] Based on the background feature dictionary, each first fusion feature vector in the background feature sample set is sparsely encoded to obtain the sparse coefficient vector corresponding to each first fusion feature vector.

[0127] It should be noted that sparse coding is specifically implemented using the existing Orthogonal Matching Pursuit (OMP) algorithm;

[0128] The first fused feature vector is reconstructed using the background feature dictionary and the corresponding sparse coefficient vector to obtain the second fused feature vector of the background region;

[0129] In implementation, the background difference correction of the original dough image includes:

[0130] The difference between the second fusion feature vector of the background region and the first fusion feature vector of the local region is calculated to obtain the difference feature vector of each local region.

[0131] The difference is calculated as follows:

[0132]

[0133] In the formula: F diff,k Let be the difference feature vector of the k-th local region. Let be the first fused feature vector of the k-th local region. The second fusion feature phase vector of the background region;

[0134] Calculate the magnitude of the differential feature vector for each local region to obtain the differential feature intensity of each local region;

[0135] The formula for calculating the modulus length is as follows:

[0136]

[0137] In the formula: F diff,j,k This represents the value of the j-th component in the differential eigenvector of the k-th local region.

[0138] For example, assuming the difference eigenvector is [0.1, 0.1], the modulus calculation is explicitly as follows:

[0139] Background correction is performed on the pixel values ​​of the original dough image based on the differential feature intensity of each local region to obtain a background-corrected dough image.

[0140] The background correction formula is as follows:

[0141] P corr =P or,i ×(1+α×Max(0,S diff,k ));

[0142] In the formula: P or,i P represents the value of the i-th pixel within the k-th local region of the original dough image. corr α represents the pixel value after background correction, α is the correction intensity coefficient (0 < α ≤ 1), which is determined based on experimental data, and Max(·) is the maximum value function.

[0143] S105: Perform multi-scale image enhancement processing on the background-corrected dough image to obtain an enhanced dough image;

[0144] In implementation, the multi-scale image enhancement processing includes:

[0145] The background-corrected dough image is denoted as I0, and a sequence of Gaussian pyramid images at multiple scales is obtained by multiple downsampling.

[0146] Specifically, if we denote the sampling as ↓, then:

[0147] I l =(I l-1 *G)↓,l=1,2,…,L;

[0148] In the formula: I l The image represents the l-th layer in the Gaussian pyramid, G represents a Gaussian kernel, for example, a Gaussian filter with a size of 5×5, L represents the total number of scale layers in the image sequence, for example, a value of 3 or 4, * represents a convolution operation, and ↓ represents that the image scale is explicitly downsampled by a factor of 2 (e.g., taking the mean in a 2×2 neighborhood or extracting odd pixels).

[0149] The background-corrected dough image is defined as I. corr (x,y), the initial scale is denoted as G0(x,y)=I corr (x,y);

[0150] Upsample each layer of the Gaussian pyramid image to restore it to the previous layer's scale, and then perform a difference operation with the original image of the previous layer.

[0151] Specifically, let upsampling be ↑, then:

[0152] L l =I l -(I l+1 ↑*G'), l=1,2,…,L-1;

[0153] In the formula: L l I represents the image at the l-th scale in the Laplacian pyramid. l+1 ↑ indicates that the image of the (l+1)th layer of the Gaussian pyramid is upsampled to the same size as the image of the lth layer. G' represents another Gaussian smoothing kernel used to eliminate artifacts generated during the upsampling process. It is also an example of a 5×5 Gaussian filter.

[0154] A sigmoid-type nonlinear function is applied to enhance the Laplacian image at each scale, and its function expression is as follows:

[0155]

[0156] In the formula: To represent the pixel values ​​of the enhanced Laplacian image, L l (x,y) represents the pixel values ​​of the Laplacian image before enhancement, a represents the slope parameter of the nonlinear function, which controls the contrast enhancement intensity and is determined based on experimental data, and b represents the mean or preset median of the Laplacian image.

[0157] Starting from the smallest scale of the pyramid, the image is reconstructed layer by layer upwards to obtain the enhanced dough image. The calculation formula is as follows:

[0158] The smallest scale image (layer L) is:

[0159]

[0160] Images at other scales are reconstructed step by step as follows:

[0161]

[0162] In the formula; This represents the reconstructed, enhanced dough image at scale l. This indicates an upsampling operation on the enhanced image. This represents the pixel values ​​of the enhanced Laplacian image;

[0163] S106: Threshold segmentation is performed on the enhanced dough image to extract candidate impurity regions, and the morphological features of the candidate impurity regions are input into a pre-trained convolutional neural network model to obtain the target impurity regions;

[0164] It should be noted that: the threshold segmentation of the enhanced dough image is performed by calculating the gray-level histogram of the enhanced dough image, and after determining the binary segmentation threshold using the Otsu algorithm based on the gray-level histogram, the enhanced dough image is binarized using the binary segmentation threshold to obtain the candidate impurity region;

[0165] The binary partitioning formula is as follows:

[0166]

[0167] In the formula: B(x,y)=1 represents the candidate impurity region, and B(x,y)=0 represents the background region;

[0168] The morphological features include, but are not limited to, area, perimeter, aspect ratio, and radian.

[0169] In implementation, the training method for the convolutional neural network model includes:

[0170] Acquire historical impurity detection training data, and divide the historical impurity detection training data into an impurity detection training set and an impurity detection test set. The historical impurity detection training data includes the morphological features of candidate impurity regions and their corresponding labels.

[0171] The label includes "0" and "1", where "0" represents a non-target impurity region and "1" represents a target impurity region;

[0172] It should be noted that the morphological features in the historical impurity detection training data were collected manually by technicians, while the labels were manually annotated by experts or technical labelers.

[0173] Initialize a convolutional neural network as a classifier, take the morphological features of the impurity detection training set as the input of the classifier, and the labeled data as the output, train the classifier to obtain the classification network;

[0174] The classification network is validated using an impurity detection test set. The classification network with an output accuracy greater than or equal to the preset accuracy is used as the trained convolutional neural network model.

[0175] Example 2

[0176] like Figure 2 As shown in the example, the parts not detailed in this embodiment are as shown in Example 1. This embodiment discloses a dough impurity detection system based on image recognition, including:

[0177] The acquisition module 201 is used to acquire an original dough image of the surface of the dough to be tested, and to divide the original dough image into multiple local regions;

[0178] The decomposition module 202 is used to calculate the covariance matrix of different color channels in each local region, and to perform eigenvalue decomposition on the covariance matrix to obtain color difference features that characterize the color differences in each local region.

[0179] The fusion module 203 is used to obtain the brightness gradient features of each local region, and fuse the color difference features and brightness gradient features of each local region to generate a first fused feature vector;

[0180] Correction module 204 is used to determine the background region based on the first fusion feature vector, generate a second fusion feature vector of the background region through sparse representation training, and perform background difference correction on the original dough image based on the second fusion feature vector of the background region to obtain a background-corrected dough image.

[0181] Enhancement module 205 is used to perform multi-scale image enhancement processing on the background-corrected dough image to obtain an enhanced dough image;

[0182] The detection module 206 is used to perform threshold segmentation on the enhanced dough image, extract candidate impurity regions, and input the morphological features of the candidate impurity regions into a pre-trained convolutional neural network model to obtain the target impurity regions.

[0183] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters, weights, and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0184] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired or wireless network. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0185] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A method for detecting dough impurities based on image recognition, characterized in that, include: Acquire an original image of the dough surface to be tested, and divide the original dough image into multiple local regions; Calculate the covariance matrix of different color channels in each local region, and perform eigenvalue decomposition on the covariance matrix to obtain color difference features that characterize the color differences in each local region. Obtain the brightness gradient features of each local region, and fuse the color difference features and brightness gradient features of each local region to generate the first fused feature vector; The background region is determined based on the first fusion feature vector, and a second fusion feature vector of the background region is generated through sparse representation training. The original dough image is then subjected to background difference correction based on the second fusion feature vector of the background region to obtain a background-corrected dough image. The background-corrected dough image is subjected to multi-scale image enhancement processing to obtain an enhanced dough image; Thresholding segmentation is performed on the enhanced dough image to extract candidate impurity regions. The morphological features of the candidate impurity regions are then input into a pre-trained convolutional neural network model to obtain the target impurity regions.

2. The dough impurity detection method based on image recognition according to claim 1, characterized in that, The calculation of the covariance matrix of different color channels in each local region includes: Calculate the degree of deviation of the pixel value of each color channel within each local region from the mean of that color channel; d i,c =|x i,c -m c |; In the formula: d i,c The degree of deviation; The non-linear mapping intensity parameter of the color channel corresponding to each local region is determined based on the degree of deviation of pixel values; The color channel values ​​of each pixel are non-linearly adjusted using a determined non-linear mapping intensity parameter to obtain the non-linearly adjusted color channel values. The covariance matrix is ​​calculated using the non-linearly adjusted color channel values.

3. The dough impurity detection method based on image recognition according to claim 2, characterized in that, The determination of the nonlinear mapping intensity parameters for the color channel corresponding to each local region includes: Calculate the standard deviation of the pixel values ​​for each color channel within each local region; The standard deviation is proportional to the mean of the color channel to obtain the ratio coefficient between the standard deviation and the mean. The formula for proportional calculation is as follows: In the formula: k c ε is the proportionality constant; ε is a very small positive number used to avoid the denominator being zero. Based on the scaling factor, the nonlinear mapping intensity parameter of the color channel is determined.

4. The dough impurity detection method based on image recognition according to claim 3, characterized in that, The eigenvalue decomposition of the covariance matrix includes: The covariance matrix of each local region is decomposed into eigenvalues ​​to obtain three eigenvalues ​​λ1, λ2 and λ3. Sort the three eigenvalues ​​in descending order to obtain the sorted eigenvalues ​​λ. max , λ mid and λ min And calculate λ max With λ mid and λ min The ratio of the sum of these values ​​serves as the color difference characteristic of each local region. The formula for calculating the proportion is as follows: In the formula; F color Represents color difference characteristics; ε is a very small positive number, λ max λ is the largest eigenvalue. mid λ is an intermediate eigenvalue. min It is the smallest eigenvalue.

5. The dough impurity detection method based on image recognition according to claim 1, characterized in that, The process of obtaining the brightness gradient features of each local region includes: The original dough image is converted to grayscale to obtain a grayscale image; The Sobel operator is used to calculate the gradient magnitude and gradient direction of each pixel in the grayscale image; Constructing the gradient direction histogram for each local region includes: range of gradient direction Divide into R equal sub-intervals; Count the number of pixels whose gradient direction falls into each sub-interval within each local region to obtain a gradient direction histogram; The nonlinear local gradient direction entropy is calculated based on the gradient direction histogram, and this local gradient direction entropy is used as a feature of the brightness gradient. The specific calculation formula is as follows: In the formula: E gard Let be the local gradient direction entropy value, ε be a minimum positive number, and log be the logarithmic function.

6. The dough impurity detection method based on image recognition according to claim 1, characterized in that, The determination of the background area includes: Calculate the magnitude of the fused feature vector for each local region to obtain the set of magnitudes for all local regions; The formula for calculating the modulus length is as follows: In the formula: L K f is the modulus length. j,k This represents the specific value of the j-th feature component in the fused feature vector of the k-th local region; Based on the set of modulus lengths, calculate the average and standard deviation of the fused feature vector modulus lengths, and then set the average μ... L and standard deviation σ L The difference is used as the modulus threshold T = μ L -σ L ; Local regions whose fused feature vector magnitude is less than the specified magnitude threshold are defined as background regions.

7. The dough impurity detection method based on image recognition according to claim 6, characterized in that, The background difference correction of the original dough image includes: The difference between the second fusion feature vector of the background region and the first fusion feature vector of the local region is calculated to obtain the difference feature vector of each local region. The difference is calculated as follows: In the formula: F diff,k Let be the difference feature vector of the k-th local region. Let be the first fused feature vector of the k-th local region. The second fusion feature phase vector of the background region; Calculate the magnitude of the differential feature vector for each local region to obtain the differential feature intensity of each local region; The formula for calculating the modulus length is as follows: In the formula: F diff,j,k This represents the value of the j-th component in the differential eigenvector of the k-th local region. Background correction is performed on the pixel values ​​of the original dough image based on the differential feature intensity of each local region to obtain a background-corrected dough image. The background correction formula is as follows: P corr =P or,i ×(1+α×Max(0,S diff,k )); In the formula: P or,i P represents the value of the i-th pixel within the k-th local region of the original dough image. corr α represents the pixel value after background correction, α is the correction intensity coefficient, and Max(·) is the maximum value function.

8. The dough impurity detection method based on image recognition according to claim 7, characterized in that, The multi-scale image enhancement process includes: The background-corrected dough image is denoted as I0, and a sequence of Gaussian pyramid images at multiple scales is obtained by multiple downsampling. Specifically, if we denote the sampling as ↓, then: I l =(I l-1 *G)↓,l=1,2,…,L; In the formula: I l denoted as the l-th layer image in the Gaussian pyramid, G represents a Gaussian kernel, L represents the total number of scale layers in the image sequence, * represents a convolution operation, and ↓ indicates that the image scale is explicitly downsampled by a factor of 2. The background-corrected dough image is defined as I. corr (x,y), the initial scale is denoted as G0(x,y)=I corr (x,y); Upsample each layer of the Gaussian pyramid image to restore it to the previous layer's scale, and then perform a difference operation with the original image of the previous layer. Specifically, let upsampling be ↑, then: L l =I l -(I l+1 ↑*G'),l=1,2,…,L-1; In the formula: L l I represents the image at the l-th scale in the Laplacian pyramid. l+1 ↑ indicates upsampling the (l+1)th layer image of the Gaussian pyramid to the same size as the lth layer image; G' represents another Gaussian smoothing kernel. A sigmoid-type nonlinear function is applied to enhance the Laplacian image at each scale, and its function expression is as follows: In the formula: To represent the pixel values ​​of the enhanced Laplacian image, L l (x,y) represents the pixel values ​​of the Laplacian image before enhancement, a represents the slope parameter of the nonlinear function, and b represents the mean or preset median of the Laplacian image. Starting from the smallest scale of the pyramid, the image is reconstructed layer by layer upwards to obtain the enhanced dough image. The calculation formula is as follows: The smallest scale image (layer L) is: Images at other scales are reconstructed step by step as follows: In the formula; This represents the reconstructed, enhanced dough image at scale l. ↑ indicates an upsampling operation on the enhanced image. This represents the pixel values ​​of the enhanced Laplacian image.

9. The dough impurity detection method based on image recognition according to claim 8, characterized in that, The morphological features include, but are not limited to, area, perimeter, aspect ratio, and radian. The training method for the convolutional neural network model includes: S106.1: Obtain historical impurity detection training data, and divide the historical impurity detection training data into an impurity detection training set and an impurity detection test set. The historical impurity detection training data includes the morphological features of candidate impurity regions and their corresponding labels. The label includes "0" and "1", where "0" represents a non-target impurity region and "1" represents a target impurity region. Initialize a convolutional neural network as a classifier, take the morphological features of the impurity detection training set as the input of the classifier, and the labeled data as the output, train the classifier to obtain the classification network; The classification network is validated using an impurity detection test set. The classification network with an output accuracy greater than or equal to the preset accuracy is used as the trained convolutional neural network model.

10. A dough impurity detection system based on image recognition, implemented based on the dough impurity detection method based on image recognition according to any one of claims 1-8, characterized in that, include: The acquisition module is used to acquire an original dough image of the surface of the dough to be tested, and to divide the original dough image into multiple local regions; The decomposition module is used to calculate the covariance matrix of different color channels in each local region, and to perform eigenvalue decomposition on the covariance matrix to obtain color difference features that characterize the color differences in each local region. The fusion module is used to obtain the brightness gradient features of each local region, and fuse the color difference features and brightness gradient features of each local region to generate the first fused feature vector; The correction module is used to determine the background region based on the first fusion feature vector, and generate a second fusion feature vector of the background region through sparse representation training. The original dough image is then subjected to background difference correction based on the second fusion feature vector of the background region to obtain a background-corrected dough image. The enhancement module is used to perform multi-scale image enhancement processing on the background-corrected dough image to obtain an enhanced dough image; The detection module is used to perform threshold segmentation on the enhanced dough image, extract candidate impurity regions, and input the morphological features of the candidate impurity regions into a pre-trained convolutional neural network model to obtain the target impurity regions.

Images