Dough impurity detection system and method based on image recognition

Abstract

The application discloses a dough impurity detection system and method based on image recognition, and relates to the technical field of food detection.The system comprises the following steps: fusing color difference features and brightness gradient features to generate a first fusion feature vector; determining a background area according to the first fusion feature vector, and generating a second fusion feature vector of the background area through sparse representation training; correcting the original dough image through background difference according to the second fusion feature vector to obtain a dough image after background correction; performing multi-scale image enhancement processing on the corrected dough image to obtain an enhanced dough image; performing threshold segmentation on the enhanced dough image to extract a candidate impurity area, and inputting the morphological features of the candidate impurity area into a convolutional neural network model to obtain a target impurity area; and the application effectively improves the accuracy, robustness and automation degree of dough surface impurity detection through feature fusion, background difference correction and multi-scale enhancement.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of food detection, and in particular to a dough impurity detection system and method based on image recognition. BACKGROUND

[0002] In the field of food industry, especially on the dough automatic production line, strict control of dough quality is of great importance. The mixing of impurities (such as foreign color particles, non-food materials, burnt or contaminated debris) in the dough not only affects the safety and quality of the final food, but also can cause damage to the equipment, and even lead to food safety accidents in severe cases. Therefore, quickly and accurately detecting and locating the impurities on the surface of the dough is an important technical problem in the food industry.

[0003] Traditional impurity detection methods generally rely on manual visual inspection or simple machine vision methods. However, manual visual inspection is limited by the fatigue level and subjectivity of the detection personnel, and has obvious instability and risk of false detection and missed detection. Conventional machine vision methods usually identify impurity regions through simple threshold segmentation, background difference or color space conversion, but due to uneven color distribution on the surface of the dough, small differences in features between impurities and background regions, these traditional methods often fail to effectively distinguish impurity regions from background regions, resulting in insufficient detection accuracy.

[0004] Specifically, the color and texture of the dough surface change weakly, especially in the case of uneven illumination in the actual industrial environment, which makes it difficult to effectively capture such subtle changes through existing simple image processing methods. At the same time, background modeling and background correction techniques usually perform simple difference based on static or single color features, which are difficult to adapt to complex and variable industrial scenes. In addition, most of the existing multi-scale image enhancement methods only use a single scale or simple filtering means, which has limited effect on the enhancement of weak texture features on the surface of the dough, further affecting the accuracy of impurity region extraction.

[0005] Therefore, there is still an urgent need for a more effective, stable and accurate dough surface impurity detection method that can more sensitively capture the subtle feature differences between impurities and background regions, and improve the accuracy and robustness of impurity region detection. SUMMARY

[0006] The purpose of the present application is to provide a dough impurity detection method and system based on image recognition to solve the problems in the background.

[0007] In order to achieve the above purpose, the present application provides the following technical solutions:

[0008] In a first aspect, the present application provides a dough impurity detection method based on image recognition, comprising:

[0009] Acquire an original image of the dough surface to be tested, and divide the original dough image into multiple local regions;

[0010] Calculate the covariance matrix of different color channels in each local region, and perform eigenvalue decomposition on the covariance matrix to obtain color difference features that characterize the color differences in each local region.

[0011] Obtain the brightness gradient features of each local region, and fuse the color difference features and brightness gradient features of each local region to generate the first fused feature vector;

[0012] The background region is determined based on the first fusion feature vector, and a second fusion feature vector of the background region is generated through sparse representation training. The original dough image is then subjected to background difference correction based on the second fusion feature vector of the background region to obtain a background-corrected dough image.

[0013] The background-corrected dough image is subjected to multi-scale image enhancement processing to obtain an enhanced dough image;

[0014] Thresholding segmentation is performed on the enhanced dough image to extract candidate impurity regions. The morphological features of the candidate impurity regions are then input into a pre-trained convolutional neural network model to obtain the target impurity regions.

[0015] Secondly, the present invention provides a dough impurity detection system based on image recognition, implemented based on the aforementioned dough impurity detection method based on image recognition, comprising:

[0016] The acquisition module is used to acquire an original dough image of the surface of the dough to be tested, and to divide the original dough image into multiple local regions;

[0017] The decomposition module is used to calculate the covariance matrix of different color channels in each local region, and to perform eigenvalue decomposition on the covariance matrix to obtain color difference features that characterize the color differences in each local region.

[0018] The fusion module is used to obtain the brightness gradient features of each local region, and fuse the color difference features and brightness gradient features of each local region to generate the first fused feature vector;

[0019] The correction module is used to determine the background region based on the first fusion feature vector, and generate a second fusion feature vector of the background region through sparse representation training. The original dough image is then subjected to background difference correction based on the second fusion feature vector of the background region to obtain a background-corrected dough image.

[0020] The enhancement module is used to perform multi-scale image enhancement processing on the background-corrected dough image to obtain an enhanced dough image;

[0021] The detection module is used to perform threshold segmentation on the enhanced dough image, extract candidate impurity regions, and input the morphological features of the candidate impurity regions into a pre-trained convolutional neural network model to obtain the target impurity regions.

[0022] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0023] This invention addresses the problem that existing technologies struggle to effectively distinguish between impurities and background areas on the surface of dough, resulting in insufficient feature representation of impurity areas and low detection accuracy. It proposes a method that obtains color difference features by nonlinearly adjusting color channel values, calculating the covariance matrix and its eigenvalue decomposition, and combining this with brightness gradient direction entropy. This significantly improves the effectiveness of feature differentiation between impurity and non-impurity areas, enhancing detection sensitivity.

[0024] To further address the problem that impurity regions in existing technologies are severely affected by background interference and are difficult to distinguish accurately, this paper proposes an automatic determination of background regions based on a fusion feature modulus threshold. The baseline features of the background regions are obtained by using sparse representation and dictionary learning, which are then used for differential correction of the original image. This effectively suppresses background interference, achieves accurate extraction of impurity region features, and significantly improves the stability and reliability of dough impurity detection.

[0025] Furthermore, this invention solves the problem of weak impurity features being easily ignored and insufficient detection accuracy in the prior art by designing a multi-scale Laplacian pyramid nonlinear enhancement method, achieving full enhancement of texture and edge details of impurity regions at different scales; and combines convolutional neural networks to automatically classify and identify the morphological features of candidate impurity regions, effectively improving the automation level and industrial applicability of impurity region detection, and meeting the actual production needs for accuracy and efficiency in dough impurity detection. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0027] Figure 1 This is a flowchart of an image recognition-based dough impurity detection method according to the present invention;

[0028] Figure 2 This is a framework diagram of an image recognition-based dough impurity detection system according to the present invention. Detailed Implementation

[0029] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided to make the description of this application more complete and comprehensive, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The drawings are merely illustrative illustrations of this application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0030] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more exemplary embodiments. Numerous specific details are provided in the following description to give a full understanding of the exemplary embodiments disclosed in this application. However, those skilled in the art will recognize that the technical solutions disclosed in this application can be practiced with one or more specific details omitted, or other methods, components, steps, etc., can be employed. In other instances, well-known structures, methods, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the disclosure of this application.

[0031] Example 1

[0032] like Figure 1 As shown, this embodiment discloses a dough impurity detection method based on image recognition, including:

[0033] S101: Obtain the original dough image of the surface of the dough to be tested, and divide the original dough image into multiple local regions;

[0034] It should be noted that the original dough image on the surface of the dough to be tested refers to the digital image data directly acquired by an industrial camera without any processing. For example, an RGB channel color digital image obtained by taking a picture with an industrial-grade RGB camera with a resolution of 2048×1536 pixels.

[0035] It should also be noted that the method of dividing the original dough image data into multiple local regions is the conventional rectangular grid division method, that is, dividing the image data into equal horizontal and vertical dimensions to obtain multiple non-overlapping rectangular regions that completely cover the entire original dough image; for example, for the original dough image data with a size of 2048×1536 pixels, it is divided into 16 regions horizontally and 12 regions vertically, resulting in 192 rectangular local regions with completely equal areas;

[0036] It should be understood that multiple local areas are determined in advance by human intervention based on the actual needs of impurity detection in the dough to be tested. The more areas there are and the smaller the area scale, the higher the corresponding detection accuracy. For example, in actual industrial applications, the area size is usually determined by a combination of factors such as dough conveying speed, impurity size requirements, or industrial camera resolution to ensure that the detection accuracy meets the actual production needs.

[0037] S102: Calculate the covariance matrix of different color channels in each local region, and perform eigenvalue decomposition on the covariance matrix to obtain the color difference features that characterize the color differences in each local region; this can more sensitively and stably characterize the difference intensity between different color channels, which is beneficial to make the color difference features between the dough impurity region and the non-impurity region more significant.

[0038] In implementation, calculating the covariance matrix of different color channels within each local region includes:

[0039] Calculate the degree of deviation of the pixel value of each color channel within each local region from the mean of that color channel;

[0040] d i,c =|x i,c -μ c |;

[0041] In the formula: d i,c The degree of deviation;

[0042] It should be noted that the deviation of color channel values ​​refers to the absolute difference between the value of a single color channel and the mean value of that color channel for each pixel in a local area.

[0043] The non-linear mapping intensity parameter of the color channel corresponding to each local region is determined based on the degree of deviation of pixel values;

[0044] Specifically, determining the nonlinear mapping intensity parameter for the color channel corresponding to each local region includes:

[0045] Calculate the standard deviation of the pixel values ​​for each color channel within each local region;

[0046] For example, for color channel c, the standard deviation σ c The calculation formula is clearly defined as follows:

[0047]

[0048] In the formula: c∈{R,G,B} represents the red, green, and blue channels; x i,c Let be the pixel value of the i-th pixel in color channel c; N is the total number of pixels in the local region;

[0049] The standard deviation is proportional to the mean of the color channel to obtain the ratio coefficient between the standard deviation and the mean.

[0050] The formula for proportional calculation is as follows:

[0051]

[0052] In the formula: k c ε is the proportionality constant; ε is a very small positive number (e.g., 10). -6 (), used to avoid the denominator being zero;

[0053] Based on the scaling factor, the nonlinear mapping intensity parameter of the color channel is determined;

[0054] It is worth noting that the nonlinear mapping intensity parameter is used for nonlinear adjustment of subsequent color channel values;

[0055] For example, the formula for calculating the nonlinear mapping intensity parameter of the i-th pixel on color channel c is:

[0056] γ i,c =1+k c ;

[0057] In the formula: γ i,c For nonlinear mapping intensity parameters;

[0058] The color channel values ​​of each pixel are non-linearly adjusted using a determined non-linear mapping intensity parameter to obtain the non-linearly adjusted color channel values.

[0059] It should be noted that adjusting the color channel values ​​through the nonlinear mapping intensity parameter aims to further highlight the differences within the color channels, which is helpful for the subsequent extraction of impurity region features.

[0060] For example, the formula for calculating the adjusted color channel value of the i-th pixel on color channel c is:

[0061]

[0062] In the formula: y i,c For color channel values, the above formula uses normalized pixel values, x i,c The value range is [0,1];

[0063] Understandably, in the above calculation, the pixel color value x i,c It has been explicitly normalized to the interval [0,1] to ensure the effectiveness of nonlinear mapping operations; for example, if the color channel value is 128 (original value range 0-255), the normalized color channel value is 128 / 255≈0.502;

[0064] The covariance matrix is ​​calculated using the non-linearly adjusted color channel values.

[0065] It should be noted that this step is based on the non-linearly adjusted color channel values ​​to calculate the covariance between each color channel, in order to reflect the more prominent differences between the color channels after non-linear adjustment.

[0066] For example, the covariance matrix C of the non-linearly adjusted color channel values ​​is defined as a 3x3 matrix, and the calculation formula is as follows:

[0067]

[0068] in:

[0069]

[0070] In the formula: m,n∈{R,G,B} represent the red, green, and blue channels; y i,m and y i,n Let m and n represent the non-linearly adjusted color channel values ​​of the i-th pixel, respectively. and Let m and n represent the mean values ​​of all pixels in color channel m after non-linear adjustment, respectively.

[0071] For example, the The calculation formula is: same The computational logic, The logic for obtaining it is the same, and will not be repeated here;

[0072] In implementation, the eigenvalue decomposition of the covariance matrix includes:

[0073] The covariance matrix of each local region is decomposed into eigenvalues ​​to obtain three eigenvalues ​​λ1, λ2 and λ3.

[0074] It should be noted that covariance matrix eigenvalue decomposition refers to performing mathematical decomposition on each 3×3 covariance matrix to obtain eigenvalues ​​that explicitly represent the characteristics of the matrix.

[0075] For example, the eigenvalue decomposition of a single covariance matrix C is explicitly performed, satisfying the following mathematical formula:

[0076] C×V=V×Λ;

[0077] In the formula: V is the eigenvector matrix; Λ is the diagonal matrix composed of eigenvalues; where, The eigenvalues ​​satisfy the descending order relation λ max ≥λ mid ≥λ min ;λ max λ is the largest eigenvalue. mid λ is an intermediate eigenvalue. min It is the smallest eigenvalue;

[0078] Sort the three eigenvalues ​​in descending order to obtain the sorted eigenvalues ​​λ. max , λ mid and λ min And calculate λ max With λ mid and λ min The ratio of the sum of these values ​​serves as the color difference characteristic of each local region.

[0079] It is worth noting that this step explicitly uses the ratio of the largest eigenvalue to the sum of the other two eigenvalues ​​as the color difference feature of each local region to highlight the significance of color channel differences.

[0080] The formula for calculating the proportion is as follows:

[0081]

[0082] In the formula; F color ε is a proportional value, representing the color difference characteristic; ε is a very small positive number (e.g., 10). -6 (), used to avoid the denominator being zero;

[0083] Understandably, the larger the calculated color difference feature value, the more significant the difference between color channels in a local area, which plays a positive role in accurately distinguishing impurity areas from non-impurity areas.

[0084] S103: Obtain the brightness gradient features of each local region, and fuse the color difference features and brightness gradient features of each local region to generate the first fused feature vector;

[0085] In implementation, obtaining the brightness gradient features of each local region includes:

[0086] The original dough image is converted to grayscale to obtain a grayscale image;

[0087] Specifically, the grayscale image conversion formula is:

[0088] I gray (x,y)=0.299×I R (x,y)+0.587×I G (x,y)+0.114×I B (x,y);

[0089] In the formula: I gray (x,y) represents the grayscale value, I R (x,y), I G (x,y) and I B (x, y) represent the pixel values ​​of the red, green, and blue channels, respectively;

[0090] The Sobel operator is used to calculate the gradient magnitude and gradient direction of each pixel in the grayscale image;

[0091] Specifically, the gradient magnitude and gradient direction are:

[0092]

[0093] In the formula: G(x,y) is the gradient magnitude, θ(x,y) is the gradient direction, and G... x (x,y) represents the gradient intensity of the pixel at position (x,y) in the horizontal (X-axis) direction, G y (x,y) represents the gradient intensity of the pixel at position (x,y) in the vertical (Y-axis) direction;

[0094] Constructing the gradient direction histogram for each local region specifically includes:

[0095] range of gradient direction Divide into R equal sub-intervals;

[0096] Count the number of pixels whose gradient direction falls into each sub-interval within each local region to obtain a gradient direction histogram;

[0097] For example, the gradient direction histogram is as follows:

[0098]

[0099] Where: n i The number of pixels falling into the i-th sub-interval, where N is the total number of pixels in the local region;

[0100] The nonlinear local gradient direction entropy is calculated based on the gradient direction histogram, and this local gradient direction entropy is used as a feature of the brightness gradient. The specific calculation formula is as follows:

[0101]

[0102] In the formula: E gard Here, ε is the local gradient direction entropy value, and ε is a minimum positive number (e.g., 10). -6 ), where log is the logarithmic function;

[0103] It should be noted that the first fused feature vector is obtained by directly concatenating and fusing the color difference feature and the brightness gradient feature;

[0104] For example, for the k-th local region, the fused feature vector is:

[0105]

[0106] In the formula: Indicates color difference characteristics. This represents the brightness gradient feature.

[0107] S104: Determine the background region based on the first fusion feature vector, and generate the second fusion feature vector of the background region through sparse representation training. Perform background difference correction on the original dough image based on the second fusion feature vector of the background region to obtain the background-corrected dough image.

[0108] It should be noted that this step explicitly uses the first fusion feature vector magnitude to automatically determine the background region, so as to ensure that the determined background region accurately reflects the features of the background part in the dough image;

[0109] In implementation, determining the background area includes:

[0110] Calculate the magnitude of the fused feature vector for each local region to obtain the set of magnitudes for all local regions;

[0111] The formula for calculating the modulus length is as follows:

[0112]

[0113] In the formula: L K f is the modulus length. j,k This represents the specific value of the j-th feature component in the fused feature vector of the k-th local region;

[0114] For example, for the k-th local region, its first fused feature vector is: The formula for calculating the modulus is as follows: Among them, f 1,k For the color difference feature of the k-th local region, f 2,k The brightness gradient feature of the k-th local region;

[0115] Based on the set of modulus lengths, calculate the average and standard deviation of the fused feature vector modulus lengths, and then set the average μ... L and standard deviation σ L The difference is used as the modulus threshold T = μ L -σ L ;

[0116] The formula for the average value of the modulus is as follows:

[0117]

[0118] The formula for the standard deviation of the modulus is as follows:

[0119]

[0120] Local regions whose fused feature vector magnitude is less than the magnitude threshold are defined as background regions;

[0121] In implementation, the step of generating the second fused feature vector of the background region through sparse representation training includes:

[0122] Multiple first fusion feature vectors are extracted from the defined background region to form a background feature sample set;

[0123] Based on the aforementioned background feature sample set, a background feature dictionary is trained using a sparse representation dictionary learning algorithm.

[0124] It should be noted that the sparse representation dictionary learning algorithm adopts the K-SVD algorithm. The K-SVD algorithm is a classic sparse dictionary learning algorithm. Its essence is to continuously update the dictionary matrix and sparse coefficients through an alternating iterative optimization method, so that the training data samples can be accurately represented by the atoms in the dictionary in the sparsest way possible.

[0125] Specifically, the K-SVD algorithm includes the following steps: First, randomly initialize the dictionary D or randomly select samples from the training data as initial dictionary atoms; Second, fix the dictionary D and use the Orthogonal Matching Pursuit (OMP) algorithm to calculate the sparse representation coefficients A for each sample; Third, fix the sparse coefficient matrix A and update the dictionary atoms one by one: update only one atom d at a time. j The method is to explicitly update the dictionary atoms using singular value decomposition (SVD). The specific update method is as follows: locate all training samples that use this atom representation; calculate the representation error; perform SVD on the representation error matrix to update the dictionary atoms and their corresponding sparse coefficients; in the fourth step, repeat steps 2 and 3 until the convergence condition is met (the error is small enough or the iteration limit is reached).

[0126] Based on the background feature dictionary, each first fusion feature vector in the background feature sample set is sparsely encoded to obtain the sparse coefficient vector corresponding to each first fusion feature vector.

[0127] It should be noted that sparse coding is specifically implemented using the existing Orthogonal Matching Pursuit (OMP) algorithm;

[0128] The first fused feature vector is reconstructed using the background feature dictionary and the corresponding sparse coefficient vector to obtain the second fused feature vector of the background region;

[0129] In practice, the background difference correction of the original dough image includes:

[0130] The difference between the second fusion feature vector of the background region and the first fusion feature vector of the local region is calculated to obtain the difference feature vector of each local region.

[0131] The difference is calculated as follows:

[0132]

[0133] In the formula: F diff,k Let be the difference feature vector of the k-th local region. Let be the first fused feature vector of the k-th local region. The second fusion feature phase vector of the background region;

[0134] Calculate the magnitude of the differential feature vector for each local region to obtain the differential feature intensity of each local region;

[0135] The formula for calculating the modulus length is as follows:

[0136]

[0137] In the formula: F diff,j,k This represents the value of the j-th component in the differential eigenvector of the k-th local region.

[0138] For example, assuming the difference eigenvector is [0.1, 0.1], the modulus calculation is explicitly as follows:

[0139] Background correction is performed on the pixel values ​​of the original dough image based on the differential feature intensity of each local region to obtain a background-corrected dough image.

[0140] The background correction formula is as follows:

[0141] P corr =P or,i ×(1+α×Max(0,S diff,k ));

[0142] In the formula: P or,i P represents the value of the i-th pixel within the k-th local region of the original dough image. corr α represents the pixel value after background correction, α is the correction intensity coefficient (0 < α ≤ 1), which is determined based on experimental data, and Max(·) is the maximum value function.

[0143] S105: Perform multi-scale image enhancement processing on the background-corrected dough image to obtain an enhanced dough image;

[0144] In implementation, the multi-scale image enhancement processing includes:

[0145] The background-corrected dough image is denoted as I0, and a sequence of Gaussian pyramid images at multiple scales is obtained by multiple downsampling.

[0146] Specifically, if we denote the sampling as ↓, then:

[0147] I l =(I l-1 *G)↓,l=1,2,…,L;

[0148] In the formula: I l The image represents the l-th layer in the Gaussian pyramid, G represents a Gaussian kernel, for example, a Gaussian filter with a size of 5×5, L represents the total number of scale layers in the image sequence, for example, a value of 3 or 4, * represents a convolution operation, and ↓ represents that the image scale is explicitly downsampled by a factor of 2 (e.g., taking the mean in a 2×2 neighborhood or extracting odd pixels).

[0149] The background-corrected dough image is defined as I. corr (x,y), the initial scale is denoted as G0(x,y)=I corr (x,y);

[0150] Upsample each layer of the Gaussian pyramid image to restore it to the previous layer's scale, and then perform a difference operation with the original image of the previous layer.

[0151] Specifically, let upsampling be ↑, then:

[0152] L l =I l -(I l+1 ↑*G'), l=1,2,…,L-1;

[0153] In the formula: L l I represents the image at the l-th scale in the Laplacian pyramid. l+1 ↑ indicates that the image of the (l+1)th layer of the Gaussian pyramid is upsampled to the same size as the image of the lth layer. G' represents another Gaussian smoothing kernel used to eliminate artifacts generated during the upsampling process. It is also an example of a 5×5 Gaussian filter.

[0154] A sigmoid-type nonlinear function is applied to enhance the Laplacian image at each scale, and its function expression is as follows:

[0155]

[0156] In the formula: To represent the pixel values ​​of the enhanced Laplacian image, L l (x,y) represents the pixel values ​​of the Laplacian image before enhancement, a represents the slope parameter of the nonlinear function, which controls the contrast enhancement intensity and is determined based on experimental data, and b represents the mean or preset median of the Laplacian image.

[0157] Starting from the smallest scale of the pyramid, the image is reconstructed layer by layer upwards to obtain the enhanced dough image. The calculation formula is as follows:

[0158] The smallest scale image (layer L) is:

[0159]

[0160] Images at other scales are reconstructed step by step as follows:

[0161]

[0162] In the formula; This represents the reconstructed, enhanced dough image at scale l. This indicates an upsampling operation on the enhanced image. This represents the pixel values ​​of the enhanced Laplacian image;

[0163] S106: Threshold segmentation is performed on the enhanced dough image to extract candidate impurity regions, and the morphological features of the candidate impurity regions are input into a pre-trained convolutional neural network model to obtain the target impurity regions;

[0164] It should be noted that: the threshold segmentation of the enhanced dough image is performed by calculating the gray-level histogram of the enhanced dough image, and after determining the binary segmentation threshold using the Otsu algorithm based on the gray-level histogram, the enhanced dough image is binarized using the binary segmentation threshold to obtain the candidate impurity region;

[0165] The binary partitioning formula is as follows:

[0166]

[0167] In the formula: B(x,y)=1 represents the candidate impurity region, and B(x,y)=0 represents the background region;

[0168] The morphological features include, but are not limited to, area, perimeter, aspect ratio, and radian.

[0169] In implementation, the training method for the convolutional neural network model includes:

[0170] Acquire historical impurity detection training data, and divide the historical impurity detection training data into an impurity detection training set and an impurity detection test set. The historical impurity detection training data includes the morphological features of candidate impurity regions and their corresponding labels.

[0171] The label includes "0" and "1", where "0" represents a non-target impurity region and "1" represents a target impurity region.

[0172] It should be noted that the morphological features in the historical impurity detection training data were collected manually by technicians, while the labels were manually annotated by experts or technical labelers.

[0173] Initialize a convolutional neural network as a classifier, take the morphological features of the impurity detection training set as the input of the classifier, and the labeled data as the output, train the classifier to obtain the classification network;

[0174] The classification network is validated using an impurity detection test set. The classification network with an output accuracy greater than or equal to the preset accuracy is used as the trained convolutional neural network model.

[0175] Example 2

[0176] like Figure 2 As shown in the example, the parts not detailed in this embodiment are as shown in Example 1. This embodiment discloses a dough impurity detection system based on image recognition, including:

[0177] The acquisition module 201 is used to acquire an original dough image of the surface of the dough to be tested, and to divide the original dough image into multiple local regions;

[0178] The decomposition module 202 is used to calculate the covariance matrix of different color channels in each local region, and to perform eigenvalue decomposition on the covariance matrix to obtain color difference features that characterize the color differences in each local region.

[0179] The fusion module 203 is used to obtain the brightness gradient features of each local region, and fuse the color difference features and brightness gradient features of each local region to generate a first fused feature vector;

[0180] Correction module 204 is used to determine the background region based on the first fusion feature vector, generate a second fusion feature vector of the background region through sparse representation training, and perform background difference correction on the original dough image based on the second fusion feature vector of the background region to obtain a background-corrected dough image.

[0181] Enhancement module 205 is used to perform multi-scale image enhancement processing on the background-corrected dough image to obtain an enhanced dough image;

[0182] The detection module 206 is used to perform threshold segmentation on the enhanced dough image, extract candidate impurity regions, and input the morphological features of the candidate impurity regions into a pre-trained convolutional neural network model to obtain the target impurity regions.

[0183] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters, weights, and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0184] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired or wireless network. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0185] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A method for detecting dough impurities based on image recognition, characterized in that, include: Acquire an original image of the dough surface to be tested, and divide the original dough image into multiple local regions; Calculate the covariance matrix of different color channels in each local region, and perform eigenvalue decomposition on the covariance matrix to obtain color difference features that characterize the color differences in each local region. The eigenvalue decomposition of the covariance matrix includes: Eigenvalue decomposition is performed on the covariance matrix of each local region to obtain three eigenvalues. , and ; Sort the three feature values ​​in descending order to obtain the sorted feature values. , and and calculate and as well as The ratio of the sum of these values ​​serves as the color difference characteristic of each local region. The formula for calculating the proportion is as follows: ； In the formula; Indicates color difference characteristics; It is a very small positive number. The largest eigenvalue, For intermediate eigenvalues, It is the smallest eigenvalue; Obtain the brightness gradient features of each local region, and fuse the color difference features and brightness gradient features of each local region to generate the first fused feature vector; The background region is determined based on the first fusion feature vector, and a second fusion feature vector of the background region is generated through sparse representation training. The original dough image is then subjected to background difference correction based on the second fusion feature vector of the background region to obtain a background-corrected dough image. The determination of the background region includes: Calculate the magnitude of the fused feature vector for each local region to obtain the set of magnitudes for all local regions; The formula for calculating the modulus length is as follows: ； In the formula: For the length of the module, This represents the specific value of the j-th feature component in the fused feature vector of the k-th local region; Based on the set of modulus lengths, calculate the average and standard deviation of the fused feature vector modulus lengths, and then set the average value... and standard deviation The difference is used as the modulus threshold. ; Local regions whose fused feature vector magnitude is less than the magnitude threshold are defined as background regions; The background-corrected dough image is subjected to multi-scale image enhancement processing to obtain an enhanced dough image; Thresholding segmentation is performed on the enhanced dough image to extract candidate impurity regions. The morphological features of the candidate impurity regions are then input into a pre-trained convolutional neural network model to obtain the target impurity regions.

2. The dough impurity detection method based on image recognition according to claim 1, characterized in that, The calculation of the covariance matrix of different color channels in each local region includes: Calculate the degree of deviation of the pixel value of each color channel within each local region from the mean of that color channel; ； In the formula: The degree of deviation; The non-linear mapping intensity parameter of the color channel corresponding to each local region is determined based on the degree of deviation of pixel values; The color channel values ​​of each pixel are non-linearly adjusted using a determined non-linear mapping intensity parameter to obtain the non-linearly adjusted color channel values. The covariance matrix is ​​calculated using the non-linearly adjusted color channel values.

3. The dough impurity detection method based on image recognition according to claim 2, characterized in that, The determination of the nonlinear mapping intensity parameters for the color channel corresponding to each local region includes: Calculate the standard deviation of the pixel values ​​for each color channel within each local region; The standard deviation is proportional to the mean of the color channel to obtain the ratio coefficient between the standard deviation and the mean. The formula for proportional calculation is as follows: ； In the formula: This is the proportionality coefficient; It is a very small positive number, used to avoid the denominator being zero; Based on the scaling factor, the nonlinear mapping intensity parameter of the color channel is determined.

4. The dough impurity detection method based on image recognition according to claim 1, characterized in that, The process of obtaining the brightness gradient features of each local region includes: The original dough image is converted to grayscale to obtain a grayscale image; The Sobel operator is used to calculate the gradient magnitude and gradient direction of each pixel in the grayscale image; Constructing the gradient direction histogram for each local region includes: range of gradient direction Divide into R equal sub-intervals; Count the number of pixels whose gradient direction falls into each sub-interval within each local region to obtain a gradient direction histogram; The nonlinear local gradient direction entropy is calculated based on the gradient direction histogram, and this local gradient direction entropy is used as a feature of the brightness gradient. The specific calculation formula is as follows: ； In the formula: The entropy value is the local gradient direction. It is a very small positive number. It is a logarithmic function.

5. The dough impurity detection method based on image recognition according to claim 1, characterized in that, The background difference correction of the original dough image includes: The difference between the second fusion feature vector of the background region and the first fusion feature vector of the local region is calculated to obtain the difference feature vector of each local region. The difference is calculated as follows: ； In the formula: Let be the difference feature vector of the k-th local region. Let be the first fused feature vector of the k-th local region. The second fusion feature phase vector of the background region; Calculate the magnitude of the differential feature vector for each local region to obtain the differential feature intensity of each local region; The formula for calculating the modulus length is as follows: ； In the formula: The value of the j-th component in the differential eigenvector of the k-th local region; Background correction is performed on the pixel values ​​of the original dough image based on the differential feature intensity of each local region to obtain a background-corrected dough image. The background correction formula is as follows: ； In the formula: This represents the value of the i-th pixel within the k-th local region of the original dough image. These are the pixel values ​​after background correction. To correct the strength coefficient, It is a function for maximizing the value.

6. The dough impurity detection method based on image recognition according to claim 1, characterized in that, The multi-scale image enhancement processing includes: The background-corrected dough image is denoted as A sequence of Gaussian pyramid images at multiple scales was obtained through multiple downsampling operations. Specifically, record the sampling as ,but: ； In the formula: This represents the first in the Gaussian pyramid. Each layer of image, Describe a Gaussian kernel, This indicates the total number of scale layers in the image sequence. This represents the convolution operation. This indicates that the image scale is explicitly downsampled by a factor of 2. Define the background-corrected dough image as The initial scale is denoted as ; Upsample each layer of the Gaussian pyramid image to restore it to the previous layer's scale, and then perform a difference operation with the original image of the previous layer. Specifically, the upsampling is recorded as ,but: ； In the formula: This indicates the first in the Laplace pyramid. Images at various scales This indicates that the first Gaussian pyramid will be... Upsampling of layer image to the 1st The layer images are the same size. Indicates another Gaussian smoothing kernel; A sigmoid-type nonlinear function is applied to enhance the Laplacian image at each scale, and its function expression is as follows: ； In the formula: To represent the pixel values ​​of the enhanced Laplacian image, This represents the pixel values ​​of the Laplacian image before enhancement. The slope parameter represents the nonlinear function. This represents the mean or preset median of the Laplacian plot. Starting from the smallest scale of the pyramid, the image is reconstructed layer by layer upwards to obtain the enhanced dough image. The calculation formula is as follows: The smallest scale image (layer L) is: ； Images at other scales are reconstructed step by step as follows: ； In the formula; Indicates the reconstructed first... Scale-enhanced image of the dough. This indicates an upsampling operation on the enhanced image. This represents the pixel values ​​of the enhanced Laplacian image.

7. The dough impurity detection method based on image recognition according to claim 1, characterized in that, The morphological features include area, perimeter, aspect ratio, and radian. The training method for the convolutional neural network model includes: Acquire historical impurity detection training data, and divide the historical impurity detection training data into an impurity detection training set and an impurity detection test set. The historical impurity detection training data includes the morphological features of candidate impurity regions and their corresponding labels. The label includes "0" and "1", where "0" represents a non-target impurity region and "1" represents a target impurity region. Initialize a convolutional neural network as a classifier, take the morphological features of the impurity detection training set as the input of the classifier, and the labeled data as the output, train the classifier to obtain the classification network; The classification network is validated using an impurity detection test set. The classification network with an output accuracy greater than or equal to the preset accuracy is used as the trained convolutional neural network model.

8. A dough impurity detection system based on image recognition, implemented based on the dough impurity detection method based on image recognition according to any one of claims 1-7, characterized in that, include: The acquisition module is used to acquire an original dough image of the surface of the dough to be tested, and to divide the original dough image into multiple local regions; The decomposition module is used to calculate the covariance matrix of different color channels in each local region, and to perform eigenvalue decomposition on the covariance matrix to obtain color difference features that characterize the color differences in each local region. The fusion module is used to obtain the brightness gradient features of each local region, and fuse the color difference features and brightness gradient features of each local region to generate the first fused feature vector; The correction module is used to determine the background region based on the first fusion feature vector, and generate a second fusion feature vector of the background region through sparse representation training. The original dough image is then subjected to background difference correction based on the second fusion feature vector of the background region to obtain a background-corrected dough image. The enhancement module is used to perform multi-scale image enhancement processing on the background-corrected dough image to obtain an enhanced dough image; The detection module is used to perform threshold segmentation on the enhanced dough image, extract candidate impurity regions, and input the morphological features of the candidate impurity regions into a pre-trained convolutional neural network model to obtain the target impurity regions.

Images