Real-time sesame seed candy forming defect detection method and device based on AI vision

Abstract

The invention relates to the technical field of AI vision, in particular to a sesame seed candy forming defect real-time detection method and device based on AI vision. The method comprises the following steps: respectively collecting multimode images of qualified sesame seed candies, generating a qualified characteristic fingerprint set, and calculating sugar body light transmission uniformity and sesame adhesion density as a domain parameter set; constructing a probability distribution model, and setting an anomaly judgment threshold value and a domain parameter early warning threshold value; collecting a multi-mode image of the to-be-detected sesame seed candy in real time, extracting a to-be-detected feature fingerprint, and calculating the light-transmitting uniformity of a to-be-detected candy body and the sesame adhesion density; calculating a comprehensive abnormal score, and judging a defect; and capturing a low-confidence sample based on the comprehensive anomaly score, obtaining an artificial correction feedback sample, and updating a probability distribution model, an anomaly judgment threshold value and a domain parameter early warning threshold value by utilizing the feedback sample through online incremental learning. According to the invention, the detection cost of a high-yield production line can be reduced, and the model deployment period is shortened.

Description

Technical Field

[0001] This invention relates to the field of AI vision technology, and in particular to a method and device for real-time detection of defects in the forming of sesame candy based on AI vision. Background Technology

[0002] Before deploying a traditional AI detection system for defects in sesame candy forming, hundreds or thousands of defect samples need to be collected to train the model. However, defect samples are scarce on high-yield production lines, resulting in extremely high sample collection costs and long cycles, which seriously affects the efficiency of system deployment. Moreover, if the defect samples are not fully covered, the trained model is prone to misjudgment and missed judgment during detection, making it difficult to meet the production line's needs for accurate and efficient detection. Summary of the Invention

[0003] This invention addresses the problems of extremely high sample collection costs and long cycles in existing technologies, which severely affect system deployment efficiency and are prone to misjudgment and missed detection during testing, making it difficult to meet the precise and efficient testing needs of production lines. It provides a real-time detection method and device for sesame candy forming defects based on AI vision to solve these problems.

[0004] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: In a first aspect, the present invention provides a real-time detection method for defects in the forming of sesame candy based on AI vision, comprising: acquiring multi-mode images of qualified sesame candy samples under backlight and coaxial light modes respectively; extracting multi-mode image features to generate a qualified feature fingerprint set; and calculating the candy body light transmittance uniformity and sesame attachment density of each sample as a domain parameter set; constructing a probability distribution model based on the qualified feature fingerprint set; setting an anomaly judgment threshold and a domain parameter warning threshold by combining the distribution of the candy body light transmittance uniformity and sesame attachment density; acquiring multi-mode images of the sesame candy to be inspected in real time, extracting the feature fingerprint to be inspected, and calculating the candy body light transmittance uniformity and sesame attachment density; inputting the feature fingerprint to be inspected into the probability distribution model to calculate an initial anomaly score; generating a process consistency score by combining the candy body light transmittance uniformity and sesame attachment density; fusing the initial anomaly score and the process consistency score to obtain a comprehensive anomaly score and determining the defect; capturing low-confidence samples based on the comprehensive anomaly score and obtaining manually corrected feedback samples; and updating the probability distribution model, anomaly judgment threshold, and domain parameter warning threshold using online incremental learning through the feedback samples.

[0005] Optionally, a multi-mode image set of qualified sesame candy samples under backlight and coaxial light modes is acquired, including: deploying an imaging unit integrating a backlight illumination module and a coaxial light illumination module at the detection station, wherein the backlight illumination module is used to acquire the outline silhouette image of the sesame candy as a backlight image, and the coaxial light illumination module is used to acquire the surface texture image of the sesame candy as a coaxial light image; sequentially triggering the two illumination modules, simultaneously acquiring two paired images of the same sesame candy sample as multi-mode images, and repeatedly acquiring multi-mode images until a preset number is reached to form the multi-mode image set.

[0006] The process involves extracting multimodal image features to generate a qualified feature fingerprint set, and calculating the uniformity of light transmission and sesame attachment density of each sample as a domain parameter set. This includes: binarizing and extracting contours from the backlit image, calculating a shape descriptor feature vector, where the shape descriptor includes the Fourier descriptor and Hu moments of the contour; performing texture analysis and color space conversion on the coaxial light image, calculating a surface descriptor feature vector, where the surface descriptor includes the local binary mode histogram and color moments in HSV space; calculating the coefficient of variation of pixel grayscale values ​​within the sesame candy body region of the backlit image as the uniformity of light transmission; separating sesame particles from the candy background using a segmentation algorithm based on color and texture features in the coaxial light image, calculating the percentage of sesame coverage area to the visible area of ​​the sesame candy surface as the sesame attachment density; and normalizing and concatenating the shape descriptor feature vector, surface descriptor feature vector, uniformity of light transmission, and sesame attachment density to generate the qualified feature fingerprint set.

[0007] Optionally, a probability distribution model is constructed based on the qualified feature fingerprint set, and an anomaly judgment threshold and a neighborhood parameter warning threshold are set in combination with the distribution of the glycosome light transmittance uniformity and sesame attachment density. This includes: fitting the qualified feature fingerprint set with a Gaussian mixture model to obtain the probability distribution model; calculating the log-likelihood value of all qualified feature fingerprints and taking its fifth percentile as the anomaly judgment threshold; statistically analyzing the distribution range of the glycosome light transmittance uniformity value and the distribution range of the sesame attachment density value of qualified samples, and expanding the upper and lower limits of the distribution by a predetermined proportion to form the neighborhood parameter warning threshold.

[0008] The process of fitting the qualified feature fingerprint set with a Gaussian mixture model to obtain the probability distribution model includes: standardizing the qualified feature fingerprint set so that the mean of each feature dimension is 0 and the variance is 1; determining the number K of Gaussian components in the Gaussian mixture model based on the Bayesian information criterion, and initializing the mean, covariance matrix, and mixing weights of each Gaussian component using the results of the K-means clustering algorithm; when initializing the covariance matrix of each component of the Gaussian mixture model, setting the initial variance values ​​of the feature dimensions corresponding to the glycosome light transmittance uniformity and sesame attachment density to be greater than the initial variance values ​​of other feature dimensions; and using the expectation-maximization algorithm to fit the model, calculating the posterior probability of each feature fingerprint belonging to each Gaussian component in the E-step, and updating the mixing weights, mean, and covariance matrix of each Gaussian component based on the posterior probability in the M-step, iterating until convergence.

[0009] Optionally, a process consistency score is generated by combining the uniformity of sugar transmittance and the density of sesame seeds. The initial anomaly score and the process consistency score are then fused to obtain a comprehensive anomaly score, and defects are determined. This includes: when any real-time domain parameter exceeds the corresponding warning threshold, it is determined as a process consistency deviation; the degree of deviation is quantified and converted into a process consistency score through a preset mapping function; the initial anomaly score and the process consistency score are fused according to preset weights to generate a comprehensive anomaly score; the preset weights increase the weight ratio of the process consistency score when there is a process consistency deviation, and the increase is proportional to the degree of deviation; based on the comprehensive anomaly score and the preset comprehensive judgment threshold, the final defect judgment result is output, and the defect information corresponding to the out-of-limit domain parameter is output simultaneously.

[0010] Optionally, low-confidence samples are captured based on the comprehensive anomaly score, and manual correction feedback is obtained, including: pre-setting a low-confidence interval for the comprehensive anomaly score; when the comprehensive anomaly score of the sesame candy to be tested falls into the low-confidence interval, the multi-modal image, feature fingerprint, neighborhood parameters, and judgment result of the sesame candy to be tested are automatically saved as judgment information and marked as a low-confidence sample; the judgment information of the low-confidence sample is displayed through a human-computer interaction interface, and the operator's input of confirmation, misjudgment correction, and missed detection labeling information is received, as well as confirmation or correction of the observed values ​​of candy transmittance uniformity and sesame attachment density; the corrected judgment information provided by the operator is stored in the calibration sample set as correction feedback.

[0011] Optionally, based on the comprehensive anomaly score, low-confidence samples are captured and manually corrected feedback samples are obtained. These feedback samples are then used to update the probability distribution model, anomaly judgment threshold, and domain parameter warning threshold through online incremental learning. This includes: initiating an incremental update process when the number of samples in the calibration sample set reaches a set batch size; adding the feature fingerprints of the calibration sample set to the qualified feature fingerprint set, and updating the existing domain parameter set using the domain parameters corrected by the feature fingerprints of the calibration sample set; employing an incremental learning algorithm based on elastic weight consolidation to fine-tune the probability distribution model parameters while constraining the probability distribution model to retain its memory of historical qualified sample distributions; and recalculating the anomaly judgment threshold and domain parameter warning threshold based on the updated qualified feature fingerprint set and domain parameter set to complete the collaborative update.

[0012] Optionally, the update of the field parameter warning threshold includes: after each update of the sesame adhesion density warning threshold, calculating the center value of the new sesame adhesion density warning threshold range; comparing the center value with the preset process target value of the sesame spreading device on the production line; if the center value continues to deviate from the preset process target value, generating a maintenance prompt that the sesame spreading uniformity needs to be calibrated; when the sugar body light transmittance uniformity warning threshold narrows by a preset range, prompting that the sugar body light transmittance uniformity has improved, and retrieving and storing the sugar boiling process parameters for the corresponding time period for review.

[0013] Secondly, the present invention provides a real-time detection device for defects in the forming of sesame candy based on AI vision, comprising: The multi-modal feature extraction module is used to collect multi-modal images of qualified sesame candy samples under backlight and coaxial light modes, extract multi-modal image features to generate a qualified feature fingerprint set, and calculate the candy body transmittance uniformity and sesame attachment density of each sample as a domain parameter set. The parameter threshold warning module is used to construct a probability distribution model based on the qualified feature fingerprint set, and set anomaly judgment threshold and domain parameter warning threshold in combination with the distribution of the glycobody light transmittance uniformity and sesame attachment density. The key value acquisition module is used to acquire multi-modal images of the sesame candy under test in real time, extract the feature fingerprint of the test, and calculate the light transmittance uniformity and sesame attachment density of the test candy. The defect type determination module is used to input the feature fingerprint to be inspected into the probability distribution model to calculate the initial anomaly score, and combine the light transmittance uniformity of the sugar body and the sesame adhesion density to generate a process consistency score. The initial anomaly score and the process consistency score are then fused to obtain a comprehensive anomaly score, and the defect is determined. The dynamic feedback update module is used to capture low-confidence samples based on the comprehensive anomaly score and obtain manually corrected feedback samples. The feedback samples are then used to update the probability distribution model, anomaly judgment threshold, and domain parameter warning threshold through online incremental learning.

[0014] By implementing this invention, it is possible to acquire multi-modal images of qualified sesame candy samples under backlight and coaxial light modes, extract multi-modal image features to generate a qualified feature fingerprint set, and calculate the candy body transmittance uniformity and sesame attachment density of each sample as a domain parameter set. The multi-modal images can comprehensively capture the contour and surface information of the sesame candy, the feature fingerprint set establishes a standard for normal samples, and the domain parameters focus on key production quality indicators, providing dual data support for subsequent anomaly judgment.

[0015] By implementing this invention, a probability distribution model can be constructed based on the qualified feature fingerprint set. An anomaly judgment threshold and a domain parameter warning threshold can be set by combining the distribution of the glycosome transmittance uniformity and sesame attachment density. The probability distribution model accurately describes the feature patterns of normal samples. The anomaly judgment criteria are clearly defined by the reasonably set dual thresholds, avoiding misjudgment or missed judgment caused by a single threshold, and providing a reliable basis for detection.

[0016] By implementing this invention, it is possible to acquire multi-modal images of the sesame candy to be tested in real time, extract the fingerprint of the features to be tested, calculate the light transmittance uniformity and sesame adhesion density of the candy to be tested, maintain consistency with the qualified sample collection, ensure that the data to be tested can be compared with the standard data, and adapt the real-time acquisition characteristics to the high-efficiency testing needs of the production line, laying the foundation for subsequent rapid judgment.

[0017] By implementing this invention, it is possible to input the fingerprint of the feature to be inspected into the probability distribution model to calculate an initial anomaly score, and combine the uniformity of light transmission of the sugar body and the density of sesame attachment to generate a process consistency score. The initial anomaly score and the process consistency score are then fused to obtain a comprehensive anomaly score, and defects are identified. The fusion of the dual scores takes into account both feature deviation and production process consistency, thereby improving the accuracy of defect identification. Exceeding limit defect information is output simultaneously, which facilitates rapid identification of the problem type.

[0018] By implementing this invention, it is possible to capture low-confidence samples based on the comprehensive anomaly score and obtain manually corrected feedback samples. The probability distribution model, anomaly judgment threshold, and domain parameter warning threshold are updated through online incremental learning using the feedback samples. The model bias is corrected with the help of manual feedback. Incremental learning allows the model to adapt to subtle changes in the production process, continuously optimizes the threshold, and enables the detection system to be adaptive and maintain high detection accuracy over a long period of time.

[0019] In summary, by implementing this invention, an initial detection system can be built using only qualified samples without relying on defective samples, enabling real-time and accurate detection of defects in sesame candy forming. At the same time, the system performance is continuously optimized through online incremental learning to adapt to changes in the production process, balancing detection efficiency and accuracy, reducing the detection cost of high-yield production lines, and shortening the model deployment cycle. Attached Figure Description

[0020] Figure 1 A flowchart illustrating the real-time detection method for sesame candy forming defects based on AI vision provided by this invention; Figure 2 This is a schematic diagram of the structure of the AI ​​vision-based real-time detection device for defects in the molding of sesame candy provided by the present invention.

[0021] In the attached diagram, the components represented by each number are as follows: Multi-modal feature extraction module 11, parameter threshold early warning module 12, key value acquisition module 13, defect type determination module 14, dynamic feedback update module 15. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0024] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0025] Example 1, as Figure 1 As shown, this embodiment of the invention provides a method and apparatus for real-time detection of defects in sesame candy forming based on AI vision, including: S100: Collect multi-mode images of qualified sesame candy samples under backlight and coaxial light modes respectively, extract multi-mode image features to generate qualified feature fingerprint set, and calculate the candy body transmittance uniformity and sesame attachment density of each sample as the domain parameter set. S200: Construct a probability distribution model based on the qualified feature fingerprint set, and set an anomaly judgment threshold and a domain parameter warning threshold by combining the distribution of the glycobody light transmittance uniformity and sesame attachment density; S300: Real-time acquisition of multi-modal images of the sesame candy to be tested, extraction of the fingerprint of the feature to be tested, and calculation of the light transmittance uniformity and sesame adhesion density of the candy to be tested; S400: Input the fingerprint of the feature to be inspected into the probability distribution model to calculate the initial anomaly score, and combine the light transmittance uniformity of the sugar body and the sesame adhesion density to generate a process consistency score. Combine the initial anomaly score and the process consistency score to obtain a comprehensive anomaly score, and determine the defect. S500: Based on the comprehensive anomaly score, capture low-confidence samples and obtain manually corrected feedback samples. Use the feedback samples to update the probability distribution model, anomaly judgment threshold, and domain parameter warning threshold through online incremental learning.

[0026] In step S100 of this embodiment, the acquisition of a multi-mode image set of qualified sesame candy samples under backlight and coaxial light modes includes: An imaging unit integrating a backlight illumination module and a coaxial light illumination module is deployed at the inspection station. The backlight illumination module is used to acquire the outline silhouette image of the sesame candy as a backlight image, and the coaxial light illumination module is used to acquire the surface texture image of the sesame candy as a coaxial light image. Two lighting modules are triggered sequentially to simultaneously acquire two paired images of the same sesame candy sample as multi-mode images. The acquisition of multi-mode images is repeated until a preset number is reached to form the multi-mode image set.

[0027] In this embodiment, the core purpose of step S100 is to acquire image data that comprehensively reflects the characteristics of qualified sesame candy, providing a foundation for subsequent generation of qualified feature fingerprint sets and construction of detection models. Backlit images can capture sesame candy contour-related features, while coaxial light images can record surface texture and sesame distribution-related information. The combination of these two types of images can cover key dimensions such as sesame candy shape and surface condition, ensuring comprehensive identification of defects during subsequent detection.

[0028] Specifically, to achieve the above objectives, an imaging unit integrating a backlight illumination module and a coaxial light illumination module needs to be deployed at the inspection station. The backlight illumination module is used to acquire the outline silhouette image of the candy as a backlight image, and the coaxial light illumination module is used to acquire the surface texture image of the candy as a coaxial light image.

[0029] Specifically, an imaging device integrating two types of lighting modules needs to be installed at the inspection station. The backlight module generates a silhouette image of the candy's outline, i.e., a backlit image. This image clearly shows the candy's shape and outline, indicating whether there are any missing corners, deformations, or other issues.

[0030] The coaxial lighting module generates a surface texture image of the sesame candy, also known as a coaxial light image. This image clearly shows the texture details of the sesame candy surface and the adhesion of the sesame seeds, such as whether the sesame seeds are evenly distributed.

[0031] Next, the two lighting modules are triggered in sequence to simultaneously acquire two paired images of the same sesame candy sample as multi-mode images. The acquisition of multi-mode images is repeated until a preset number is reached to form the multi-mode image set.

[0032] That is, the backlight illumination module and the coaxial light illumination module are triggered in sequence. While each module is triggered, the same sesame candy sample is photographed to obtain a set of paired images containing the backlight image and the coaxial light image. This set of images is the multi-mode image.

[0033] Repeat the above acquisition process until the number of multimodal images acquired reaches the preset number, ultimately forming a multimodal image set. The preset number needs to be determined based on the actual detection accuracy requirements; for example, it can be set to 500 sets to ensure that the sample data is sufficiently representative.

[0034] In step S100 of this embodiment, multimodal image features are extracted to generate a qualified feature fingerprint set, and the glycoplasma transmittance uniformity and sesame seed adhesion density of each sample are calculated as a domain parameter set, including: The backlit image is binarized and its contour is extracted. The shape descriptor feature vector is calculated, and the shape descriptor includes the Fourier descriptor and Hu moment of the contour. Texture analysis and color space conversion are performed on coaxial light images to calculate surface descriptor feature vectors, wherein the surface descriptor includes a local binary mode histogram and color moments in HSV space; In the main area of ​​the candy in the backlit image, the coefficient of variation of pixel grayscale values ​​is calculated as the light transmission uniformity of the candy. In coaxial light images, a segmentation algorithm based on color and texture features is used to separate sesame particles from the background of the candy. The percentage of the sesame-covered area to the visible area of ​​the candy surface is calculated as the sesame attachment density. The shape descriptor feature vector, surface descriptor feature vector, glycoplasma transmittance uniformity, and sesame attachment density are normalized and spliced ​​to generate the qualified feature fingerprint set.

[0035] In this embodiment, the purpose of the above steps is to extract key information from the multi-modal image that can accurately characterize the features of qualified sesame candy, forming a qualified feature fingerprint set and a domain parameter set, providing data support for subsequent construction of probability distribution models and setting judgment thresholds. Shape descriptors reflect the shape characteristics of the sesame candy, surface descriptors reflect surface texture and color characteristics, and the uniformity of light transmission and sesame seed adhesion density reflect process consistency-related characteristics. The combination of these four elements can comprehensively define the characteristic standards for qualified sesame candy, ensuring that defects deviating from the standards can be accurately identified subsequently.

[0036] To achieve the above objectives, it is first necessary to binarize and extract the contour of the backlit image, and calculate the shape descriptor feature vector, which includes the Fourier descriptor and Hu moment of the contour.

[0037] First, the backlit image is binarized to convert it into a black and white binary form, highlighting the boundary between the candy and the background. Then, a contour extraction algorithm is used to separate the contour lines of the candy.

[0038] Based on the extracted contours, shape descriptor feature vectors are calculated. Shape descriptors include Fourier descriptors and Hu moments. Fourier descriptors can compress contour information; for example, 10 Fourier coefficients can characterize the overall contour shape of the candy, determining whether there are issues such as bending or missing corners. Hu moments are rotation, scaling, and translation invariant. For instance, regardless of how much the candy is rotated or scaled in the image, the Hu moment values ​​stably reflect its geometric features, avoiding the impact of changes in placement on feature extraction.

[0039] Secondly, texture analysis and color space conversion are performed on the coaxial light image to calculate the surface descriptor feature vector, which includes the local binary mode histogram and color moments in the HSV space.

[0040] This involves performing texture analysis on coaxial light images to capture information such as the roughness and texture direction of the candy surface. Simultaneously, color space conversion is performed, transforming the image from RGB to HSV space.

[0041] Next, the surface descriptor feature vector is calculated. The surface descriptor includes the local binary mode histogram and the color moments in HSV space. The local binary mode histogram can characterize the details of surface texture, such as whether there are bumps or depressions on the surface of the candy. It is generated by statistically analyzing the gray-level contrast of pixels in local areas. For example, if the peak of the local binary mode histogram of a certain area is concentrated in the low gray-level difference range, it indicates that the surface of that area is smooth. The color moments in HSV space include the first moment mean, the second moment variance, and the third moment skewness. For example, the mean of the H channel in HSV space is 30, reflecting that the candy is generally pale yellow; the variance of the S channel is 5, indicating that the color saturation distribution of the candy is relatively uniform.

[0042] Next, within the main area of ​​the candy in the backlit image, the coefficient of variation of pixel grayscale values ​​is calculated as the uniformity of light transmission of the candy.

[0043] In the backlit image, the main area of ​​the candy is determined through contour extraction, excluding background and edge interference areas. Then, the grayscale values ​​of all pixels within this area are calculated, followed by the coefficient of variation (COP). The COP is calculated by dividing the standard deviation by the mean. For example, a qualified candy sample with a mean grayscale value of 150 and a standard deviation of 7.5 has a COP of 0.05. A smaller COP indicates more uniform light transmission. Conversely, a sample with a mean grayscale value of 140 and a standard deviation of 28 has a COP of 0.2, indicating poor uniformity of light transmission and potential defects such as internal bubbles or impurities.

[0044] Then, in the coaxial light image, a segmentation algorithm based on color and texture features is used to separate the sesame particles from the sugar background, and the percentage of the sesame-covered area to the visible area of ​​the sesame candy surface is calculated as the sesame attachment density.

[0045] In coaxial light images, a segmentation algorithm based on color and texture features is used. By utilizing the color differences between sesame seeds and sugar bodies (e.g., sesame seeds are darker brown while sugar bodies are lighter yellow) and the differences in texture (e.g., sesame seeds have a rougher surface), sesame seed particles are separated from the sugar body background.

[0046] Next, calculate the total coverage area of ​​the sesame seeds and the total area of ​​the visible area on the surface of the sesame candy. The ratio of the two is the sesame seed adhesion density. For example, if the total visible area of ​​the sesame candy surface is 200 square millimeters and the total coverage area of ​​the separated sesame seeds is 50 square millimeters, then the sesame seed adhesion density is 25%; if the sesame seed coverage area is 70 square millimeters, then the sesame seed adhesion density is 35%.

[0047] Finally, the shape descriptor feature vector, surface descriptor feature vector, glycoplasma transmittance uniformity, and sesame attachment density are normalized and spliced ​​to generate the qualified feature fingerprint set.

[0048] This involves normalizing the shape descriptor feature vector, surface descriptor feature vector, glycoparticle transmittance uniformity, and sesame attachment density. The min-max normalization method is used to map all feature values ​​to the interval between 0 and 1. For example, if a sample's Fourier descriptor vector has a value of 50 in one dimension, and the minimum value for that dimension across all samples is 10 and the maximum value is 100, the normalized value is 0.44; and the sesame attachment density is 30%, which is normalized to 0.5.

[0049] The normalized shape descriptor feature vector, surface descriptor feature vector, glycoparticle transmittance uniformity, and sesame attachment density are concatenated sequentially to form the feature fingerprint of a single qualified sample. For example, the concatenation results in a high-dimensional vector containing 10 Fourier descriptor coefficients, 7 Hu moment values, 20 local binary mode histogram features, 9 HSV color moment features, 1 glycoparticle transmittance uniformity value, and 1 sesame attachment density value. The feature fingerprints of all qualified samples combined together constitute the qualified feature fingerprint set.

[0050] In step S200 of this application embodiment, a probability distribution model is constructed based on the qualified feature fingerprint set, and an anomaly judgment threshold and a domain parameter warning threshold are set in combination with the distribution of glycosome light transmittance uniformity and sesame attachment density, including: The qualified feature fingerprint set is fitted using a Gaussian mixture model to obtain the probability distribution model; Calculate the log-likelihood value of all qualified feature fingerprints, and take the fifth percentile as the anomaly determination threshold. The distribution ranges of the uniformity of light transmittance of sugar bodies and the distribution range of the density of sesame attachment in qualified samples are statistically analyzed, and the upper and lower limits of the distribution are expanded by a predetermined proportion to form the warning threshold of the field parameter.

[0051] In this embodiment, the purpose of step S200 is to establish a characteristic distribution standard for qualified sesame candy and set clear and reasonable judgment thresholds to provide a basis for subsequent defect judgment of sesame candy to be inspected. The probability distribution model quantifies the distribution law of qualified features, the anomaly judgment threshold is used to determine whether the features to be inspected deviate from the qualified distribution, and the domain parameter warning threshold focuses on the acceptable range of key process indicators. The three together constitute the core judgment system for detection, ensuring the accuracy and pertinence of defect identification.

[0052] To achieve the above objectives, it is first necessary to use a Gaussian mixture model to fit the qualified feature fingerprint set to obtain the probability distribution model.

[0053] In step S200 of this application embodiment, a Gaussian mixture model is used to fit the qualified feature fingerprint set to obtain the probability distribution model, including: The qualified feature fingerprint set is standardized so that the mean of each feature dimension is 0 and the variance is 1. The number K of Gaussian components in the Gaussian mixture model is determined based on the Bayesian information criterion, and the mean, covariance matrix and mixing weight of each Gaussian component are initialized using the results of the K-means clustering algorithm. When initializing the covariance matrix of each component of the Gaussian mixture model, the initial variance values ​​of the feature dimensions corresponding to the glycoside light transmittance uniformity and sesame attachment density are set to be greater than the initial variance values ​​of other feature dimensions. The model is fitted using the expectation-maximization algorithm. In the E-step, the posterior probability of each feature fingerprint belonging to each Gaussian component is calculated. In the M-step, the mixing weight, mean, and covariance matrix of each Gaussian component are updated based on the posterior probability. The iteration continues until convergence.

[0054] Specifically, to obtain the probability distribution model, the first step is to standardize the qualified feature fingerprint set so that the mean of each feature dimension is 0 and the variance is 1.

[0055] This involves standardizing each feature dimension of the qualified feature fingerprint set, resulting in a mean of 0 and a variance of 1 for each feature dimension after standardization. For example, a feature dimension might be glycosome transmittance uniformity, with an original value range of 0.03 to 0.1. After standardization, this dimension's value will be mapped to the range of -1.2 to 1.8. Another feature dimension might be Fourier descriptor coefficients, with an original value range of 10 to 50. After standardization, these coefficients will also be mapped to the range of -1.5 to 2.1, thus achieving dimensional uniformity for different feature dimensions.

[0056] The second step is to determine the number K of Gaussian components in the Gaussian mixture model based on the Bayesian information criterion, and use the results of the K-means clustering algorithm to initialize the mean, covariance matrix and mixing weight of each Gaussian component. When initializing the covariance matrix of each component of the Gaussian mixture model, the initial variance values ​​of the feature dimensions corresponding to the glycoside light transmittance uniformity and sesame attachment density are set to be greater than the initial variance values ​​of other feature dimensions. This involves calculating the model score for different numbers of Gaussian components based on the Bayesian information criterion, and selecting the optimal number of Gaussian components, K. For example, if a qualified feature fingerprint set has 1000 samples and a feature dimension of 50, the model score is highest when K=4 after calculation using the Bayesian information criterion, thus determining that the Gaussian mixture model consists of 4 Gaussian components.

[0057] The standardized qualified feature fingerprint set was clustered using the K-means clustering algorithm to obtain K cluster centers. These cluster centers were used as the initial mean of each Gaussian component, and the covariance matrix and mixing weights of each component were initialized simultaneously.

[0058] When initializing the covariance matrix, the initial variance values ​​of the feature dimensions corresponding to sugar body light transmittance uniformity and sesame attachment density are set to be greater than those of other feature dimensions. For example, the initial variance of other feature dimensions is set to 1, the initial variance of the dimension corresponding to sugar body light transmittance uniformity is set to 3, and the initial variance of the dimension corresponding to sesame attachment density is set to 3.5, highlighting the influence weight of these two key process parameters.

[0059] The second step is to use the expectation-maximization algorithm to fit the model. In the E step, the posterior probability of each feature fingerprint belonging to each Gaussian component is calculated. In the M step, the mixing weight, mean and covariance matrix of each Gaussian component are updated based on the posterior probability. The process is iterated until convergence.

[0060] Step E involves calculating the posterior probability of each qualified feature fingerprint belonging to each Gaussian component. For example, the posterior probability of a feature fingerprint belonging to the first Gaussian component is 0.6, to the second is 0.2, to the third is 0.15, and to the fourth is 0.05. This posterior probability reflects the degree of matching between the feature fingerprint and each component.

[0061] The M-step involves updating the mixture weights, mean, and covariance matrix of each Gaussian component based on the posterior probabilities of all feature fingerprints. For example, the mixture weight of the first Gaussian component is updated from the initial 0.25 to 0.3, the mean is adjusted to a new vector based on the weighted average of the posterior probabilities, and the covariance matrix is ​​also recalculated based on the posterior probabilities and the new mean.

[0062] Repeat the E-step and M-step until the model parameters no longer change significantly, i.e., convergence is achieved. For example, after 30 iterations, if the change in the mean of each component is less than 0.001, it is considered convergence, and a stable probability distribution model is finally obtained.

[0063] Furthermore, it is necessary to calculate the log-likelihood value of all qualified feature fingerprints and take its fifth percentile as the anomaly determination threshold.

[0064] Specifically, this requires iterating through all eligible fingerprint features, inputting each fingerprint feature into the constructed Gaussian mixture model, and calculating the log-likelihood value of each fingerprint feature in the model. The larger the log-likelihood value, the more closely the fingerprint feature conforms to the distribution of eligible samples. For example, if the log-likelihood value of one eligible fingerprint feature is -10, and the log-likelihood value of another fingerprint feature is -18, the former is closer to the eligible distribution than the latter.

[0065] Next, collect the log-likelihood values ​​of all qualified fingerprint features and sort them in ascending order. The value at the fifth percentile after sorting is selected as the anomaly detection threshold. For example, if there are 1000 qualified samples, and the 50th value (1000 × 5% = 50) after sorting the log-likelihood values ​​is -16, then the anomaly detection threshold is -16. Subsequent samples with a log-likelihood value lower than -16 will be initially identified as anomalies.

[0066] Finally, it is necessary to statistically analyze the distribution range of the uniformity of light transmittance of sugar bodies and the distribution range of the density of sesame attachment in qualified samples, and expand the upper and lower limits of the distribution by a predetermined ratio to form the warning threshold of the field parameter.

[0067] This involves statistically analyzing the light transmittance uniformity values ​​of all qualified samples to determine their distribution range, i.e., the minimum and maximum values. For example, after statistical analysis, the minimum light transmittance uniformity of the glycosomes is found to be 0.04, and the maximum value is 0.09, with a distribution range of 0.04 to 0.09.

[0068] Next, the sesame seed adhesion density values ​​of all qualified samples were statistically analyzed to determine their distribution range, i.e., the minimum and maximum values. For example, the statistical analysis showed that the minimum sesame seed adhesion density was 22%, the maximum was 38%, and the distribution range was 22% to 38%.

[0069] Set a predetermined ratio, which is determined based on the stability and fault tolerance requirements of the production process. For example, set the predetermined ratio to 8%.

[0070] The upper and lower limits of the distribution are expanded according to a predetermined ratio, and the warning thresholds for the domain parameters are calculated. For example, in the above example, the lower limit of the warning threshold for glycoparticle transmittance uniformity is 0.04×(1-8%)=0.0368, and the upper limit is 0.09×(1+8%)=0.0972; the lower limit of the warning threshold for sesame attachment density is 22%×(1-8%)=20.24%, and the upper limit is 38%×(1+8%)=41.04%.

[0071] In step S300 of the embodiment of this application, it is necessary to acquire multi-mode images of the sesame candy to be tested in real time, extract the feature fingerprint to be tested, and calculate the light transmittance uniformity and sesame attachment density of the candy to be tested. In this embodiment, the purpose of step S300 is to obtain the core feature data of the sesame candy to be inspected, providing a basis for comparison in subsequent defect determination. By acquiring multi-modal images, extracting feature fingerprints, and calculating key process parameters, the features of the sesame candy sample to be inspected can be quantitatively compared with the standard features of qualified samples, ensuring accurate determination of whether the sesame candy to be inspected has defects.

[0072] To achieve this step, firstly, at the detection station, an imaging unit integrating backlight and coaxial light illumination modules needs to be used to sequentially trigger the two illumination modules on the sesame candy to be tested, acquiring the backlight image and coaxial light image of the same sesame candy sample to be tested, forming a paired multi-mode image.

[0073] Then, following the same method as for qualified samples, the backlit image is binarized and contour extracted, and the shape descriptor vector composed of the Fourier descriptor and Hu moments is calculated; the coaxial light image is textured and color space converted, and the surface descriptor vector composed of the local binary mode histogram and HSV color moments is calculated; the two types of vectors are normalized and then concatenated to obtain the fingerprint of the feature to be detected.

[0074] Then, in the main area of ​​the sesame candy to be tested in the backlit image, the coefficient of variation of the pixel gray value is calculated as the uniformity of light transmission of the candy. For example, the coefficient of variation of a certain sesame candy sample is 0.12. In the coaxial light image, the sesame seeds and the candy are separated by the segmentation algorithm, and the proportion of sesame seed coverage area is calculated as the sesame seed attachment density. For example, the sesame seed attachment density of a certain sesame candy sample is 18%.

[0075] In step S400 of this application embodiment, the fingerprint feature to be inspected needs to be input into the probability distribution model to calculate the initial anomaly score, and the process consistency score is generated by combining the light transmittance uniformity of the sugar body and the sesame adhesion density. The initial anomaly score and the process consistency score are then fused to obtain the comprehensive anomaly score, and the defect is determined. In step S400 of this application embodiment, a process consistency score is generated by combining the uniformity of sugar light transmission and the density of sesame seeds. This initial anomaly score is then fused with the process consistency score to obtain a comprehensive anomaly score, and defects are determined, including: When any real-time domain parameter exceeds the corresponding warning threshold, it is determined as a process consistency deviation. The degree of deviation is quantified and converted into a process consistency score through a preset mapping function. The initial anomaly score and the process consistency score are merged according to a preset weight to generate a comprehensive anomaly score; the preset weight increases the weight ratio of the process consistency score when the process consistency deviates, and the increase is proportional to the degree of deviation. The final defect judgment result is output based on the comprehensive anomaly score and the preset comprehensive judgment threshold, and the defect information corresponding to the out-of-limit domain parameters is output simultaneously.

[0076] In this embodiment, the purpose of step S400 is to accurately determine whether the sesame candy to be inspected has defects by fusing multi-dimensional scores. The initial anomaly score reflects the degree of deviation between the features to be inspected and the qualified distribution, while the process consistency score focuses on the compliance of key process parameters. The comprehensive anomaly score obtained by fusing the two with dynamic weights can take into account both feature matching degree and process stability, making defect judgment more comprehensive and accurate, while also clarifying the process problems associated with the defects.

[0077] To achieve the above objectives, firstly, the fingerprint of the feature to be detected needs to be input into the probability distribution model to calculate the initial anomaly score. That is, the fingerprint of the feature to be detected is input into the constructed probability distribution model, the log-likelihood value of the fingerprint is calculated by the probability distribution model, and this value is then converted into an initial anomaly score. The initial anomaly score is negatively correlated with the log-likelihood value; the lower the log-likelihood value, the higher the initial anomaly score. For example, if the log-likelihood value of a sample of sesame candy to be detected is -18, which is lower than the anomaly detection threshold of -16, its initial anomaly score is set to 85; if the log-likelihood value of another sample of sesame candy to be detected is -14, which is higher than the threshold, its initial anomaly score is set to 30.

[0078] Next, when any real-time domain parameter exceeds the corresponding warning threshold, it is determined to be a process consistency deviation. The degree of deviation is quantified and converted into a process consistency score through a preset mapping function.

[0079] This involves comparing the uniformity of light transmittance and the density of sesame seeds attached to the tested sesame candy sample with the corresponding warning thresholds for relevant parameters. If any parameter exceeds the warning threshold, it is considered a deviation from process consistency. For example, the warning threshold for uniform light transmittance is 0.0368 to 0.0972, while the tested sesame candy sample has a value of 0.11, exceeding the upper limit; the warning threshold for sesame seed attachment density is 20.24% to 41.04%, while the tested sesame candy sample has a value of 19%, exceeding the lower limit. Both are considered deviations from process consistency.

[0080] The degree of deviation is quantified as follows: Degree of deviation = (Value of parameter to be tested - Threshold boundary value) / Threshold range. For example, the degree of deviation of the light transmittance uniformity of the above-mentioned glycosomes = (0.11-0.0972) / (0.0972-0.0368)≈0.202.

[0081] The deviation is then converted into a process consistency score using a preset mapping function. The mapping function can be set to a linear mapping, where the greater the deviation, the higher the process consistency score. For example, a deviation of 0 corresponds to a score of 0, a deviation of 0.5 corresponds to a score of 100, and the process consistency score corresponding to the above sugar transmittance uniformity is 40.4 points.

[0082] Then, the initial anomaly score and the process consistency score need to be merged according to a preset weight to generate a comprehensive anomaly score; the preset weight increases the weight ratio of the process consistency score when the process consistency deviates, and the increase is proportional to the degree of deviation.

[0083] First, preset weights are set. Under normal circumstances, the initial weight for the anomaly score is 0.6, and the weight for the process consistency score is 0.4. When a deviation from process consistency exists, the weight of the process consistency score is increased proportionally to the degree of deviation. For example, if the deviation in the light transmittance uniformity of the aforementioned sugar is 0.202, the weight of the process consistency score is increased to 0.4 + 0.202 × 0.3 = 0.4606, and the initial weight for the anomaly score is adjusted accordingly to 0.5394. The basis for weight adjustment can be historical data statistics or process sensitivity analysis.

[0084] The overall anomaly score is calculated based on the adjusted anomaly score weights. Overall anomaly score = initial anomaly score × initial weight + process consistency score × process weight. For example, the above-mentioned sample of sesame candy has an initial anomaly score of 85 and a process consistency score of 40.4. The overall anomaly score = 85 × 0.5394 + 40.4 × 0.4606 ≈ 65.7.

[0085] Finally, the final defect judgment result needs to be output based on the comprehensive anomaly score and the preset comprehensive judgment threshold, and the defect information corresponding to the out-of-limit domain parameters should be output simultaneously.

[0086] Specifically, a preset comprehensive judgment threshold is set, for example, 70 points. If the comprehensive abnormality score of the sample to be tested is higher than 70 points, it is judged as defective; if it is lower than or equal to 70 points, it is judged as qualified. At the same time, the defect information corresponding to the parameters in the out-of-limit area is output synchronously. For example, if the light transmittance uniformity of the sample exceeds the warning threshold, the relevant defect information of light transmittance non-uniformity of the sample is output.

[0087] In step S500 of this application embodiment, capturing low-confidence samples based on the comprehensive anomaly score and obtaining manual correction feedback includes: Preset the low confidence interval for the overall anomaly score; When the overall anomaly score of the sesame candy to be tested falls into the low confidence interval, the multimodal image, feature fingerprint, neighborhood parameters and judgment result of the sesame candy to be tested are automatically saved as judgment information and marked as a low confidence sample. The system displays the judgment information of the low-confidence samples through a human-computer interaction interface, receives confirmation, misjudgment correction, and missed detection labeling information input by the operator, and confirms or corrects the observed values ​​of sugar body light transmittance uniformity and sesame attachment density. The corrected judgment information provided by the operator is stored as correction feedback in the calibration sample set.

[0088] In this embodiment, the purpose of step S500 is to screen out low-confidence samples with uncertain judgment results, obtain accurate feedback through manual correction, and provide reliable data for subsequent incremental updates of the model and thresholds. This avoids misjudgments and missed detections caused by initial model settings or production fluctuations, and improves the long-term accuracy and adaptability of the detection system.

[0089] To achieve the above steps, it is first necessary to preset a low confidence interval for the overall anomaly score; that is, to define a low confidence interval based on the overall judgment threshold, which is located near the critical range between qualified and defective. For example, if the overall judgment threshold is 70 points, the low confidence interval can be set to 60 to 80 points. Samples whose scores fall into this interval have a lower reliability in determining whether they have defects.

[0090] Next, when the overall anomaly score of the test candy falls into the low confidence interval, the multimodal image, feature fingerprint, neighborhood parameters and judgment result of the test candy need to be automatically saved as judgment information and marked as a low confidence sample.

[0091] The system monitors the overall anomaly score of the sample in real time. When the score falls into a preset low-confidence range, the system automatically marks the sample as a low-confidence sample. It simultaneously saves key judgment information for the sample, including multi-modal images (backlit and coaxial images), the feature fingerprint to be detected, two domain parameters (candy body transmittance uniformity and sesame seed adhesion density), and the current preliminary judgment result. For example, if a sample has an overall anomaly score of 68, falling into the low-confidence range of 60 to 80, the system will save its clearly outlined backlit image, its distinctly textured coaxial image, its feature fingerprint composed of Fourier descriptors, a candy body transmittance uniformity of 0.105, a sesame seed adhesion density of 39%, and a preliminary qualified result.

[0092] Then, the judgment information of the low confidence sample is displayed through the human-computer interaction interface, and the operator inputs confirmation, misjudgment correction, and missed detection labeling information, as well as confirmation or correction of the observed values ​​of sugar body light transmittance uniformity and sesame attachment density. This involves displaying all saved information about low-confidence samples to the operator through a human-computer interaction interface. The operator then makes a manual judgment based on the image and parameter information, providing corresponding feedback. This feedback includes confirming the accuracy of the preliminary judgment, correcting misjudgments (such as initial judgments of compliance indicating actual defects), marking missed defects, and confirming or correcting the observed values ​​of glycogen transmittance uniformity and sesame attachment density. For example, if the operator finds that the actual sesame attachment density of the above sample is 42%, exceeding the warning threshold and judging it as a defect, they would input the corrected judgment result and the corrected sesame attachment density value.

[0093] Finally, the corrected judgment information provided by the operator is stored in the calibration sample set as correction feedback. This means that all the operator-corrected judgment information, including the corrected parameter values ​​and the final judgment results, is uniformly stored in the calibration sample set to provide data for subsequent model and threshold updates.

[0094] In step S500 of this embodiment, low-confidence samples are captured based on the comprehensive anomaly score, and manually corrected feedback samples are obtained. The probability distribution model, anomaly detection threshold, and domain parameter warning threshold are updated using online incremental learning based on the feedback samples, including: When the number of samples in the calibration sample set reaches the set batch size, the incremental update process is initiated. Add the feature fingerprints of the calibration sample set to the qualified feature fingerprint set, and update the existing domain parameter set using the domain parameters corrected by the feature fingerprints of the calibration sample set. An incremental learning algorithm based on elastic weight consolidation is adopted to fine-tune the parameters of the probability distribution model while constraining the probability distribution model to retain the memory of the distribution of historical qualified samples. Based on the updated qualified feature fingerprint set and domain parameter set, the anomaly judgment threshold and domain parameter early warning threshold are recalculated to complete the collaborative update.

[0095] In this embodiment, the purpose of step S500 is to utilize reliable feedback samples after manual correction to dynamically optimize the core components of the detection system through online incremental learning. This allows the probability distribution model, anomaly detection threshold, and domain parameter warning threshold to adapt to fluctuations in production processes or changes in sample characteristics, continuously improving the accuracy and stability of the detection system and reducing misjudgments and missed detections during long-term use.

[0096] To achieve the above objectives, the incremental update process must first be initiated when the number of samples in the calibration sample set reaches the set batch size.

[0097] This means setting the batch size as the update trigger condition. When the number of samples accumulated in the calibration sample set reaches the batch size, the incremental update process is automatically started. For example, if the batch size is set to 50, when the number of samples in the calibration sample set increases from 49 to 50, the system triggers an update command and begins subsequent parameter optimization operations.

[0098] Next, the feature fingerprints of the calibration sample set are added to the qualified feature fingerprint set, and the existing domain parameter set is updated using the domain parameters corrected by the feature fingerprints of the calibration sample set.

[0099] The feature fingerprints of all samples in the calibration sample set are added to the original qualified feature fingerprint set, expanding the sample coverage of qualified features. For example, if the original qualified feature fingerprint set has 1000 samples and the calibration sample set has 50 samples, the updated qualified feature fingerprint set will contain 1050 samples. Then, the manually corrected glycosome transmittance uniformity and sesame seed adhesion density are extracted from the calibration sample set. These corrected neighborhood parameters are used to update the existing neighborhood parameter set, ensuring that the neighborhood parameters reflect the process characteristics of real qualified samples. For example, if the mean of glycosome transmittance uniformity in the original neighborhood parameter set is 0.06, and the corrected mean in the calibration sample set is 0.062, the mean of the updated neighborhood parameter set will be adjusted to 0.062.

[0100] Then, an incremental learning algorithm based on elastic weight consolidation is adopted to fine-tune the parameters of the probability distribution model while constraining the probability distribution model to retain the memory of the historical distribution of qualified samples. This involves employing an incremental learning algorithm based on elastic weight consolidation to fine-tune the parameters of the probability distribution model. This algorithm adds constraints to the weight parameters corresponding to the distribution of historical qualified samples in the model, preventing the model from forgetting historical features due to the addition of new samples. For example, the weight parameters representing the contour features of historical samples are kept stable through elastic weight constraints, while parameters related to the features of new samples are fine-tuned, allowing the model to adapt to the features of new samples without losing historical memory.

[0101] Finally, based on the updated qualified feature fingerprint set and the domain parameter set, the anomaly judgment threshold and the domain parameter early warning threshold are recalculated to complete the collaborative update.

[0102] In step S500 of this application embodiment, the updating of the field parameter early warning threshold includes: After each update of the sesame attachment density warning threshold, the center value of the new sesame attachment density warning threshold range is calculated; The center value is compared with the preset process target value of the sesame spreading device on the production line. If the center value continues to deviate from the preset process target value, a maintenance prompt indicating that the uniformity of sesame spreading needs to be calibrated is generated. When the warning threshold for the uniformity of light transmission of sugar body narrows by a preset range, it indicates that the uniformity of light transmission of sugar body has improved, and the sugar boiling process parameters for the corresponding time period are retrieved and associated for reference.

[0103] Specifically, to achieve the aforementioned collaborative update, it is first necessary to collaboratively update the anomaly detection threshold and the domain parameter early warning threshold.

[0104] Based on the updated qualified feature fingerprint set, the log-likelihood value of all feature fingerprints is recalculated, and the fifth percentile is taken as the new anomaly detection threshold. For example, if the updated qualified feature fingerprint set has 1050 samples, and the fifth percentile of the log-likelihood value after sorting is -15.2, then the original threshold of -15.5 is used. Based on the updated neighborhood parameter set, the distribution range of glycosome transmittance uniformity and sesame attachment density is recalculated, and a new neighborhood parameter warning threshold is obtained by expanding it according to a preset ratio. For example, if the updated sesame attachment density distribution range is 21% to 39%, and the preset ratio is 10%, then the new warning threshold is 18.9% to 42.9%.

[0105] Then, after each update of the sesame attachment density warning threshold, the center value of the new sesame attachment density warning threshold range is calculated.

[0106] The central value of the new warning threshold range is calculated using the formula (lower warning threshold + upper warning threshold) / 2. For example, if the new warning threshold is 18.9% to 42.9%, the central value is (18.9% + 42.9%) / 2 = 30.9%.

[0107] Next, the center value is compared with the preset process target value of the sesame spreading device on the production line. If the center value continues to deviate from the preset process target value, a maintenance prompt indicating that the uniformity of sesame spreading needs to be calibrated is generated.

[0108] This involves retrieving the preset process target value of the sesame spreading device on the production line. This value represents the ideal standard set for production, such as 30%.

[0109] Set a criterion for continuous deviation, such as if the deviation between the center value and the target value exceeds 2% after three consecutive updates. If this condition is met, a maintenance prompt indicating that the sesame seed distribution uniformity needs calibration is generated. For example, if the center value is 33%, 33.5%, and 34% for three consecutive updates, and the deviation from the target value of 30% exceeds 2%, a maintenance prompt is triggered.

[0110] In addition, when the warning threshold for the uniformity of light transmission of sugar body narrows by a preset range, it indicates that the uniformity of light transmission of sugar body has improved, and the sugar boiling process parameters for the corresponding time period are retrieved and associated for reference.

[0111] Specifically, a preset range needs to be set as the threshold narrowing judgment standard, such as 10%, and the calculation method is (threshold range before update - threshold range after update) / threshold range before update × 100%.

[0112] Calculate the threshold narrowing range. For example, before the update, the glycosome transmittance uniformity warning threshold range was 0.03496 to 0.1026 with a width of 0.06764; after the update, the range is 0.036 to 0.098 with a width of 0.062. The narrowing range is (0.06764-0.062) / 0.06764×100%≈8.34%, which is less than the preset range of 10% and no prompt is triggered. If the updated range is 0.038 to 0.095 with a width of 0.057, the narrowing range is (0.06764-0.057) / 0.06764×100%≈15.73%, which exceeds the preset range.

[0113] When the narrowing reaches the preset range, it indicates that the uniformity of light transmission of the sugar body has improved. At the same time, it retrieves the sugar cooking process parameters for the corresponding time period, including sugar cooking temperature of 110℃, sugar cooking time of 20 minutes, stirring speed of 30 rpm, etc., and stores them for later retrieval.

[0114] Example 2, as Figure 2 As shown, based on the same inventive concept as the AI ​​vision-based real-time detection method for sesame candy forming defects provided in Embodiment 1, this embodiment of the invention also provides an AI vision-based real-time detection device for sesame candy forming defects, including: The multi-mode feature extraction module 11 is used to collect multi-mode images of qualified sesame candy samples under backlight and coaxial light modes respectively, extract multi-mode image features to generate qualified feature fingerprint set, and calculate the candy body transmittance uniformity and sesame attachment density of each sample as a domain parameter set. The parameter threshold warning module 12 is used to construct a probability distribution model based on the qualified feature fingerprint set, and set an anomaly judgment threshold and a domain parameter warning threshold in combination with the distribution of the glycobody light transmittance uniformity and sesame attachment density. Key value acquisition module 13 is used to acquire multi-mode images of the sesame candy to be tested in real time, extract the fingerprint of the feature to be tested, and calculate the light transmittance uniformity and sesame attachment density of the candy to be tested. The defect type determination module 14 is used to input the feature fingerprint to be inspected into the probability distribution model to calculate the initial anomaly score, and combine the light transmittance uniformity of the sugar body and the sesame adhesion density to generate a process consistency score. The initial anomaly score and the process consistency score are fused to obtain a comprehensive anomaly score, and the defect is determined. The dynamic feedback update module 15 is used to capture low-confidence samples based on the comprehensive anomaly score and obtain manually corrected feedback samples, and use the feedback samples to update the probability distribution model, anomaly judgment threshold and domain parameter warning threshold through online incremental learning.

[0115] Furthermore, the multi-modal feature extraction module 11 includes the following execution steps: An imaging unit integrating a backlight illumination module and a coaxial light illumination module is deployed at the inspection station. The backlight illumination module is used to acquire the outline silhouette image of the sesame candy as a backlight image, and the coaxial light illumination module is used to acquire the surface texture image of the sesame candy as a coaxial light image. Two lighting modules are triggered sequentially to simultaneously acquire two paired images of the same sesame candy sample as multi-mode images. The acquisition of multi-mode images is repeated until a preset number is reached to form the multi-mode image set.

[0116] The backlit image is binarized and its contour is extracted. The shape descriptor feature vector is calculated, and the shape descriptor includes the Fourier descriptor and Hu moment of the contour. Texture analysis and color space conversion are performed on coaxial light images to calculate surface descriptor feature vectors, wherein the surface descriptor includes a local binary mode histogram and color moments in HSV space; In the main area of ​​the candy in the backlit image, the coefficient of variation of pixel grayscale values ​​is calculated as the light transmission uniformity of the candy. In coaxial light images, a segmentation algorithm based on color and texture features is used to separate sesame particles from the background of the candy. The percentage of the sesame-covered area to the visible area of ​​the candy surface is calculated as the sesame attachment density. The shape descriptor feature vector, surface descriptor feature vector, glycoplasma transmittance uniformity, and sesame attachment density are normalized and spliced ​​to generate the qualified feature fingerprint set.

[0117] Furthermore, the parameter threshold early warning module 12 includes the following execution steps: The qualified feature fingerprint set is fitted using a Gaussian mixture model to obtain the probability distribution model; Calculate the log-likelihood value of all qualified feature fingerprints, and take the fifth percentile as the anomaly determination threshold. The distribution ranges of the uniformity of light transmittance of sugar bodies and the distribution range of the density of sesame attachment in qualified samples are statistically analyzed, and the upper and lower limits of the distribution are expanded by a predetermined proportion to form the warning threshold of the field parameter.

[0118] The probability distribution model is obtained by fitting the qualified feature fingerprint set with a Gaussian mixture model, including: The qualified feature fingerprint set is standardized so that the mean of each feature dimension is 0 and the variance is 1. The number K of Gaussian components in the Gaussian mixture model is determined based on the Bayesian information criterion, and the mean, covariance matrix and mixing weight of each Gaussian component are initialized using the results of the K-means clustering algorithm. When initializing the covariance matrix of each component of the Gaussian mixture model, the initial variance values ​​of the feature dimensions corresponding to the glycoside light transmittance uniformity and sesame attachment density are set to be greater than the initial variance values ​​of other feature dimensions. The model is fitted using the expectation-maximization algorithm. In the E-step, the posterior probability of each feature fingerprint belonging to each Gaussian component is calculated. In the M-step, the mixing weight, mean, and covariance matrix of each Gaussian component are updated based on the posterior probability. The iteration continues until convergence.

[0119] Furthermore, the defect type determination module 14 includes the following execution steps: When any real-time domain parameter exceeds the corresponding warning threshold, it is determined as a process consistency deviation. The degree of deviation is quantified and converted into a process consistency score through a preset mapping function. The initial anomaly score and the process consistency score are merged according to a preset weight to generate a comprehensive anomaly score; the preset weight increases the weight ratio of the process consistency score when the process consistency deviates, and the increase is proportional to the degree of deviation. The final defect judgment result is output based on the comprehensive anomaly score and the preset comprehensive judgment threshold, and the defect information corresponding to the out-of-limit domain parameters is output simultaneously.

[0120] Furthermore, the dynamic feedback update module 15 includes the following execution steps: Preset the low confidence interval for the overall anomaly score; When the overall anomaly score of the sesame candy to be tested falls into the low confidence interval, the multimodal image, feature fingerprint, neighborhood parameters and judgment result of the sesame candy to be tested are automatically saved as judgment information and marked as a low confidence sample. The system displays the judgment information of the low-confidence samples through a human-computer interaction interface, receives confirmation, misjudgment correction, and missed detection labeling information input by the operator, and confirms or corrects the observed values ​​of sugar body light transmittance uniformity and sesame attachment density. The corrected judgment information provided by the operator is stored as correction feedback in the calibration sample set.

[0121] When the number of samples in the calibration sample set reaches the set batch size, the incremental update process is initiated. Add the feature fingerprints of the calibration sample set to the qualified feature fingerprint set, and update the existing domain parameter set using the domain parameters corrected by the feature fingerprints of the calibration sample set. An incremental learning algorithm based on elastic weight consolidation is adopted to fine-tune the parameters of the probability distribution model while constraining the probability distribution model to retain the memory of the distribution of historical qualified samples. Based on the updated qualified feature fingerprint set and domain parameter set, the anomaly judgment threshold and domain parameter early warning threshold are recalculated to complete the collaborative update.

[0122] The updating of the domain parameter early warning threshold includes: After each update of the sesame attachment density warning threshold, the center value of the new sesame attachment density warning threshold range is calculated; The center value is compared with the preset process target value of the sesame spreading device on the production line. If the center value continues to deviate from the preset process target value, a maintenance prompt indicating that the uniformity of sesame spreading needs to be calibrated is generated. When the warning threshold for the uniformity of light transmission of sugar body narrows by a preset range, it indicates that the uniformity of light transmission of sugar body has improved, and the sugar boiling process parameters for the corresponding time period are retrieved and associated for reference.

[0123] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0124] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0126] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0127] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0128] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Clearly, those skilled in the art can make various alterations and variations to the invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the invention and its equivalents, the invention is also intended to include these modifications and variations.

Claims

1. A real-time detection method for sugar forming defects based on AI vision, characterized in that, The method comprises the following steps: Collecting multi-mode images of qualified sesame sugar samples under backlight and coaxial light modes respectively, extracting multi-mode image features to generate a qualified feature fingerprint set, and calculating the sugar body light transmission uniformity and sesame adhesion density of each sample as a domain parameter set; Based on the qualified feature fingerprint set, a probability distribution model is constructed, and the distribution of the sugar body light transmission uniformity and the sesame adhesion density is combined to set an abnormality judgment threshold and a domain parameter warning threshold; Real-time collection of multi-mode images of the inspected sesame sugar, extraction of the inspected feature fingerprint, calculation of the inspected sugar body light transmission uniformity and sesame adhesion density; The inspected feature fingerprint is input into the probability distribution model to calculate an initial abnormality score, and the sugar body light transmission uniformity and the sesame adhesion density are combined to generate a process consistency score, the initial abnormality score and the process consistency score are fused to obtain a comprehensive abnormality score, and defects are judged; Based on the comprehensive abnormality score, low confidence samples are captured and artificial correction feedback samples are obtained, and the feedback samples are used to update the probability distribution model, the abnormality judgment threshold and the domain parameter warning threshold through online incremental learning. 2.The AI vision-based real-time detection method for sugar forming defects according to claim 1, characterized in that, A multi-mode image set of qualified sesame sugar samples is collected under backlight and coaxial light modes, comprising: Deploying an imaging unit integrated with a backlight illumination module and a coaxial light illumination module at a detection station, the backlight illumination module is used to obtain the outline silhouette image of the sesame sugar as the backlight image, and the coaxial light illumination module is used to obtain the surface texture image of the sesame sugar as the coaxial light image; Triggering the two illumination modules in turn, and simultaneously collecting two paired images of the same sesame sugar sample as multi-mode images, and repeating the collection of multi-mode images to reach a preset number to form the multi-mode image set. 3.The AI vision-based real-time detection method for sugar forming defects according to claim 2, characterized in that, Extracting multi-mode image features to generate a qualified feature fingerprint set, and calculating the sugar body light transmission uniformity and sesame adhesion density of each sample as a domain parameter set, comprising: Performing binaryzation and contour extraction on the backlight image, calculating a shape descriptor feature vector, and the shape descriptor includes Fourier descriptor and Hu moment of the contour; Performing texture analysis and color space conversion on the coaxial light image, calculating a surface descriptor feature vector, and the surface descriptor includes local binary pattern histogram and color moment in HSV space; In the main body area of the backlight image, the coefficient of variation of the pixel gray value is calculated as the sugar body light transmission uniformity; In the coaxial light image, a segmentation algorithm based on color and texture features is used to separate sesame particles and sugar body background, and the percentage of sesame coverage area to the visible area of sesame surface is calculated as the sesame adhesion density; The shape descriptor feature vector, the surface descriptor feature vector, the sugar body light transmission uniformity and the sesame adhesion density are normalized and spliced to generate the qualified feature fingerprint set. 4.The AI vision-based real-time detection method for sugar forming defects according to claim 1, characterized in that, Based on the qualified feature fingerprint set, a probability distribution model is constructed, and the distribution of the sugar body light transmission uniformity and the sesame adhesion density is combined to set an abnormality judgment threshold and a domain parameter warning threshold, comprising: A Gaussian mixture model is used to fit the qualified feature fingerprint set to obtain the probability distribution model; The log-likelihood values of all qualified feature fingerprints are calculated, and the fifth percentile is taken as the abnormality judgment threshold; The distribution range of the sugar body light transmission uniformity value of the qualified sample and the distribution range of the sesame adhesion density value are extended by a predetermined proportion to form the field parameter early warning threshold.

5. The AI vision-based real-time detection method for sugar forming defects according to claim 4, characterized in that, The Gaussian mixture model is used to fit the qualified feature fingerprint set to obtain the probability distribution model, including: The qualified feature fingerprint set is standardized to make the mean value of each feature dimension 0 and the variance 1; The number K of Gaussian components in the Gaussian mixture model is determined based on the Bayesian information criterion, and the results of the K-means clustering algorithm are used to initialize the mean value, covariance matrix and mixing weight of each Gaussian component. When initializing the covariance matrix of each component of the Gaussian mixture model, the initial variance value of the corresponding feature dimension of the sugar body light transmission uniformity and the sesame adhesion density is set to be greater than the initial variance value of other feature dimensions; The expectation maximization algorithm is used for model fitting. In the E step, the posterior probability of each feature fingerprint belonging to each Gaussian component is calculated. In the M step, the mixing weight, mean value and covariance matrix of each Gaussian component are updated based on the posterior probability, and the iteration is converged. 6.The AI vision-based real-time detection method for sugar forming defects according to claim 1, characterized in that, The sugar body light transmission uniformity and the sesame adhesion density are combined to generate a process consistency score. The initial anomaly score and the process consistency score are fused to obtain a comprehensive anomaly score, and defects are determined, including: When any real-time field parameter exceeds the corresponding early warning threshold, it is determined that the process consistency deviates, the deviation degree is quantified, and the process consistency score is generated by a preset mapping function; The initial anomaly score and the process consistency score are fused according to a preset weight to generate a comprehensive anomaly score. When the process consistency deviates, the weight ratio of the process consistency score is increased, and the increase is proportional to the deviation degree; Based on the comprehensive anomaly score and the preset comprehensive determination threshold, the final defect determination result is output, and the defect information corresponding to the out-of-limit field parameter is also output. 7.The AI vision-based real-time detection method for sugar forming defects according to claim 1, characterized in that, Based on the comprehensive anomaly score, low-confidence samples are captured and artificial correction feedback is obtained, including: A low-confidence interval of the comprehensive anomaly score is preset; When the comprehensive anomaly score of the sugar sample falls within the low-confidence interval, the multi-mode image, feature fingerprint, field parameter and determination result of the sugar sample are automatically saved as determination information and marked as low-confidence samples; The determination information of the low-confidence samples is displayed through a human-computer interaction interface, and confirmation, misjudgment correction and missed detection labeling information input by an operator are received, as well as confirmation or correction of the sugar body light transmission uniformity and sesame adhesion density observation values; The corrected determination information provided by the operator is stored in the calibration sample set as correction feedback. 8.The AI vision-based real-time detection method for sugar forming defects according to claim 1, characterized in that, Based on the comprehensive anomaly score, low-confidence samples are captured and artificial correction feedback samples are obtained. The feedback samples are used to update the probability distribution model, anomaly determination threshold and field parameter early warning threshold through online incremental learning, including: When the number of samples in the calibration sample set reaches a set batch size, the incremental update process is started; The feature fingerprint of the calibration sample set is added to the qualified feature fingerprint set, and the existing field parameter set is updated using the field parameter corrected by the feature fingerprint of the calibration sample set. An incremental learning algorithm based on elastic weight consolidation is adopted to fine-tune the parameters of the probability distribution model while preserving the memory of the probability distribution model for the distribution of historical qualified samples; Based on the updated qualified feature fingerprint set and the domain parameter set, the abnormality judgment threshold and the domain parameter warning threshold are recalculated to complete the collaborative update. 9.The AI vision-based real-time detection method for sugar forming defects according to claim 1, characterized in that, The updating of the domain parameter warning threshold comprises: After updating the sesame adhesion density warning threshold each time, the center value of the new sesame adhesion density warning threshold range is calculated; The center value is associated and compared with the preset process target value of the sesame spreading device on the production line. If the center value deviates from the preset process target value continuously, a maintenance prompt that the sesame spreading uniformity needs to be calibrated is generated; When the sugar body light transmission uniformity warning threshold is narrowed by a preset amplitude, the sugar body light transmission uniformity is improved, and the corresponding period of the sugar boiling process parameters is retrieved and associated for review.

10. AI vision-based real-time detection device for sugar forming defects, characterized in that, Comprise: A multi-mode feature extraction module is used to extract multi-mode images of qualified sesame candy samples under backlight and coaxial light modes, extract multi-mode image features to generate a qualified feature fingerprint set, and calculate the sugar body light transmission uniformity and sesame adhesion density of each sample as a domain parameter set; A parameter threshold warning module is used to construct a probability distribution model based on the qualified feature fingerprint set, set an abnormality judgment threshold and a domain parameter warning threshold in combination with the distribution of the sugar body light transmission uniformity and the sesame adhesion density; A key value acquisition module is used to acquire multi-mode images of the to-be-inspected sesame candy in real time, extract to-be-inspected feature fingerprints, and calculate the to-be-inspected sugar body light transmission uniformity and sesame adhesion density; A defect type judgment module is used to input the to-be-inspected feature fingerprints into the probability distribution model to calculate an initial abnormality score, generate a process consistency score in combination with the sugar body light transmission uniformity and the sesame adhesion density, fuse the initial abnormality score and the process consistency score to obtain a comprehensive abnormality score, and judge the defect; A dynamic feedback updating module is used to capture low-confidence samples based on the comprehensive abnormality score and obtain artificial correction feedback samples, and update the probability distribution model, the abnormality judgment threshold, and the domain parameter warning threshold through online incremental learning using the feedback samples.

Images