Food impurity online screening method and system based on visual recognition

Abstract

The application discloses a food impurity online screening method and system based on visual recognition, which pre-trains a general feature encoder offline through a large-scale multi-category food image set, establishes an efficient feature extraction basis, dynamically constructs a product prototype Gaussian mixture model online based on K standard product images to accurately depict the normal distribution, and fuses global probability density and local manifold curvature information when modeling an abnormal score to construct a comprehensive and robust abnormality discrimination basis, effectively solving the missed detection problem of traditional methods for hidden normal product multi-modal distribution abnormalities. In this way, the rapid adaptation capability of the screening system to new product categories is significantly improved, the new product model deployment time and cost are greatly shortened, and the accuracy and system robustness of impurity detection are significantly improved.

Description

Technical Field

[0001] This application relates to the field of intelligent sorting, and more specifically, to a method and system for online sorting of food impurities based on visual recognition. Background Technology

[0002] Food safety and quality are crucial cornerstones for people's health and social stability. Impurities introduced during food production not only affect product quality but also pose potential threats to consumer health. Therefore, efficient and accurate online impurity sorting of food is essential. Traditional manual sorting is inefficient and susceptible to subjective factors, failing to meet the stringent speed and precision requirements of modern food production. This has prompted the industry to continuously seek automated and intelligent solutions.

[0003] Currently, while online sorting technology for food impurities based on visual recognition has made some progress, it still faces significant challenges. Existing methods mostly employ a supervised learning paradigm, relying on a large number of known impurity samples for model training. This leads to the problem of rapid cold start for new product categories; that is, when faced with a small number of new product category samples, it is difficult to quickly build a high-precision sorting model, severely hindering rapid production line switchover and new product testing. The root cause is that traditional classification models misidentify unknown impurities as qualified products, and their feature extractors are often task-specific and difficult to generalize across product categories. This results in significant costs and time required for model retraining when facing different food categories or novel impurities. This limitation makes existing technologies inflexible and unadaptable to the diverse and rapidly changing production needs of the food industry. There is an urgent need for a new online sorting method and system for food impurities that can effectively overcome the shortcomings of existing technologies, especially achieving rapid adaptation and high-precision sorting for new product categories under conditions of small sample sizes, thereby improving the intelligence level and economic efficiency of food production.

[0004] Therefore, an optimized online sorting method for food impurities based on visual recognition is needed. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a visual recognition-based online food impurity sorting method and system. It establishes an efficient feature extraction foundation by offline pre-training a general-purpose feature encoder on a large-scale, multi-category food image set. For new food categories, based on K standard product images, a Gaussian mixture model of the product prototype is dynamically constructed online to accurately characterize its normal distribution. In anomaly score modeling, global probability density and local manifold curvature information are integrated to construct a comprehensive and robust anomaly discrimination criterion, effectively solving the problem of missed detection of anomalies hidden in the multimodal distribution of normal products using traditional methods. In this way, the sorting system's ability to quickly adapt to new food categories is significantly improved, the deployment time and cost of new product models are greatly shortened, and the accuracy and robustness of impurity detection are significantly enhanced.

[0006] According to one aspect of this application, a method for online sorting of food impurities based on visual recognition is provided, comprising:

[0007] A pre-trained encoder model is obtained by offline pre-training a general feature encoder based on a large-scale, multi-category food image set.

[0008] Based on K standard product images of new product categories and the pre-trained encoder model, an online dynamic prototype of the new product category is constructed to obtain a Gaussian mixture model of the product prototype.

[0009] Real-time material images captured by a high-speed industrial camera are input into a pre-trained encoder model to obtain material feature vectors;

[0010] Based on the Gaussian mixture model of the product prototype, anomaly scores are explicitly modeled on the material feature vector to obtain anomaly scores;

[0011] A triage decision is made based on a comparison between the anomaly score and the triage decision threshold to obtain a triage execution signal.

[0012] According to another aspect of this application, a visual recognition-based online food impurity sorting system is provided, comprising:

[0013] The offline pre-training module is used to perform offline pre-training of a general feature encoder based on a large-scale multi-category food image set to obtain a pre-trained encoder model.

[0014] The online construction module for dynamic prototypes of new product categories is used to construct dynamic prototypes of new product categories online based on K standard product images of new product categories and the pre-trained encoder model to obtain a Gaussian mixture model of the product prototype.

[0015] The material feature extraction module is used to input real-time material images captured by a high-speed industrial camera into a pre-trained encoder model to obtain material feature vectors.

[0016] The explicit anomaly score modeling module is used to explicitly model the anomaly scores of material feature vectors based on the Gaussian mixture model of the product prototype to obtain anomaly scores.

[0017] The sorting decision module is used to make sorting decisions based on the comparison between the abnormal score and the sorting decision threshold to obtain the sorting execution signal.

[0018] Compared with existing technologies, this application provides a visual recognition-based online food impurity sorting method and system. It establishes an efficient feature extraction foundation by offline pre-training a general feature encoder on a large-scale, multi-category food image set. For new food categories, it dynamically constructs a product prototype Gaussian mixture model online based on K standard product images to accurately characterize its normal distribution. In anomaly score modeling, it integrates global probability density and local manifold curvature information to construct a comprehensive and robust anomaly discrimination criterion, effectively solving the problem of missed detection of anomalies hidden in the multimodal distribution of normal products using traditional methods. In this way, it significantly improves the sorting system's ability to quickly adapt to new food categories, greatly shortens the deployment time and cost of new product models, and significantly improves the accuracy and robustness of impurity detection. Attached Figure Description

[0019] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0020] Figure 1 This is a flowchart of a visual recognition-based online food impurity sorting method according to an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of the data flow of the online food impurity sorting method based on visual recognition according to an embodiment of this application;

[0022] Figure 3 This is a block diagram of a visual recognition-based online food impurity sorting system according to an embodiment of this application. Detailed Implementation

[0023] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0024] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0025] While this application makes various references to certain modules of the systems according to embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The modules described are merely illustrative, and different aspects of the systems and methods may use different modules.

[0026] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0027] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0028] The technical solution of this application proposes an online sorting method for food impurities based on visual recognition. Figure 1 This is a flowchart of a visual recognition-based online food impurity sorting method according to an embodiment of this application. Figure 2 This is a system architecture diagram of a visual recognition-based online food impurity sorting method according to an embodiment of this application. Figure 1 and Figure 2 As shown, the online food impurity sorting method based on visual recognition according to an embodiment of this application includes the following steps: S1, offline pre-training of a general feature encoder based on a large-scale multi-category food image set to obtain a pre-trained encoder model; S2, online construction of a new category dynamic prototype based on K new category standard product images and the pre-trained encoder model to obtain a product prototype Gaussian mixture model; S3, inputting real-time material images captured by a high-speed industrial camera into the pre-trained encoder model to obtain material feature vectors; S4, explicit modeling of anomaly scores on the material feature vectors based on the product prototype Gaussian mixture model to obtain anomaly scores; S5, making a sorting decision based on a comparison between the anomaly scores and a sorting decision threshold to obtain a sorting execution signal.

[0029] Specifically, S1 involves offline pre-training of a general feature encoder model based on a large-scale, multi-category food image set to obtain a pre-trained encoder model. It should be understood that in the complex scenario of online food impurity sorting, facing a wide variety of foods with diverse forms and potential minute impurities, a system capable of accurately and robustly understanding the visual features of food is crucial for success. By utilizing a large-scale, multi-category food image set for offline pre-training of the general feature encoder, the system can fully learn and master the inherent, discriminative visual patterns and features in food images before actual deployment. This pre-training process enables the feature encoder to extract high-level, semantically rich general feature representations from massive amounts of food data, thus avoiding the huge overhead and inefficiency of training the model from scratch at the beginning of each new category sorting task. It provides a strong feature foundation for subsequent online construction of dynamic prototypes for K new category standard product images, ensuring that the model can effectively capture the normal visual characteristics of food, laying a solid foundation for subsequent explicit modeling of abnormal scores, and significantly improving the sorting system's recognition accuracy and generalization ability for various food impurities.

[0030] Among them, a general feature encoder is a machine learning model, typically a deep neural network (such as a convolutional neural network), whose main function is to encode raw, high-dimensional food image data (e.g., pixel values) into a low-dimensional feature vector with stronger semantic information and discriminative power. Offline pre-training is a model training strategy that refers to the process of initially training a model in an independent environment using a large amount of data before the actual application system is put into operation. This process does not involve real-time interaction and can consume a lot of computing resources and time to ensure that the model achieves high performance and generalization ability before deployment. Compared with online training, offline training emphasizes the separation of training from actual operation.

[0031] This process is a typical deep learning model training paradigm. Its main goal is to teach a general feature encoder how to map high-dimensional food image data into a low-dimensional but information-rich feature vector space. This process is performed offline, meaning it does not affect real-time sorting operations and can be fully implemented in environments with sufficient computing resources. Specifically, the process begins with data preparation and cleaning. A large-scale, multi-category food image set is the core of the training data for this step, containing images of various normal food categories. These images undergo preprocessing, such as size normalization, color correction, and data augmentation (e.g., random cropping, flipping, rotation, color jittering), to improve the model's generalization ability and robustness. Next, the architecture of the general feature encoder is selected and initialized. General feature encoders typically employ model architectures such as deep convolutional neural networks (CNNs). These networks usually choose models pre-trained on large general image datasets like ImageNet as initial weights, such as ResNet, EfficientNet, and Vision Transformer, to serve as a starting point for knowledge transfer, accelerating the training process and improving performance. Finally, the offline training process is implemented. In the offline pre-training phase, a large-scale, multi-category food image set is fed into a general feature encoder for training. The training objective is to enable the encoder to learn a general representation that can effectively distinguish different food categories and capture the visual features of food. This is typically achieved through supervised or self-supervised learning. During training, the optimizer (such as Adam or SGD) iteratively updates the encoder's network parameters based on the gradient of the loss function until the model's performance on the validation set reaches expectations or the loss function converges; finally, the pre-trained encoder model is generated. After completing the offline training, the parameters of the general feature encoder are fixed, resulting in the pre-trained encoder model. This model can receive new food image inputs and output their corresponding feature vectors. These feature vectors are highly abstract and effectively encoded representations of the food's visual content, providing standardized, high-quality input for subsequent online sorting tasks.

[0032] Specifically, in step S2, a dynamic prototype of the new food category is constructed online based on K standard images of new food categories and the pre-trained encoder model to obtain a Gaussian mixture model of the product prototype. It should be understood that food sorting systems need to possess high flexibility and adaptability to cope with constantly changing market demands and the emergence of new food categories. While the pre-trained encoder model can extract general visual features, it does not inherently possess the ability to distinguish between "normal" and "abnormal" specific new food categories. To enable the system to identify and sort new food categories, a mathematical description of the normal state of that new category must be established, i.e., a product prototype. In the technical solution of this application, by utilizing a small number (K) standard images of new food categories and combining them with the powerful feature extraction capabilities of the pre-trained encoder model, the system can quickly and accurately construct a unique prototype model for each new category without requiring extensive manual annotation and time-consuming retraining. This dynamic construction mechanism allows the system to adapt to new products in a short time, significantly reducing deployment and maintenance costs, and ensuring the timeliness and accuracy of sorting. As a powerful probabilistic model, the Gaussian mixture model of the product prototype can capture the complex distribution of the characteristics of the standard products of the new category in the feature space, providing a reliable benchmark for subsequent explicit modeling of anomaly scores. This enables the system to effectively distinguish between new and old categories and accurately identify any deviations or impurities in the new category.

[0033] In practice, firstly, K standard product images of the new product category are input into a pre-trained encoder model to obtain a set of feature vectors. Among these, K images representing the standard state of the new product category are selected as baseline data for building the prototype model. These images are then input one by one into the pre-trained encoder model. The pre-trained encoder model receives these images and extracts their features, converting each image into a high-dimensional feature vector. Ultimately, the feature vectors corresponding to all K standard product images together constitute a feature vector set. This set is the digital representation of the new product category's standard products in the feature space, laying the data foundation for subsequent statistical modeling.

[0034] Next, the optimal number of prototype components is determined based on the feature vector set to obtain the optimal number of components. The Gaussian Mixture Model (GMM) is a weighted superposition of multiple Gaussian distributions (or components), each component representing a latent pattern or cluster in the data. Determining the optimal number of components is crucial for constructing an effective GMM. Specifically, first, a range for the number of candidate components is defined. That is, a reasonable integer range, such as from 1 to a maximum value M, is predefined as the possible number of components in the GMM. Then, for each candidate component number within this range, the feature vector set is used as training data to initially fit the GMM. For each set of candidate component number m (where m is within the candidate component number range), the feature vector set is used as input data, and a Gaussian Mixture Model with m components is initially trained using methods such as the Expectation-Maximization (EM) algorithm. Subsequently, the Bayesian Information Criterion (BIC) score of each Gaussian Mixture Model is calculated. That is, for each initially fitted Gaussian Mixture Model, its Bayesian Information Criterion (BIC) score is calculated. Bayesian Information Criterion (BIC) is a criterion for model selection that penalizes model complexity while measuring goodness of fit, aiming to find a model that can explain the data well without being overly complex. Then, the number of candidate components corresponding to the minimum Bayesian Information Criterion score is selected as the optimal number of components. That is, among all the BIC scores corresponding to the number of candidate components, the candidate component number with the smallest BIC score is selected and determined as the optimal number of components for the Gaussian Mixture Model. The minimum BIC score generally indicates a good balance between model complexity and goodness of fit.

[0035] Then, a Gaussian mixture model is fitted based on the feature vector set and the optimal number of components to obtain a fitted GMM object. Once the optimal number of components is determined, the Gaussian mixture model fitting process is performed again using the entire feature vector set and this optimal number of components to obtain a stable and optimized GMM object that can accurately describe the distribution of feature vectors of the new category standard in the feature space. The fitting process also typically uses the expectation-maximization (EM) algorithm, which iteratively estimates the posterior probability of the components and updates the component parameters (weights, mean, covariance) until convergence.

[0036] Subsequently, the fitted Gaussian Mixture Model (GMM) object is finalized to obtain the product prototype Gaussian Mixture Model. In other words, the GMM object is solidified into a product prototype model. This step involves fine-tuning, regularizing, or storing the parameters of the fitted GMM to ensure its stability and robustness in subsequent online sorting. The finalized GMM model becomes the product prototype Gaussian Mixture Model, which fully contains the probability distribution information of the new category's standard product image in the feature space, providing a precise mathematical description for subsequent anomaly detection.

[0037] Specifically, in step S3, the real-time material image captured by a high-speed industrial camera is input into a pre-trained encoder model to obtain a material feature vector. It should be understood that, firstly, the real-time requirement of the sorting task demands that the system be able to quickly acquire and process images of continuously flowing materials on the production line. High-speed industrial cameras can meet this stringent timeliness requirement, ensuring that no material to be inspected is missed. Therefore, in the technical solution of this application, real-time material images are acquired using a high-speed industrial camera. Secondly, the original image data (pixel matrix) contains a large amount of redundant information and has extremely high dimensionality. Directly using it for subsequent anomaly detection would impose a huge computational burden and make it difficult to extract discriminative deep semantic information. Therefore, in the technical solution of this application, the real-time material image is further input into a pre-trained encoder model to obtain a material feature vector. By inputting these real-time captured images into a pre-trained encoder model that has been trained on large-scale data, the raw pixel data can be effectively compressed and converted into a low-dimensional, high-information-density material feature representation. This feature representation is an abstract representation of key visual attributes in the image (such as color, texture, and shape), filtering out irrelevant details while retaining the core information needed to distinguish normal from abnormal materials. This conversion not only significantly reduces the computational complexity of subsequent anomaly score modeling, but more importantly, it provides the system with a standardized and robust representation, enabling the subsequent anomaly detection module to make accurate judgments based on these refined features. This ensures that the sorting system can identify various impurities and defects in food in a timely and accurate manner.

[0038] In practice, the first step is to capture images in real time using high-speed industrial cameras. On the production line, as food materials awaiting sorting pass through designated inspection areas, high-speed industrial cameras continuously capture real-time images of the materials at an extremely high frame rate. These high-speed industrial cameras are vision devices specifically designed for industrial production environments. Their core features include the ability to capture images at extremely high frame rates (number of images captured per second) and high resolution, while also possessing excellent stability, durability, and compatibility with other industrial automation equipment. This ensures that even when the production line is operating at high speed, clear and complete real-time images of each material to be inspected can still be captured, providing a high-quality data source for subsequent visual analysis. These cameras are typically equipped with high-performance sensors and fast data transmission interfaces to ensure clear, blur-free images are captured even when materials are moving at high speeds. Specifically, the captured images can be color images, grayscale images, or even multispectral images, depending on the sorting requirements. Image capture is usually synchronized with the physical triggering mechanisms of the production line (such as photoelectric sensors, encoders, etc.) to ensure that each captured image precisely corresponds to one or a batch of materials to be inspected.

[0039] Secondly, image preprocessing operations are performed on the acquired real-time images. The preprocessing process includes image size normalization, color space conversion, pixel value normalization, and background removal or region cropping;

[0040] The preprocessed image is then input into a pre-trained encoder model. This encoder model was previously pre-trained offline using a large-scale, multi-category food image set; its network parameters are fixed and will not be trained or updated again. It operates as an efficient feature extractor in inference mode.

[0041] Finally, the model outputs a material feature vector. As the preprocessed material image flows through the layers of the pre-trained encoder model, the model's convolutional, pooling, and fully connected layers (or their equivalents) abstract and transform the image layer by layer. Ultimately, the model outputs a fixed-length numerical vector, the material feature vector, which is a highly condensed and abstract representation of the original image content, capturing the most essential and discriminative visual features of the image. For example, if the encoder is a convolutional neural network, its feature vector is typically the output of its penultimate layer (before the classification head), which is then passed to the next module for explicit anomaly score modeling.

[0042] Specifically, S4, based on the Gaussian mixture model of the product prototype, explicitly models the anomaly score of the material feature vector to obtain the anomaly score. It should be understood that the fundamental technical reason for the existing technology in calculating the anomaly score lies in the singularity of the discrimination criterion. Specifically, this method only evaluates the probability density value of the feature vector of the material under test in isolation under a pre-constructed probability model, which essentially ignores the local topological structure relationship of the feature vector on its probability manifold. This significantly increases the risk of missed detection for certain anomalies. In particular, anomalies whose feature vectors are precisely embedded in low-probability regions—i.e., "valley zones"—between multiple modes in the multimodal distribution of the genuine product are difficult for traditional methods to effectively distinguish. In other words, the original mechanism can only perceive the "altitude" of the feature point on the probabilistic topographic map, but completely lacks the utilization of the local curvature information of the point's location. Therefore, this mechanism cannot effectively distinguish between a normal edge sample located in a low-probability plain and a structurally abnormal sample hidden among multiple normal modes, thus limiting the accuracy and reliability of the sorting system. To address the aforementioned technical shortcomings, this technical approach proposes a dual evaluation mechanism integrating global probability density and local manifold curvature. Through this mechanism, the system can not only identify anomalies that significantly deviate from the normal distribution in the feature space (such as isolated impurities), but also accurately capture materials within the normal category distribution but with abnormal local structures (such as substandard products or gradual defects intermediate between normal and standard products). This comprehensive, multi-dimensional anomaly evaluation framework significantly improves the sorting system's ability to identify a wider range of food impurities and defects with more subtle characteristics, thereby achieving higher quality control standards.

[0043] In practice, firstly, the density anomaly score of the material feature vector relative to the Gaussian mixture model of the product prototype is calculated. It should be understood that the macroscopic evaluation capability of the global probability distribution in the original mechanism is retained and utilized as the basis for anomaly determination. Specifically, for the input material feature vector... The density anomaly score is obtained by calculating the negative of the log-likelihood function value of the material feature vector under the constructed Gaussian mixture model of the product prototype. This process involves first calculating the log-likelihood function value of the material feature vector under the Gaussian mixture model of the product prototype; then, taking the negative of the log-likelihood function value as the density anomaly score. In specific food sorting scenarios, the density anomaly score measures the degree to which the visual characteristics of the current material deviate from all known typical patterns of authentic products. This effectively identifies obvious impurities whose characteristics differ significantly from all normal categories and occupy isolated positions in the feature space. Specifically, this process is expressed by the formula:

[0044] ,

[0045] in, Indicates density anomaly score; It is the natural logarithm function; , and These are the weights, mean vector, and covariance matrix of the c-th Gaussian component in the Gaussian mixture model of the product prototype. Representative eigenvector The probability density value under this Gaussian component;

[0046] Next, the curvature anomaly score is obtained by calculating the local manifold curvature anomaly score of the material feature vector relative to the Gaussian mixture model of the product prototype. This step aims to compensate for the information loss caused by relying solely on probability density and to capture subtle anomalies hidden between normal data patterns. In this process, firstly, the log-likelihood function of the material feature vector relative to the Gaussian mixture model of the product prototype is calculated; secondly, the second derivative of the log-likelihood function at the material feature vector is calculated to obtain the Hessian matrix; then, the Hessian matrix is ​​decomposed into eigenvalues, and its maximum eigenvalue is taken and processed by a nonlinear function (such as ReLU) to obtain the curvature anomaly score. In a sorting scenario, the curvature anomaly score treats the probability distribution as a multidimensional terrain, and the maximum eigenvalue of the Hessian matrix precisely quantifies the steepest convexity of this terrain at the location of the material feature point. A positive maximum eigenvalue strongly suggests that the point is located in a valley between two or more normal pattern peaks, which is precisely the typical characteristic of defective products or gradual defects that fall between normal and normal products. In this way, such structural anomalies can be accurately quantified and identified. Specifically, the process can be expressed by the following formula:

[0047] ,

[0048] in, The curvature anomaly score; This represents the operation of finding the largest eigenvalue of a matrix; Log-likelihood function exist The Hessian matrix at that location;

[0049] Then, the density anomaly score and curvature anomaly score are fused to obtain the final anomaly score. That is, global and local information are organically combined to form a comprehensive and robust final criterion. Specifically, the density anomaly score and curvature anomaly score obtained in the first two steps are weighted and summed to generate the final anomaly score. In application scenarios, the significance of this fusion lies in constructing a complementary decision-making system: the density score is responsible for eliminating outliers, while the curvature score focuses on identifying defective products mixed in with normal products. By adjusting the weights, the focus can be flexibly shifted to the detection of different types of anomalies based on the defect characteristics of different batches of materials. This results in a comprehensive score with high sensitivity to various anomalies, greatly improving the accuracy of sorting decisions and the system's generalization ability. Specifically, this process is expressed by the formula:

[0050] ,

[0051] Where S is the final outlier score; α and β are preset non-negative weighting coefficients used to adjust the importance of the two scores.

[0052] This approach significantly improves the accuracy and robustness of the visual recognition-based online food impurity sorting system. Specifically, by creatively introducing and fusing local curvature information from probabilistic manifolds with traditional global probability density information, a two-dimensional, multi-perspective anomaly assessment framework is constructed. This framework not only inherits the powerful detection capability for obvious foreign objects, but more importantly, it successfully solves the detection blind spots for intervalve or structural anomalies present in the original technology. Ultimately, this enables the sorting system to more reliably identify a wider range of food impurities and defects with more subtle characteristics, thereby achieving higher quality control standards in practical industrial applications.

[0053] Specifically, in step S5, a sorting decision is made based on a comparison between the anomaly score and the sorting decision threshold to obtain a sorting execution signal. It should be understood that although the preceding steps have successfully extracted features from the real-time material image and calculated a quantified anomaly score, these values ​​do not directly indicate the specific physical operation. To translate intelligent analysis into practical action, the system needs a clear judgment criterion to determine whether the material is qualified. In the technical solution of this application, by comparing the final anomaly score of each material with a pre-set sorting decision threshold, the system can objectively determine whether the material meets the "normal" standard. This comparison mechanism simplifies the complex image analysis and probabilistic modeling results into a binary decision (qualified or unqualified), thereby driving the actuator to perform the corresponding physical sorting.

[0054] In practice, the system first receives the final anomaly score, a comprehensive evaluation value obtained by fusing density anomaly scores and local manifold curvature anomaly scores. This score quantifies the degree of deviation between the material and the normal category standard. Next, it obtains the sorting decision threshold. The system acquires a pre-set sorting decision threshold, a key parameter typically determined through experiments and statistical analysis based on actual production needs, product quality standards, acceptable false positive and false negative rates, etc., defining the boundary between normal and abnormal materials. For example, the threshold can be selected by analyzing the anomaly score distribution of normal and abnormal samples on the validation set to maximize the F1 score or balance recall and precision. Then, the received final anomaly score is directly compared with the sorting decision threshold. Specifically, if the final anomaly score of the material is greater than or equal to the sorting decision threshold, the material is judged as an abnormal product; if the final anomaly score is less than the sorting decision threshold, the material is judged as a normal product. Finally, a sorting execution signal is generated and sent. Based on the comparison result, the system generates a corresponding sorting execution signal. Specifically, if a product is identified as defective, the system generates an anomaly signal and sends it to the corresponding physical actuator. For example, this signal can trigger pneumatic nozzles, robotic arms, or other sorting devices to remove the defective material from the production line or direct it to a waste collection area. If the product is identified as normal, the system does not send an anomaly signal, and the material continues to pass normally on the production line. The entire process is highly automated, ensuring high efficiency and immediacy in online sorting.

[0055] In summary, the online food impurity sorting method based on visual recognition according to the embodiments of this application is explained. It establishes an efficient feature extraction foundation by offline pre-training a general feature encoder on a large-scale multi-category food image set. For new categories, a Gaussian mixture model of the product prototype is dynamically constructed online based on K standard product images to accurately characterize its normal distribution. When modeling anomaly scores, global probability density and local manifold curvature information are integrated to construct a comprehensive and robust anomaly discrimination criterion, effectively solving the problem of missed detection of anomalies hidden in the multimodal distribution of normal products by traditional methods. In this way, the sorting system's ability to quickly adapt to new categories is significantly improved, the deployment time and cost of new product models are greatly shortened, and the accuracy and robustness of impurity detection are significantly improved.

[0056] Furthermore, an online food impurity sorting system based on visual recognition is also provided.

[0057] Figure 3 This is a block diagram of a visual recognition-based online food impurity sorting system according to an embodiment of this application. Figure 3As shown, the visual recognition-based online food impurity sorting system 300 according to an embodiment of this application includes: an offline pre-training module 310, used to perform offline pre-training of a general feature encoder based on a large-scale multi-category food image set to obtain a pre-trained encoder model; an online construction module 320 for new category dynamic prototypes, used to perform online construction of new category dynamic prototypes based on K new category standard product images and the pre-trained encoder model to obtain a product prototype Gaussian mixture model; a material feature extraction module 330, used to input real-time material images captured by a high-speed industrial camera into the pre-trained encoder model to obtain material feature vectors; an anomaly score explicit modeling module 340, used to perform anomaly score explicit modeling on the material feature vectors based on the product prototype Gaussian mixture model to obtain anomaly scores; and a sorting decision module 350, used to make sorting decisions based on the comparison between the anomaly scores and a sorting decision threshold to obtain a sorting execution signal.

[0058] As described above, the visual recognition-based online food impurity sorting system 300 according to the embodiments of this application can be implemented in various wireless terminals, such as servers with a visual recognition-based online food impurity sorting algorithm. In one possible implementation, the visual recognition-based online food impurity sorting system 300 according to the embodiments of this application can be integrated into the wireless terminal as a software module and / or hardware module. For example, the visual recognition-based online food impurity sorting system 300 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the visual recognition-based online food impurity sorting system 300 can also be one of many hardware modules of the wireless terminal.

[0059] Alternatively, in another example, the visual recognition-based online food impurity sorting system 300 and the wireless terminal can also be separate devices, and the visual recognition-based online food impurity sorting system 300 can be connected to the wireless terminal via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0060] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for online sorting and detection of food impurities based on visual recognition, characterized in that, include: A pre-trained encoder model is obtained by offline pre-training a general feature encoder based on a large-scale, multi-category food image set. Based on K standard product images of new product categories and the pre-trained encoder model, an online dynamic prototype of the new product category is constructed to obtain a Gaussian mixture model of the product prototype. Real-time material images captured by a high-speed industrial camera are input into a pre-trained encoder model to obtain material feature vectors; Based on the Gaussian mixture model of the product prototype, anomaly scores are explicitly modeled on the material feature vectors to obtain anomaly scores, including: Calculating the density anomaly score of the material feature vector relative to the Gaussian mixture model of the product prototype includes: calculating the log-likelihood function value of the material feature vector under the Gaussian mixture model of the product prototype; and taking the negative of the log-likelihood function value as the density anomaly score. The calculation of the local manifold curvature anomaly score of the material eigenvector relative to the Gaussian mixture model of the product prototype includes: calculating the log-likelihood function of the material eigenvector relative to the Gaussian mixture model of the product prototype; calculating the second derivative of the log-likelihood function at the material eigenvector to obtain the Hessian matrix; performing eigenvalue decomposition on the Hessian matrix and taking its largest eigenvalue as the curvature anomaly score. The density anomaly score and the curvature anomaly score are fused to obtain the final anomaly score; A triage decision is made based on a comparison between the anomaly score and the triage decision threshold to obtain a triage execution signal.

2. The online food impurity sorting method based on visual recognition according to claim 1, characterized in that, Based on K standard product images of new product categories and the pre-trained encoder model, an online dynamic prototype of the new product category is constructed to obtain a Gaussian mixture model of the product prototype, including: Input the images of K new category standard products into the pre-trained encoder model to obtain a set of feature vectors; The optimal number of prototype components is determined based on the feature vector set to obtain the optimal number of components; Gaussian mixture prototype model fitting is performed based on feature vector set and optimal component number to obtain fitted GMM object; The prototype model parameters of the fitted GMM object are finalized to obtain the product prototype Gaussian mixture model.

3. The online food impurity sorting method based on visual recognition according to claim 2, characterized in that, The optimal number of prototype components is determined based on the feature vector set, including: Set the range for the number of candidate components; For each candidate component number within the range of the candidate component number, a Gaussian mixture model is initially fitted using the feature vector set as training data. Calculate the Bayesian information criterion score for each Gaussian mixture model; The number of candidate components corresponding to the minimum Bayesian information criterion score is selected as the optimal number of components.

4. The online food impurity sorting method based on visual recognition according to claim 1, characterized in that, The density anomaly score and the curvature anomaly score are fused to obtain the final anomaly score, including: fusing the density anomaly score and the curvature anomaly score using the following formula: ， Where S is the final anomaly score; and These are preset non-negative weighting coefficients used to adjust the importance of the two scores.

5. A visual recognition-based online food impurity sorting system, characterized in that, include: The offline pre-training module is used to perform offline pre-training of a general feature encoder based on a large-scale multi-category food image set to obtain a pre-trained encoder model. The online construction module for dynamic prototypes of new product categories is used to construct dynamic prototypes of new product categories online based on K standard product images of new product categories and the pre-trained encoder model to obtain a Gaussian mixture model of the product prototype. The material feature extraction module is used to input real-time material images captured by a high-speed industrial camera into a pre-trained encoder model to obtain material feature vectors. The explicit anomaly score modeling module is used to explicitly model the anomaly scores of material feature vectors based on the product prototype Gaussian mixture model to obtain anomaly scores, including: Calculating the density anomaly score of the material feature vector relative to the Gaussian mixture model of the product prototype includes: calculating the log-likelihood function value of the material feature vector under the Gaussian mixture model of the product prototype; and taking the negative of the log-likelihood function value as the density anomaly score. The calculation of the local manifold curvature anomaly score of the material eigenvector relative to the Gaussian mixture model of the product prototype includes: calculating the log-likelihood function of the material eigenvector relative to the Gaussian mixture model of the product prototype; calculating the second derivative of the log-likelihood function at the material eigenvector to obtain the Hessian matrix; performing eigenvalue decomposition on the Hessian matrix and taking its largest eigenvalue as the curvature anomaly score. The density anomaly score and the curvature anomaly score are fused to obtain the final anomaly score; The sorting decision module is used to make sorting decisions based on the comparison between the abnormal score and the sorting decision threshold to obtain the sorting execution signal.

Images