Novel food foreign matter real-time detection method based on flexible feature matching learning

Abstract

The invention discloses a novel food foreign matter real-time detection method based on flexible feature matching learning. The novel food foreign matter real-time detection method comprises the following steps: step 1, new category identification and extension: inputting a food foreign matter image data set into a new foreign matter learning module; 2, transmitting the task to a task isolation module; step 3, flexible matching of foreign matter knowledge; 4, generating a foreign matter candidate characteristic matrix; 5, focusing a foreign matter area; and 6, a self-adaptive weight calculation module is used for dynamically adjusting the weight of the model according to the feedback of the previous stage and the importance of the task. According to the method, rapid identification and learning of new foreign matter categories are realized through a method based on flexible feature matching learning, catastrophic forgetting is avoided, the model is optimized in the detection process, and the cross-scene adaptability and the real-time feedback capability are improved. The accuracy, efficiency and adaptability of food foreign matter detection can be effectively improved, and the method has wide application prospects and particularly has important commercial value in real-time detection scenes such as food production lines.

Description

Technical Field

[0001] The invention relates to the technical field of artificial intelligence and pattern recognition, and in particular to a novel real-time detection method for food foreign matter based on flexible feature matching learning. Background Art

[0002] As food safety issues become increasingly serious, how to effectively detect and remove foreign matter in food has become an important issue that the global food industry needs to solve. In traditional food production and packaging lines, foreign matter detection in food usually relies on manual inspection or automated equipment based on simple rules. These methods are not only inefficient, but also have high false detection and missed detection rates. Especially in large-scale production environments, current detection methods cannot meet the needs of efficient and accurate detection. Therefore, automated non-destructive testing systems based on X-ray imaging technology have gradually become an important solution in the food industry.

[0003] At present, many deep learning technologies have been widely used in food foreign body detection, especially convolutional neural networks (CNN) and object detection algorithms such as YOLO. However, with the continuous expansion of food production lines and the diversification of foreign body types, existing detection systems often handle multiple scenarios at the same time, and detection models often face the problem of "catastrophic forgetting", that is, when detecting new types of foreign bodies, the old foreign body category recognition capabilities are often forgotten, resulting in a decrease in detection accuracy. In addition, with the changes in the production environment (such as different types of food, etc.), existing detection models are also difficult to quickly adapt to new tasks.

[0004] In order to meet these challenges, the present invention proposes a new food dangerous foreign body detection method based on flexible feature matching learning. This method can continuously learn new foreign body categories while maintaining the ability to recognize old categories, thereby avoiding the catastrophic forgetting of traditional models and effectively improving the adaptability of the detection system. This technology can be widely used in the automated detection of food production lines, and can detect new and original food foreign bodies in real time and accurately. Summary of the invention

[0005] The present invention aims to solve the problems of existing food foreign body detection methods in terms of poor adaptability to a single scenario, inability to quickly identify new foreign body categories, catastrophic forgetting, and insufficient feedback capabilities. Existing methods are usually unable to handle the detection of cross-scenario, multi-type foreign bodies, and when new foreign body categories appear, the entire model must be retrained, resulting in a long training cycle, high costs, and easy loss of recognition capabilities for old foreign bodies. The present invention proposes a food foreign body detection technology combined with flexible feature matching learning to achieve rapid recognition and learning of new foreign body categories, while avoiding catastrophic forgetting, and optimizing the model in the detection process to enhance its cross-scenario adaptability and real-time feedback capabilities. This technology can effectively improve the accuracy, efficiency and adaptability of food foreign body detection, and has broad application prospects, especially in real-time detection scenarios such as food production lines. It has important commercial value.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A novel real-time detection method for food foreign matter based on flexible feature matching learning includes the following steps:

[0008] Step 1: New Category Identification and Expansion

[0009] Input the image data into the new foreign object learning module to determine whether the image contains a new foreign object category;

[0010] Step 2: Pass in the task isolation module

[0011] If the input image is determined to contain a new foreign object in step 1, the image and the new foreign object category are passed to the task isolation module;

[0012] Step 3: Flexible matching of foreign body knowledge

[0013] Enter the foreign object knowledge flexible feature matching learning network, transfer the knowledge learned by the existing model to the new task model, and use the soft labels of the existing model as the training targets of the new task network. The new task network quickly learns new feature representations and retains the accuracy of the existing task.

[0014] Step 4: Generation of foreign body candidate feature matrix

[0015] The image data enters the first stage of the target detection module, and the candidate feature matrix is ​​generated; through the RegionProposal Network, a candidate feature matrix containing foreign objects is generated; multiple candidate boxes are generated based on the input foreign object features, and each candidate box is calculated to see whether it contains the target food foreign object; a list of candidate regions is output, and these regions will enter the next stage for fine classification and regression;

[0016] Step 5: Focus on the foreign body area

[0017] After the candidate feature matrix is ​​generated in the first stage, the image enters the second stage of the target detection module to focus on the foreign body area; the candidate area is classified to determine whether the area is a target foreign body, and the bounding box coordinates of the target are accurately located through the regression model;

[0018] Step 6: Adaptive weight calculation module

[0019] Finally, we enter the adaptive weight calculation module to dynamically adjust the model weights based on the feedback from the previous stage and the importance of the task.

[0020] Preferably, in step 1, the input is an X-ray imaging grayscale image dataset of food foreign matter, which includes various types of food foreign matter images, such as mustard tuber and braised food images containing glass, metal, and plastic foreign matter.

[0021] Preferably, in step 1, the judgment process is implemented by a binary classification method, which determines whether it is a new category based on the high-dimensional feature representation of the input image; the specific process is as follows:

[0022]

[0023] Among them, P(C new |X) is the probability of the new foreign body category; f(X,θ) is the feature output calculated by the input image X and the model parameter θ; exp is an exponential function used to output the probability of the new foreign body category.

[0024] Preferably, in step 2, the role of the task isolation module is to ensure that the learning between the new foreign object category and the existing foreign object category does not interfere with each other.

[0025] Preferably, in step 3, the difference between the outputs of the teacher network and the student network is measured by KL divergence, and the student network is optimized thereby. The specific calculation formula is as follows:

[0026]

[0027] Among them, D KL is the KL divergence, which is used to measure the previous network P T and the new network P S The probability difference of the output; P T (y i ) and P S (y i ) are respectively the previous network and the new network for the foreign object category y i The probability distribution of ; log is a logarithmic function used to calculate the difference between probability distributions;

[0028]

[0029] in, and Represent the loss functions of the new task and the old task respectively; λ is a hyperparameter that controls the weight ratio of the new task and the old task.

[0030] Preferably, in step 4, the calculation method is:

[0031] P object =softmax(W rpn f(X,θ rpn ))

[0032] Among them, P object is the probability that the candidate region contains the target; f(X,θ rpn ) is the feature map calculated by input image X and parameter θ; W rpn is the weight matrix of the network, used to generate candidate regions;

[0033] Perform matrix operations on candidate regions to extract and filter feature regions:

[0034] R=Matrix_Multiplication(A,B)

[0035] Among them, R is the generated candidate feature matrix, representing each candidate region in the image; A is the feature map obtained by the convolutional network of image X; B is the feature weight matrix used for feature extraction.

[0036] Preferably, in step 5, the calculation method is:

[0037]

[0038] in, is the classification loss, which represents the prediction error of the target category; is the positioning loss, which represents the bounding box regression error; It is the confidence loss, which measures the confidence of the candidate box.

[0039] Preferably, in step 6, the learning rate and weight ratio are adjusted to ensure that new tasks and new foreign body categories can be sufficiently learned without affecting the accuracy of existing tasks; the optimized weights are fed back to the knowledge distillation module to further adjust the learning progress and effect of the student network.

[0040] Compared with the prior art, the present invention has achieved the following technical effects:

[0041] (1) The present invention adopts a flexible knowledge matching learning method, which is more advantageous than the existing method in processing new foreign objects. It can process new foreign objects faster and more accurately, and will not affect the detection accuracy of the model for other foreign objects.

[0042] (2) The present invention adopts the method of generating foreign matter characteristic matrix and focusing on foreign matter area, which overcomes the shortcomings of existing methods in identifying foreign matter and improves the recognition rate of new food foreign matter;

[0043] (3) The present invention can avoid catastrophic forgetting and optimize the model in the detection process to improve its cross-scenario adaptability and real-time feedback capabilities;

[0044] (4) The present invention has broad application prospects, especially in real-time detection scenarios such as food production lines, and has important commercial value. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is an algorithm flow chart of a novel food foreign body real-time detection method based on flexible feature matching learning of the present invention;

[0046] Figure 2 This is a diagram showing the results of food dangerous foreign matter detection by the novel food foreign matter real-time detection method based on flexible feature matching learning of the present invention;

[0047] Among them, (a) the glass ball foreign body was in the trotter sample; (b) the glass ball foreign body was in the roast duck sample; (c) the stone, occipital bone, and hard chicken bone were in the chicken breast sample 1; (d) the stone, occipital bone, and hard chicken bone were in the chicken breast sample 2;

[0048] Figure 3 A foreign body knowledge flexible matching learning network structure diagram of the novel food foreign body real-time detection method based on flexible feature matching learning of the present invention;

[0049] Figure 4 This is a partial diagram of a food foreign body image data set of the novel food foreign body real-time detection method based on flexible feature matching learning of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] like Figure 1 As shown, the present invention discloses a novel real-time detection method for food foreign matter based on flexible feature matching learning, comprising the following steps:

[0052] Step 1: New Category Identification and Expansion

[0053] The input is a dataset of grayscale images of food foreign bodies in X-ray imaging. First, the image data is input into the new foreign body learning module, which determines whether the image contains new foreign body categories. This process is implemented by a binary classification method, which determines whether it is a new category based on the high-dimensional feature representation of the input image. The specific process is as follows:

[0054]

[0055] Among them, P(C new |X) is the probability of the new foreign body category; f(X,θ) is the feature output calculated by the input image X and the model parameter θ; exp is an exponential function used to output the probability of the new foreign body category;

[0056] Step 2: Task Isolation Module

[0057] If the input image is determined to contain a new foreign object in step 1, the image and the new task (new foreign object category) are passed to the task isolation module; the role of this module is to ensure that the learning between the new task (new foreign object category) and the old task (existing foreign object category) does not interfere with each other; to ensure this, the learning of the new foreign object category will be independent of the old model task; the specific implementation is to assign dedicated network parameters to the new task, so as to avoid the influence of the new task on the learning of the old model task;

[0058] Step 3: Flexible matching of foreign body knowledge

[0059] After task isolation in step 2, enter the foreign knowledge flexible matching learning network; the task of this network is to transfer the knowledge learned by the existing model to the new task model; by using the output of the existing model (i.e., soft label) as the training target of the new task network, the new task network can quickly learn new feature representations and retain the accuracy of the existing task; this process measures the difference between the output of the teacher network and the student network through KL divergence, and optimizes the student network accordingly;

[0060]

[0061] Among them, D KL is the KL divergence, which is used to measure the previous network P T and the new network P S The probability difference of the output; P T (y i ) and P S (y i ) are respectively the previous network and the new network for the foreign object category y i The probability distribution of ; log is a logarithmic function used to calculate the difference between probability distributions;

[0062]

[0063] in, and Represent the loss functions of the new task and the old task respectively; λ is a hyperparameter that controls the weight ratio of the new task and the old task;

[0064] Step 4: Foreign body candidate feature matrix generation

[0065] After completing the flexible knowledge matching learning in step 3, the image data enters the first stage of the target detection module, namely, the candidate feature matrix generation. Through the Region Proposal Network (RPN), this stage generates a candidate feature matrix that may contain foreign objects. The RPN generates multiple candidate boxes based on the input foreign object features, and calculates and determines whether each candidate box contains the target object (food foreign object). The output is a list of candidate regions, which will enter the next stage for fine classification and regression.

[0066] P object =softmax(W rpn f(X,θ rpn ))

[0067] Among them, P object is the probability that the candidate region contains the target; f(X,θ rpn ) is the feature map calculated by input image X and parameter θ; W rpn is the weight matrix of the network, used to generate candidate regions;

[0068] Perform matrix operations on candidate regions to extract and filter feature regions:

[0069] R=Matrix_Multiplication(A,B)

[0070] Among them, R is the generated candidate feature matrix, representing each candidate region in the image; A is the feature map obtained by the convolutional network of image X; B is the feature weight matrix used for feature extraction;

[0071] Step 5: Focus on the foreign body area

[0072] After generating the candidate feature matrix in the first stage, the image enters the second stage of the target detection module to focus on the foreign object area. In this stage, the candidate area is classified to determine whether the area is a target object (foreign object), and the position of the target (i.e., the bounding box coordinates) is accurately located through the regression model.

[0073]

[0074] in, is the classification loss, which indicates the prediction error of the target category. is the positioning loss, which represents the bounding box regression error; It is the confidence loss, which measures the confidence of the candidate box;

[0075] Step 6: Adaptive weight calculation module

[0076] In the final stage, the adaptive weight calculation module is entered; this module dynamically adjusts the weights of the model according to the feedback from the previous stage and the importance of the task; by adjusting the learning rate and weight ratio, it ensures that new tasks (new foreign object categories) can be sufficiently learned without affecting the accuracy of existing tasks; the optimized weights are fed back to the knowledge distillation module to further adjust the learning progress and effect of the student network.

[0077] Embodiment 1:

[0078] 1. Experimental conditions

[0079] The present invention is based on the central processing unit of Inter(R)Core(TM)i5-12400F processor, main frequency 2.5GHz CPU, NVIDIA GeForce RTX 4060Ti, 16GB video memory, and Windows 10 Professional operating system; the experimental environment uses PyCharm 2023 as the main development tool, and cooperates with Anaconda 2023 environment for data processing and model training. GPU acceleration uses CUDA 11.3 to improve training speed and computing efficiency; software environment: Python 3.8, TensorFlow 2.5, PyTorch 1.9, OpenCV 4.5.1

[0080] The methods compared in the experiment are as follows:

[0081] R-CNN: As a detection method based on region proposal, CNN provides higher accuracy, but is slightly slower than the YOLO series.

[0082] YOLOv3: As a classic single-stage target detection algorithm, YOLOv3 has high performance and speed in the field of target detection.

[0083] 2. Experimental Dataset

[0084] The experimental data set uses a food dangerous foreign body data set collected by the X-ray detector. Food foreign body data set: This data set contains various types of food foreign body images (such as mustard tuber and stewed food images containing glass, metal, and plastic foreign bodies) for detecting different foreign body categories. The images in the data set are obtained using X-ray imaging technology, and each image contains corresponding labels (such as category identification and location box).

[0085] The dataset contains 1,000 images of food foreign bodies, 70% of which are used for training and 30% for testing.

[0086] 3. Experimental evaluation indicators

[0087] In order to evaluate the performance of the food foreign body detection method of the present invention, the following simplified evaluation indexes were used:

[0088] Accuracy: The proportion of samples correctly classified by the evaluation model to the total samples.

[0089] Frame rate (FPS): The number of image frames that can be processed per second, used to evaluate the real-time performance of the model.

[0090] 4 Comparative Experiments

[0091] The detection effect of dangerous foreign matter in food is as follows: Figure 2 shown.

[0092] To verify the effectiveness of the method of the present invention, we conducted comparative experiments with the following mainstream target detection algorithms. The results are shown in Table 1.

[0093] Table 1 Accuracy and speed of detection of foreign matter in new foods

[0094] method R-CNN Yolov3 The present invention Accuracy 62.2％ 94.3％ 98.9％ FPS (frames per second) 24.8 35.1 48.2

[0095] As can be seen from Table 1, the method based on flexible knowledge matching learning adopted by the present invention can maintain high detection accuracy when adding new foreign body categories, and avoids the catastrophic forgetting problem in traditional methods, which verifies the advancement of the present invention. The method of the present invention has high real-time performance and is suitable for application in food foreign body detection scenarios that require high real-time performance.

[0096] like Figure 3 As shown, it is a foreign body knowledge flexible matching learning network structure diagram of the novel food foreign body real-time detection method based on flexible feature matching learning of the present invention;

[0097] like Figure 4 , which is a partial diagram of a food foreign body image data set of the novel food foreign body real-time detection method based on flexible feature matching learning of the present invention.

[0098] The above description is only a preferred embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any slight modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A novel real-time detection method for food foreign matter based on flexible feature matching learning, characterized in that: The steps include: Step 1: New Category Identification and Expansion Input the image data into the new foreign object learning module to determine whether the image contains new foreign object categories; Step 2: Pass in the task isolation module If the input image is determined to contain a new foreign object in step 1, the image and the new foreign object category are passed to the task isolation module; Step 3: Flexible matching of foreign body knowledge Enter the foreign object knowledge flexible matching learning network, transfer the knowledge learned by the existing model to the new task model, and use the soft labels of the existing model as the training targets of the new task network. The new task network quickly learns new feature representations and retains the accuracy of the existing task. Step 4: Generation of foreign body candidate feature matrix The image data enters the first stage of the target detection module to generate a candidate feature matrix; through the RegionProposalNetwork, a candidate feature matrix containing foreign objects is generated; multiple candidate boxes are generated based on the input foreign object features, and each candidate box is calculated to see whether it contains the target food foreign object; a list of candidate regions is output, and these regions will enter the next stage for fine classification and regression; Step 5: Focus on the foreign area After the candidate feature matrix is ​​generated in the first stage, the image enters the second stage of the target detection module to focus on the foreign body area; the candidate area is classified to determine whether the area is a target foreign body, and the bounding box coordinates of the target are accurately located through the regression model; Step 6: Adaptive weight calculation module Finally, we enter the adaptive weight calculation module to dynamically adjust the model weights based on the feedback from the previous stage and the importance of the task.

2. The novel food foreign body real-time detection method based on flexible feature matching learning according to claim 1 is characterized in that: In step 1, the input is an X-ray imaging grayscale image dataset of food foreign bodies, which includes various types of food foreign body images, such as mustard tuber and braised food images containing glass, metal, and plastic foreign bodies.

3. The novel food foreign body real-time detection method based on flexible feature matching learning according to claim 1 is characterized in that: In step 1, the judgment process is implemented by a binary classification method, which determines whether it is a new category based on the high-dimensional feature representation of the input image; The specific process is as follows: Among them, P(C new |X) is the probability of the new foreign body category; f(X,θ) is the feature output calculated by the input image X and the model parameter θ; exp is an exponential function used to output the probability of the new foreign body category.

4. The novel food foreign body real-time detection method based on flexible feature matching learning according to claim 1 is characterized in that: In step 2, the role of the task isolation module is to ensure that the learning between the new foreign object category and the existing foreign object category does not interfere with each other.

5. The novel food foreign body real-time detection method based on flexible feature matching learning according to claim 1 is characterized in that: In step 3, the difference between the output of the teacher network and the student network is measured by KL divergence, and the student network is optimized in this way. The specific calculation formula is as follows: Among them, D KL is the KL divergence, which is used to measure the previous network P T and the new network P S The probability difference of the output; P T( y i) and P S( y i) They are respectively the previous network and the new network for the foreign object category y i The probability distribution of ; log is a logarithmic function used to calculate the difference between probability distributions; in, and Represent the loss functions of the new task and the old task respectively; λ is a hyperparameter that controls the weight ratio of the new task and the old task.

6. The novel food foreign body real-time detection method based on flexible feature matching learning according to claim 1 is characterized in that: In step 4, the calculation method is: P object =softmax(W rpn ·f(X,θ rpn )) Among them, P object is the probability that the candidate region contains the target; f(X,θ rpn ) is the feature map calculated by input image X and parameter θ; W rpn is the weight matrix of the network, used to generate candidate regions; Perform matrix operations on candidate regions to extract and filter feature regions: R=Matrix_Multiplication(A,B) Among them, R is the generated candidate feature matrix, representing each candidate region in the image; A is the feature map obtained by the convolutional network of image X; B is the feature weight matrix used for feature extraction.

7. The novel food foreign body real-time detection method based on flexible feature matching learning according to claim 1 is characterized in that: In step 5, the calculation method is: in, is the classification loss, which represents the prediction error of the target category; is the positioning loss, which represents the bounding box regression error; It is the confidence loss, which measures the confidence of the candidate box.

8. The novel food foreign body real-time detection method based on flexible feature matching learning according to claim 1 is characterized in that: In step 6, the learning rate and weight ratio are adjusted to ensure that new tasks and new foreign object categories can be sufficiently learned without affecting the accuracy of existing tasks; the optimized weights will be fed back to the knowledge distillation module to further adjust the learning progress and effect of the student network.

Images