Food foreign matter detection method and system and application

Abstract

The invention relates to a food foreign matter detection method and system and application, and the method comprises the steps: collecting a food image which is updated in a rolling manner; performing preprocessing and data expansion on the collected food image, inputting the food image into a detection model, detecting foreign matters in the food image by using the detection model, and identifying the detected foreign matters; according to the system, an image collection unit is used for collecting food images updated in a rolling mode, an image processing unit is used for processing the collected food images, and a detection unit provided with a detection model is used for conducting foreign matter detection on the processed food images; the method is applied to detection of foreign matters in caviar. According to the invention, foreign matters in the food can be rapidly detected by the detection model through collection of the food image, mixing of new foreign matters is prevented, the detection efficiency is high, the effect is good, and the labor cost is low.

Description

Technical Field

[0001] The present invention relates to the technical field of general image data processing or generation, and in particular to a food foreign body detection method, system and application. Background Art

[0002] Food safety means that food is non-toxic and harmless, meets the necessary nutritional requirements, and does not cause any acute, subacute or chronic harm to human health.

[0003] In the process of food safety management, in addition to the detection of invisible hazards in food, the detection of visible foreign matter should be more stringent.

[0004] Foreign matter in food is usually mixed in through poor management of the production workshop, or mixed in production supplies, or mixed in raw materials / primary packaging materials and other situations. Among them, the mixing of raw materials is very common. When the raw materials come from fishing and processing sites, they contain some debris. Incomplete and incomplete cleaning during processing will cause the mixing of foreign matter or foreign matter itself. Take caviar as an example. During the processing of caviar, tiny foreign matter may be mixed in it. These foreign matter will have a serious negative impact on the quality and taste of the product. Once the foreign matter in the caviar is not distinguished, it will not only destroy the overall eating experience, but also pose a certain risk to the health of consumers.

[0005] In the prior art, screening of foreign matter in food generally requires manual processing, and workers need to be extra careful during the operation to prevent new foreign matter from being mixed in, which greatly affects the overall work efficiency. As for foreign matter detection in images of foods such as caviar, due to the presence of a large number of repeated units in the image, once a foreign matter is integrated into it, it is extremely difficult to be identified, and the recognition efficiency is low. Summary of the invention

[0006] The present invention solves the problems existing in the prior art and provides a food foreign matter detection method, system and application.

[0007] The technical solution adopted by the present invention is a method for detecting foreign matter in food. The method collects food images, which are updated in a rolling manner. The collected food images are preprocessed and data expanded, and then input into a detection model, and the detection model is used to detect foreign matter in the food images, and the detected foreign matter is marked.

[0008] Preferably, training the detection model comprises the following steps:

[0009] S1 obtains a preset number of normal food images and pre-processes the images;

[0010] S2 pre-trains the detection model with a residual network;

[0011] S3 adjusts the spatial features of the detection model based on the loss function;

[0012] S4 uses KNN algorithm to calculate the distance between the feature and the center point and judge the abnormal feature;

[0013] S5 loops through S3 until the training is completed to obtain a detection model.

[0014] Preferably, in S1, the normal food image is cropped by sliding window to obtain a plurality of sub-images with reduced resolution and size.

[0015] Preferably, there is an overlapping area between adjacent sub-graphs.

[0016] Preferably, in S2, the detection model adopts a VGG network, and a plurality of residual blocks are provided between the convolution layer and the pooling layer of the VGG network;

[0017] After pre-training of the residual network, a vector set of training samples is obtained, which is used to describe the characteristic relationship of the training samples in space.

[0018] Preferably, in S3, a loss function is defined:

[0019]

[0020] Among them, the mean of the vector set obtained after pre-training is used as the center point C, L compact (θ * ) is the fine-tuning weight θ * i The feature tight loss is a custom loss function obtained by continuously adjusting parameters, θ i is the pre-trained weight, λ is the hyperparameter, i is the number of the pre-trained weight, F θi is the value of the Fisher matrix with diagonal elements θ.

[0021] Preferably,

[0022] Among them, E (x,y)∈Dpretrain is the expectation, which means the pre-training dataset D pretrain All samples (x, y) in the calculation are expected to be performed. It is about the model parameter θ for the pre-training loss function L pretrain The gradient of (x,y).

[0023] Preferably, in S4, the feature representations ψ(x1), ψ(x2), ..., ψ(xN) of the training set samples are extracted through the residual network, the distance between the center point C and the vectors in the feature representation is measured, and after obtaining the distance between each vector and the center point C, the density of normal data (food information) around the sample ψ(xi) is measured by the KNN algorithm with a scoring function, and a high abnormality score is assigned to a low-density area;

[0024] After the loop is completed, a maximum value R of a normal image from the center C is obtained; within the radius R is normal data (food information), and outside the radius R is foreign data (invalid information, such as desktop, debris, etc.).

[0025] A system using the food foreign matter detection method, the system comprising:

[0026] An image acquisition unit, used for acquiring food images, where the food images are updated in a rolling manner;

[0027] an image processing unit, used for processing the collected food image based on the image processing method in the food foreign matter detection method;

[0028] A detection unit is configured with a detection model for detecting foreign matter in the processed food image.

[0029] An application of the food foreign matter detection method is applied to the detection of foreign matter in caviar.

[0030] The present invention relates to a method, system and application for detecting foreign matter in food. The method collects food images, which are updated in a rolling manner. The collected food images are pre-processed and data expanded before being input into a detection model, and foreign matter in the food images is detected by the detection model, and the detected foreign matter is marked. The system collects the rolling updated food images by an image acquisition unit, processes the collected food images by an image processing unit, and performs foreign matter detection on the processed food images by a detection unit equipped with a detection model. The method is applied to the detection of foreign matter in caviar.

[0031] The beneficial effect of the present invention is that foreign matter can be quickly detected by collecting food images using a detection model to prevent the mixing of new foreign matter, with high detection efficiency, good effect and low labor cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flow chart of the present invention for training a detection model;

[0033] Figure 2 is a structural block diagram of the system of the present invention;

[0034] Figure 3It is a schematic diagram of a circle with a radius of R obtained by S3 in the present invention, wherein the inside of the circle is the food feature and the outside of the circle is the foreign body feature. DETAILED DESCRIPTION

[0035] The present invention is further described in detail below in conjunction with embodiments, but the protection scope of the present invention is not limited thereto.

[0036] The present invention relates to a method for detecting foreign matter in food. The method collects food images, which are updated in a rolling manner. The collected food images are preprocessed and data expanded before being input into a detection model. The detection model is used to detect foreign matter in the food images, and the detected foreign matter is marked.

[0037] In the present invention, it should be first explained that rolling update includes but is not limited to turning over the food and continuously detecting whether there are foreign objects on its surface image, pre-laying the food and continuously detecting whether there are foreign objects on each conveying surface, etc.

[0038] In the present invention, the detection model needs to be trained. The model adopts the PANDA model. After cropping the picture, the model is pre-trained with a residual network, and the model spatial features are fine-tuned with a loss function. Finally, the KNN algorithm is used to calculate the distance between the feature and the center point and judge the abnormal features.

[0039] Training the detection model includes the following steps:

[0040] S1 obtains a preset number of normal food images and pre-processes the images;

[0041] In S1, the normal food image is cropped by sliding window to obtain a plurality of sub-images with reduced resolution and size.

[0042] There are overlapping areas between adjacent sub-images.

[0043] In the present invention, the data set is the core component of the data-driven deep learning algorithm. In most cases, the size of the image used for training does not exceed 500×500 pixels. The reason is that using large-sized images for training will cause data loss and occupy too much graphics card memory. However, in engineering applications, images taken with high-definition cameras or drones generally exceed 1920×1080 pixels. It is generally believed that 4k UHD (3840×2160) is considered high-resolution as a new industrial standard, and images exceeding this size are considered super-large or very high-resolution. Some super-large images may even exceed 4096×3112. Therefore, for larger-sized images, image cropping is required.

[0044] In the specific implementation process, the ultra-high resolution image is cropped into multiple sub-images by sliding window, and then target detection is performed on each sub-image, and finally the results of all sub-images are spliced ​​and filtered. For example, for a 16000×16000 pixel image, a 416×416 pixel sliding window is used, and overlap is not considered. Finally, about 1500 sub-images are generated. When the sliding window is cropped, about 15% of the overlapping area needs to be retained to avoid a foreign object that happens to be at the edge of the window and is cut into two pieces, resulting in a small number of pixels occupied by the object itself and being difficult to detect after being truncated.

[0045] S2 pre-trains the detection model with a residual network;

[0046] In S2, the detection model adopts a VGG network, and a number of residual blocks are arranged between the convolution layer and the pooling layer of the VGG network;

[0047] After pre-training of the residual network, a vector set of training samples is obtained, which is used to describe the characteristic relationship of the training samples in space.

[0048] In the present invention, the goal of pre-training is to enable the model to learn general feature representations by training the model on a related but larger task. By using large-scale data for training, the model can learn more generalized feature representations and thus perform better on specific tasks. Part or all of the pre-trained model is used for new tasks to improve the performance of the model.

[0049] In the present invention, a residual network (ResNet) is used for pre-training. In the implementation process, the residual network is generally constructed by directly stacking residual blocks. After pre-training, a vector set of training samples is obtained, which describes the characteristic relationship of the training samples in space.

[0050] S3 adjusts the spatial features of the detection model based on the loss function;

[0051] In S3, define the loss function,

[0052]

[0053] Among them, the mean of the vector set obtained after pre-training is used as the center point C, L compact (θ * ) is the fine-tuning weight θ * i The feature close loss, θ i is the pre-trained weight, λ is the hyperparameter, i is the number of the pre-trained weight, F θi is the value of the Fisher matrix with diagonal elements θ.

[0054]

[0055] Among them, E (x,y)∈Dpretrain is the expectation, which means the pre-training dataset D pretrain All samples (x, y) in the calculation are expected to be performed. It is about the model parameter θ for the pre-training loss function L pretrain The gradient of (x,y).

[0056] In the present invention, feature fine-tuning enables the model to process different input data more accurately by learning the feature representation of the input data, and automatically adjusts the parameters of the model to adapt to different feature spaces during the training process, effectively solving the problem of model performance degradation due to different feature distributions of the input data.

[0057] In this invention, the idea of ​​continuous learning is adopted to fine-tune the features by setting the loss function, and the auxiliary tasks are pre-trained by multiple small batch samples through elastic weight consolidation (EWC). The diagonal of the Fisher information matrix F of all weight parameters in the network is calculated, and the value of the Fisher matrix of the diagonal element θ satisfies Pre-train weights θ for each network i ∈ψ0 and fine-tune the weights This can be viewed as a measure of the curvature of the loss as a weight function, with larger values ​​implying high curvature and inelastic weights. Combining this regularization with a loss weighted by a factor λ makes it easier to tune the loss function, where λ is a hyperparameter (λ = 10 4 ),satisfy,

[0058]

[0059] The mean of the vector set obtained after pre-training is taken as the center point C, and the vector set is adjusted using the loss function so that the representation of the normal vector is as close to the center point C as possible, while the representation of the abnormal vector is as far away from the center point C as possible.

[0060] S4 uses KNN algorithm to calculate the distance between the feature and the center point and judge the abnormal feature;

[0061] In S4, the feature representations ψ(x1), ψ(x2), ..., ψ(xN) of the training set samples are extracted, and the distance between the center point C and the vectors in the feature representation is measured. After obtaining the distance between each vector and the center point C, the density of normal data around the sample ψ(xi) is measured by the KNN algorithm with a scoring function, and high anomaly scores are assigned to low-density areas.

[0062] After the loop is completed, the maximum value R of a normal image from the center C is obtained.

[0063] In the present invention, the abnormal features are judged here taking into account that there are other interfering contents besides food in the sample images in training, such as tables, plastic bags and other things. Therefore, it is necessary to calculate the distance from the center point through the KNN algorithm. If the distance is too far, it will be identified as a foreign object (non-food). When training the model, it is necessary to judge whether the generated radius is reasonable, because if the radius is too large, it is very easy to regard foreign objects as food, otherwise, food will be regarded as foreign objects.

[0064] In the present invention, in the feature fine-tuning stage, the feature representations ψ(x1), ψ(x2), ..., ψ(xN) of the training set samples can be extracted, and the distance between the center point C and the vectors in the feature representation can be measured. In statistical learning, the distance between the two can be quantified. In this embodiment, the Euclidean distance is used. After the Euclidean distance between each vector and the center point C is obtained, a scoring function is further learned to describe the abnormality of the sample. Through the KNN algorithm, the scoring function measures the density of normal data around the sample ψ(x) and assigns high abnormality scores to low-density areas, that is, the data corresponding to the food is concentrated in the circle as much as possible, which is high density, while the foreign matter is less than the normal data, forming low-density data, such as Figure 3 shown.

[0065] After the loop is finished, the maximum value R of a normal image from the center C is obtained.

[0066] S5 loops through S3 until the training is completed to obtain a detection model.

[0067] After training the PANDA model to obtain the PANDA inference model, the inference model can be deployed to the detection system, the image to be detected is input, the image is cropped, and the cropped image is input into the model to give the foreign body detection result.

[0068] The present invention also relates to a system using the food foreign matter detection method, the system comprising:

[0069] An image acquisition unit, used for acquiring food images, where the food images are updated in a rolling manner;

[0070] an image processing unit, used for processing the collected food image based on the image processing method in the food foreign matter detection method;

[0071] A detection unit is configured with a detection model for detecting foreign matter in the processed food image.

[0072] The invention also relates to an application of the food foreign matter detection method, which is applied to the detection of foreign matter in caviar.

[0073] In the present invention, taking caviar as an example, the method is particularly suitable for application in foods with obvious repetitive characteristics, and caviar, as a single-particle, high-density food, is particularly suitable for the method. Based on this, technicians in this field can choose to apply the method to the detection of other similar foods or products.

[0074] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0078] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0079] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for detecting foreign matter in food, characterized in that: The method collects food images, which are updated in a rolling manner; the collected food images are preprocessed and data expanded, and then input into a detection model, so that foreign matter in the food images is detected by the detection model, and the detected foreign matter is marked.

2. A method for detecting foreign matter in food according to claim 1, characterized in that: Training the detection model includes the following steps: S1 obtains a preset number of normal food images and pre-processes the images; S2 pre-trains the detection model with a residual network; S3 adjusts the spatial features of the detection model based on the loss function; S4 uses KNN algorithm to calculate the distance between the feature and the center point and judge the abnormal feature; S5 loops through S3 until the training is completed to obtain a detection model.

3. A method for detecting foreign matter in food according to claim 2, characterized in that: In S1, the normal food image is cropped by sliding window to obtain a plurality of sub-images with reduced resolution and size.

4. A method for detecting foreign matter in food according to claim 3, characterized in that: There are overlapping areas between adjacent sub-images.

5. A method for detecting foreign matter in food according to claim 2, characterized in that: In S2, the detection model adopts a VGG network, and a number of residual blocks are arranged between the convolution layer and the pooling layer of the VGG network; After pre-training of the residual network, a vector set of training samples is obtained, which is used to describe the characteristic relationship of the training samples in space.

6. A method for detecting foreign matter in food according to claim 2, characterized in that: In S3, define the loss function, Among them, the mean of the vector set obtained after pre-training is used as the center point C, L compact (θ * ) is the fine-tuning weight θ * i The feature close loss, θ i is the pre-trained weight, λ is a hyperparameter, i is the number of the weight after pre-training, F θi is the value of the Fisher matrix with the diagonal elements θ.

7. A method for detecting foreign matter in food according to claim 6, characterized in that: Among them, E (x,y)∈Dpretrain is the expectation, which means the pre-training dataset D pretrain All samples (x, y) in the calculation are expected to be calculated. It is about the model parameter θ for the pre-training loss function L pretrain The gradient of (x,y).

8. A method for detecting foreign matter in food according to claim 2, characterized in that: In S4, the feature representations ψ(x1), ψ(x2), ..., ψ(xN) of the training set samples are extracted, and the distance between the center point C and the vectors in the feature representation is measured. After obtaining the distance between each vector and the center point C, the density of normal data around the sample ψ(xi) is measured by the KNN algorithm with a scoring function, and high anomaly scores are assigned to low-density areas. After the loop is completed, the maximum value R of a normal image from the center C is obtained.

9. A system using the food foreign matter detection method according to any one of claims 1 to 8, characterized in that: The system comprises: An image acquisition unit, used for acquiring food images, where the food images are updated in a rolling manner; an image processing unit, used for processing the collected food image based on the image processing method in the food foreign matter detection method; A detection unit is configured with a detection model for detecting foreign matter in the processed food image.

10. An application of the method for detecting foreign matter in food according to any one of claims 1 to 8, characterized in that: Applied to the detection of foreign matter in caviar.

Images