Food rapid identification and quality inspection image system and method
By using technical means such as image capture, subject detection model and feature extraction model in the food detection system, the problem of rapid identification and quality inspection of stores food is solved, and the efficiency and quality inspection capabilities of new products are improved, ensuring the system's identification efficiency and quality inspection accuracy.
Patent Information
- Application Number
- CN202411561075.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-04
- Publication Date
- 2025-02-25
AI Technical Summary
It is difficult for the existing technology to achieve rapid identification and improve the efficiency and quality inspection capabilities of new products in store food inspection, especially in scenarios where lighting conditions change and new products are updated quickly.
A food rapid identification and quality inspection image system is adopted, and the rapid identification and quality inspection of food is achieved through image capture, subject detection model, feature extraction model, identification feature library and quality inspection feature library, combined with color feature map retrieval, texture feature map retrieval and similarity calculation.
It has achieved the efficiency of rapid identification and quality inspection of food, can quickly respond to the replacement of new and old foods, greatly improve the identification efficiency and application capabilities, and ensure the fairness and accuracy of quality inspection.
Smart Images
Figure CN119510401A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of food detection, and in particular to a food rapid identification and quality inspection image system and method. Background Art
[0002] Due to the complex environment of the food finishing process room in the store, different lighting conditions in different stores, and the rapid update and iteration of new and old foods, in addition, there are many quantities and types of food completed at one time, which poses severe challenges to the AI recognition speed, new product recognition efficiency and quality inspection capabilities. At present, the classic model detection based on parallel bounding box regression and category classification tasks cannot meet the scenario of fast new product updates. Because this model architecture requires the number of new product annotations to be comparable to the scale of other existing food category annotations, and since new products have not yet been put on the shelves, large-scale data cannot be collected, making this method unfeasible. In addition, in the business process, food completion and quality inspection are performed on the same program, that is, quality inspection scoring is performed while taking pictures for recognition. Therefore, it is difficult for existing technical methods to balance the model recognition speed, new product recognition efficiency, quality inspection efficiency and model accuracy. Summary of the invention
[0003] The object of the present invention is to provide a food rapid identification and quality inspection image system and method to solve the problems raised in the above background technology.
[0004] To achieve the above object, the present invention provides the following technical solutions:
[0005] A method for rapid food identification and quality inspection image system, including image capture, the output end of the image capture is electrically connected to a subject detection model, the output end of the subject detection model is electrically connected to a feature extraction model, the input end of the feature extraction model is electrically connected to an identification feature library and a quality inspection feature library, the output end of the feature extraction model is electrically connected to a color feature map retrieval, a texture feature map retrieval and a similarity calculation, and the output ends of the color feature map retrieval and the texture feature map retrieval are electrically connected to a quality inspection score.
[0006] A method for rapid food recognition and quality inspection image capture, the image capture comprising the following steps:
[0007] S1. The camera installation position should be vertical to the center of the food tray, and the height should be between cm. The food image data is obtained by taking pictures with an industrial camera installed in the food visual recognition hardware system.
[0008] As a preferred solution of the present invention, the subject detection model includes the following steps:
[0009] S2. Based on yolov8, yolov8_biasloss is proposed to enhance the detection capability of long-tail data and difficult data. The loss function expression of yolov8_biasloss is as follows:
[0010]
[0011] In the formula, f a and f m They represent the α factor and adjustment factor respectively, loss_fcn is the CEWithLogitsLoss loss function, β represents the target box size factor, and h min and w min is the minimum height and width of the food target size allowed, h and w represent the actual height and width of the food target. If the detection error occurs in a small food object, the penalty will increase. The food annotation is divided into one or more attributes, including small food targets, medium food targets, and large food targets. In the formula Represents the proportion of samples belonging to the foreground category c, showing The larger it is, the more the model training penalizes the wrong samples for detection.
[0012] As a preferred solution of the present invention, the feature extraction model includes the following steps:
[0013] S3. ArcMargin loss is used. The mathematical expression is as follows: In the inference stage, we design the number of neurons in the last fully connected layer of the network to be 128, which means that the image feature value output by the feature extraction network is 128D. This feature value represents the representational features of the food target image and can be mainly used for similarity calculation to determine the food name.
[0014]
[0015] In the formula, θ j is the angle between the weight and the feature, the batch size and number of categories are N and n respectively, and m is the target angle The angular margin penalty on , s is the characteristic scale.
[0016] As a preferred solution of the present invention, the identification feature library includes the following steps:
[0017] S4. For the same food in different styles, select one image in different styles to register in the feature library, so as to increase the food feature representation capability. The feature value is obtained by cutting the food target from the original image and sending it to the feature extraction model to obtain a 128D feature value. The feature value and its corresponding food name are stored in the database to complete the design of the recognition feature library.
[0018] As a preferred solution of the present invention, the quality inspection feature library includes the following steps:
[0019] S5. A professional food chef selects a picture of the best food quality as the standard food, and compares and scores the quality inspection items of the food under inspection with the standard food. The quality inspection feature library stores the color feature map and texture feature map of the standard product. These feature libraries are used to perform similarity calculation with the corresponding feature maps extracted after inputting the feature extraction model of the food under inspection, thereby completing the quality similarity calculation;
[0020] The color feature map retrieval and texture feature map retrieval include the following steps:
[0021] S6. During the training phase, feature maps representing color features and texture features are retrieved by visualizing shallow feature maps and deep feature maps respectively.
[0022] 7. As a preferred solution of the present invention, similarity calculation includes the following steps:
[0023] S7. The 128D feature value output by the feature extraction model represents the category information of the identified food. The cosine value of the angle between the two vectors is calculated based on the cosine similarity after the feature value is vectorized. The cosine value represents the similarity between the two vectors. The larger the cosine value, the more similar the two vectors are, that is, the more similar the two food target images are. The cosine similarity calculation formula is as follows:
[0024]
[0025] In the formula, A and B represent two eigenvectors, A i and B i They represent the components of vectors A and B respectively, n represents the dimension of the feature vector, and the value of n is 128.
[0026] As a preferred solution of the present invention, the quality inspection scoring includes the following steps:
[0027] S8. The mathematical formula for food quality inspection scoring is as follows:
[0028]
[0029] In the formula, and β represent the color and texture score adjustment weight factors respectively, where
[0030] score color and score texture The score represents the cosine similarity calculation results of the color feature map and texture feature map of the inspected food and the feature map of the corresponding food in the quality inspection feature library. color and score textureThe value range is [0,1], score represents the final score of the inspected food, and the value range is [0,100]. Assume that Set to 0.4, β to 0.6, and calculate the cosine similarity to get the score color =0, score texture =1, it means that the color quality inspection gets 0 points, the texture quality inspection gets full marks, and the final food quality inspection score is 60 points.
[0031] Compared with the prior art, the present invention has the following beneficial effects:
[0032] 1. In the present invention, by adopting a food object subject detection model and a food object feature extraction model, the subject detection model is intended to detect all food objects in the image captured by the camera. The feature extraction model is intended to extract category features and color problem features of the food objects output by the subject detection model and perform similarity calculation with the label features in the image library to obtain the correct classification and quality inspection score of the food objects.
[0033] 2. In the present invention, by designing a lightweight dual model, rapid food recognition and rapid response to the replacement of old and new foods are achieved, which greatly improves the recognition efficiency and application ability of the system. In addition, the method based on prior color and texture feature retrieval has strong robustness and replicability, and can well filter environmental and light noise through feature convolution, ensuring the fairness and accuracy of the system's food color and texture quality inspection. The successful implementation of this method in the Chinese food scenario shows that this system can be replicated and promoted to industries such as baked foods, dairy foods, and catering. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 It is a schematic diagram of the framework structure of the present invention;
[0035] Figure 2 A method diagram for color and texture feature map retrieval according to the present invention;
[0036] Figure 3 This is the effect diagram of this system being applied to Chinese food identification and quality inspection analysis.
[0037] In the figure: 1. Image capture; 2. Subject detection model; 3. Feature extraction model; 4. Recognition feature library; 5. Quality inspection feature library; 6. Color feature map retrieval; 7. Texture feature map retrieval; 8. Similarity calculation; 9. Quality inspection score. DETAILED DESCRIPTION
[0038] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0039] For examples, see Figure 1-Figure 3 , the present invention provides a technical solution:
[0040] A food rapid identification and quality inspection image system comprises an image capture device 1, wherein the output end of the image capture device 1 is electrically connected to a subject detection model 2, the output end of the subject detection model 2 is electrically connected to a feature extraction model 3, the input end of the feature extraction model 3 is electrically connected to an identification feature library 4 and a quality inspection feature library 5, respectively, the output end of the feature extraction model 3 is electrically connected to a color feature map retrieval device 6, a texture feature map retrieval device 7 and a similarity calculation device 8, and the output ends of the color feature map retrieval device 6 and the texture feature map retrieval device 7 are electrically connected to a quality inspection score device 9.
[0041] A method for rapid food recognition and quality inspection image, image capture 1 includes the following steps:
[0042] S1. The camera installation position should be vertical to the center of the food tray and the height should be between 70-80cm. The food image data is obtained by taking pictures with an industrial camera installed in the food visual recognition hardware system. The subject detection model 2 includes the following steps:
[0043] S2. Based on yolov8, yolov8_biasloss is proposed to enhance the detection capability of long-tail data and difficult data. The loss function expression of yolov8_biasloss is as follows:
[0044]
[0045] In the formula, f a and f m They represent the α factor and adjustment factor respectively, loss_fcn is the CEWithLogitsLoss loss function, β represents the target box size factor, and h min and w min is the minimum height and width of the food target size allowed, h and w represent the actual height and width of the food target. If the detection error occurs in a small food object, the penalty will increase. The food annotation is divided into one or more attributes, including small food targets, medium food targets, and large food targets. In the formula Represents the proportion of samples belonging to the foreground category c, showing The larger it is, the more the model training penalizes the wrong samples for detection.
[0046] Among them, the method based on prior color and texture feature retrieval is highly robust and reproducible, and can effectively filter out environmental and lighting noise through feature convolution, ensuring the fairness and accuracy of the system in food color and texture quality inspection.
[0047] In this embodiment, Figure 2 As shown, the feature extraction model 3 includes the following steps:
[0048] S3. ArcMargin loss is used. The mathematical expression is as follows: In the inference stage, we design the number of neurons in the last fully connected layer of the network to be 128, which means that the image feature value output by the feature extraction network is 128D. This feature value represents the representational features of the food target image and can be mainly used for similarity calculation to determine the food name.
[0049]
[0050] In the formula, θ j is the angle between the weight and the feature, the batch size and number of categories are N and n respectively, and m is the target angle The angular margin penalty on , s is the characteristic scale,
[0051] Identifying the feature library 4 includes the following steps:
[0052] S4. Select a different style of the same food for feature library registration, so as to increase the food feature characterization capability. The feature value is obtained by cutting the food target from the original picture and sending it to the feature extraction model to obtain the 128D feature value. The feature value and its corresponding food name are stored in the database to complete the design of the recognition feature library. The quality inspection feature library 5 includes the following steps:
[0053] S5. A professional food chef selects a picture of the best food quality as the standard food, and compares and scores the quality inspection items of the food under inspection with the standard food. The quality inspection feature library stores the color feature map and texture feature map of the standard product. These feature libraries are used to perform similarity calculation with the corresponding feature maps extracted after inputting the feature extraction model of the food under inspection, thereby completing the quality similarity calculation;
[0054] The color feature map retrieval 6 and the texture feature map retrieval 7 include the following steps:
[0055] S6. During the training phase, feature maps representing color features and texture features are retrieved by visualizing shallow feature maps and deep feature maps respectively.
[0056] Among them, the subject detection model 2 aims to detect all food targets in the image taken by the camera, and the feature extraction model 3 aims to extract category features and color problem features of the food targets output by the subject detection model 2 and calculate the similarity with the label features in the image library to obtain the correct classification and recognition and quality inspection score of the food targets.
[0057] In this embodiment, Figure 3 As shown, the similarity calculation 8 includes the following steps:
[0058] S7. The 128D feature value output by the feature extraction model represents the category information of the identified food. The cosine value of the angle between the two vectors is calculated based on the cosine similarity after the feature value is vectorized. The cosine value represents the similarity between the two vectors. The larger the cosine value, the more similar the two vectors are, that is, the more similar the two food target images are. The cosine similarity calculation formula is as follows:
[0059]
[0060] In the formula, A and B represent two eigenvectors, A i and B i They represent the components of vectors A and B respectively, n represents the dimension of the feature vector, and the value of n is 128.
[0061] Scoring 9 includes the following steps:
[0062] S8. The mathematical formula for food quality inspection scoring is as follows:
[0063]
[0064] In the formula, and β represent the color and texture score adjustment weight factors respectively, where
[0065] score color and score texture The score represents the cosine similarity calculation results of the color feature map and texture feature map of the inspected food and the feature map of the corresponding food in the quality inspection feature library. color and score texture The value range is [0,1], score represents the final score of the inspected food, and the value range is [0,100]. Assume that Set to 0.4, β to 0.6, and calculate the cosine similarity to get the score color =0, score texture =1, it means that the color quality inspection gets 0 points, the texture quality inspection gets full marks, and the final food quality inspection score is 60 points.
[0066] Workflow of the present invention: When the food rapid identification and quality inspection image system and method designed by the present scheme are working, the image capture 1 is mainly responsible for acquiring the image data obtained by the system triggering the photo taking from the device camera; the subject detection model 2 mainly performs food target detection from the captured image, and the feature extraction model 3 performs deep feature extraction on the food target. The intermediate color feature map and texture feature map retrieved in the feature extraction process are used to calculate with the pre-registered quality inspection feature library to obtain the quality inspection score. At the same time, the final feature value output by the feature extraction model 3 and the pre-registered recognition feature library 4 are similarly calculated, and the food recognition result is obtained by sorting according to the recognition similarity value.
[0067] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A food rapid identification and quality inspection image system, comprising an image capture device (1), characterized in that: The output end of the image capture (1) is electrically connected to a subject detection model (2), the output end of the subject detection model (2) is electrically connected to a feature extraction model (3), the input end of the feature extraction model (3) is electrically connected to an identification feature library (4) and a quality inspection feature library (5), the output end of the feature extraction model (3) is electrically connected to a color feature map retrieval (6), a texture feature map retrieval (7) and a similarity calculation (8), and the output ends of the color feature map retrieval (6) and the texture feature map retrieval (7) are electrically connected to a quality inspection score (9).
2. A method for rapid food recognition and quality inspection image, applicable to the rapid food recognition and quality inspection image system of claim 1, characterized in that: The image capturing (1) comprises the following steps: S1. The camera should be installed vertically at the center of the food tray and at a height of (70-80) cm. The food image data is obtained by taking photos with an industrial camera installed in the food visual recognition hardware system.
3. A method for rapid food identification and quality inspection according to claim 2, characterized in that: The subject detection model (2) comprises the following steps: S2. Based on yolov8, yolov8_biasloss is proposed to enhance the detection capability of long-tail data and difficult data. The loss function expression of yolov8_biasloss is as follows: In the formula, f a and f m They represent the α factor and adjustment factor respectively, loss_fcn is the CEWithLogitsLoss loss function, β represents the target box size factor, and h min and w min is the minimum height and width of the food target size allowed, h and w represent the actual height and width of the food target. If the detection error occurs in a small food object, the penalty will increase. The food annotation is divided into one or more attributes, including small food targets, medium food targets, and large food targets. In the formula Represents the proportion of samples belonging to the foreground category, showing The larger it is, the more the model training penalizes the wrong samples for detection.
4. A method for rapid food identification and quality inspection according to claim 2, characterized in that: The feature extraction model (3) comprises the following steps: S3. ArcMargin loss is used. The mathematical expression is as follows: In the inference stage, we design the number of neurons in the last fully connected layer of the network to be 128, which means that the image feature value output by the feature extraction network is 128D. This feature value represents the representational features of the food target image and can be mainly used for similarity calculation to determine the food name. In the formula, θ j is the angle between the weight and the feature, the batch size and number of categories are N and n respectively, and m is the target angle The angular margin penalty on , s is the characteristic scale.
5. A method for rapid food identification and quality inspection according to claim 2, characterized in that: The identification feature library (4) comprises the following steps: S4. For the same food in different styles, select one image in different styles to register in the feature library, so as to increase the food feature representation capability. The feature value is obtained by cutting the food target from the original image and sending it to the feature extraction model to obtain a 128D feature value. The feature value and its corresponding food name are stored in the database to complete the design of the recognition feature library.
6. A method for rapid food identification and quality inspection according to claim 2, characterized in that: The quality inspection feature library (5) comprises the following steps: S5. A professional food chef selects a picture of the best food quality as the standard food, and compares and scores the quality inspection items of the food under inspection with the standard food. The quality inspection feature library stores the color feature map and texture feature map of the standard product. These feature libraries are used to perform similarity calculation with the corresponding feature maps extracted after inputting the feature extraction model of the food under inspection, thereby completing the quality similarity calculation; The color feature map retrieval (6) and the texture feature map retrieval (7) include the following steps: S6. During the training phase, feature maps representing color features and texture features are retrieved by visualizing shallow feature maps and deep feature maps respectively.
7. A method for rapid food identification and quality inspection according to claim 2, characterized in that: The similarity calculation (8) comprises the following steps: S7. The 128D feature value output by the feature extraction model represents the category information of the identified food. The cosine value of the angle between the two vectors is calculated based on the cosine similarity after the feature value is vectorized. The cosine value represents the similarity between the two vectors. The larger the cosine value, the more similar the two vectors are, that is, the more similar the two food target images are. The cosine similarity calculation formula is as follows: In the formula, A and B represent two eigenvectors, A i and B i They represent the components of vectors A and B respectively, n represents the dimension of the feature vector, and the value of n is 128.
8. A method for rapid food identification and quality inspection according to claim 2, characterized in that: Quality inspection score (9) The following steps are involved: S8. The mathematical formula for food quality inspection scoring is as follows: In the formula, and β represent the color and texture score adjustment weight factors respectively, where score color and score texture The score represents the cosine similarity calculation results of the color feature map and texture feature map of the inspected food and the feature map of the corresponding food in the quality inspection feature library. color and score texture The value range is [0,1], score represents the final score of the inspected food, and the value range is [0,100]. Assume that Set to 0.4, β to 0.6, and calculate the cosine similarity to get the score color =0, score texture =1, it means that the color quality inspection gets 0 points, the texture quality inspection gets full marks, and the final food quality inspection score is 60 points.