Qualitative and quantitative detection method and system for food maturity based on image deep learning

Abstract

The present invention discloses a method and system for qualitatively and quantitatively detecting the maturity of dishes based on image deep learning. The method comprises: S1, measuring the physical and chemical property index data of dishes of different maturity levels, analyzing the data, and determining the processing technology of dishes of different maturity levels; S2, building an image acquisition system and a dish maturity database based on cooking environment conditions and image construction requirements; S3, using the dish maturity database to train an image deep learning model to achieve target detection and instance segmentation of dishes of different maturity levels, and respectively build a maturity judgment system; S4, based on the probability density function and the dish maturity prediction results output by the maturity judgment system, quantitatively detecting the physical and chemical properties of dishes of different maturity levels. The present invention, based on the self-learning and self-thinking capabilities of the deep learning model, applies it to the intelligent discrimination of dish maturity, effectively solving the current problem of no relevant standards and reliance on experience for maturity discrimination.

Description

Technical Field

[0001] The present invention belongs to the technical field of food detection, and specifically relates to a method and system for qualitative and quantitative detection of dish maturity based on image deep learning. Background Art

[0002] Overcooking can reduce nutritional value and produce toxic and harmful substances. For example, if frying is too long, the fat can oxidize rapidly due to the heat, producing peroxides that are harmful to the body. Therefore, properly determining the doneness of food during cooking is a crucial issue in the food industry.

[0003] Although traditional maturity detection methods can determine the maturity of dishes, they have problems such as damage to samples, expensive instruments and equipment, and high operating thresholds.

[0004] Chinese patent CN202211454574.7 discloses a "control method and automatic cooking machine," which sets cooking process parameters based on parameters such as the type and weight of the ingredients before cooking to ensure that the dishes are cooked until mature. This patent selects the cooking process based on the state of the ingredients before cooking to control the maturity of the ingredients, and cannot sense them in real time, which has significant limitations. Chinese patent CN202410586342.X discloses an "intelligent cooking method and related device based on ingredient maturity perception," which uses multimodal equipment to obtain multiple types of information. After linear fitting with simple functions, the obtained features are compared with an existing database to obtain the freshness and maturity of the cooking ingredients. However, the model in this patent relies on existing databases, lacks independent thinking capabilities, and requires the support of a large database. The database is prepared based on online recipes and has not been scientifically analyzed. In addition, multimodal detection requires expensive equipment and complex operations, which limits its convenience and scalability. Summary of the Invention

[0005] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and propose a method and system for qualitative and quantitative detection of dish maturity based on image deep learning.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The qualitative and quantitative detection method for food maturity based on image deep learning includes the following steps:

[0008] S1. Measure the physical and chemical properties of dishes at different maturity levels, analyze the data, and determine the processing technology for dishes at different maturity levels;

[0009] S2. Build an image acquisition system and a food maturity database based on cooking environment conditions and image construction requirements;

[0010] S3. Use the dish maturity database to train an image deep learning model to achieve object detection and instance segmentation for dishes of different maturity levels, and build maturity judgment systems for each.

[0011] S4. Based on the probability density function and the maturity prediction results of the dish output by the maturity judgment system, quantitative detection of the physical and chemical properties of dishes of different maturity levels is performed.

[0012] The present invention also includes a qualitative and quantitative detection system for dish maturity based on image deep learning. The system adopts the qualitative and quantitative detection method for dish maturity provided by the present invention. The system includes: a dish processing technology determination module, an image acquisition system, a dish maturity database construction module, a maturity judgment system construction module, and a dish physical and chemical property quantitative detection module;

[0013] The dish processing technology determination module is used to measure the physical and chemical properties of dishes of different maturity levels, analyze the data, and determine the processing technology of dishes of different maturity levels;

[0014] Image acquisition system, used to collect images of dishes at different levels of maturity;

[0015] The dish maturity database construction module uses the images collected by the image acquisition system to build a dish maturity database;

[0016] The maturity judgment system construction module uses the dish maturity database to train the image deep learning model to achieve target detection and instance segmentation of dishes of different maturity levels, and build maturity judgment systems respectively;

[0017] The quantitative detection module of the physical and chemical properties of dishes performs quantitative detection of the physical and chemical properties of dishes of different maturity levels based on the probability density function and the dish maturity prediction results output by the maturity judgment system.

[0018] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0019] 1. Existing non-deep learning maturity recognition models do not have transfer capabilities, and the application of deep learning models is relatively simple. Most of them stay at the direct replacement of model databases, resulting in many models being screened out before they can fully exert their capabilities. The present invention introduces transfer learning in target detection, retains some parameters and feature extraction capabilities of excellent models, and compares various models on this basis, greatly improving the model screening efficiency and the final application effect. In instance segmentation, the characteristics of the cooking process of dishes in traditional cooking scenarios are reasonably analyzed, and based on this characteristic, the Yolov8 model is improved to construct the Yolo_doneness model, which innovates the deep learning application method.

[0020] 2. Existing image-based deep learning intelligent discrimination applications are limited to qualitative analysis of black box properties from input to output. The present invention develops a quantitative detection method for the physical and chemical properties of food maturity based on probability density functions. It uses the physical and chemical properties measured when the maturity standards are established in advance to achieve non-destructive and rapid detection of physical and chemical properties, promote the development of nutritional analysis calculations and special recipe recommendations, food quality analysis and other functions, help scientific researchers to conduct rough and rapid analysis, enrich the functions of corresponding software and equipment, and enhance consumers' sense of gain.

[0021] 3. Existing cooking maturity recognition programs and methods are mostly concentrated in large high-end equipment such as smart microwave ovens and smart ovens. They are expensive and not universal. The present invention can be obtained and used by various users by building a smartphone application, and has a broad application market and development space. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of the method of the present invention;

[0023] Figure 2 Schematic diagram of the physical and chemical property index data of the dishes measured in the examples;

[0024] Figure 3 is the correlation analysis result of the physicochemical properties of steaks at different maturity levels in the embodiment;

[0025] Figure 4 Schematic diagram of the number and grouping of images in constructing a steak maturity database in an embodiment;

[0026] Figure 5 3. This is a comparison chart of the accuracy and loss values ​​of each transfer learning model in the steak maturity recognition training process in the embodiment;

[0027] Figure 6 is the confusion matrix of the performance results of GoogLeNet and DenseNet121 on the steak maturity test set in the embodiment;

[0028] Figure 7 This is a .yaml file diagram of the Yolov8 model main architecture improvement module and the self-added module fusion;

[0029] Figure 8 This is a schematic diagram of the quantitative detection of physical and chemical properties of vegetables of different maturity levels. DETAILED DESCRIPTION

[0030] The present invention will be described in further detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0031] like Figure 1 As shown in FIG, the method for qualitative and quantitative detection of food maturity based on image deep learning includes the following steps:

[0032] S1. Measure the physical and chemical properties of dishes at different maturity levels, analyze the data, and determine the processing technology for dishes at different maturity levels;

[0033] Among them, the physical and chemical properties of dishes of different maturity were measured. The measured physical and chemical properties included cooking loss, myofibrillar protein content, total thiol content, myoglobin content, malondialdehyde content, texture, color difference, water phase distribution, electron microscope observation images, volatile components of dish aroma, and soluble components of juice after cooking.

[0034] The data analysis specifically includes:

[0035] The measured data of physical and chemical properties of dishes with different maturity levels were analyzed based on statistical analysis software such as SPSS, including: using correlation analysis to obtain changes in indicators with the same trend, determining the consistency of macroscopic and microscopic changes in dish properties, increasing the interpretability of changes in physical and chemical properties during the cooking process of dishes, and preliminarily determining which physical and chemical properties have the same effectiveness in the maturity judgment task; further using principal component analysis to determine the main influencing factors of dish maturity to reduce the amount of data required to be processed in the process of determining the maturity classification of dishes; finally, using cluster analysis to classify the physical and chemical properties of dishes corresponding to the set cooking time gradient, and determining the processing technology of dishes with different maturity levels according to the cooking time corresponding to the classified categories (the processing technology includes a series of processes involved in the traditional dish cooking process, such as cooking conditions, sample preparation, and cooking time).

[0036] S2. Build an image acquisition system and a food maturity database based on cooking environment conditions and image construction requirements; the image acquisition system includes a cooking module, an image acquisition module, a basic framework, and an auxiliary equipment module;

[0037] Among them, the cooking module includes an induction cooker and other required cooking tools; the image acquisition module includes an industrial camera and a manipulator for fixing and adjusting the camera position; the basic skeleton is a square frame built of stainless steel; the auxiliary equipment module includes a fan, a scattering lamp and gauze. The fan is used to remove the water mist and smoke generated during the database construction process. The scattering lamp serves as a light source to eliminate database fluctuations caused by different light. The gauze serves as an isolation to prevent external light sources from entering and causing image distortion. The cooking module is set at the bottom of the basic skeleton, the image acquisition module is fixed to the upper part of the basic skeleton with screws, and the gauze is fixed to the five surfaces of the basic skeleton except the bottom surface with clips, and a fan exhaust port is left on the side.

[0038] The construction of the dish maturity database is as follows:

[0039] Using the constructed image acquisition system, according to the determined processing technology of dishes of different maturity, the cooking and dish image acquisition of dishes of different maturity are completed, the images of dishes of different maturity are stored and the random seeds are used to complete the division of training set and test set.

[0040] S3. Use the dish maturity database to train an image deep learning model to achieve object detection and instance segmentation for dishes of different maturity levels, and build maturity judgment systems for each.

[0041] The object detection task is implemented through transfer learning. Based on PyTorch, several existing image deep learning models are selected and trained using the constructed food maturity database. The model with the fastest convergence, highest accuracy, and most stable model operation is selected as the optimal object detection model.

[0042] The instance segmentation task is implemented based on the Yolov8 model. It is adaptively improved based on the characteristics of the cooking process, and the Yolo_doneness model with better performance in mature recognition is constructed for instance segmentation.

[0043] The Yolov8 model is adaptively improved according to the characteristics of the cooking process, specifically including:

[0044] ContextGuide convolution is used to replace the C2f layer of the feature extraction part of the original Yolov8 model, Deformable convolution replaces the traditional convolution of the original Yolov8 model, TripletAttention mechanism is used to improve the C2f layer of the detection head of the original Yolov8 model, SA_attention detection head replaces the detection head of the original Yolov8 model, and the AOD-PONO-Net end-to-end dehazing detection module is deployed as a pre-processing means before the data is input into Yolo_doneness. The combination of ASF and SDI and the SEAM attention module are deployed in the data fusion part before the detection head of the original Yolov8 model. Finally, the constructed models are integrated to construct a Yolo_doneness model with specific applicability to the maturity of dishes.

[0045] Based on the optimal target detection model and Yolo_doneness model, maturity judgment systems are constructed respectively.

[0046] S4. Based on the probability density function and the maturity prediction results of the dish output by the maturity judgment system, quantitative testing of the physical and chemical properties of dishes of different maturity levels is performed;

[0047] Quantitative detection of the physical and chemical properties of vegetables at different maturity levels includes methods for target detection tasks in transfer learning models and methods for instance segmentation tasks in Yolo_doneness models;

[0048] Among them, in the method for target detection task of transfer learning model, a probability density function is introduced, and the inverse of the numerical difference of a certain physical and chemical property between two similar maturity levels is used as the probability density. The confidence corresponding to the three labels with the highest confidence levels are obtained according to the target detection model, and the comprehensive maturity label of the measured dish is obtained by weighted calculation with the label as the weight, and the corresponding physical and chemical index range is selected. Based on the probability density range and the corresponding physical and chemical property values ​​in the dish maturity database, the physical and chemical property value of the measured dish is quantitatively analyzed.

[0049] Among them, the method used for the instance segmentation task of the Yolo_doneness model first uses the instance segmentation mask area of ​​the same maturity of the same type of dishes as the weight to calculate the weighted confidence of the dishes with the same maturity. Then, the instance segmentation mask area of ​​different maturity of the same type of dishes is used as the weight, and with the help of the maturity label, the comprehensive maturity label of the same type of dishes is calculated. Finally, based on this label, the quantitative value of a certain physical and chemical indicator of the dish is calculated using the method used for the target detection task of the transfer learning model.

[0050] Example

[0051] This embodiment takes the qualitative and quantitative detection of steak maturity as an example, and includes the following steps:

[0052] S1, measuring the physical and chemical properties of steaks of different maturity, analyzing the data, and obtaining the processing technology of steak samples of different maturity; in this embodiment, Figure 2 As shown, including:

[0053] Cut the steak into 6cm×4cm×1.5cm blocks. Set the induction cooker power to 80W, add an appropriate amount of olive oil to the frying pan, and use an infrared thermometer to wait until the bottom temperature of the pan reaches 105°C before placing the prepared steak into the pan. Prepare steak samples of different degrees of maturity with a frying time of 2-12 minutes.

[0054] Steak samples were cut into 1.5 cm cubes along the muscle fiber direction. Texture information was measured using a P36 meat texture analyzer, with a pre-test speed of 1.5 mm / s and a post-test speed of 2 mm / s over a distance of 30 mm. The firmness, springiness, chewiness, and stickiness of each sample were calculated as indicators of doneness. Three replicates were performed for each experimental condition, and the average value was calculated.

[0055] Use a colorimeter to measure the color of the cross section of the steak. Cut the steak sample along the cross section and use a colorimeter to measure the redness, greenness, yellowness, blueness and brightness of the center of the steak and calculate them as a * , b * , L *Three random cross sections of each sample at each maturity were tested, and the average of the three test results was taken.

[0056] Using a low-field NMR instrument and a 40mm probe, the steak samples were cooled, the center cut into 1.5cm cubes, and wrapped in plastic wrap for later use. The sample was then placed into a 40mm NMR tube and placed in the tube. The T2 relaxation time and signal intensity of the steaks were measured using a CPMG sequence to determine the distribution of the aqueous phase in steaks of varying maturity.

[0057] After removing the fascia and fat, the steaks of different maturity levels were broken. 3.0 g of meat sample was placed in a centrifuge tube and 25 mL of a pre-prepared 0.01 mol / L sodium phosphate buffer solution (potassium monohydrogen phosphate: dipotassium hydrogen phosphate = 1:1 (molar ratio)) was added and homogenized at 8000 rpm for 2 minutes. The sample was then placed in a 4°C refrigerator and allowed to stand for 1 hour. After removal, the sample was centrifuged in a refrigerated centrifuge at 4°C and 7800 rpm for 15 minutes. The supernatant was filtered through a PES filter with a pore size of 0.45 μm and the absorbance of each sample at 525, 545, 565, and 572 nm was measured using a spectrophotometer. The three myoglobin contents were calculated as follows:

[0058]

[0059] Among them, A 525 is the absorbance of the sample at 525 nm, A 545 is the absorbance of the sample at 545 nm, A 565 is the absorbance of the sample at 565 nm, A 572 is the absorbance of the sample at 572 nm.

[0060] This embodiment designed a series of physical and chemical properties and micro-indicator tests from both macroscopic and microscopic aspects. The specific measurement data are shown in Table 1 below. According to Table 1, the overall physical and chemical property indicators are relatively significant in correlation, and there is also consistency between the macroscopic and microscopic aspects. The overall curve is relatively flat from 2 to 4 minutes, increases significantly after 6 minutes, and tends to be flat after 8 minutes. From this, it can be judged that the steak cooking process changes gradually significantly after 4 minutes, and the microstructure of the steak changes most rapidly and obviously after 6 minutes. Therefore, on this basis, the degree of doneness is graded as one-well-done, three-well-done, five-well-done, and seven-well-done according to the cooking time of 2 minutes, 4 minutes, 6 minutes, and 8 minutes. A longer heating interval of 12 minutes is fully cooked. This embodiment studies the relationship between cooking time and steak maturity on the basis of controlling other variables, but the quantitative detection method of the present invention is suitable for the study of various indicators of dish cooking and processing technology.

[0061]

[0062] Table 1

[0063] The measured physical and chemical property index data were input into SPSS27 statistical analysis software for processing. The original parameters of different physical and chemical indexes were input into the software and the cooking time was used as their label. The bivariate correlation in the analysis function of the software was used to calculate the mean and variance. The Pearson correlation coefficient was used for two-tailed test and the correlation was verified under the confidence interval of 0.05 and 0.01 respectively. Figure 3 The following are the correlation analysis results of the physicochemical properties of steaks at different maturity levels. Figure 3 It can be seen that the content of hemoglobin and the red-blue intensity (a * ) is significantly correlated, brightness (L * ) was significantly correlated with metmyoglobin and deoxymyoglobin, and the chromaticity and protein content achieved an organic unity to a certain extent. The macroscopic and microscopic results were consistent, which further proved that the unevenness of steak maturity during the frying process and the mutation at 4min and 6min were objectively present.

[0064] On this basis, in order to obtain a more standard steak maturity sample and reduce the complexity of the judgment to improve efficiency, we selected more representative physical and chemical indicators to establish an objective maturity judgment method. Figure 3 It can be seen that the content of deoxy and metmyoglobin is related to L * The values ​​were significantly correlated; the content of oxymyoglobin and a * The values ​​of cooking loss and L are significantly correlated; all texture parameters are significantly correlated; myofibrillar protein has the highest correlation with cooking loss. * In order to conduct more accurate or in-depth research on the steak maturity judgment model, the value, oxymyoglobin content and hardness are used as representative indicators.

[0065] S2. Build an image acquisition system and steak maturity database based on cooking environment conditions and image construction requirements. In this embodiment, specifically:

[0066] In order to control the same height for each sampling and photographing, and to effectively isolate ambient light and image noise, a simple image acquisition system was built, including an induction cooker, an iron pot, a shooting table, an industrial camera, and a manipulator for fixing and adjusting the camera position. The outside of the system was wrapped with gauze, the shooting table was supported by a tripod, and an operating window was set up with white cloth on the outside, which was connected to a pipe with an exhaust fan to extract the fumes.

[0067] During the image acquisition process, we randomly selected angles between different steaks, and obtained a total of more than 1,700 steak maturity images. We then used random seeds to complete the division of the training set and test set. The number of data for the five different maturity levels is similar, and the number of images in the training set and test set for each category is as follows: Figure 4As shown in the figure, a balanced amount of data can improve the robustness and balance of the model and prevent the occurrence of simplified analysis.

[0068] S3. Use the constructed steak maturity database to train an image deep learning model to achieve object detection and instance segmentation of steaks of different maturity levels, and build maturity judgment systems for each.

[0069] Among them, the target detection task is achieved through transfer learning. Based on Pytorch, several existing image deep learning models are selected and trained using the constructed dish maturity database. The model with the fastest convergence, highest accuracy and most stable model operation is selected as the optimal target detection model; in this embodiment, specifically:

[0070] Models such as AlexNet, VGG19, and ResNet34 were selected for training in the steak maturity database. During the training process, the feature extraction part of the model was frozen, and only the fully connected layer of the model was trained.

[0071] During the training process, batch=32, lr=0.003, epoch=30. The best model is the maximum accuracy of each batch in the same training round, and the relevant weights are saved. The training results of each model are as follows Figure 5 As shown in the figure, AlexNet and VGG19 models cannot converge effectively, SqueezeNet can only reach 80% accuracy, and ResNet34 can improve the accuracy to more than 98% within 30 rounds. However, the remaining GoogLeNet and DenseNet121 have good fitting effects, and both reach nearly 100% accuracy and stability in the seventh round of training. Figure 6 As shown in the figure, the confusion matrix of the performance results of the GoogLeNet and DenseNet121 models on the steak maturity test set shows that the first accuracy of the DenseNet121 model for sample prediction is significantly higher than that of GoogLeNet. The heat map on the main diagonal is darker in color, indicating better prediction performance. Therefore, DenseNet121 is used as the best steak maturity judgment model.

[0072] The instance segmentation task is implemented based on the Yolov8 model, which is adaptively improved according to the characteristics of the cooking process to construct the Yolo_doneness model with better maturity recognition performance for instance segmentation. In this embodiment, the specific steps are:

[0073] Based on the characteristics of overlapping, small sample size, susceptibility to interference, and small size of samples in the cooking process, the Yolo model for maturity recognition was constructed based on the adaptive modification of the Yolov8 model. The ContextGuide convolution replaces the C2f layer in the feature extraction part of the original Yolov8 model to improve the model's ability to connect and fuse contextual information. The Deformable module replaces the traditional convolution in the original Yolov8 model to improve the model's adaptability to geometric transformations, allowing the model to adapt to the differences in size, angle, and shape of the same dish due to different segmentation methods during cooking. To adapt to the details of color and texture reflected in the varying maturity of dishes, the TripletAttention module is introduced to improve the C2f information extraction module in the model's Neck, reducing its attention to the length and width dimensions and increasing its attention to the channel dimension. The ASF+SDI architecture improves the Neck architecture of the original model, improving model computation speed and streamlining the model. This reduces the number of floating-point calculations and enhances the model's ability to transfer and fuse feature extraction, making the model lightweight, integrated, and computationally convenient without affecting the model's recognition capabilities. To adapt to situations where dishes have dense and overlapping ingredients, the SA_attention detection head is introduced. The self-attention mechanism module and convolution module are rationally configured to achieve efficient dish feature extraction with minimal parameters and reduce the occurrence of disconnected and ignored segmented samples. Finally, the above modules are reasonably arranged and stitched together to build the Yolo steak maturity instance segmentation model.

[0074] On this basis, the AOD-PONO-Net end-to-end defogging detection module is introduced to solve the problem of blurred images caused by water vapor condensation on the camera lens during the cooking process. With the help of the SEAM attention module in the Yolo-Face model, a weighted loss compensation module is added to correct the instance segmentation results to solve the problem of stacked ingredients occluding during the cooking process. Finally, the modules are fused based on the Yolov8 environment to develop the actual segmentation model Yolo_doneness suitable for judging the doneness of steak. Figure 7 The figure shows a .yaml file diagram of the Yolov8 model main architecture improvement module and the self-added module fusion.

[0075] Based on the optimal target detection model DenseNet121 and Yolo_doneness model, steak doneness judgment systems were constructed respectively.

[0076] S4. Based on the probability density function and the steak maturity prediction results output by the maturity judgment system, quantitative detection of the physical and chemical properties of steaks of different maturity levels is performed.

[0077] like Figure 8As shown in the figure, the quantitative detection of the physical and chemical properties of vegetables of different maturity levels includes a method based on the target detection task of the transfer learning model and a method based on the instance segmentation task of the Yolo_doneness model.

[0078] Among them, in the method of target detection task based on transfer learning model, a probability density function is introduced, and the inverse of the numerical difference of a certain physical and chemical property between two similar maturity levels is used as the probability density. The confidence corresponding to the three labels with the highest confidence levels are obtained according to the target detection model, and the comprehensive maturity label of the measured dish is obtained by weighted calculation with the label as the weight, and the corresponding physical and chemical index range is selected. Based on the probability density range and the corresponding physical and chemical property values ​​in the dish maturity database, the physical and chemical property value of the measured dish is quantitatively analyzed.

[0079] Among them, the method based on the instance segmentation task of the Yolo_doneness model first uses the instance segmentation mask area of ​​the same maturity of the same type of dishes as the weight to calculate the weighted confidence of the dishes with the same maturity. Then, the instance segmentation mask area of ​​different maturity of the same type of dishes is used as the weight, and with the help of maturity labels, the comprehensive maturity label of the same dish is calculated. Finally, based on this label, the quantitative value of a certain physical and chemical indicator of the dish is calculated using the method based on the transfer learning model target detection task.

[0080] In this embodiment, specifically:

[0081] Assume that the detection result of a certain steak target is that the steak image recognition result is 80.22% well-done, 10.67% medium-rare, and 4.80% well-done, and the myofibrillar protein content corresponding to the three degrees of maturity are 24.15 mg / g, 151.34 mg / g, and 36.50 mg / g respectively.

[0082] In this example, the labels for well-done, medium-rare, and rare are 5, 2, and 4, respectively; the measured doneness of the steak is calculated according to the following formula:

[0083]

[0084] Among them, A is the sample prediction maturity label, A i Output the maturity label of the measured sample for the model, B i The model outputs the corresponding confidence level of the maturity of the measured sample.

[0085] Based on this formula, the maturity label of the tested steak sample is calculated to be 4.4164, and the maturity range of the sample is determined to fall between well-done and medium-rare. The probability density functions of the two are used to calculate the myofibrillar protein content of the tested sample based on the probability density functions, creating the following formula:

[0086] m=(m1-m2)×(A-A2)+m2

[0087] Among them, m1 is the myofibrillar protein content with a larger maturity label, m2 is the myofibrillar protein content with a smaller maturity label, A is the sample predicted maturity label, and A2 is the smaller maturity label.

[0088] Based on this formula, the myofibrillar protein content of the tested steak was calculated to be 31.36 mg / g. Without using a probability density function, the sample content would be calculated based solely on the confidence level and the myofibrillar protein content corresponding to each maturity level, resulting in a value of 37.27 mg / g. This value fluctuates significantly due to the properties of the three-mature samples, which are unrelated to the actual maturity of the tested sample. This method can effectively avoid such situations caused by excessively large or small values.

[0089] In the instance segmentation task, the quantitative analysis of the physical and chemical properties of vegetables of different maturity levels is improved based on the methods related to the target detection task. The calculation is based on the output of the instance segmentation model and the mask segmentation area. The specific calculation formula is as follows:

[0090]

[0091] Among them, C is the confidence level of the weighted sample of the same ingredient and the same maturity, C i is the confidence of the i-th mask area of ​​the maturity category, S i is the i-th mask area of ​​the maturity.

[0092] The confidence of the comprehensive label of the same ingredient is then weighted using the following formula:

[0093]

[0094] Among them, D is a comprehensive sample label of a certain food, C j S is the confidence level of the sample of the same ingredient at the same maturity after weighting the jth maturity of the ingredient. j is the total mask area under the jth maturity, D j The i-th maturity label of the ingredient.

[0095] Finally, the calculated final sample confidence of a certain food ingredient is brought into the quantitative detection formula of physical and chemical properties applicable to the target detection task to realize the quantitative analysis of a physical and chemical property in the instance segmentation task:

[0096] M=(M1-M2)×(D-D2)+M2

[0097] Among them, M is the physical and chemical property value that needs to be measured, M1 is the physical and chemical property value with a larger maturity label, M2 is the physical and chemical property value with a smaller maturity label, D is the comprehensive sample label of a certain food ingredient, and D2 is the smaller maturity label.

[0098] In another embodiment, a qualitative and quantitative detection system for dish maturity based on image deep learning is also provided. The system adopts the qualitative and quantitative detection method for dish maturity described in the above embodiment. The system includes: a dish processing technology determination module, an image acquisition system, a dish maturity database construction module, a maturity judgment system construction module, and a dish physical and chemical property quantitative detection module;

[0099] The dish processing technology determination module is used to measure the physical and chemical properties of dishes of different maturity levels, analyze the data, and determine the processing technology of dishes of different maturity levels;

[0100] Image acquisition system, used to collect images of dishes at different levels of maturity;

[0101] The dish maturity database construction module uses the images collected by the image acquisition system to build a dish maturity database;

[0102] The maturity judgment system construction module uses the dish maturity database to train the image deep learning model to achieve target detection and instance segmentation of dishes of different maturity levels, and build maturity judgment systems respectively;

[0103] The quantitative detection module of the physical and chemical properties of dishes performs quantitative detection of the physical and chemical properties of dishes of different maturity levels based on the probability density function and the dish maturity prediction results output by the maturity judgment system.

[0104] It should also be noted that, in this specification, terms such as "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or apparatus comprising the element.

[0105] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A qualitative and quantitative detection method for food maturity based on image deep learning, characterized by: The following steps are involved: S1. Measure the physical and chemical properties of dishes at different maturity levels, analyze the data, and determine the processing technology for dishes at different maturity levels; S2. Build an image acquisition system and a food maturity database based on cooking environment conditions and image construction requirements; S3. Use the dish maturity database to train an image deep learning model to achieve object detection and instance segmentation for dishes of different maturity levels, and build maturity judgment systems for each. S4. Based on the probability density function and the maturity prediction results of the dish output by the maturity judgment system, quantitative testing of the physical and chemical properties of dishes of different maturity levels is performed; In step S3, the object detection task is implemented through transfer learning. Based on PyTorch, several existing image deep learning models are selected and trained using the constructed dish maturity database. The model with the fastest convergence, highest accuracy, and most stable model operation is selected as the optimal object detection model. The instance segmentation task is implemented based on the Yolov8 model. It is adaptively improved based on the characteristics of the cooking process, and the Yolo_doneness model with better performance in mature recognition is constructed for instance segmentation. Based on the optimal target detection model and Yolo_doneness model, a maturity judgment system is constructed respectively; Based on the Yolov8 model, it is adaptively improved according to the characteristics of the cooking process, including: ContextGuide convolution is used to replace the C2f layer of the feature extraction part of the original Yolov8 model, Deformable convolution replaces the traditional convolution of the original Yolov8 model, TripletAttention mechanism is used to improve the C2f layer of the detection head of the original Yolov8 model, SA_attention detection head replaces the detection head of the original Yolov8 model, and the AOD-PONO-Net end-to-end dehazing detection module is deployed as a pre-processing means before the data is input into Yolo_doneness. The combination of ASF and SDI and the SEAM attention module are deployed in the data fusion part before the detection head of the original Yolov8 model. Finally, the constructed models are integrated to construct a Yolo_doneness model with specific applicability to the maturity of dishes.

2. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 1 is characterized in that: In step S1, the physical and chemical property index data of dishes of different maturity levels are measured. The measured physical and chemical property indexes include cooking loss, myofibrillar protein content, total thiol content, myoglobin content, malondialdehyde content, texture, color difference, water phase distribution, electron microscope observation image, volatile components of dish aroma, and soluble components of dish juice after cooking.

3. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 1 is characterized in that: In step S1, the data is analyzed, including: The measured physical and chemical properties of vegetables of different maturity levels were analyzed using statistical analysis software, including: The correlation analysis method was used to obtain the changes in the same trend indicators, determine the consistency of the macroscopic and microscopic changes in the properties of the dishes, increase the interpretability of the changes in the physical and chemical properties of the dishes during the cooking process, and preliminarily determine which physical and chemical properties have the same effectiveness in the maturity judgment task; further, the principal component analysis method was used to determine the main influencing factors of the maturity of the dishes to reduce the amount of data required to be processed in the process of determining the maturity of the dishes; finally, the cluster analysis method was used to classify the physical and chemical properties of the dishes corresponding to the set cooking time gradient, and determine the processing technology of dishes with different maturity levels according to the cooking time corresponding to the classified categories.

4. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 1 is characterized in that: In step S2, the image acquisition system includes a cooking module, an image acquisition module, a basic skeleton, and an auxiliary equipment module; Among them, the cooking module includes an induction cooker and other required cooking tools; the image acquisition module includes an industrial camera and a manipulator for fixing and adjusting the camera position; the basic skeleton is a square frame built of stainless steel; the auxiliary equipment module includes a fan, a scattering lamp and gauze. The fan is used to remove the water mist and smoke generated during the database construction process. The scattering lamp serves as a light source to eliminate data fluctuations caused by different light. The gauze serves as an isolation to prevent external light sources from entering and causing image distortion. The cooking module is set at the bottom of the basic skeleton, the image acquisition module is fixed to the upper part of the basic skeleton with screws, and the gauze is fixed to the five surfaces of the basic skeleton except the bottom surface with clips, and a fan exhaust port is left on the side.

5. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 4 is characterized in that: The construction of the dish maturity database is as follows: Using the constructed image acquisition system, according to the determined processing technology of dishes of different maturity, the cooking and dish image acquisition of dishes of different maturity are completed, the images of dishes of different maturity are stored and the random seeds are used to complete the division of training set and test set.

6. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 1 is characterized in that: Quantitative detection of physical and chemical properties of vegetables at different maturity levels includes methods based on target detection tasks using transfer learning models and methods based on instance segmentation tasks using the Yolo_doneness model; In the method of target detection task based on transfer learning model, a probability density function is introduced, and the inverse of the difference in the numerical value of a certain physical and chemical property between two similar maturity levels is used as the probability density. The confidence corresponding to the three labels with the highest confidence levels are obtained according to the target detection model, and the comprehensive maturity label of the measured dish is obtained by weighted calculation with the label as the weight, and the corresponding physical and chemical index range is selected. Based on the probability density range and the corresponding physical and chemical property values ​​in the dish maturity database, the physical and chemical property value of the measured dish is quantitatively analyzed.

7. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 6 is characterized in that: Based on the method of instance segmentation task of Yolo_doneness model, the weighted confidence of dishes of the same maturity is calculated by taking the instance segmentation mask area of ​​dishes of the same type with the same maturity as the weight. Then, the comprehensive maturity label of the same dish is calculated with the help of maturity label using the instance segmentation mask area of ​​dishes of the same type with different maturity as the weight. Finally, the quantitative value of a certain physical and chemical indicator of the dish is calculated based on this label using the method of target detection task based on transfer learning model.

8. A qualitative and quantitative detection system for food maturity based on image deep learning, characterized by: The system adopts the method according to any one of claims 1 to 7, and the system comprises: a dish processing technology determination module, an image acquisition system, a dish maturity database construction module, a maturity judgment system construction module, and a dish physical and chemical property quantitative detection module; The dish processing technology determination module is used to measure the physical and chemical properties of dishes of different maturity levels, analyze the data, and determine the processing technology of dishes of different maturity levels; Image acquisition system, used to collect images of dishes at different levels of maturity; The dish maturity database construction module uses the images collected by the image acquisition system to build a dish maturity database; The maturity judgment system construction module uses the dish maturity database to train the image deep learning model to achieve target detection and instance segmentation of dishes of different maturity levels, and build maturity judgment systems respectively; The quantitative detection module of the physical and chemical properties of dishes conducts quantitative detection of the physical and chemical properties of dishes of different maturity levels based on the probability density function and the maturity prediction results of the dishes output by the maturity judgment system.

Images