Vegetable maturity qualitative and quantitative detection method and system based on image deep learning

Abstract

The invention discloses a dish maturity qualitative and quantitative detection method and system based on image deep learning, and the method comprises the steps: S1, carrying out the physicochemical property index data measurement of dishes with different maturity, carrying out the analysis of the data, and determining the processing technology of the dishes with different maturity; s2, constructing an image acquisition system and a dish maturity database based on cooking environment conditions and image construction requirements; s3, training an image deep learning model by utilizing the dish maturity database, realizing target detection and instance segmentation of dishes with different maturity, and respectively constructing maturity judgment systems; and S4, based on the probability density function and a dish maturity prediction result output by the maturity judgment system, performing quantitative detection on physicochemical properties of dishes with different maturity. Based on the self-learning and self-thinking ability of the deep learning model, the deep learning model is applied to intelligent determination of the maturity of the dishes, and the problems that no relevant standard exists at present and maturity determination depends on experience are effectively solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of food detection, and specifically relates to a qualitative and quantitative detection method and system for the maturity of dishes based on image deep learning. Background Art

[0002] Overcooking may reduce the nutritional value of dishes and produce toxic and harmful substances. For example, if the frying time is too long, the fat will be oxidized by heat and produce peroxides that are harmful to the body. Therefore, it is an important issue in the food industry to reasonably judge the maturity of dishes during cooking.

[0003] Although traditional maturity detection methods can determine the maturity of dishes, they have problems such as causing damage to samples, expensive instruments and equipment, and high operating thresholds.

[0004] Chinese patent CN202211454574.7 discloses "a control method and an automatic cooking machine", which sets the cooking process parameters according to the type and weight of the ingredients before cooking to determine whether the dishes are cooked until mature. The patent selects the cooking process based on the state of the ingredients before cooking to control the maturity of the ingredients, and cannot be perceived in real time, which has great limitations. Chinese patent CN202410586342.X discloses "an intelligent cooking method and related device based on the perception of the maturity of ingredients", which uses multimodal equipment to obtain a variety of information, and compares the obtained features with the existing database after linear fitting with simple functions to obtain the freshness and maturity of the cooking ingredients. However, the model in the patent relies on the existing database, lacks independent thinking ability, and requires a huge database support. The database is prepared based on online recipes and has not been scientifically analyzed; in addition, multimodal detection requires expensive equipment and complex operations, which limits its convenience and scalability. Summary of the invention

[0005] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to propose a method and system for qualitative and quantitative detection of food maturity based on image deep learning.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The qualitative and quantitative detection method of food maturity based on image deep learning includes the following steps:

[0008] S1. Determine the physical and chemical property index data of dishes with different maturity, analyze the data, and determine the processing technology of dishes with different maturity;

[0009] S2. Build an image acquisition system and a food maturity database based on cooking environment conditions and image construction requirements;

[0010] S3. Use the dish maturity database to train the image deep learning model to achieve target detection and instance segmentation of dishes of different maturity levels, and build maturity judgment systems respectively;

[0011] S4. Based on the probability density function and the maturity prediction results of the dishes output by the maturity judgment system, quantitative detection of the physical and chemical properties of dishes of different maturity is carried out.

[0012] The present invention also includes a qualitative and quantitative detection system for maturity of dishes based on image deep learning. The system adopts the qualitative and quantitative detection method for maturity of dishes provided by the present invention. The system includes: a dish processing technology determination module, an image acquisition system, a dish maturity database construction module, a maturity judgment system construction module, and a quantitative detection module for physical and chemical properties of dishes;

[0013] The dish processing technology determination module is used to measure the physical and chemical property index data of dishes of different maturity, analyze the data, and determine the processing technology of dishes of different maturity;

[0014] Image acquisition system, used to collect images of dishes at different degrees of maturity;

[0015] The dish maturity database construction module uses the images collected by the image acquisition system to build a dish maturity database;

[0016] The maturity judgment system construction module uses the dish maturity database to train the image deep learning model, realizes the target detection and instance segmentation of dishes of different maturity, and constructs the maturity judgment system respectively;

[0017] The quantitative detection module of the physical and chemical properties of dishes conducts quantitative detection of the physical and chemical properties of dishes of different maturity levels based on the probability density function and the prediction results of the maturity of dishes output by the maturity judgment system.

[0018] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0019] 1. Existing non-deep learning maturity recognition models do not have migration capabilities, and the application of deep learning models is relatively simple, and most of them stay at the direct replacement of the model database, resulting in many models being screened out without fully exerting their capabilities. The present invention introduces transfer learning in target detection, retains some parameters and feature extraction capabilities of excellent models, and compares various models on this basis, greatly improving the model screening efficiency and the final application effect. In instance segmentation, the characteristics of the cooking process of dishes in traditional cooking scenarios are reasonably analyzed, and the Yolov8 model is improved based on this characteristic to construct the Yolo_doneness model, which innovates the deep learning application method.

[0020] 2. Existing image-based deep learning intelligent discrimination applications are limited to qualitative analysis of black box properties from input to output. The present invention develops a quantitative detection method for the physical and chemical properties of food maturity based on probability density functions. It uses the physical and chemical properties measured when the maturity standards are established in advance to achieve non-destructive and rapid detection of physical and chemical properties, promote the development of nutritional analysis calculations and special recipe recommendations, food quality analysis and other functions, help scientific researchers to conduct rough and rapid analysis, enrich the functions of corresponding software and equipment, and enhance consumers' sense of gain.

[0021] 3. Existing cooking maturity recognition programs and methods are mostly concentrated in large high-end equipment such as smart microwave ovens and smart ovens. They are expensive and not universal. The present invention can be obtained and used by various users by constructing a smart phone application, and has a broad application market and development space. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of the method of the present invention;

[0023] Figure 2 It is a schematic diagram of the physical and chemical property index data of the dishes measured in the embodiment;

[0024] Figure 3 is the correlation analysis result of the physicochemical properties of steaks at different maturity levels in the embodiment;

[0025] Figure 4 Schematic diagram of the number and grouping of steak maturity database images in the embodiment;

[0026] Figure 5 is a comparison chart of the accuracy and loss value of each transfer learning model in the steak maturity recognition training process in the embodiment;

[0027] Figure 6 is the confusion matrix of the performance results of GoogLeNet and DenseNet121 on the steak maturity test set in the embodiment;

[0028] Figure 7 This is a .yaml file diagram of the Yolov8 model main architecture improvement module and the self-added module fusion;

[0029] Figure 8 It is a schematic diagram of the quantitative detection of the physical and chemical properties of vegetables of different maturity levels. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0031] like Figure 1 As shown, the qualitative and quantitative detection method of food maturity based on image deep learning includes the following steps:

[0032] S1. Determine the physical and chemical property index data of dishes with different maturity, analyze the data, and determine the processing technology of dishes with different maturity;

[0033] Among them, the physical and chemical properties of dishes of different maturity were measured, including cooking loss, myofibrillar protein content, total thiol content, myoglobin content, malondialdehyde content, texture, color difference, water phase distribution, electron microscope observation images, volatile components of dish aroma and soluble components of dish juice after cooking.

[0034] The data analysis specifically includes:

[0035] The measured data of the physical and chemical properties of dishes with different maturity were analyzed based on statistical analysis software such as SPSS, including: using the correlation analysis method to obtain the changes of the same trend indicators, determining the consistency of the macroscopic and microscopic changes in the properties of the dishes, increasing the interpretability of the changes in the physical and chemical properties of the dishes during the cooking process, and preliminarily determining which physical and chemical properties have the same effectiveness in the maturity judgment task; further using the principal component analysis method to determine the main influencing factors of the maturity of the dishes to reduce the amount of data to be processed in the process of determining the maturity of the dishes; finally, using the cluster analysis method to classify the physical and chemical properties of the dishes corresponding to the set cooking time gradient, and determining the processing technology of dishes with different maturity according to the cooking time corresponding to the classified categories (the processing technology includes a series of processes involving the cooking process of traditional dishes, such as cooking conditions, sample preparation, and cooking time).

[0036] S2. Building an image acquisition system and a food maturity database based on cooking environment conditions and image construction requirements; wherein the image acquisition system includes a cooking module, an image acquisition module, a basic framework, and an auxiliary equipment module;

[0037] Among them, the cooking module includes an induction cooker and other necessary cooking tools; the image acquisition module includes an industrial camera and a manipulator for fixing and adjusting the camera position; the basic skeleton is a square frame built of stainless steel; the auxiliary equipment module includes a fan, a scattering lamp and gauze. The fan is used to remove the water mist and smoke generated during the database construction process. The scattering lamp is used as a light source to eliminate database fluctuations caused by different light. The gauze is used as an isolation to prevent external light sources from entering and causing image distortion. The cooking module is set at the bottom of the basic skeleton, the image acquisition module is fixed to the upper part of the basic skeleton by screws, and the gauze is fixed to the five surfaces of the basic skeleton except the bottom surface by clips, and a fan exhaust port is left on the side.

[0038] The construction of the dish maturity database is as follows:

[0039] By using the constructed image acquisition system, according to the determined processing technology of dishes with different maturity, the cooking of dishes with different maturity and the acquisition of dish images are completed, the images of dishes with different maturity are stored and the division of training set and test set is completed by using random seeds.

[0040] S3. Use the dish maturity database to train the image deep learning model to achieve target detection and instance segmentation of dishes of different maturity levels, and build maturity judgment systems respectively;

[0041] Among them, the target detection task is achieved through transfer learning. Based on Pytorch, several existing image deep learning models are selected and trained using the constructed food maturity database. The model with the fastest convergence, the highest accuracy and the most stable model operation is selected as the best target detection model;

[0042] The instance segmentation task is implemented based on the Yolov8 model, which is adaptively improved according to the characteristics of the cooking process, and the Yolo_doneness model with better maturity recognition performance is constructed for instance segmentation;

[0043] The Yolov8 model is adaptively improved according to the characteristics of the cooking process, specifically including:

[0044] ContextGuide convolution is used to replace the C2f layer of the feature extraction part of the original Yolov8 model, Deformable convolution replaces the traditional convolution of the original Yolov8 model, TripletAttention mechanism is used to improve the C2f layer of the detection head of the original Yolov8 model, SA_attention detection head replaces the detection head of the original Yolov8 model, and the AOD-PONO-Net end-to-end dehazing detection module is deployed as a pre-processing means before the data is input into Yolo_doneness. The combination of ASF and SDI and the SEAM attention module are deployed in the data fusion part before the detection head of the original Yolov8 model. Finally, the constructed models are integrated to construct a Yolo_doneness model with specific applicability to the maturity of dishes.

[0045] Based on the optimal target detection model and Yolo_doneness model, maturity judgment systems are constructed respectively.

[0046] S4. Based on the probability density function and the prediction results of the maturity of the dishes output by the maturity judgment system, quantitative detection of the physical and chemical properties of dishes of different maturity is performed;

[0047] Quantitative detection of the physical and chemical properties of dishes of different maturity levels includes methods for target detection tasks of transfer learning models and methods for instance segmentation tasks of Yolo_doneness models;

[0048] Among them, in the method for transfer learning model target detection task, a probability density function is introduced, and the inverse of the numerical difference of a certain physical and chemical property between two similar maturity levels is used as the probability density. The confidence levels corresponding to the three labels with the highest confidence levels are obtained according to the target detection model, and the labels are used as weights for weighted calculation to obtain the comprehensive maturity labels of the measured dishes and select the corresponding physical and chemical index intervals. Based on the probability density interval and the corresponding physical and chemical property values ​​in the dish maturity database, the physical and chemical property values ​​of the measured dishes are quantitatively analyzed.

[0049] Among them, the method used for the instance segmentation task of the Yolo_doneness model first uses the instance segmentation mask area of ​​the same maturity of the same type of dishes as the weight to calculate the weighted confidence of the dishes with the same maturity. Then, the instance segmentation mask area of ​​different maturity of the same type of dishes is used as the weight. With the help of maturity labels, the comprehensive maturity label of the same dish is calculated. Finally, based on this label, the quantitative value of a physical and chemical indicator of the dish is calculated using the method for the target detection task of the transfer learning model.

[0050] Example

[0051] This embodiment takes the qualitative and quantitative detection of steak maturity as an example, and includes the following steps:

[0052] S1. Determine the physical and chemical property index data of steaks of different maturity, analyze the data, and obtain the processing technology of steak samples of different maturity; in this embodiment, Figure 2 As shown, including:

[0053] Cut the steak into 6cm×4cm×1.5cm blocks, set the induction cooker power to 80W, add a proper amount of olive oil to the pan, use an infrared thermometer and put the prepared steak into the pan when the temperature of the pan bottom reaches 105℃, and prepare steak samples of different degrees of maturity with a frying time of 2-12min.

[0054] The steak samples were divided into small cubes with a side length of 1.5 cm along the direction of the muscle fibers. The samples were pressed down perpendicular to the muscle fibers using a P36 probe using a meat texture analyzer to determine the texture information. The pre-test speed was 1.5 mm / s, the test and post-test speeds were 2 mm / s, and the test distance was 30 mm. The hardness, elasticity, chewiness, and stickiness of each sample were calculated as reference indicators of maturity. Three parallel tests were performed under each experimental condition to obtain the average value.

[0055] Use a colorimeter to measure the color of the cross section of the steak sample. Cut the steak sample along the cross section, and use a colorimeter to measure the red-green, yellow-blue and brightness of the center of the steak and calculate them as a * , b * , L *Three random cross sections were taken from each sample at each maturity for testing, and the average of the three test results was taken.

[0056] The test was conducted using a low-field NMR instrument and a 40mm probe. After the prepared steak sample was cooled, the middle part was cut into small cubes with a side length of 1.5cm and wrapped with plastic wrap for later use. The sample was then placed in a 40mm NMR tube and the NMR tube was placed in. The CPMG sequence was used to detect the T2 relaxation time and signal intensity of the steak to obtain the water phase distribution law of steaks of different maturity.

[0057] After removing the fascia and fat, the steaks of different maturity were broken, 3.0g of meat samples were placed in a centrifuge tube, 25mL of the previously prepared 0.01mol / L sodium phosphate buffer solution (potassium hydrogen phosphate: potassium hydrogen phosphate = 1:1 (molar ratio)) was added and homogenized at 8000rpm for 2min, and then placed in a refrigerator at 4℃ for 1h. After taking out, centrifuge in a refrigerated centrifuge at 4℃ and 7800rpm for 15min. After filtering the supernatant with a PES filter membrane with a pore size of 0.45μm, the absorbance of each sample at 525, 545, 565, and 572nm was measured using a spectrophotometer, and the three myoglobin contents were calculated as follows:

[0058]

[0059] Among them, A 525 is the absorbance of the sample at 525 nm, A 545 is the absorbance of the sample at 545 nm, A 565 is the absorbance of the sample at 565 nm, A 572 is the absorbance of the sample at 572 nm.

[0060] This embodiment designs a series of physical and chemical properties and microscopic index tests from both macroscopic and microscopic aspects. The specific measurement data are shown in Table 1 below. According to Table 1, the overall physical and chemical property index correlation is relatively significant, and there is also consistency between the macroscopic and microscopic aspects. The overall curve is relatively flat at 2-4 minutes, increases significantly after 6 minutes, and tends to be flat after 8 minutes. From this, it can be judged that the steak cooking process gradually changes significantly at 4 minutes, and the microscopic structure of the steak changes most rapidly and obviously at 6 minutes. Therefore, on this basis, the degree of doneness is graded into one-mature, three-mature, five-mature, and seven-mature according to cooking for 2 minutes, 4 minutes, 6 minutes, and 8 minutes. A longer heating interval of 12 minutes is fully cooked. This embodiment studies the relationship between cooking time and steak maturity on the basis of controlling other variables, but the quantitative detection method of the present invention is suitable for the study of various indicators of dish cooking and processing technology.

[0061]

[0062] Table 1

[0063] The measured physical and chemical property index data were input into SPSS27 statistical analysis software for processing. The original parameters of different physical and chemical indexes were input into the software and the cooking time was used as their label. The bivariate correlation in the analysis function of the software was used to calculate the mean and variance. The Pearson correlation coefficient was used for two-tailed test, and the correlation was verified at the confidence interval of 0.05 and 0.01 respectively. Figure 3 The following are the correlation analysis results of the physicochemical properties of steaks at different maturity levels. Figure 3 It can be seen that the hemoglobin content is related to the red and blue intensity (a * ) is significantly correlated, brightness (L * ) was significantly correlated with metmyoglobin and deoxymyoglobin. The chromaticity and protein content achieved an organic unity to a certain extent. The macroscopic and microscopic results were consistent, which further proved that the unevenness of steak maturity during the frying process and the mutation at 4min and 6min were objectively present.

[0064] On this basis, in order to obtain a more standard steak maturity sample and reduce the complexity of the judgment to improve efficiency, more representative physical and chemical indicators were selected to establish an objective maturity judgment method. Figure 3 It can be seen that the content of deoxy and metmyoglobin is related to L * The values ​​were significantly correlated; the content of oxymyoglobin and a * The values ​​of cooking loss and L are significantly correlated; the texture parameters are all significantly correlated; myofibrillar protein has the highest correlation with cooking loss. * In order to conduct more accurate or in-depth research on the steak maturity judgment model, the value, oxymyoglobin content and hardness are used as representative indicators.

[0065] S2. Building an image acquisition system and steak maturity database based on cooking environment conditions and image construction requirements; in this embodiment, specifically:

[0066] In order to control the same height for each sampling and photographing, and to effectively isolate the ambient light and image noise, a simple image acquisition system was built, including an induction cooker, an iron pot, a shooting table, an industrial camera, and a manipulator for fixing and adjusting the camera position. The outside of the system was wrapped with gauze, the shooting table was supported by a tripod, and an operating window was set up with white cloth on the outside, which was connected to a pipe with an exhaust fan to extract the fumes.

[0067] During the image acquisition process, we randomly selected angles between different steaks, and obtained more than 1,700 steak maturity images. Then, we used random seeds to divide the training set and test set. The number of data for the five different maturity levels is similar, and the number of images in the training set and test set for each category is as follows: Figure 4As shown in the figure, a balanced amount of data can improve the robustness and balance of the model and prevent the occurrence of simplified analysis.

[0068] S3. Use the constructed steak maturity database to train the image deep learning model to achieve target detection and instance segmentation of steaks of different maturity levels, and build maturity judgment systems respectively;

[0069] Among them, the target detection task is implemented through transfer learning. Based on Pytorch, several existing image deep learning models are selected, and the constructed dish maturity database is used for training. The model with the fastest convergence, the highest accuracy and the most stable model operation is selected as the optimal target detection model; in this embodiment, specifically:

[0070] Models such as AlexNet, VGG19, and ResNet34 were selected for training in the steak maturity database. During the training process, the feature extraction part of the model was frozen, and only the fully connected layer of the model was trained.

[0071] During the training process, batch=32, lr=0.003, epoch=30. The best model is the maximum accuracy of each batch in the same training round, and the relevant weights are saved. The training results of each model are as follows Figure 5 As shown in the figure, AlexNet and VGG19 models cannot converge effectively, SqueezeNet can only reach 80% accuracy, and ResNet34 can increase the accuracy to more than 98% within 30 rounds. However, the remaining GoogLeNet and DenseNet121 have good fitting effects, and both have reached nearly 100% accuracy and stability in the seventh training. Figure 6 As shown in the figure, it is the confusion matrix of the performance results of GoogLeNet and DenseNet121 models on the steak maturity test set. The first accuracy of the DenseNet121 model for sample prediction is significantly higher than that of GoogLeNet. The heat map on the main diagonal is darker in color and has better prediction performance. Therefore, DenseNet121 is used as the best steak maturity judgment model.

[0072] The instance segmentation task is implemented based on the Yolov8 model, which is adaptively improved according to the characteristics of the cooking process, and a Yolo_doneness model with better maturity recognition performance is constructed for instance segmentation; in this embodiment, specifically:

[0073] Based on the characteristics of overlapping, small sample sizes, susceptibility to interference, and small size of samples in the cooking process, the Yolo model for maturity recognition was constructed based on the adaptive transformation of the Yolov8 model. The ContextGuide convolution is introduced to replace the C2f layer of the feature extraction part of the original Yolov8 model to improve the connection and fusion capabilities of the model context information; the Deformable module is introduced to replace the traditional convolution in the original Yolov8 model to improve the adaptability of the model geometric transformation, so that the model can adapt to the different sizes, angles, and shapes of the same dishes due to different segmentation methods in the cooking of dishes; in order to make the model adapt to the details of the color and texture of the dishes with different maturity changes, the TripletAttention module is introduced to improve the C2f information extraction module in the Neck of the model to reduce its attention to the length and width dimensions and improve its attention to the channel dimension; the ASF+SDI architecture is introduced to improve the Neck architecture of the original model, improve the model calculation speed and simplify the model, reduce the model calculation floating point number while enhancing the model feature extraction transfer and fusion capabilities, and give the model the possibility of lightweight integration and convenient calculation without affecting the model recognition ability; in order to make the model adapt to the situation where the ingredients of the dishes are dense and overlapped, the SA_attention detection head is introduced, and the self-attention mechanism module and the convolution module are reasonably configured to achieve efficient extraction of dish features with as few parameters as possible, and reduce the occurrence of disconnection and neglect of segmented samples. Finally, the above modules are reasonably arranged and stitched together to build the Yolo steak maturity instance segmentation model.

[0074] On this basis, the AOD-PONO-Net end-to-end defogging detection module is introduced to solve the problem of blurred images caused by liquefaction of water vapor on the camera lens during the cooking process. With the help of the SEAM attention module in the Yolo-Face model, a weighted loss compensation module is added to correct the instance segmentation results to solve the problem of stacking and occlusion of ingredients during the cooking process. Finally, the modules are fused based on the Yolov8 environment, and finally the actual segmentation model Yolo_doneness suitable for judging the maturity of steak is developed. Figure 7 The figure shows a .yaml file diagram of the Yolov8 model main architecture improvement module and the self-added module fusion.

[0075] Based on the best target detection model DenseNet121 and Yolo_doneness model, steak maturity judgment systems were constructed respectively.

[0076] S4. Based on the probability density function and the steak maturity prediction results output by the maturity judgment system, quantitative detection of the physical and chemical properties of steaks of different maturity levels is performed.

[0077] like Figure 8As shown in the figure, the quantitative detection of the physical and chemical properties of vegetables of different maturity levels includes a method based on the target detection task of the transfer learning model and a method based on the instance segmentation task of the Yolo_doneness model.

[0078] Among them, in the method of target detection task based on transfer learning model, a probability density function is introduced, and the inverse of the numerical difference of a certain physical and chemical property between two similar maturity levels is taken as the probability density. The confidence corresponding to the three labels with the highest confidence levels are obtained according to the target detection model, and the comprehensive maturity label of the measured dish is obtained by weighted calculation with the label as the weight, and the corresponding physical and chemical index interval is selected. Based on the probability density interval and the corresponding physical and chemical property values ​​in the dish maturity database, the physical and chemical property value of the measured dish is quantitatively analyzed.

[0079] Among them, the method based on the instance segmentation task of the Yolo_doneness model first uses the instance segmentation mask area of ​​the same maturity of the same type of dishes as the weight to calculate the weighted confidence of the dishes with the same maturity. Then, the instance segmentation mask area of ​​different maturity of the same type of dishes is used as the weight. With the help of maturity labels, the comprehensive maturity label of the same dish is calculated. Finally, according to this label, the quantitative value of a certain physical and chemical index of the dish is calculated using the method based on the transfer learning model target detection task.

[0080] In this embodiment, specifically:

[0081] Assume that the detection result of a certain steak target is that the steak image recognition result is 80.22% for well-done, 10.67% for medium-rare, and 4.80% for medium-rare, while the myofibrillar protein contents corresponding to the three degrees of maturity are 24.15 mg / g, 151.34 mg / g, and 36.50 mg / g, respectively.

[0082] In this embodiment, the labels for well done, medium done, and medium done are 5, 2, and 4 respectively; the measured doneness of the steak is obtained according to the following formula:

[0083]

[0084] Among them, A is the sample prediction maturity label, A i Output the maturity label of the measured sample for the model, B i The model outputs the corresponding confidence level of the maturity of the measured sample.

[0085] Based on this formula, the maturity label of the tested steak sample is calculated to be 4.4164, and the maturity range of the sample is judged to be between well-done and medium-rare. The probability density functions of the two are used, and then the myofibrillar protein content of the tested sample is calculated according to the probability density function to create the following formula:

[0086] m=(m1-m2)×(A-A2)+m2

[0087] Among them, m1 is the myofibrillar protein content with a larger maturity label, m2 is the myofibrillar protein content with a smaller maturity label, A is the sample predicted maturity label, and A2 is the smaller maturity label.

[0088] Based on this formula, the myofibrillar protein content of the tested steak is calculated to be 31.36 mg / g. If the probability density function is not used, the sample content is calculated based on the confidence level and the myofibrillar protein content corresponding to each maturity level to be 37.27 mg / g. This value fluctuates greatly due to the properties of the three-mature samples that are not related to the actual maturity of the tested sample. The method of this embodiment can effectively avoid the occurrence of such situations caused by such super large and super small values.

[0089] In the instance segmentation task, the quantitative analysis of the physical and chemical properties of vegetables with different maturity levels is improved on the methods related to the target detection task. The calculation is performed based on the output results of the instance segmentation model and the mask segmentation area. The specific calculation formula is as follows:

[0090]

[0091] Among them, C is the confidence level of the weighted samples of the same food material and the same maturity, C i is the confidence of the i-th mask area of ​​the maturity category, S i is the i-th mask area of ​​the maturity.

[0092] The confidence of the comprehensive label of the same ingredient is then weighted using the following formula:

[0093]

[0094] Among them, D is a comprehensive sample label of a certain food, C j S is the confidence of the sample of the same ingredient with the same maturity after weighting the jth maturity of the ingredient. j is the total mask area at the jth maturity, D j It is the i-th maturity label of the ingredient.

[0095] Finally, the calculated final sample confidence of a certain food ingredient is brought into the quantitative detection formula of physical and chemical properties applicable to the target detection task to realize the quantitative analysis of a physical and chemical property in the instance segmentation task:

[0096] M=(M1-M2)×(D-D2)+M2

[0097] Among them, M is the value of the physical and chemical properties that need to be measured, M1 is the value of the physical and chemical properties with a larger maturity label, M2 is the value of the physical and chemical properties with a smaller maturity label, D is the comprehensive sample label of a certain food ingredient, and D2 is a smaller maturity label.

[0098] In another embodiment, a qualitative and quantitative detection system for maturity of dishes based on image deep learning is also provided. The system adopts the qualitative and quantitative detection method for maturity of dishes described in the above embodiment. The system includes: a dish processing technology determination module, an image acquisition system, a dish maturity database construction module, a maturity judgment system construction module, and a quantitative detection module for physical and chemical properties of dishes;

[0099] The dish processing technology determination module is used to measure the physical and chemical property index data of dishes of different maturity, analyze the data, and determine the processing technology of dishes of different maturity;

[0100] Image acquisition system, used to collect images of dishes at different degrees of maturity;

[0101] The dish maturity database construction module uses the images collected by the image acquisition system to build a dish maturity database;

[0102] The maturity judgment system construction module uses the dish maturity database to train the image deep learning model, realizes the target detection and instance segmentation of dishes of different maturity, and constructs the maturity judgment system respectively;

[0103] The quantitative detection module of the physical and chemical properties of dishes conducts quantitative detection of the physical and chemical properties of dishes of different maturity levels based on the probability density function and the prediction results of the maturity of dishes output by the maturity judgment system.

[0104] It should also be noted that in this specification, terms such as "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0105] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A qualitative and quantitative detection method for food maturity based on image deep learning, characterized in that: The following steps are involved: S1. Determine the physical and chemical property index data of dishes with different maturity, analyze the data, and determine the processing technology of dishes with different maturity; S2. Build an image acquisition system and a food maturity database based on cooking environment conditions and image construction requirements; S3. Use the dish maturity database to train the image deep learning model to achieve target detection and instance segmentation of dishes of different maturity levels, and build maturity judgment systems respectively; S4. Based on the probability density function and the maturity prediction results of the dishes output by the maturity judgment system, quantitative detection of the physical and chemical properties of dishes of different maturity is carried out.

2. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 1 is characterized in that: In step S1, the physical and chemical property index data of dishes of different maturity are measured, and the measured physical and chemical property indexes include cooking loss, myofibrillar protein content, total thiol content, myoglobin content, malondialdehyde content, texture, color difference, water phase distribution, electron microscope observation image, volatile components of dish aroma and soluble components of dish juice after cooking.

3. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 1 is characterized in that: In step S1, the data is analyzed, including: The measured physical and chemical properties of vegetables of different maturity levels were analyzed based on statistical analysis software, including: The correlation analysis method was used to obtain the changes in indicators with the same trend, determine the consistency of macroscopic and microscopic changes in dish properties, increase the interpretability of changes in physical and chemical properties during the cooking process, and preliminarily determine which physical and chemical properties have the same effectiveness in the maturity judgment task; the principal component analysis method was further used to determine the main influencing factors of dish maturity in order to reduce the amount of data required to be processed in the process of determining the maturity of dishes; finally, the cluster analysis method was used to classify the physical and chemical properties of dishes corresponding to the set cooking time gradient, and determine the processing technology of dishes with different maturity levels according to the cooking time corresponding to the classified categories.

4. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 1 is characterized in that: In step S2, the image acquisition system includes a cooking module, an image acquisition module, a basic framework, and an auxiliary equipment module; Among them, the cooking module includes an induction cooker and other necessary cooking tools; the image acquisition module includes an industrial camera and a manipulator for fixing and adjusting the camera position; the basic skeleton is a square frame built of stainless steel; the auxiliary equipment module includes a fan, a scattering lamp and gauze. The fan is used to remove the water mist and smoke generated during the database construction process. The scattering lamp is used as a light source to eliminate data fluctuations caused by different light. The gauze is used as an isolation to prevent external light sources from entering and causing image distortion. The cooking module is set at the bottom of the basic skeleton, the image acquisition module is fixed to the upper part of the basic skeleton by screws, and the gauze is fixed to the five surfaces of the basic skeleton except the bottom surface by clips, and a fan exhaust port is left on the side.

5. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 4 is characterized in that: The construction of the dish maturity database is as follows: By using the constructed image acquisition system, according to the determined processing technology of dishes with different maturity, the cooking of dishes with different maturity and the acquisition of dish images are completed, the images of dishes with different maturity are stored and the division of training set and test set is completed by using random seeds.

6. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 1 is characterized in that: In step S3, the target detection task is implemented through transfer learning. Based on Pytorch, several existing image deep learning models are selected and trained using the constructed dish maturity database. The model with the fastest convergence, the highest accuracy and the most stable model operation is selected as the best target detection model. The instance segmentation task is implemented based on the Yolov8 model, which is adaptively improved according to the characteristics of the cooking process, and the Yolo_doneness model with better maturity recognition performance is constructed for instance segmentation; Based on the optimal target detection model and Yolo_doneness model, maturity judgment systems are constructed respectively.

7. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 6 is characterized in that: Based on the Yolov8 model, it is adaptively improved according to the characteristics of the cooking process, including: ContextGuide convolution is used to replace the C2f layer of the feature extraction part of the original Yolov8 model, Deformable convolution replaces the traditional convolution of the original Yolov8 model, TripletAttention mechanism is used to improve the C2f layer of the detection head of the original Yolov8 model, SA_attention detection head replaces the detection head of the original Yolov8 model, and the AOD-PONO-Net end-to-end dehazing detection module is deployed as a pre-processing method before the data is input into Yolo_doneness. The combination of ASF and SDI and the SEAM attention module are deployed in the data fusion part before the detection head of the original Yolov8 model. Finally, the constructed models are integrated to construct a Yolo_doneness model with specific applicability to the maturity of dishes.

8. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 7 is characterized in that: The quantitative detection of the physical and chemical properties of vegetables of different maturity levels includes the method of target detection task based on the transfer learning model and the method of instance segmentation task based on the Yolo_doneness model; In the method of target detection task based on transfer learning model, probability density function is introduced, and the inverse of the numerical difference of a certain physical and chemical property between two similar maturity levels is taken as probability density. The confidence corresponding to the three labels with the highest confidence levels are obtained according to the target detection model, and the comprehensive maturity label of the measured dish is obtained by weighted calculation with the label as weight, and the corresponding physical and chemical index interval is selected. Based on the probability density interval and the corresponding physical and chemical property values ​​in the dish maturity database, the physical and chemical property value of the measured dish is quantitatively analyzed.

9. The method for qualitative and quantitative detection of food maturity based on image deep learning according to claim 8, characterized in that: Based on the method of instance segmentation task of Yolo_doneness model, the weighted confidence of dishes with the same maturity is calculated by taking the instance segmentation mask area of ​​the same type of dishes with the same maturity as the weight. Then, the comprehensive maturity label of the same dish is calculated with the help of maturity label by taking the instance segmentation mask area of ​​different maturity of the same type of dishes as the weight. Finally, the quantitative value of a certain physical and chemical index of the dish is calculated according to this label using the method of target detection task based on transfer learning model.

10. A qualitative and quantitative detection system for food maturity based on image deep learning, characterized in that: The system adopts the method described in any one of claims 1 to 9, and the system comprises: a dish processing technology determination module, an image acquisition system, a dish maturity database construction module, a maturity judgment system construction module, and a dish physical and chemical property quantitative detection module; The dish processing technology determination module is used to measure the physical and chemical property index data of dishes of different maturity, analyze the data, and determine the processing technology of dishes of different maturity; Image acquisition system, used to collect images of dishes at different degrees of maturity; The dish maturity database construction module uses the images collected by the image acquisition system to build a dish maturity database; The maturity judgment system construction module uses the dish maturity database to train the image deep learning model, realizes the target detection and instance segmentation of dishes of different maturity, and constructs the maturity judgment system respectively; The quantitative detection module of the physical and chemical properties of dishes conducts quantitative detection of the physical and chemical properties of dishes of different maturity levels based on the probability density function and the prediction results of the maturity of dishes output by the maturity judgment system.

Images