Complex environment-oriented Chinese food dish detection and identification method

Abstract

The invention belongs to the field of computer vision, deep learning and intelligent catering management, and particularly relates to a Chinese food dish detection and identification method oriented to a complex environment. The invention provides a Chinese food dish detection and identification method oriented to a complex environment, which is based on an improved YOLOv8 architecture and is combined with an attention mechanism, so that the Chinese food dish detection and identification method has the capability of performing efficient detection and identification in a complex dining table environment, and can output the category and bounding box information of each dish in real time. The problems of multi-target Chinese style dish detection and image classification in a complex scene are solved, automatic calculation of the calorie value of the dish is realized, a comprehensive solution is provided for intelligent ordering and nutrition analysis, and the method has a wide application prospect.

Description

Technical Field

[0001] The present invention belongs to the fields of computer vision, deep learning and intelligent catering management, and specifically relates to a method for detecting and identifying Chinese food dishes in complex environments. Background Art

[0002] In modern catering services and health management, real-time and accurate identification and analysis of Chinese dishes are of great value, especially for efficient detection and classification of a variety of dishes under cluttered backgrounds and complex lighting conditions, which has broad application prospects for realizing smart ordering, automatic nutritional assessment and health management.

[0003] Current dish detection technology mainly relies on traditional computer vision methods, which often show problems of insufficient detection accuracy and poor adaptability in complex environments with changing lighting, background interference, and changing dish shapes. With the rapid development of deep learning technology, especially convolutional neural networks (CNNs), significant progress has been made in the field of target detection. These methods have significantly improved detection performance by leveraging their end-to-end detection strategies and powerful feature extraction capabilities. However, traditional CNNs are based on local convolution operations and cannot effectively capture global contextual information in images, resulting in certain limitations in multi-target detection and complex scene recognition. Summary of the invention

[0004] In view of the deficiencies of the prior art, the present invention aims to solve the problems of automatic detection, classification and calorie value calculation of multi-target Chinese dishes in complex scenes, and is applied to the fields of smart ordering, health management, nutritional analysis, etc. in the catering industry. The present invention provides a method for detecting and identifying Chinese dishes in complex environments, focusing on solving the problems of multi-target Chinese dish detection and image classification in complex scenes, and realizing the automatic calculation of the calorie value of dishes. This method improves the existing YOLOv8 target detection framework by integrating a deep convolutional neural network (CNN) with the Transformer attention mechanism, so that it has the ability to perform efficient detection and recognition in complex dining table environments.

[0005] The present invention mainly includes the following core modules: a backbone network (Backbone Network), which is used to extract preliminary features of the input image; a convolutional Transformer block (ConvTransformerBlock), which is used to extract higher-level contextual information from the preliminary features; a multi-scale attention module (MultiScaleAttention), which fuses features from different scales to enhance multi-target detection; an attention head (AttentionHead), which is used to generate category labels and parameter information of bounding boxes of detected targets.

[0006] In order to achieve the purpose of the present invention, the present invention adopts the following technical scheme:

[0007] A method for detecting and identifying Chinese food in a complex environment, the steps are as follows:

[0008] Step S1: Data acquisition and preprocessing:

[0009] S1-1. First, select common Chinese dishes and obtain images of Chinese dishes as the initial data set;

[0010] S1-2, then preprocessing the images in the initial data set to obtain preprocessed images, and the data set composed of the preprocessed images is recorded as data set 1;

[0011] S2: Build ResAttention-YOLO network model:

[0012] The object detection model based on the YOLOv8 framework uses the ResNet101 network pre-trained on the ISIA Food-500 dataset as the backbone network. The ResAttention-YOLO network model is constructed by introducing the Transformer module, multi-scale attention module and multi-head self-attention mechanism.

[0013] The pre-training process of the ResNet101 network pre-trained on the ISIA Food-500 dataset is as follows: based on the ResNet101 network, ISIA Food-500 is used for pre-training, and the pre-training parameters are set as follows: learning rate (Learning Rate) = 5e-5; batch size (Batch Size) = 32; number of training rounds (Epochs) = 4; weight decay (Weight Decay) = 1e-4; after the pre-training is completed, the convolutional layer in the backbone network is frozen; and the ResNet101 network pre-trained on the ISIA Food-500 dataset is obtained;

[0014] S2-1. Feature extraction: The pre-processed image in step 1 is subjected to feature extraction through the ResNet101 network pre-trained on the ISIA Food-500 dataset, and the pre-processed image is mapped into a multi-scale feature map, which is recorded as feature Figure 1 ;

[0015] S2-2, feature transformation: transform the features obtained in step S2-1 Figure 1 Input to the Transformer module for feature transformation, and obtain the feature map after Transformer transformation, which is recorded as feature Figure 2 ;

[0016] The Transformer module includes: a one-dimensional convolutional layer, a multi-head self-attention mechanism, a layer normalization layer and a feedforward network; the specific operation of the feature transformation is: using a one-dimensional convolutional layer to transform the multi-scale features output by ResNet101 Figure 1 Project it into a high-dimensional feature space, and then capture the global context information of the image projected into the high-dimensional feature space through a multi-head self-attention mechanism to obtain enhanced features Figure 2 ;

[0017] S2-3, MultiScaleAttention module fusion processing:

[0018] A multi-scale attention module is introduced in the feature fusion stage; the multi-scale attention module includes a self-attention layer, layer normalization, and a feed-forward network, which can fuse local and global information extracted from feature maps of different scales;

[0019] The feature of step S2-1 Figure 1 and the characteristics of step S2-2 Figure 2 Transform to the same dimension and use the multi-scale attention module to transform the features to the same dimension Figure 1 With features Figure 2 Fusion is performed to obtain the fused feature map, which is recorded as feature Figure 3 ;

[0020] S2-4, target detection and classification (AttentionHead module):

[0021] The multi-head attention mechanism is introduced to improve the original prediction head of YOLO. The improved detection head is named AttentionHead and is used for target detection and classification. The YOLO mentioned here refers to YOLO v8.

[0022] The fused features obtained in step S2-3 Figure 3 Input to AttentionHead for target detection and classification, which is responsible for identifying and classifying dish targets and obtaining parameter information of the bounding box;

[0023] Based on the operations of steps S2-1 to S2-4, by introducing the Transformer module, the multi-scale attention module and the multi-head self-attention mechanism, the final constructed model is recorded as the ResAttention-YOLO network model;

[0024] S3, model training: input data set 1 into the ResAttention-YOLO network model, update parameters, and save the trained convolutional neural network structure and parameters; the training parameters are: initial learning rate (Learning Rate) = 5e-5, which drops to 1e-5 after 4 rounds; batch size (Batch Size) = 64; number of training rounds (Epochs) = 8; weight decay (Weight Decay) = 1e-4; the training platform system is ubuntu18.0, and the GPU is NVIDIA A100;

[0025] S4. Sample test and output results:

[0026] Select a dish to be identified, and obtain a preprocessed image as a sample to be identified according to the data acquisition and preprocessing operations in step S1;

[0027] Then the sample to be identified is input into the ResAttention-YOLO network model to obtain the category label and bounding box parameters of the dish to be identified;

[0028] S5. Calorie calculation:

[0029] (1) Establishment of calorie database: Label different dishes in data set 1, establish category labels, and search for the calorie value of the corresponding dish to obtain the basic calorie value of the corresponding dish; calculate and summarize the basic calorie value of all dishes according to the corresponding labels to obtain a calorie database;

[0030] (2) The area of ​​the bounding box is obtained according to the dish category label detected by S4 and the parameters of the bounding box, and the basic calorie value corresponding to the detected dish category label is searched in the corresponding calorie database; then the basic calorie value is multiplied by the area of ​​the corresponding dish bounding box to obtain the calorie value of each dish in the dish to be identified; finally, the calorie values ​​of each dish are accumulated and the total calorie value of the dish to be identified is output.

[0031] Preferably, in step S1, the preprocessing method includes normalization processing (to reduce the impact of illumination changes), data enhancement processing (such as rotation, scaling, flipping, to enhance the robustness of the model to complex environments), and noise reduction processing (to reduce the impact of environmental noise).

[0032] Preferably, the feature in step S2-1 is Figure 1 The extraction formula is as follows:

[0033] Y1=Conv(W1,x)+ReLU(BN(Conv(W2,x)))

[0034] Where Conv is the convolution layer; BN is the batch normalization layer; ReLU is the activation function; W1 and W2 are the weights of the convolution kernel; x is the preprocessed image; Y1 is the feature Figure 1 ;

[0035] Preferably, the output feature in S2-2 is Figure 2 The formula is as follows:

[0036] Y2=LayerNorm(Y1+Attention(Q,K,V))

[0037]

[0038] Q,K,V=XWq,XWk,XWv

[0039] Among them, Q is the query matrix; K is the key matrix; V is the value matrix; dk is the dimension of the key matrix; Wq, Wk, Wv are the weight matrices obtained during the model training process; Y1 is the feature Figure 1 ; Y2 is the characteristic Figure 2 .

[0040] Preferably, the fusion processing steps in S2-3 are as follows:

[0041] (1) The features Figure 1 and Features Figure 2 The formula for transforming to the same dimension is:

[0042] z1=Conv2D(w1,Y1),z2=Conv2D(w2,Y2)

[0043] Where Y1 is the feature Figure 1 , Y2 is the characteristic Figure 2 ; w1, w2 are convolution kernel weights; z1, z2 are converted feature maps;

[0044] (2) Apply self-attention layer to realize information interaction between feature maps:

[0045]

[0046] where · represents the dot product, T represents transpose, is the scaling factor, and the softmax function is used to generate the weight distribution; z1 and z2 are the feature maps after the conversion in step (1);

[0047] (3) Fusion of the output of the self-attention layer with the original feature map:

[0048] z=LayerNorm(z1+Attention(z1,z2))

[0049] LayerNorm represents the layer normalization operation, and z is the feature Figure 3 .

[0050] The present invention also provides a computer device, which includes a storage module, a processing module and a display module. The processing module, the storage module and the display module are electrically connected to realize information transmission and control; the storage module stores program instructions that can be executed by the processing module, and the processing module calls the program instructions to execute the method as described above, and displays the final result through the display module.

[0051] Beneficial effects of the present invention:

[0052] The present invention is based on an improved YOLO architecture, combined with an attention mechanism, to output the category and bounding box information of each dish in real time, and further calculate the calorie value of the dish based on the detection results, providing intuitive nutritional information. The present invention takes into account the diversity and real-time requirements in complex environments, and ensures rapid response and efficient detection capabilities in practical applications by optimizing the model structure and algorithm flow. In addition, combined with the detection results, the present invention further realizes the automatic calculation function of the calorie value of the dish, providing a comprehensive solution for smart ordering and nutritional analysis.

[0053] The specific advantages are as follows:

[0054] High detection accuracy: The method of the present invention integrates the Transformer attention mechanism, which enables the model to capture long-distance dependencies in the image and improves the accuracy of detection and recognition in complex environments.

[0055] Strong multi-tasking capability: The present invention can not only detect dishes, but also classify dishes and calculate calorie values ​​at the same time, providing an integrated solution that effectively supports the simultaneous processing of multiple tasks.

[0056] Adapting to complex environments: The present invention enhances the model's ability to detect multi-target and multi-scale targets through the design of a multi-scale attention module, which is particularly suitable for complex detection tasks of Chinese food dishes.

[0057] Automatic calorie calculation: The present invention uses the detection results in combination with the calorie database to automatically calculate the calorie value of dishes, providing important data support for the user's dietary health management.

[0058] Automatic recognition: The system can automatically recognize images of Chinese food dishes without the need for users to manually enter information, thus improving ease of use.

[0059] Detailed evaluation: The system can generate detailed dish evaluation reports to help users fully understand the nutritional value, freshness, meat and vegetable combination, calories, vitamin and mineral content, and Chinese medicine therapeutic effects of the dishes, so as to make healthier dietary choices.

[0060] Personalized suggestions: The system combines the user's individual information and traditional Chinese medicine concepts to provide personalized dietary suggestions and health guidance to enhance the user's dietary experience and health management level.

[0061] Wide scope of application: The system is applicable to all kinds of Chinese food, whether it is home-cooked dishes or restaurant specialties, and can accurately identify and evaluate them, with broad application prospects and practical value.

[0062] Continuous learning: The system has the ability to continuously learn and dynamically update, and can continuously improve the accuracy of identifying and evaluating new dishes and new cooking methods, ensuring the cutting-edge nature and practicality of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is the image recognition flow chart.

[0064] Figure 2 Schematic diagram of image recognition, input image, output dish category label and bounding box parameters.

[0065] Figure 3 This is the overall system architecture diagram.

[0066] Figure 4 This is a case study of the dish dataset.

[0067] Figure 5 It is the display interface for operating the computer equipment background information management system. DETAILED DESCRIPTION

[0068] The present invention is explained in more detail by means of the following embodiments. The following embodiments are merely illustrative and the present invention is not limited to these embodiments.

[0069] It should be understood that the terms described in the present invention are only for describing specific embodiments and are not intended to limit the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art in the art to which the present invention relates.

[0070] Although the present invention only describes preferred methods and materials, any methods and materials similar or equivalent to those described herein may also be used in the practice or testing of the present invention. All documents mentioned in this specification are incorporated by reference to disclose and describe methods and / or materials related to the documents. The raw materials, reagents, etc. used in the present invention can be obtained through conventional channels such as commercially available unless otherwise specified. In the event of a conflict with any incorporated document, the content of this specification shall prevail.

[0071] YOLO, the full name of which is You Only Look Once: Unified, Real-Time Object Detection, is a real-time target detection algorithm.

[0072] like Figure 1 The figure shows the operation flow of the present invention. For details of the operation, please refer to Example 1.

[0073] Embodiment 1:

[0074] A method for detecting and identifying Chinese food in a complex environment, comprising the following steps:

[0075] S1: Data acquisition and preprocessing: We selected n = 100 common Chinese dishes and took 1000 images of each dish as the initial dataset. The images in the initial dataset cover different shooting angles, lighting conditions, background complexity, and the diversity of various dishes (such as Figure 4 The collected data is annotated, including the bounding box and category label of each dish.

[0076] Then the images in the initial data set are preprocessed: first, data enhancement is performed to increase the diversity and robustness of the data; the enhancement methods are random flipping, rotation, and color adjustment (brightness, contrast, saturation); then noise reduction is performed to remove environmental noise and interference factors to ensure that the model can focus on the dishes themselves; the images obtained after preprocessing form a data set again, recorded as data set 1.

[0077] Divide dataset 1 into training set, validation set, and test set to ensure that the category distribution of each dataset is consistent; the ratio of training set, validation set, and test set is 8:1:1.

[0078] S2: Constructing the ResAttention-YOLO network model: Based on the object detection model of the YOLOv8 framework, the ResNet101 network pre-trained on the ISIA Food-500 dataset is used as the backbone network. The ResAttention-YOLO network model is constructed by introducing the Transformer module, multi-scale attention module and multi-head self-attention mechanism;

[0079] The pre-training process of the ResNet101 network pre-trained on the ISIA Food-500 dataset is as follows: based on the ResNet101 network, ISIA Food-500 is used for pre-training, and the pre-training parameters are set as follows: learning rate (Learning Rate) = 5e-5; batch size (Batch Size) = 32; number of training rounds (Epochs) = 4; weight decay (Weight Decay) = 1e-4; after the pre-training is completed, the convolutional layer in the backbone network is frozen; and the ResNet101 network pre-trained on the ISIA Food-500 dataset is obtained;

[0080] S2-1: Feature extraction: The image preprocessed in step S1 is extracted using the ResNet101 network pre-trained on the ISIA Food-500 dataset, and the preprocessed image is mapped into a multi-scale feature map, denoted as feature Figure 1 , used for subsequent high-order feature transformation.

[0081] feature Figure 1 The extraction formula is as follows:

[0082] Y1=Conv(W1,x)+ReLU(BN(Conv(W2,x)))

[0083] Where Conv is the convolution layer; BN is the batch normalization layer; ReLU is the activation function; W1 and W2 are the weights of the convolution kernel; x is the preprocessed image; Y1 is the feature Figure 1 ;

[0084] S2-2: Add a Transformer module to the feature extraction part to extract the features obtained in step S2-1 Figure 1 Input to the Transformer module for feature transformation, and obtain the feature map after Transformer transformation, which is recorded as feature Figure 2 ;

[0085] The Transformer module includes one-dimensional convolution, multi-head self-attention mechanism, layer normalization and feedforward network.

[0086] The specific operation of feature transformation is: use a one-dimensional convolutional layer to transform the multi-scale features output by ResNet101 Figure 1 Project it into a high-dimensional feature space, and then capture the global context information of the image projected into the high-dimensional feature space through a multi-head self-attention mechanism to obtain enhanced features Figure 2 ;

[0087] Output characteristics Figure 2 The formula is as follows:

[0088] Y2=LayerNorm(Y1+Attention(Q,K,V))

[0089]

[0090] Q,K,V=XWq,XWk,XWv

[0091] Where Q is the query matrix; K is the key matrix; V is the value matrix; dk is the dimension of the key matrix

[0092] Wq, Wk, Wv are weight matrices obtained during model training; Y1 is the feature Figure 1 ; Y2 is the characteristic Figure 2 .

[0093] S2-3: In the feature fusion stage, a multi-scale attention mechanism is introduced; the multi-scale attention module includes a self-attention layer, layer normalization, and a feed-forward network, which can fuse local and global information extracted from feature maps of different scales; the feature map of step S2-1 is Figure 1 and the characteristics of step S2-2 Figure 2 Transform to the same dimension and use the multi-scale attention module to transform the features to the same dimension Figure 1 With features Figure 2 Fusion is performed to obtain the fused feature map, recorded as feature Figure 3 ;

[0094] The multi-scale attention module includes a self-attention layer, a layer normalization layer and a feed-forward network;

[0095] The fusion processing steps are as follows:

[0096] (1) The features Figure 1 and Features Figure 2 The formula for transforming to the same dimension is:

[0097] z1=Conv2D(w1,Y1),z2=Conv2D(w2,Y2)

[0098] Where Y1 is the feature Figure 1 , Y2 is the characteristic Figure 2 ; w1, w2 are convolution kernel weights; z1, z2 are converted feature maps;

[0099] (2) Apply self-attention layer to realize information interaction between feature maps:

[0100]

[0101] where · represents the dot product, T represents transpose, is the scaling factor, and the softmax function is used to generate the weight distribution;

[0102] (3) Fusion of the output of the self-attention layer with the original feature map:

[0103] z=LayerNorm(z1+Attention(z1,z2))

[0104] LayerNorm represents the layer normalization operation, and z is the feature Figure 3 .

[0105] This part combines multi-scale information and multi-head attention mechanism of neural network module, which is designed to process two input feature maps Y1 and Y2, which usually come from different scales or have different semantic information. Through the Transformer module, Y1 can be weighted based on the contextual information of Y2, thereby enhancing the feature representation of Y1.

[0106] By using 1x1 convolution for embedding dimension transformation and output recovery, the module maintains a relatively lightweight design, reducing computational complexity and the number of parameters. Since the inputs Y1 and Y2 can come from different network layers or have different scales, the module is highly flexible and can be easily integrated into various neural network architectures. By weighting Y1 through the attention mechanism, the module is able to enhance the feature representation of Y2 to include more contextual information, which is very helpful for improving the performance of subsequent tasks.

[0107] S2-4: After multi-scale feature fusion, the native detection head of the YOLOv8 model is improved by introducing the multi-head self-attention mechanism (Multi-Head Self-Attention). The improved detection head is named AttentionHead and is used for target detection and classification.

[0108] The fused features obtained in step S2-3 Figure 3 Input to the AttentionHead for target detection and classification component, which is responsible for identifying and classifying the dish target and obtaining the parameter information of the bounding box; Figure 2 Schematic diagram of image recognition, input image, output dish category label and bounding box parameters.

[0109] Based on the operations of steps S2-1 to S2-4, by introducing the Transformer module, the multi-scale attention module and the multi-head self-attention mechanism, the final constructed model is recorded as the ResAttention-YOLO network model;

[0110] The detection head passes the characteristic Figure 3Generate bounding boxes and category labels for each target. This part combines the Multi-Head Self-Attention mechanism to enhance feature representation, especially in capturing global context information. First, the input features Figure 3 The embedding dimension is transformed, and the multi-head attention mechanism is used to capture the long-distance dependencies in the feature map, thereby enhancing the feature expression of the image. Then, the original number of channels of the feature map is restored through the convolution layer and directly used for target detection. Through the multi-head self-attention mechanism, AttentionHead can effectively enhance the ability to capture global information in the feature map and improve detection accuracy. Finally, the module outputs the classification and regression task prediction results of each target through AttentionHead, and generates the target's bounding box, category label, and confidence. This method supports multi-task learning, enabling the model to solve detection and classification tasks simultaneously in a single task, thereby improving the generalization ability and practicality of the model.

[0111] S3: Model training: Input the dataset 1 into the ResAttention-YOLO network model, update the parameters, and save the trained convolutional neural network structure and parameters; the training parameters are: initial learning rate = 5e-5, which drops to 1e-5 after 4 rounds; batch size = 64; number of training rounds (Epochs) = 8; weight decay = 1e-4;

[0112] The training platform system is Ubuntu 18.0, the GPU is NVIDIA A100, and the Python version is 3.11.8

[0113] S4: Predict the test sample and output:

[0114] Select a dish to be identified, and obtain a preprocessed image as a sample to be identified according to the data acquisition and preprocessing operations in step S1;

[0115] Then the sample to be identified is input into the ResAttention-YOLO model to obtain the category label and bounding box parameters of the dish to be identified.

[0116] S5: Calorie Calculation:

[0117] (1) Establishment of calorie database: Label the different dishes in data set 1, establish category labels, and search for the calorie value of the corresponding dish to obtain the basic calorie value of the corresponding dish; calculate and summarize the basic calorie value of all dishes according to the corresponding labels to obtain a calorie database;

[0118] (2) Obtain the area of ​​the bounding box according to the dish category label detected by S4 and the parameters of the bounding box, and search the calorie database for the basic calorie value corresponding to the detected dish category label, and then multiply it by the area of ​​the dish bounding box to obtain the calorie value of each dish in the dish to be identified;

[0119] The calorie value of each dish is accumulated and the total calorie value of the dish to be identified is output for users to manage their diet and control their health.

[0120] Calorie calculation: Calories = BaseCalories(i) × Area

[0121] Among them, BaseCalories(i) is the basic calorie value of the dish numbered i in the database, and Area is the area of ​​the detected dish boundary box.

[0122] Figure 3 This is the overall architecture diagram of the system, showing the overall architecture of the Chinese food dish recognition and evaluation system based on convolutional neural network and natural language processing; it mainly consists of two parts: image recognition (left side) and dish nutritional value calculation (right side); image recognition is based on receiving Chinese food dish images uploaded by users, and identifying them through the convolutional neural network model to determine the type of dish; the recognition result is then transmitted to the natural language processing module, which uses the statistical dish nutritional value information to generate the calorie calculation result about the dish.

[0123] A computer device includes a storage module, a processing module and a display module. The processing module is electrically connected to the storage module and the display module to realize information transmission and control. The storage module stores program instructions that can be executed by the processing module. The processing module calls the program instructions to execute the above method and display the final result through the display module. Display interface Figure 5 As shown, the interface is divided into two main parts: the function navigation bar on the left and the dish information management area on the right; it is designed to efficiently manage various information of Chinese food dishes, including the classification, basic information, cost, status, etc. of the dishes.

[0124] Experiment and verification: Classification accuracy test;

[0125] Four commonly used models in the prior art are used to identify the types of samples and compare the classification results. The four models are YOLOv4, YOLOv5, SSD and YOLOv8 models.

[0126] mAP (mean average precision) and mIoU (the degree of overlap between the predicted bounding box and the true bounding box) are used as the main evaluation indicators.

[0127] The model's detection and recognition capabilities in complex environments are verified through the test set, and the results are analyzed to evaluate the robustness and accuracy of the model. An example of the experimental results is shown in Table 1;

[0128] Table 1: Test results

[0129]

[0130]

[0131] It can be seen from the detection results in Table 1 that the present invention significantly improves the detection and recognition performance by optimizing the model structure and parameter settings.

[0132] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A method for detecting and identifying Chinese food in a complex environment, characterized in that: The following steps are involved: Step S1: Data acquisition and preprocessing: S1-1. First, select common Chinese dishes and obtain images of Chinese dishes as the initial data set; S1-2, then preprocessing the images in the initial data set to obtain preprocessed images, and the data set composed of the preprocessed images is recorded as data set 1; S2: Constructing the ResAttention-YOLO network model: Based on the target detection model of the YOLOv8 framework, the ResNet101 network pre-trained on the ISIAFood-500 dataset is used as the backbone network. The ResAttention-YOLO network model is constructed by introducing the Transformer module, multi-scale attention module and multi-head self-attention mechanism; The pre-training process of the ResNet101 network pre-trained on the ISIA Food-500 dataset is as follows: based on the ResNet101 network, ISIA Food-500 is used for pre-training, and the pre-training parameters are set as follows: learning rate = 5e-5; batch size = 32; number of training rounds = 4; weight decay = 1e-4; after the pre-training is completed, the convolutional layers in the backbone network are frozen; and the ResNet101 network pre-trained on the ISIA Food-500 dataset is obtained; S2-1, feature extraction: extract features from the image preprocessed in step S1 using the ResNet101 network pre-trained on the ISIAFood-500 dataset, and map the preprocessed image into a multi-scale feature map, denoted as feature map 1; S2-2, feature transformation: Input the feature map 1 obtained in step S2-1 into the Transformer module for feature transformation to obtain the feature map after Transformer transformation, which is recorded as feature map 2; The Transformer module includes: a one-dimensional convolution layer, a multi-head self-attention mechanism, a layer normalization layer and a feedforward network; the specific operation of the feature transformation is: using a one-dimensional convolution layer to project the multi-scale feature map 1 output by ResNet101 into a high-dimensional feature space, and then using a multi-head self-attention mechanism to capture the global context information of the image projected into the high-dimensional feature space, to obtain an enhanced feature map 2; S2-3, fusion processing: A multi-scale attention module is introduced in the feature fusion stage; the multi-scale attention module includes a self-attention layer, layer normalization, and a feed-forward network, which can fuse local and global information extracted from feature maps of different scales; The feature map 1 of step S2-1 and the feature map 2 of step S2-2 are transformed to the same dimension, and the feature map 1 and the feature map 2 transformed to the same dimension are fused through a multi-scale attention module to obtain a fused feature map, which is recorded as feature map 3; S2-4 Target detection and classification: The multi-head attention mechanism is introduced to improve the original prediction head of YOLO. The improved detection head is named AttentionHead and is used for target detection and classification. The YOLO mentioned here refers to YOLOv8. The fused feature map 3 obtained in step S2-3 is input into the AttentionHead for target detection and classification, which is responsible for identifying and classifying the dish targets and obtaining the parameter information of the bounding box; Based on the operations of steps S2-1 to S2-4, by introducing the Transformer module, the multi-scale attention module and the multi-head self-attention mechanism, the final constructed model is recorded as the ResAttention-YOLO network model; S3, model training: input the data set 1 into the ResAttention-YOLO network model, update the parameters, and save the trained convolutional neural network structure and parameters; the training parameters are: initial learning rate = 5e-5, which drops to 1e-5 after 4 rounds; batch size = 64; number of training rounds = 8; weight decay = 1e-4; the training platform system is ubuntu18.0, and the GPU is NVIDIAA100; S4. Sample testing: Select a dish to be identified, and obtain a preprocessed image as a sample to be identified according to the data acquisition and preprocessing operations in step S1; Then the sample to be identified is input into the ResAttention-YOLO network model to obtain the category label and bounding box parameters of the dish to be identified; S5. Calorie calculation: (1) Establishment of calorie database: Label different dishes in data set 1, establish category labels, and search for the calorie value of the corresponding dish to obtain the basic calorie value of the corresponding dish; calculate and summarize the basic calorie value of all dishes according to the corresponding labels to obtain a calorie database; (2) The area of ​​the bounding box is obtained according to the dish category label detected by S4 and the parameters of the bounding box, and the basic calorie value corresponding to the detected dish category label is searched in the corresponding calorie database; then the basic calorie value is multiplied by the area of ​​the corresponding dish bounding box to obtain the calorie value of each dish in the dish to be identified; finally, the calorie values ​​of each dish are accumulated and the total calorie value of the dish to be identified is output.

2. According to claim 1, a method for detecting and identifying Chinese food in a complex environment is characterized in that: In step S1, the preprocessing method includes normalization processing, data enhancement processing, and noise reduction processing.

3. The method for detecting and identifying Chinese food in a complex environment according to claim 1, characterized in that: The extraction formula of the feature map 1 in step S2-1 is as follows: Y1=Conv(W1,x)+ReLU(BN(Conv(W2,x))) Where Conv is the convolution layer; BN is the batch normalization layer; ReLU is the activation function; W1 and W2 are the weights of the convolution kernel; x is the preprocessed image; Y1 is feature map 1.

4. The method for detecting and identifying Chinese food in a complex environment according to claim 1, characterized in that: The formula for outputting feature map 2 in S2-2 is as follows: Y2=LayerNorm(Y1+Attention(Q,K,V)) Q,K,V=XWq,XWk,XWv Where Q is the query matrix; K is the key matrix; V is the value matrix; dk is the dimension of the key matrix; Wq, Wk, and Wv are weight matrices obtained during model training; Y1 is feature map 1; and Y2 is feature map 2.

5. The method for detecting and identifying Chinese food in a complex environment according to claim 1, characterized in that: The fusion processing steps in S2-3 are as follows: (1) The formula for transforming feature map 1 and feature map 2 to the same dimension is: z1=Conv2D(w1,Y1),z2=Conv2D(w2,Y2) Where Y1 is feature map 1, Y2 is feature map 2; w1 and w2 are convolution kernel weights; z1 and z2 are converted feature maps; (2) Apply self-attention layer to realize information interaction between feature maps: where · represents the dot product, T represents transpose, is the scaling factor, and the softmax function is used to generate the weight distribution; z1 and z2 are the feature maps after the conversion in step (1); (3) Fusion of the output of the self-attention layer with the original feature map: z=LayerNorm(z1+Attention(z1,z2)) Where LayerNorm represents the layer normalization operation and z is the feature map 3.

6. A computer device, characterized in that: It includes a storage module, a processing module and a display module. The processing module is electrically connected to the storage module and the display module to realize information transmission and control. The storage module stores program instructions that can be executed by the processing module. The processing module calls the program instructions to execute any method described in claims 1-5, and displays the final result through the display module.

Images