Method for detection and identification of chinese dishes in complex environment

Abstract

The present invention belongs to the fields of computer vision, deep learning and intelligent catering management, and specifically relates to a method for detection and identification of Chinese dishes in a complex environment. The present invention provides the method for detection and identification of Chinese dishes in a complex environment. On the basis of an improved YOLOv8 architecture and in combination with an attention mechanism, the method is endowed with the capability of performing efficient detection and identification in complex dining table environments, and can output the category and bounding box information of each dish in real time, thus solving the problems of multi-target Chinese dish detection and image classification in complex scenarios; and the method achieves automatic calculation of the calorie values of dishes, and provides comprehensive solutions for smart dish ordering and nutrition analysis, and thus has a broad application prospect.

Description

A method for detecting and identifying Chinese dishes in complex environments Technical Field

[0001] This invention belongs to the fields of computer vision, deep learning and intelligent catering management, and specifically relates to a method for detecting and recognizing Chinese dishes in complex environments. Background Technology

[0002] In modern catering services and health management, real-time and accurate identification and analysis of Chinese dishes is of great value. In particular, the efficient detection and classification of various dishes under cluttered backgrounds and complex lighting conditions has broad application prospects for realizing intelligent ordering, automatic nutritional assessment and health management.

[0003] Current food detection technologies primarily rely on traditional computer vision methods. These methods often exhibit insufficient detection accuracy and poor adaptability in complex environments with varying lighting conditions, background interference, and diverse food shapes. With the rapid development of deep learning technology, especially the significant progress made by Convolutional Neural Networks (CNNs) in object detection, these methods have significantly improved detection performance by utilizing their end-to-end detection strategies and powerful feature extraction capabilities. However, traditional CNNs, based on local convolution operations, cannot effectively capture global contextual information in images, resulting in limitations in multi-object detection and complex scene recognition. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention aims to solve the problem of automated detection, classification, and calorie calculation of multiple Chinese dishes in complex scenarios, with applications in areas such as intelligent ordering, health management, and nutritional analysis within the catering industry. This invention provides a method for detecting and recognizing Chinese dishes in complex environments, focusing on solving the problem of multi-target Chinese dish detection and image classification in complex scenarios, and automatically calculating the calorie value of the dishes. This method improves the existing YOLOv8 object detection framework by integrating a deep convolutional neural network (CNN) and a Transformer attention mechanism, enabling it to perform efficient detection and recognition in complex dining environments.

[0005] This invention mainly includes the following core modules: a backbone network for extracting preliminary features from the input image; a ConvTransformer block for extracting higher-level contextual information from the preliminary features; a multi-scale attention module for fusing features from different scales to enhance multi-target detection; and an attention head for generating class labels and bounding box parameter information for the detected targets.

[0006] To achieve the objectives of this invention, the following technical solution is adopted:

[0007] A method for detecting and recognizing Chinese dishes in complex environments, comprising the following steps:

[0008] Step S1, Data Acquisition and Preprocessing:

[0009] S1-1 First, select common Chinese dishes and obtain images of Chinese dishes as the initial dataset;

[0010] S1-2. Then, the images in the initial dataset are preprocessed to obtain preprocessed images. The dataset composed of the preprocessed images is denoted as dataset 1.

[0011] S2: Construct the ResAttention-YOLO network model:

[0012] The object detection model based on the YOLOv8 framework uses a ResNet101 network pre-trained on the ISIAFood-500 dataset as the backbone network, and constructs the ResAttention-YOLO network model by introducing a Transformer module, a multi-scale attention module and a multi-head self-attention mechanism.

[0013] The ResNet101 network pre-trained on the ISIAFood-500 dataset was pre-trained as follows: Based on the ResNet101 network, it was pre-trained using ISIA Food-500. The pre-training parameters were set as follows: Learning Rate = 5e-5; Batch Size = 32; Epochs = 4; Weight Decay = 1e-4; After pre-training, the convolutional layers in the backbone network were frozen; thus, the ResNet101 network pre-trained on the ISIAFood-500 dataset was obtained.

[0014] S2-1, Feature Extraction: The image after preprocessing in step one is preprocessed by using a ResNet101 network pre-trained on the ISIAFood-500 dataset. The preprocessed image is then mapped to a multi-scale feature map, denoted as feature map 1.

[0015] S2-2, Feature Transformation: Input the feature map 1 obtained in step S2-1 into the Transformer module to perform feature transformation, and obtain the feature map after Transformer transformation, denoted as feature map 2;

[0016] The Transformer module includes: a one-dimensional convolutional layer, a multi-head self-attention mechanism, a layer normalization layer, and a feedforward network; the specific operation of feature transformation is as follows: the multi-scale feature map 1 output by ResNet101 is projected onto a high-dimensional feature space using a one-dimensional convolutional layer, and then the global context information of the image projected onto the high-dimensional feature space is captured through the multi-head self-attention mechanism to obtain the enhanced feature map 2.

[0017] S2-3, MultiScaleAttention module fusion processing:

[0018] A multi-scale attention module is introduced in the feature fusion stage; the multi-scale attention module includes a self-attention layer, layer normalization, and a feedforward network, which can fuse local and global information extracted from feature maps of different scales;

[0019] The feature map 1 in step S2-1 and the feature map 2 in step S2-2 are transformed to the same dimension, and the feature map 1 and feature map 2 transformed to the same dimension are fused through the multi-scale attention module to obtain the fused feature map, which is denoted as feature map 3.

[0020] S2-4, Object Detection and Classification (AttentionHead Module):

[0021] A multi-head attention mechanism is introduced to improve the native prediction head of YOLO. The improved detection head is named AttentionHead and is used for object detection and classification. YOLO refers to YOLO v8.

[0022] The fused feature map 3 obtained in steps S2-3 is input into AttentionHead for target detection and classification, which is responsible for identifying and classifying food targets and obtaining the parameter information of the bounding box;

[0023] Based on the operations in steps S2-1 to S2-4, by introducing the Transformer module, the multi-scale attention module, and the multi-head self-attention mechanism, the final model is denoted as the ResAttention-YOLO network model.

[0024] S3. Model Training: Input dataset 1 into the ResAttention-YOLO network model, update the parameters, and save the trained convolutional neural network structure and parameters. The training parameters are: initial learning rate = 5e-5, decreasing to 1e-5 after 4 epochs; batch size = 64; number of training epochs = 8; weight decay = 1e-4; the training platform system is Ubuntu 18.0, and the GPU is NVIDIA A100.

[0025] S4. Sample testing and output results:

[0026] Select the dish to be identified, and follow the data acquisition and preprocessing operation in step S1 to obtain the preprocessed image as the sample to be identified;

[0027] Then, the sample to be identified is input into the ResAttention-YOLO network model to obtain the category label and bounding box parameters of the dish to be identified;

[0028] S5. Calorie Calculation:

[0029] (1) Establishment of calorie database: Label the different dishes in dataset 1, establish category labels, and find the calorie value of the corresponding dishes to obtain the basic calorie value of the corresponding dishes; calculate and summarize the basic calorie value of all dishes according to the corresponding labels to obtain the calorie database;

[0030] (2) Based on the parameters of the dish category label and bounding box detected by S4, obtain the bounding box area, and find the basic calorie value corresponding to the detected dish category label in the calorie database; then multiply the basic calorie value by the area of ​​the corresponding dish bounding box to obtain the calorie value of each dish in the dish to be identified; finally, accumulate the calorie values ​​of each dish and output the total calorie value of the dish to be identified.

[0031] Preferably, in step S1, the preprocessing method includes normalization (to reduce the impact of illumination changes), data augmentation (such as rotation, scaling, and flipping to enhance the robustness of the model to complex environments), and noise reduction (to reduce the impact of environmental noise).

[0032] Preferably, the extraction formula for feature map 1 in step S2-1 is as follows: Y1=Conv(W1,x)+ReLU(BN(Conv(W2,x)))

[0033] Where Conv is the convolutional layer; BN is the batch normalization layer; ReLU is the activation function; W1 and W2 are the weights of the convolutional kernel; x is the preprocessed image; Y1 is feature map 1;

[0034] Preferably, the formula for outputting feature map 2 in S2-2 is as follows: Y2=LayerNorm(Y1+Attention(Q,K,V)) Q,K,V=XWq,XWk,XWv

[0035] Where Q is the query matrix; K is the key matrix; V is the value matrix; and dk is the dimension of the key matrix.

[0036] Wq, Wk, and Wv are weight matrices obtained during model training; Y1 is feature map 1; Y2 is feature map 2.

[0037] Preferably, the fusion processing steps in S2-3 are as follows:

[0038] (1) Formulas for transforming feature map 1 and feature map 2 to the same dimension: z1 = Conv2D(w1,Y1), z2 = Conv2D(w2,Y2)

[0039] Where Y1 is feature map 1, Y2 is feature map 2; w1 and w2 are convolution kernel weights; z1 and z2 are the transformed feature maps;

[0040] (2) Applying a self-attention layer to achieve information exchange between feature maps:

[0041] Where · represents the dot product, T Indicates transpose. is the scaling factor, and the softmax function is used to generate the weight distribution; z1 and z2 are the feature maps after transformation in step (1);

[0042] (3) Fuse the output of the self-attention layer with the original feature map: z = LayerNorm(z1 + Attention(z1, z2))

[0043] Where LayerNorm represents the layer normalization operation, and z is the feature map 3.

[0044] The present invention also provides a computer device, comprising a storage module, a processing module, and a display module, wherein the processing module is electrically connected to the storage module and the display module, and is capable of transmitting and controlling information; the storage module stores program instructions that can be executed by the processing module, and the processing module can execute the method described above by calling the program instructions, and display the final result through the display module.

[0045] The beneficial effects of this invention are:

[0046] This invention, based on an improved YOLO architecture and incorporating an attention mechanism, outputs the category and bounding box information of each dish in real time. Furthermore, it calculates the calorie value of each dish based on the detection results, providing intuitive nutritional information. This invention considers the diversity and real-time requirements of complex environments, ensuring rapid response and efficient detection capabilities in practical applications by optimizing the model structure and algorithm flow. In addition, based on the detection results, this invention further realizes the automatic calculation function of dish calorie values, providing a comprehensive solution for intelligent ordering and nutritional analysis.

[0047] The specific advantages are as follows:

[0048] High detection accuracy: By integrating the Transformer attention mechanism, the method of this invention enables the model to capture long-distance dependencies in images, thereby improving the accuracy of detection and recognition in complex environments.

[0049] Strong multitasking capability: This invention can not only detect dishes, but also classify dishes and calculate calorie values ​​at the same time, providing an integrated solution that effectively supports simultaneous multitasking.

[0050] Adaptable to complex environments: This invention enhances the model's ability to detect multiple targets and targets at multiple scales through the design of a multi-scale attention module, making it particularly suitable for complex detection tasks involving Chinese dishes.

[0051] Automated calorie calculation: This invention uses detection results combined with a calorie database to automatically calculate the calorie value of dishes, providing important data support for users' dietary health management.

[0052] Automatic recognition: The system can automatically recognize images of Chinese dishes, eliminating the need for users to manually input information and improving ease of use.

[0053] Detailed assessment: The system can generate detailed food evaluation reports to help users fully understand the nutritional value, freshness, meat and vegetable balance, calories, vitamin and mineral content, as well as the effects of traditional Chinese medicine, so as to make healthier food choices.

[0054] Personalized recommendations: The system combines users' individual information with traditional Chinese medicine concepts to provide personalized dietary advice and health guidance, improving users' dietary experience and health management level.

[0055] Wide range of applications: The system is applicable to all kinds of Chinese dishes, whether they are home-style dishes or restaurant specialties, and can accurately identify and evaluate them. It has broad application prospects and practical value.

[0056] Continuous learning: The system has the ability to continuously learn and dynamically update, which can continuously improve the accuracy of identifying and evaluating new dishes and cooking methods, ensuring the system's cutting-edge nature and practicality. Attached Figure Description

[0057] Figure 1 is a flowchart of the image recognition process.

[0058] Figure 2 is a schematic diagram of image recognition, which takes an image as input and outputs the parameters of the dish category label and the bounding box.

[0059] Figure 3 shows the overall system architecture.

[0060] Figure 4 shows a case study of a food dataset.

[0061] Figure 5 shows the operation interface of the computer equipment background information management system. Detailed Implementation

[0062] The invention will be explained in more detail by means of the following embodiments. The following embodiments are merely illustrative and the invention is not limited to these embodiments.

[0063] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Unless otherwise stated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.

[0064] While this invention describes only preferred methods and materials, any methods and materials similar or equivalent to those described herein may be used in the implementation or testing of this invention. All references to this specification are incorporated by way of citation to disclose and describe the methods and / or materials associated with those references. Unless otherwise specified, all raw materials, reagents, etc., used in this invention are available through commercial or other conventional channels. In the event of any conflict with any incorporated references, the contents of this specification shall prevail.

[0065] YOLO, short for You Only Look Once: Unified, Real-Time Object Detection, is a real-time object detection algorithm.

[0066] Figure 1 shows a schematic diagram of the operation process of the present invention. For detailed operation, please refer to Embodiment 1.

[0067] Example 1:

[0068] A method for detecting and identifying Chinese dishes in complex environments includes the following steps:

[0069] S1: Data Acquisition and Preprocessing: n = 100 common Chinese dishes were selected, and 1000 images of each dish were taken as the initial dataset. The images in the initial dataset covered different shooting angles, lighting conditions, background complexity, and the diversity of various dishes (as shown in Figure 4). The collected data were labeled, including the bounding box and category label for each dish.

[0070] Then, the images in the initial dataset are preprocessed: First, data augmentation is performed to increase the diversity and robustness of the data; the augmentation methods are random flipping, rotation, and color adjustment (brightness, contrast, saturation); then, noise reduction is performed to remove environmental noise and interference factors, ensuring that the model can focus on the dish itself; the images obtained after preprocessing are combined into a dataset again, denoted as dataset 1.

[0071] Dataset 1 is divided into training, validation, and test sets to ensure that the class distribution of each dataset is consistent; the ratio of training, validation, and test sets is 8:1:1.

[0072] S2: Constructing the ResAttention-YOLO network model: Based on the YOLOv8 framework, the object detection model uses the ResNet101 network pre-trained on the ISIA Food-500 dataset as the backbone network. The ResAttention-YOLO network model is constructed by introducing the Transformer module, the multi-scale attention module and the multi-head self-attention mechanism.

[0073] The ResNet101 network pre-trained on the ISIA Food-500 dataset was pre-trained as follows: Based on the ResNet101 network, it was pre-trained using ISIA Food-500. The pre-training parameters were set as follows: Learning Rate = 5e-5; Batch Size = 32; Epochs = 4; Weight Decay = 1e-4; After pre-training, the convolutional layers in the backbone network were frozen; thus, the ResNet101 network pre-trained on the ISIA Food-500 dataset was obtained.

[0074] S2-1: Feature Extraction: The image preprocessed in step S1 is processed by using a ResNet101 network pre-trained on the ISIA Food-500 dataset to extract features. The preprocessed image is mapped to a multi-scale feature map, denoted as Feature Map 1, which is used for subsequent high-order feature transformations.

[0075] The formula for extracting feature map 1 is as follows: Y1=Conv(W1,x)+ReLU(BN(Conv(W2,x)))

[0076] Where Conv is the convolutional layer; BN is the batch normalization layer; ReLU is the activation function; W1 and W2 are the weights of the convolutional kernel; x is the preprocessed image; Y1 is feature map 1;

[0077] S2-2: Add a Transformer module to the feature extraction part. Input the feature map 1 obtained in step S2-1 into the Transformer module to perform feature transformation and obtain the feature map after Transformer transformation, which is denoted as feature map 2.

[0078] The Transformer module includes one-dimensional convolution, multi-head self-attention mechanism, layer normalization, and feedforward network.

[0079] The specific operation of feature transformation is as follows: a one-dimensional convolutional layer is used to project the multi-scale feature map 1 output by ResNet101 onto a high-dimensional feature space, and then a multi-head self-attention mechanism is used to capture the global context information of the image projected onto the high-dimensional feature space to obtain the enhanced feature map 2.

[0080] The formula for outputting feature map 2 is as follows: Y2=LayerNorm(Y1+Attention(Q,K,V)) Q,K,V=XWq,XWk,XWv

[0081] Where Q is the query matrix; K is the key matrix; V is the value matrix; dk is the dimension of the key matrix; Wq, Wk, and Wv are the weight matrices obtained during model training; Y1 is feature map 1; and Y2 is feature map 2.

[0082] S2-3: A multi-scale attention mechanism is introduced in the feature fusion stage; the multi-scale attention module includes a self-attention layer, layer normalization, and a feedforward network, which can fuse local and global information extracted from feature maps of different scales; feature map 1 in step S2-1 and feature map 2 in step S2-2 are transformed to the same dimension, and the multi-scale attention module is used to fuse feature map 1 and feature map 2 to obtain the fused feature map, denoted as feature map 3;

[0083] The multi-scale attention module includes a self-attention layer, a layer normalization layer, and a feedforward network;

[0084] The fusion process is as follows:

[0085] (1) Formulas for transforming feature map 1 and feature map 2 to the same dimension: z1 = Conv2D(w1,Y1), z2 = Conv2D(w2,Y2)

[0086] Where Y1 is feature map 1, Y2 is feature map 2; w1 and w2 are convolution kernel weights; z1 and z2 are the transformed feature maps;

[0087] (2) Applying a self-attention layer to achieve information exchange between feature maps:

[0088] Where · represents the dot product, T Indicates transpose. It is a scaling factor; the softmax function is used to generate the weight distribution.

[0089] (3) Fuse the output of the self-attention layer with the original feature map: z = LayerNorm(z1 + Attention(z1, z2))

[0090] Where LayerNorm represents the layer normalization operation, and z is the feature map 3.

[0091] This section combines a neural network module with multi-scale information and a multi-head attention mechanism, designed to process two input feature maps Y1 and Y2, which may typically come from different scales or have different semantic information. Through the Transformer module, Y1 can be attention-weighted based on the contextual information of Y2, thereby enhancing the feature representation of Y1.

[0092] By using 1x1 convolutions for embedding dimension transformation and output recovery, this module maintains a relatively lightweight design, reducing computational complexity and the number of parameters. Since inputs Y1 and Y2 can come from different network layers or have different scales, this module offers high flexibility and can be easily integrated into various neural network architectures. By weighting Y1 through an attention mechanism, this module enhances the feature representation of Y2, incorporating more contextual information, which is highly beneficial for improving the performance of subsequent tasks.

[0093] S2-4: After multi-scale feature fusion, the native detection head of the YOLOv8 model is improved by introducing a multi-head self-attention mechanism. The improved detection head is named AttentionHead and is used for object detection and classification.

[0094] The fused feature map 3 obtained in steps S2-3 is input into the AttentionHead for target detection and classification, which is responsible for identifying and classifying dish targets and obtaining the parameter information of the bounding box; Figure 2 is a schematic diagram of image recognition, which takes an image as input and outputs dish category labels and bounding box parameters.

[0095] Based on the operations in steps S2-1 to S2-4, by introducing the Transformer module, the multi-scale attention module, and the multi-head self-attention mechanism, the final model is denoted as the ResAttention-YOLO network model.

[0096] This detection head generates bounding boxes and class labels for each target using feature map 3. This part incorporates a multi-head self-attention mechanism to enhance feature representation, particularly excelling in capturing global contextual information. First, the embedding dimension of the input feature map 3 is transformed, and the multi-head attention mechanism is used to capture long-distance dependencies in the feature map, thereby enhancing the image's feature representation. Then, the original number of channels in the feature map is recovered through convolutional layers and directly used for target detection. Through the multi-head self-attention mechanism, AttentionHead effectively enhances the ability to capture global information in the feature map, improving detection accuracy. Finally, this module outputs the classification and regression prediction results for each target through AttentionHead, generating the target's bounding box, class label, and confidence score. This method supports multi-task learning, enabling the model to simultaneously solve detection and classification tasks in a single task, thereby improving the model's generalization ability and practicality.

[0097] S3: Model Training: Input dataset 1 into the ResAttention-YOLO network model, update the parameters, and save the trained convolutional neural network structure and parameters; the training parameters are: initial learning rate = 5e-5, decreasing to 1e-5 after 4 rounds; batch size = 64; number of training rounds = 8; weight decay = 1e-4;

[0098] The training platform uses Ubuntu 18.0, an NVIDIA A100 GPU, and Python version 3.11.8.

[0099] S4: Predict and output the results for the test samples:

[0100] Select the dish to be identified, and follow the data acquisition and preprocessing operation in step S1 to obtain the preprocessed image as the sample to be identified;

[0101] The sample to be identified is then input into the ResAttention-YOLO model to obtain the category label and bounding box parameters of the dish to be identified.

[0102] S5: Calorie Calculation

[0103] (1) Establishment of calorie database: Label the different dishes in dataset 1, establish category labels, and find the calorie value of the corresponding dishes to obtain the basic calorie value of the corresponding dishes; calculate and summarize the basic calorie value of all dishes according to the corresponding labels to obtain the calorie database;

[0104] (2) Based on the parameters of the dish category label and bounding box detected by S4, the bounding box area is obtained. The corresponding calorie database is used to find the basic calorie value corresponding to the detected dish category label. Then, the bounding box area is multiplied to obtain the calorie value of each dish in the dish to be identified.

[0105] It accumulates the calorie values ​​of each dish and outputs the total calorie value of the dish to be identified, allowing users to manage their diet and control their health.

[0106] Calorie calculation: Calories = BaseCalories(i) × Area

[0107] Where BaseCalories(i) is the base calorie value of dish number i in the database, and Area is the area of ​​the detected dish bounding box.

[0108] Figure 3 shows the overall system architecture, illustrating the overall architecture of the Chinese dish recognition and evaluation system based on convolutional neural networks and natural language processing. It mainly consists of two parts: image recognition (left) and dish nutritional value calculation (right). Image recognition is based on receiving images of Chinese dishes uploaded by users and identifying them through a convolutional neural network model to determine the type of dish. The recognition results are then transmitted to the natural language processing module, which uses the statistically analyzed nutritional value information of the dishes to generate calorie calculation results for the dishes.

[0109] A computer device includes a storage module, a processing module, and a display module. The processing module is electrically connected to the storage module and the display module, enabling information transmission and control. The storage module stores program instructions executable by the processing module. The processing module invokes the program instructions to execute the method described above, and the final result is displayed through the display module. The display interface is shown in Figure 5, and is divided into two main parts: a function navigation bar on the left and a menu information management area on the right. It aims to efficiently manage various information about Chinese dishes, including dish classification, basic information, cost, and status.

[0110] Experiments and validation: Classification accuracy test;

[0111] Four commonly used models in existing technologies were used to identify the categories of samples and the classification results were compared. The four models are YOLOv4, YOLOv5, SSD and YOLOv8.

[0112] mAP (mean accuracy) and mIoU (the degree of overlap between predicted and ground truth bounding boxes) were used as the main evaluation metrics.

[0113] The model's detection and recognition capabilities in complex environments were validated using a test set, and the results were analyzed to evaluate the model's robustness and accuracy. Examples of experimental results are shown in Table 1.

[0114] Table 1: Test Results

[0115] As can be seen from the detection results in Table 1, the present invention significantly improves detection and recognition performance by optimizing the model structure and parameter settings.

[0116] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for detecting and identifying Chinese dishes in complex environments, characterized in that, Includes the following steps: Step S1, Data Acquisition and Preprocessing: S1-1 First, select common Chinese dishes and obtain images of Chinese dishes as the initial dataset; S1-2. Then, the images in the initial dataset are preprocessed to obtain preprocessed images. The dataset composed of the preprocessed images is denoted as dataset 1. S2: Constructing the ResAttention-YOLO network model: Based on the YOLOv8 framework, the object detection model uses the ResNet101 network pre-trained on the ISIA Food-500 dataset as the backbone network. The ResAttention-YOLO network model is constructed by introducing the Transformer module, the multi-scale attention module and the multi-head self-attention mechanism. The ResNet101 network pre-trained on the ISIA Food-500 dataset was pre-trained as follows: Based on the ResNet101 network, it was pre-trained using ISIA Food-500. The pre-training parameters were set as follows: learning rate = 5e-5; batch size = 32; number of training epochs = 4; weight decay = 1e-4; after pre-training, the convolutional layers in the backbone network were frozen; thus, the ResNet101 network pre-trained on the ISIA Food-500 dataset was obtained. S2-1, Feature Extraction: The image preprocessed in step S1 is processed by a ResNet101 network pretrained on the ISIA Food-500 dataset. The preprocessed image is mapped to a multi-scale feature map, denoted as Feature Map 1. S2-2, Feature Transformation: Input the feature map 1 obtained in step S2-1 into the Transformer module to perform feature transformation, and obtain the feature map after Transformer transformation, denoted as feature map 2; The Transformer module includes: a one-dimensional convolutional layer, a multi-head self-attention mechanism, a layer normalization layer, and a feedforward network; the specific operation of feature transformation is as follows: the multi-scale feature map 1 output by ResNet101 is projected onto a high-dimensional feature space using a one-dimensional convolutional layer, and then the global context information of the image projected onto the high-dimensional feature space is captured through the multi-head self-attention mechanism to obtain the enhanced feature map 2. S2-3, Fusion Processing: A multi-scale attention module is introduced in the feature fusion stage; the multi-scale attention module includes a self-attention layer, layer normalization and a feedforward network, which can fuse local and global information extracted from feature maps of different scales; The feature map 1 in step S2-1 and the feature map 2 in step S2-2 are transformed to the same dimension, and the feature map 1 and feature map 2 transformed to the same dimension are fused through the multi-scale attention module to obtain the fused feature map, which is denoted as feature map 3. S2-4 Target Detection and Classification: A multi-head attention mechanism is introduced to improve the native prediction head of YOLO. The improved detection head is named AttentionHead and is used for object detection and classification. YOLO refers to YOLOv8. The fused feature map 3 obtained in steps S2-3 is input into AttentionHead for target detection and classification, which is responsible for identifying and classifying food targets and obtaining the parameter information of the bounding box; Based on the operations in steps S2-1 to S2-4, by introducing the Transformer module, the multi-scale attention module, and the multi-head self-attention mechanism, the final model is denoted as the ResAttention-YOLO network model. S3. Model Training: Input dataset 1 into the ResAttention-YOLO network model, update the parameters, and save the trained convolutional neural network structure and parameters. The training parameters are: initial learning rate = 5e-5, decreasing to 1e-5 after 4 rounds; batch size = 64; number of training rounds = 8; weight decay = 1e-4; the training platform system is Ubuntu 18.0, and the GPU is NVIDIA A100. S4, Sample Testing: Select the dish to be identified, and follow the data acquisition and preprocessing operation in step S1 to obtain the preprocessed image as the sample to be identified; Then, the sample to be identified is input into the ResAttention-YOLO network model to obtain the category label and bounding box parameters of the dish to be identified; S5. Calorie Calculation: (1) Establishment of calorie database: Label the different dishes in dataset 1, establish category labels, and find the calorie value of the corresponding dishes to obtain the basic calorie value of the corresponding dishes; calculate and summarize the basic calorie value of all dishes according to the corresponding labels to obtain the calorie database; (2) Based on the parameters of the dish category label and bounding box detected by S4, obtain the bounding box area, and find the basic calorie value corresponding to the detected dish category label in the calorie database; then multiply the basic calorie value by the area of ​​the corresponding dish bounding box to obtain the calorie value of each dish in the dish to be identified; finally, accumulate the calorie values ​​of each dish and output the total calorie value of the dish to be identified.

2. The method for detecting and identifying Chinese dishes in complex environments according to claim 1, characterized in that, In step S1, the preprocessing methods include normalization, data augmentation, and noise reduction.

3. The method for detecting and identifying Chinese dishes in complex environments according to claim 1, characterized in that, The extraction formula for feature map 1 in step S2-1 is as follows: Y1=Conv(W1,x)+ReLU(BN(Conv(W2,x))) Where Conv is a convolutional layer; BN is a batch normalization layer; ReLU is an activation function; W1 and W2 are the weights of the convolutional kernel; x is the preprocessed image; and Y1 is feature map 1.

4. The method for detecting and identifying Chinese dishes in complex environments according to claim 1, characterized in that, The formula for outputting feature map 2 in S2-2 is as follows: Y2=LayerNorm(Y1+Attention(Q,K,V)) Q,K,V=XWq,XWk,XWv Where Q is the query matrix; K is the key matrix; V is the value matrix; and dk is the dimension of the key matrix. Wq, Wk, and Wv are weight matrices obtained during model training; Y1 is feature map 1; Y2 is feature map 2.

5. The method for detecting and identifying Chinese dishes in complex environments according to claim 1, characterized in that, The fusion processing steps in S2-3 are as follows: (1) Formula for transforming feature map 1 and feature map 2 to the same dimension: z1=Conv2D(w1,Y1),z2=Conv2D(w2,Y2) Where Y1 is feature map 1, Y2 is feature map 2; w1 and w2 are the convolution kernel weights; z1 and z2 are the transformed feature maps; (2) Applying a self-attention layer to achieve information exchange between feature maps: Where · represents the dot product, T Indicates transpose. is the scaling factor, and the softmax function is used to generate the weight distribution; z1 and z2 are the feature maps after transformation in step (1); (3) Fuse the output of the self-attention layer with the original feature map: z=LayerNorm(z1+Attention(z1,z2)) Where LayerNorm represents the layer normalization operation, and z is the feature map 3.

6. A computer device, characterized in that, It includes a storage module, a processing module, and a display module. The processing module is electrically connected to the storage module and the display module, enabling information transmission and control. The storage module stores program instructions that can be executed by the processing module. The processing module can call the program instructions to execute the method described in any one of claims 1-5, and display the final result through the display module.

Images