Lightweight food image recognition model with dual-path feature aggregation

Abstract

The invention discloses a lightweight food image recognition method and system combining local and global feature fusion, and belongs to the field of computer vision. According to the method, a dual-path feature aggregation architecture DPFA-Net is provided, and a GhostBottleneck local feature extraction module and a Mama-based PM-ViT global modeling module are fused. The PM-ViT reconstructs a feature map through Unfold-Patch, the Local Representation enhances spatial perception, and the Mama Attention is combined to model long-distance dependence, so that the feature fusion efficiency is improved. The model adopts a multi-level series structure, and the channel width is adjustable. Experiments show that the method is excellent in performance on Food-101, Food-172 and Food-256 data sets, greatly reduces the parameter quantity and the calculation quantity, improves the recognition accuracy, is suitable for resource limited scenes, and is high in practical value.

Description

Technical Field

[0001] This invention relates to a lightweight food image recognition method based on a dual-path feature aggregation network, belonging to the field of computer vision technology. Background Technology

[0002] With the development of digital technology, food computing is increasingly being applied in the fields of food science and nutrition. Food image recognition, as a key technology, has significant application value in various scenarios, such as dietary health management, intelligent upgrading and quality control in the food industry, and intelligent collaboration in the food supply chain. However, due to the performance limitations of mobile devices, how to achieve lightweight and efficient deployment of models while ensuring recognition accuracy has become a critical issue that urgently needs to be addressed. Food images generally exhibit the characteristic of "small differences between categories, but large variations within categories," thus essentially falling under the category of fine-grained image recognition and presenting a high level of discrimination difficulty. Specifically, foods within the same category can exhibit significant visual differences due to variations in raw materials, shapes, or cooking methods. For example, waffles, as a category, can vary in color, structure, and even geometric shape depending on the recipe or mold used, causing key discriminative features to be distributed across the global semantics of the entire image rather than concentrated in a specific local area. On the other hand, different categories may also exhibit high appearance similarity. For instance, while sea cucumber fried rice, stir-fried sea cucumber, and minced sea cucumber are different categories by definition, their images contain similar main components, with highly overlapping color and texture distributions, leading to significant confusion in classification tasks. This characteristic of "different shapes within the same category, and similar shapes between different categories" further highlights the fine-grained challenges in food image recognition, requiring models to not only possess strong local representation capabilities but also capture the semantic dependencies between distant components in the image, thereby achieving more accurate discriminative modeling. Traditional convolutional neural networks (CNNs) are limited by their local receptive fields, making it difficult to effectively capture long-range dependencies and cross-class similarities. To compensate for this, it's often necessary to deepen the network structure, but this significantly increases the number of model parameters and computational complexity, contradicting the principles of lightweight design. In recent years, the attention-based Vision Transformer (ViT) has shown great potential in global feature modeling, but its computational complexity increases quadratically with input resolution, making training difficult and limiting its application in lightweight scenarios. The recently proposed Vision Mamba (Vim) achieves superior global modeling capabilities compared to ViT while maintaining low computational cost, providing a new solution for lightweight vision tasks. In contrast, while convolutional neural networks have significant limitations in handling fine-grained and long-range dependencies, they have a significant advantage in extracting local features. Therefore, existing methods face the dual challenges of difficulty in co-modeling local and global features and limitations in computational resources when processing food image recognition tasks. Thus, how to achieve efficient fusion of local and global features within a lightweight framework has become a key technical bottleneck that urgently needs to be overcome in the field of food image recognition. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a lightweight hybrid neural network with dual-path feature aggregation; The technical solution provided by this invention is as follows: A lightweight food image recognition method based on dual-path feature aggregation, characterized in that it includes the following steps: Step 1: Construct a lightweight food image recognition model, including: The inverse residual module, namely the GhostBottleneck module, is constructed for local feature extraction and feature map downsampling operations. A lightweight inter-group transformer module, namely the Position Mamba Vision Transformer (PM-ViT) module, is constructed for global feature extraction and fusion of inter-channel information; Step 2: Train the model using an open-source dataset; Step 3: Deploy the trained model to the edge device.

[0004] Furthermore, the GhostBottleneck module is an important structure of the model, which contains several 1×1 and 3×3 depth-separable convolutional layers, and extracts fine-grained local features of food images through efficient convolution operations. The PM-ViT module includes a Local Representation Block (LR Block), a Mamba Attention (MA) module, a Feedforward Network (FFN), and a Separable Self-Attention (SSA) mechanism. SSA decouples attention between the spatial and channel dimensions, reducing computational complexity while enhancing modeling capabilities. This module integrates the Mamba structure with SSA to achieve efficient capture of global features and effective fusion of spatial information.

[0005] Furthermore, the lightweight food image recognition model DPFA-Net adopts a layered architecture design: The first layer consists of a convolutional module and a dual-path feature aggregation module (DualPathFeatureBlock); the convolutional module has a kernel size of 3×3, a stride of 2, and 32α output channels; the DualPathFeatureBlock module has a stride of 1, and contains 2 GhostBottleneck modules and 1 PM-VIT module, with 32α output channels. The second layer consists of a single DualPathFeatureBlock module, which contains 3 GhostBottleneck modules and 2 PM-VIT modules; the module step size is 2, and the number of output channels is 64α. The third layer consists of a single DualPathFeatureBlock module, which contains 3 GhostBottleneck modules and 3 PM-VIT modules; the module step size is 2, and the number of output channels is 96α. The fourth layer consists of a single DualPathFeatureBlock module, which contains 3 GhostBottleneck modules and 2 PM-VIT modules; the module step size is 2, and the number of output channels is 160α. The fifth layer consists of a single DualPathFeatureBlock module, which contains one GhostBottleneck and one PM-VIT module; the module step size is 2, and the number of output channels is 320α; The last layer is a global pooling fully connected layer, which consists of a global average pooling operation and a fully connected layer. The output dimension is a one-dimensional vector of num_classes, representing the probability distribution of the classes. Where α∈{0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2.0} is the model width multiplier hyperparameter, which can be set to adjust the model size.

[0006] Furthermore, the calculation formula for the GhostBottleneck module is as follows: in, The dimension of the output feature map is represented by ; GhostConv2d represents the initial GhostConv2d layer convolution kernel size k=1 and stride s=1; The function used is defined as follows: `ConvDW` indicates that a depthwise convolution is applied when `s>1`, ​​with a default kernel size of 3; `SE` indicates that the `SE` module is enabled, which is the `Squeeze-and-Excitation` module used to model channel attention, adaptively adjusting the feature weights of each channel, with a default squeezing factor of 4. Features are adjusted through global average pooling, two fully connected layers, and scaled activation; `ConvLayer2d` indicates the final activation function used. convolution.

[0007] Furthermore, the calculation process of the PM-VIT module is as follows: The input feature map is Where C is the number of channels in the input feature map; H and W are the length and width of the feature map, respectively; First, the input feature map is unfolded, i.e.: ; in This represents the feature map after unfolding, where C represents the number of channels, P represents the spatial size of a single patch (i.e., P = patch_H × patch_W, where patch_H and patch_W are the height and width of the patch, respectively), and N represents the number of patches. After unfolding, the input feature map is transformed into patches for SSA computation. The specific unfolding process involves extracting each patch from X using a sliding window. The size of each patch is... The output shape is The total number of P is calculated as follows: Each patch flattens out to A P-dimensional vector, arranged along the P-dimensional axis. Patch_H and Patch_W are the height and width of the patch, pad_H and pad_W are the padding in the height and width directions, dilation_H and dilation_W are the dilation rates in the height and width directions, and stride_H and stride_W are the stride sizes in the height and width directions. In the Unfold computation, the output feature map is then input into the LR Block, and the following calculations are performed: ; This indicates the dimension of the feature map after LR Block processing. The number of input and output channels is equal before and after processing, maintaining dimensionality consistency. Depthwise separable convolutions are used, with each input channel undergoing independent convolution to reduce the number of parameters. A 3×3 convolution kernel is used to capture local spatial information. No activation functions or normalization layers are used during computation to maintain linear transformation. after Entering the MA Block, the left branch passes through a 1 A convolutional layer and an action layer (a linear temporal modeling module built on the Mamba architecture to enhance the dynamic modeling capability of features) process the input to obtain a value map. The right branch passes through a 1 A convolution of 1 and a 3 The DWConv of 3 is then fed into SSA to obtain the Attention Map. Finally, the value map and the attention map are multiplied to obtain the final output. The following calculations are performed: ; ; ; in Indicates the input For a standard 2D convolution calculation, the kernel size is 1 and the stride is 1. The mathematical representation of standard convolution is as follows: Where i represents the batch index, and its value ranges from 1 to N; This is the output channel index; h and w are the output height and width; The weights of the convolution kernel and It represents the relative position of the convolution kernel, ranging from 0 to... and ; This represents the corresponding position value in the input feature map; The output channel corresponding to the bias term ; The stride represents the height and width; the default value is 1. and Indicates the padding for height and width; the default value is 0. Then, activation is performed in the linear layer using the hard_swish activation function, the calculation formula of which is as follows: ; in This indicates that the output is limited to... Within the range, For input values. in In this step, the number of input and output channels are equal to maintain dimensionality; a 3×3 convolution kernel is used to capture local spatial information; the convolution is set to depthwise separable convolution, with each input channel convolved independently. Its mathematical calculation formula is as follows: ; Indicates the first A convolutional kernel with 1 channel, Indicates the offset of the convolution kernel. It is a batch index. It is the output spatial location. SSA (Separable Self-Attention) is a lightweight attention mechanism that represents a separable self-attention mechanism combined with feedforward neural network computation. The final features of the left and right branches and Multiply to get The calculation formula is as follows: ; The patch after merging the two branches is passed through a 1 The output is obtained by convolution of 1, and then combined through residual connections to generate the final output. The following calculations are performed: ; Here Same as the ordinary convolution operation above, The above-mentioned feature map represents the result of a series of changes. The feature map after unfolding; Subsequently, the output is processed sequentially through an LR block and a feedforward network (FFN) to obtain... Then combine it with The summation generates the final output of PM-ViT, which is then transformed through a folding operation. The following calculations are performed: ; ; ; A feed-forward network (FFN) includes a normalization layer, convolutional layers, activation layers, and Dropout. The convolutional and activation layers are the same as described above; the mathematical representation of the normalization layer is as follows: ; Indicates along The mean of the dimension; Indicates variance; This represents the learnable parameters. The mathematical representation of Dropout is as follows: ; This indicates the result after the activation layer is activated. Indicates will After a series of processing steps, it is added to itself to form a residual connection. This represents the feature map after the model processing results have been folded.

[0008] Furthermore, the calculation steps for the global pooling fully connected layer are as follows: The input feature map is Global average pooling is performed; global average pooling averages the pixel values ​​of each channel to obtain a one-dimensional feature vector of dimension C. The specific calculation formula is as follows: Where c is the number of channels in the input feature map; h and w are the length and width of the feature map, respectively. The feature vectors obtained in the above steps are transformed into recognition results through a fully connected layer. The specific calculation formula is as follows: ; W is the classification matrix, b is the bias vector, k is the class index, and i is the batch index. This step converts the compressed features into classification scores.

[0009] The beneficial effects of this invention are as follows: By effectively fusing local and global features, this invention overcomes the limitations of traditional CNN models that focus on local feature extraction, demonstrating superior performance in global feature capture. Local feature extraction is achieved through the GhostBottleneck module, which utilizes depthwise separable convolutions and 1x1 convolutional layers to adjust the number of channels, efficiently capturing local texture features of the image while reducing computational costs. Global feature extraction is achieved through the Position MambaVision Transformer (PM-ViT) module. This module first uses the local features extracted by GhostBottleneck as input, then enhances the correlation between different patches through the Local Representation Block (LR Block), and then enters the Mamba Attention (MA) module to replace the traditional self-attention mechanism to reduce computational complexity and improve the ability to capture global information. Features are processed in two paths within the MA module and merged through Hadamard integration, and finally further enhanced by LRBlock and a feedforward network (FFN), thereby efficiently capturing long-distance spatial dependencies and global structural information in the image, improving the model's robustness to complex scenes and recognition accuracy. On the three benchmark datasets ETHZ Food-101, Vireo Food-172, and UEC Food-256, DPFA-Net achieved Top-1 accuracies of 91.46%, 91.59%, and 75.33%, respectively, while reducing the number of parameters by approximately 8.4M. Ablation studies show that on datasets such as Food-101, DPFA-Net-1.0, which integrates the GhostBottleneck and PM-ViT modules, achieved a Top-1 accuracy of 89.20%, representing improvements of 2.26% and 2.31% compared to using only local feature modules (86.94%) and global feature modules (86.89%), respectively. Replacing PM-ViT with a regular Transformer reduced the model's accuracy on this dataset to 88.85%. A significant breakthrough in computational efficiency is noteworthy; DPFA-Net-1.0 has a computational load of approximately 919M FLOPs, only 23% of that of a regular Transformer, demonstrating significant application value in resource-constrained environments. This balanced design philosophy enables the model to achieve optimal configuration in three dimensions: accuracy, number of parameters, and computational complexity. Attached Figure Description

[0010] Figure 1 This is a flowchart of the present invention. Specific implementation measures

[0011] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further explained in detail below with reference to embodiments. It should be noted that the specific embodiments described herein are only intended to explain this invention and are not intended to limit or define it. like Figure 1 As shown, a lightweight food image recognition method based on dual-path feature aggregation includes the following steps: Step 1: Build a lightweight food image recognition model on the computing platform, including: The inverse residual module, namely the GhostBottleneck module, is constructed for local feature extraction and feature map downsampling operations. A lightweight inter-group transformer module, namely the PM-VIT (Position Mamba Vision Transformer) module, is constructed for global feature extraction and fusion of inter-channel information; The GhostBottleneck module is the core structure of the lightweight model GhostNetV1, which contains several 1×1 and 3×3 depthwise separable convolutional layers.

[0012] The PM-ViT module includes a Local Representation Block (LR Block), a Mamba Attention (MA) module, a feedforward network (FFN), and a separable self-attention mechanism.

[0013] Layer the model architecture: The first layer consists of a combination of a convolutional module and a DualPathFeatureBlock module. The convolutional module has a kernel size of 3×3, a stride of 2, and 32α output channels. The DualPathFeatureBlock module has a stride of 1 and contains 2 GhostBottleneck modules and 1 PM-ViT module, with 32α output channels.

[0014] The second layer consists of a single DualPathFeatureBlock module, which contains 3 GhostBottleneck modules and 2 PM-VIT modules; the module step size is 2, and the number of output channels is 48α.

[0015] The third layer consists of a single DualPathFeatureBlock module, which contains 3 GhostBottleneck modules and 3 PM-VIT modules; the module step size is 2, and the number of output channels is 80α.

[0016] The fourth layer consists of a single DualPathFeatureBlock module, which contains 3 GhostBottleneck modules and 2 PM-VIT modules; the module step size is 2 and the number of output channels is 160α.

[0017] The fifth layer consists of a single DualPathFeatureBlock module, which contains one GhostBottleneck and one PM-VIT module; the module step size is 2, and the number of output channels is 320α.

[0018] The last layer is a global pooling fully connected layer, which consists of a global average pooling operation (pool_type defaults to "mean") and a fully connected layer. The output dimension is a one-dimensional vector of num_classes, representing the probability distribution of the classes.

[0019] Where α∈{0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2.0} is the model width multiplier hyperparameter, which can be set to adjust the model size.

[0020] The calculation formula for the GhostBottleneck module is as follows: in, The dimension of the output feature map is represented by ; GhostConv2d represents the initial GhostConv2d layer convolution kernel size k=1 and stride s=1; The activation function defaults to the hard_swish activation function; ConvDW indicates that depthwise convolution is applied when s>1, with a default kernel size of 3; SE indicates that the SE module is enabled, where the squeezing factor defaults to 4, and features are adjusted through global average pooling, two fully connected layers, and scaling activation; ConvLayer2d indicates the final activation function used. convolution.

[0021] The GhostBottleneck module extracts local features through pointwise convolution and depthwise convolution. When the stride is 2, it achieves spatial downsampling of the feature map, reducing the size of the output feature map by half.

[0022] The PM-VIT module calculation process is as follows: The input feature map is Where C is the number of channels in the input feature map; H and W are the length and width of the feature map, respectively.

[0023] First, the input feature map is unfolded, i.e.: ; in The input feature map is represented by X after unfolding, where C represents the number of channels, P represents the spatial size of a single patch (i.e., P = patch_H × patch_W, where patch_H and patch_W are the height and width of the patch, respectively), and N represents the number of patches. After unfolding, the input feature map is transformed into patches for Separable SelfAttention computation. The specific unfolding process involves extracting each patch from X using a sliding window. The size of each patch is... The output shape is The total number of P is calculated as follows: Each patch flattens out to A P-dimensional vector, arranged along the P-dimensional axis. Patch_H and Patch_W are the height and width of the patch, pad_H and pad_W are the padding in the height and width directions, dilation_H and dilation_W are the dilation rates in the height and width directions, and stride_H and stride_W are the stride sizes in the height and width directions. In the Unfold computation, the output feature map is then input into the LR Block, and the following calculations are performed: ; This indicates the dimension of the feature map after LR Block processing. The number of input and output channels is equal before and after processing, maintaining dimensionality consistency. Depthwise separable convolutions are used, with each input channel undergoing independent convolution to reduce the number of parameters. A 3×3 convolution kernel is used to capture local spatial information. No activation functions or normalization layers are used during computation to maintain linear transformation. after Entering the MA Block, the left branch passes through a 1 A convolutional layer and an Action Layer process the input to obtain a value map. The right branch passes through a 1 A convolution of 1 and a 3 The DWConv of 3 is then fed into SSA to obtain the Attention Map. Finally, the value map and the attention map are multiplied to obtain the final output. The following calculations are performed: ; ; ; in Indicates the input For a standard 2D convolution calculation, the kernel size is 1 and the stride is 1. The mathematical representation of standard convolution is as follows: Where i represents the batch index, and its value ranges from 1 to N; This is the output channel index; h and w are the output height and width; The weights of the convolution kernel and It represents the relative position of the convolution kernel, ranging from 0 to... and ; This represents the corresponding position value in the input feature map; The output channel corresponding to the bias term ; The stride represents the height and width; the default value is 1. and Indicates the padding for height and width; the default value is 0.

[0024] Then, activation is performed in the linear layer using the hard_swish activation function, the calculation formula of which is as follows: ; in This indicates that the output is limited to... Within the range, For input values. in In this step, the number of input and output channels are equal to maintain dimensionality; a 3×3 convolution kernel is used to capture local spatial information; the convolution is set to depthwise separable convolution, with each input channel convolved independently. Its mathematical calculation formula is as follows: ; Indicates the first A convolutional kernel with 1 channel, Indicates the offset of the convolution kernel. It is a batch index. It is the output spatial location. SSA (Separable Self-Attention) is a lightweight attention mechanism that represents a separable self-attention mechanism combined with feedforward neural network computation.

[0025] The final features of the left and right branches and Multiply to get The calculation formula is as follows: ; The patch after merging the two branches is passed through a 1 The output is obtained by convolution of 1, and then combined through residual connections to generate the final output. The following calculations are performed: ; Here Same as the ordinary convolution operation above, The above-mentioned feature map represents the result of a series of changes. This is the feature map after unfolding.

[0026] Subsequently, the output is processed sequentially through an LR block and a feedforward network (FFN) to obtain... Then combine it with The summation generates the final output of PM-ViT, which is then transformed through a folding operation. The following calculations are performed: ; ; ; The FFN (Feed-Forward Network) includes normalization layers, convolutional layers, activation layers, and Dropout. The convolutional and activation layers are the same as described above; the mathematical representation of the normalization layer is as follows: ; Indicates along The mean of the dimension; Indicates variance; This represents the learnable parameters. The mathematical representation of Dropout is as follows: ; This indicates the result after the activation layer is activated. Indicates will After a series of processing steps, it is added to itself to form a residual connection. This represents the feature map after the model processing results have been folded.

[0027] The calculation steps for the global pooling fully connected layer are as follows: The input feature map is Global average pooling is performed; global average pooling averages the pixel values ​​of each channel to obtain a one-dimensional feature vector of dimension C. The specific calculation formula is as follows: Where c is the number of channels in the input feature map; h and w are the length and width of the feature map, respectively.

[0028] The feature vectors obtained in the above steps are transformed into recognition results through a fully connected layer. The specific calculation formula is as follows: ; W is the classification matrix, b is the bias vector, k is the class index, and i is the batch index. This step converts the compressed features into classification scores.

[0029] Step 2: Train the model on an open-source dataset; Based on the model built in Step 1, the model can be trained on a server platform using open-source food datasets such as ETHZ Food-101, Vireo Food-172, and UEC Food-256.

[0030] The ETHZ Food-101 dataset contains 101 categories of Western food, using 75,750 images for training and 25,250 images for validation.

[0031] The Vireo Food-172 dataset contains 172 categories of Chinese food, and was trained using 66,071 images and validated using 44,170 images.

[0032] The UEC Food-256 dataset contains 256 Asian food categories, with 22,095 images used for training and 9,300 images used for validation.

[0033] To verify the effectiveness of the GhostBottleneck module in the food image recognition model and the effective role of PM-ViT in this embodiment, we conducted ablation experiments on the aforementioned dataset.

[0034] To improve model convergence efficiency and prevent getting trapped in local optima during computation, this method employs a cosine annealing strategy to adjust the learning rate during training. This allows the learning rate to vary according to the shape of a cosine function, enabling it to change periodically over a wide range. This avoids the model getting stuck in local optima and allows for fine-tuning of parameters later on. The calculation formula is as follows: in It is the learning rate for the t-th training cycle. and These are the minimum and maximum learning rates after restarting, respectively, and T is the total length of the learning rate annealing cycle.

[0035] In the training process of this method, the model parameters are updated using the gradient descent algorithm. However, factors such as gradient noise and fluctuations in training data can easily cause instability in the model parameters. To address this, this method introduces an exponential moving average (EMA) mechanism to smooth the model parameters, which can suppress such instability to some extent.

[0036] Step 3: Deploy the trained model to the edge device; determine the task deployment end and select a reasonable model width for deployment; due to the adjustable model width design, the model sizes are: 0.5M, 0.75M, 1.0M, 1.25M, 1.5M, 1.75M, 2.0M. As the number of model parameters increases, the model performance will continuously improve. Therefore, when facing resource-constrained scenarios, it is necessary to reasonably determine the model width so that the recognition method can achieve better performance in limited scenarios.

[0037] It should be understood that any parts not described in detail in this specification belong to the prior art; the above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A lightweight food image recognition method based on dual-path feature aggregation, characterized in that, It includes the following steps: Step 1: Construct a lightweight food image recognition model, including: constructing the inverse residual module, namely the GhostBottleneck module, for local feature extraction and feature map downsampling operations; A lightweight inter-group transformer module, namely the Position Mamba Vision Transformer (PM-ViT) module, is constructed for global feature extraction and fusion of inter-channel information; Step 2: Train the model using an open-source dataset; Step 3: Deploy the trained model to the edge device.

2. The lightweight food image recognition method based on dual-path feature aggregation according to claim 1, characterized in that, The GhostBottleneck module is the core structure of the lightweight model GhostNetV1, which contains several 1×1 and 3×3 depthwise separable convolutional layers, and extracts fine-grained local features of food images through efficient convolution operations. The PM-ViT module includes a Local Representation Block (LR Block), a Mamba Attention (MA) module, a feedforward network (FFN), and a Separable Self-Attention (SSA) mechanism. By fusing the Mamba structure with Separable Self-Attention, it achieves efficient capture of global features and interaction of spatial information.

3. The lightweight food image recognition method based on dual-path feature aggregation according to claim 1, characterized in that, The lightweight food image recognition model DPFA-Net adopts a layered architecture design: The first layer consists of a convolutional module and a dual-path feature aggregation module (DualPathFeatureBlock); the convolutional module has a kernel size of 3×3, a stride of 2, and 32α output channels; the DualPathFeatureBlock module has a stride of 1, and contains 2 GhostBottleneck modules and 1 PM-VIT module, with 32α output channels. The second layer consists of a single DualPathFeatureBlock module, which contains 3 GhostBottleneck modules and 2 PM-VIT modules; the module step size is 2, and the number of output channels is 64α. The third layer consists of a single DualPathFeatureBlock module, which contains 3 GhostBottleneck modules and 3 PM-VIT modules; the module step size is 2, and the number of output channels is 96α. The fourth layer consists of a single DualPathFeatureBlock module, which contains 3 GhostBottleneck modules and 2 PM-VIT modules; the module step size is 2, and the number of output channels is 160α. The fifth layer consists of a single DualPathFeatureBlock module, which contains one GhostBottleneck and one PM-VIT module; the module step size is 2, and the number of output channels is 320α; The last layer is a global pooling fully connected layer, which consists of a global average pooling operation and a fully connected layer. The output dimension is a one-dimensional vector of num_classes, representing the probability distribution of the classes. Where α∈{0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2.0} is the model width multiplier hyperparameter, which can be set to adjust the model size.

4. The lightweight food image recognition method based on dual-path feature aggregation according to claim 1, characterized in that, The calculation formula for the GhostBottleneck module is as follows: in, The dimension of the output feature map is represented by ; GhostConv2d represents the initial GhostConv2d layer convolution kernel size k=1 and stride s=1; This represents the activation function; ConvDW indicates that a depthwise convolution is applied when s>1, with a default kernel size of 3; SE indicates that the SE module is enabled, which is the Squeeze-and-Excitation module used to model channel attention, adaptively adjusting the feature weights of each channel, where the squeezing factor defaults to 4, and the features are adjusted through global average pooling, two fully connected layers, and scaling activation; ConvLayer2d indicates the final activation function used. convolution.

5. A lightweight food image recognition method based on dual-path feature aggregation according to claim 1, characterized in that, The PM-VIT module calculation process is as follows: The input feature map is... Where C is the number of channels in the input feature map; H and W are the length and width of the feature map, respectively; first, the input feature map is unfolded, i.e.: ;in The unfolded feature map represents the feature map after the unfolding process. C represents the number of channels, P represents the spatial size of a single patch (i.e., P = patch_H × patch_W, where patch_H and patch_W are the height and width of the patch, respectively), and N represents the number of patches. After the unfolding process, the input feature map is transformed into patches for Separable Self-Attention computation. The specific unfolding process involves using a sliding window to extract each patch from X, with each patch having a size of [missing information]. The output shape is The total number of P is calculated as follows: Each patch flattens out to The vector is arranged along dimension P. Patch_H and Patch_W are the height and width of the patch, pad_H and pad_W are the padding in the height and width directions, dilation_H and dilation_W are the dilation rates in the height and width directions, and stride_H and stride_W are the stride sizes in the height and width directions. In the Unfold calculation, the output feature map is then input into the LR Block as follows: ； This indicates the dimension of the feature map after LR Block processing. The number of input and output channels is equal before and after processing to maintain dimensionality consistency. Depthwise separable convolutions are used, with each input channel undergoing independent convolution to reduce the number of parameters. A 3×3 convolution kernel is used to capture local spatial information. No activation functions or normalization layers are used during computation to maintain linear transformation. Entering the MA Block, the left branch passes through a 1 A convolutional layer and an action layer (the Action Layer is a linear temporal modeling module built on the Mamba architecture to enhance the dynamic modeling capability of features) process the input to obtain a value map. ; The right branch passes through a 1 A convolution of 1 and a 3 The DWConv of 3 is then fed into SSA to obtain the Attention Map. Finally, the value map and the attention map are multiplied together to obtain the final output. The following calculations are performed: ； ； ； in Indicates the input For a standard 2D convolution calculation, with a kernel size of 1 and a stride of 1, the mathematical representation of a standard convolution is as follows: Where i represents the batch index, and its value ranges from 1 to N; This is the output channel index; h and w are the output height and width; The weights of the convolution kernel and It represents the relative position of the convolution kernel, ranging from 0 to... and ; This represents the corresponding position value in the input feature map; The output channel corresponding to the bias term ; The stride represents the height and width; the default value is 1. and This represents the padding for height and width; the default value is 0. Then, activation is performed in the linear layer using the hard_swish activation function, the calculation formula of which is as follows: ； in This indicates that the output is limited to... Within the range, For input values, in In this step, the number of input and output channels are equal, maintaining dimensionality consistency; a 3×3 convolution kernel is used to capture local spatial information; the convolution is set to depthwise separable convolution, with each input channel convolved independently. Its mathematical calculation formula is as follows: ； Indicates the first A convolutional kernel with 1 channel, Indicates the offset of the convolution kernel. It is a batch index. It is the output spatial location; SSA (Separable Self-Attention) is a lightweight attention mechanism that represents a separable self-attention mechanism and feedforward neural network computation. The final features of the left and right branches and Multiply to get The calculation formula is as follows: ； The patch after merging the two branches is passed through a 1 The output is obtained by convolution of 1, and then combined through residual connections to generate the final output. The following calculations are performed: ； Here Same as the ordinary convolution operation above, The above-mentioned feature map represents the result of a series of changes. The feature map after unfolding; Subsequently, the output is processed sequentially through an LR block and a feedforward network (FFN) to obtain... Then combine it with The summation generates the final output of PM-ViT, which is then transformed through a folding operation. The following calculations are performed: ； ； ； The FFN (Feed-Forward Network) includes normalization layers, convolutional layers, activation layers, and Dropout. The convolutional and activation layers are the same as described above; the mathematical representation of the normalization layer is as follows: ； Indicates along The mean of the dimension; Indicates variance; The mathematical representation of the learnable parameters in Dropout is as follows: ； This indicates the result after the activation layer is activated. Indicates will After a series of processing steps, it is added to itself to form a residual connection. This represents the feature map after the model processing results have been folded.

6. A lightweight food image recognition method based on dual-path feature aggregation according to claim 1, characterized in that, The calculation steps for the global pooling fully connected layer are as follows: The input feature map is Global average pooling is performed; global average pooling averages the pixel values ​​of each channel to obtain a one-dimensional feature vector of dimension C. The specific calculation formula is as follows: Where c is the number of channels in the input feature map; h and w are the length and width of the feature map, respectively; The feature vectors obtained in the above steps are transformed into recognition results through a fully connected layer. The specific calculation formula is as follows: ; W is the classification matrix, b is the bias vector, k is the class index, and i is the batch index. This step converts the compressed features into classification scores.

Images