Food identification method and system based on image identification

Abstract

The invention belongs to the technical field of food identification, and discloses a food identification method and system based on image identification. The method comprises the following steps: collecting a plurality of historical food image data based on a cloud data center, and constructing a food identification model by using an image identification and transfer learning fusion algorithm according to the plurality of historical food image data; on the basis of a user terminal, downloading a food identification model, and updating the food identification model to obtain a customized food identification model; and based on the user terminal, collecting real-time food product image data, and according to the real-time food product image data, performing food product identification by using the customized food product identification model to obtain a real-time food product identification result. According to the method, the problems of high data dependence, insufficient generalization ability and poor model adaptability in the prior art are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of food recognition, and in particular relates to a food recognition method and system based on image recognition. Background Art

[0002] With the development of science and technology and the improvement of people's living standards, the demand for food recognition is growing. For example, when tracking nutrition, users upload food pictures by taking photos, and the application automatically recognizes the food and estimates the nutritional content. It provides food recognition functions for health management applications to help users monitor the types of food and calories they consume. When managing catering, restaurants use food recognition systems to automate the ordering process. Customers only need to take photos to complete the ordering. When recommending recipes, similar recipes are recommended or cooking suggestions are provided based on the food pictures uploaded by users. Based on the above needs, how to combine image recognition technology to realize food recognition is a key research direction in the fields of catering services, health management, etc., aiming to improve the accuracy of food recognition and user experience.

[0003] The existing food recognition technology has the following defects:

[0004] 1) High data dependence: Existing food recognition technologies often rely on a large number of labeled data sets, which are expensive and time-consuming to collect and annotate. For some rare food categories, it may be difficult to collect enough training samples, resulting in poor recognition performance of the model on these categories.

[0005] 2) Insufficient generalization ability: Many food recognition models are trained under specific conditions, such as specific lighting, angles, or background environments, which limits the generalization ability of the model. When the model encounters test data that is inconsistent with the distribution of training data, the recognition accuracy may drop significantly;

[0006] 3) Poor model adaptability: Existing models are often difficult to adapt to the food recognition needs of different users or different regions, because these needs may involve different food types and cultural differences. The model update and customization process is complicated, which is not conducive to quickly adapting to new food categories or user preferences. Summary of the invention

[0007] In order to solve the problems of high data dependence, insufficient generalization ability and poor model adaptability in the prior art, the present invention aims to provide a food recognition method and system based on image recognition.

[0008] The technical solution adopted by the present invention is:

[0009] A method for identifying food based on image recognition comprises the following steps:

[0010] Based on the cloud data center, some historical food image data are collected, and based on these historical food image data, a food recognition model is constructed using the image recognition and transfer learning fusion algorithm;

[0011] Based on the user terminal, download the meal recognition model, and update the meal recognition model to obtain a customized meal recognition model;

[0012] Based on the user terminal, real-time food image data is collected, and based on the real-time food image data, a customized food recognition model is used to perform food recognition to obtain real-time food recognition results.

[0013] Furthermore, based on the cloud data center, a number of historical meal image data are collected, and based on the historical meal image data, an image recognition and transfer learning fusion algorithm is used to construct a meal recognition model, including the following steps:

[0014] Based on the cloud data center, a number of historical meal image data are collected, and the number of historical meal image data are preprocessed to obtain a number of preprocessed historical meal image data;

[0015] Using an image recognition algorithm, an initial food recognition architecture is constructed, and using a transfer learning algorithm, the initial food recognition architecture is improved to obtain an improved food recognition architecture;

[0016] Use swarm intelligence optimization algorithm to optimize the improved food recognition architecture and obtain the initial food recognition model;

[0017] According to a number of preprocessed historical food image data, the initial food recognition model is trained and optimized to obtain a final food recognition model, and model metadata of the final food recognition model is extracted.

[0018] Furthermore, the image recognition algorithm is the LFTCNN-RF algorithm, the transfer learning algorithm is the DANN algorithm, and the swarm intelligence optimization algorithm is the ICPO optimization algorithm;

[0019] Furthermore, the food recognition model is constructed based on the LFTCNN-DANN-RF algorithm, and the food recognition model includes a feature extractor constructed based on the LFTCNN algorithm, a domain adversary constructed based on the DANN algorithm, and a label predictor constructed based on the RF algorithm, and the feature extractor is connected to the domain adversary and the label predictor respectively;

[0020] The feature extractor includes a first feature channel and a second feature channel both constructed based on the CNN algorithm. The domain adversary includes a gradient reversal layer and a domain classification layer. The first feature channel and the second feature channel are respectively connected to the label predictor and the gradient reversal layer, and the gradient reversal layer is connected to the domain classification layer.

[0021] Furthermore, the improved food recognition architecture is optimized using a swarm intelligence optimization algorithm to obtain an initial food recognition model, including the following steps:

[0022] The objective function is to minimize the prediction error, and the network parameters of the improved food recognition architecture are encoded as the encoding vector of the ICPO individual in the ICPO optimization algorithm;

[0023] The objective function is used as the fitness function of ICPO individuals, and the ICPO population parameters and the maximum number of iterations are set;

[0024] Initialization is performed according to the coding vector of the ICPO individual and the ICPO population parameters to generate a number of initial ICPO individuals of the initial ICPO population;

[0025] According to the fitness function and ICPO population parameters, the ICPO optimization algorithm is used to iteratively update the initial ICPO population and retain the best individuals;

[0026] If the number of iterations of iterative optimization reaches the maximum number of iterations or the fitness value of the optimal individual meets the requirements, the optimal solution corresponding to the optimal individual is output;

[0027] Decode the encoding vector of the optimal ICPO individual to obtain the optimal network parameters of the improved food recognition architecture;

[0028] According to the optimal network parameters, the improved food recognition architecture is optimized to obtain the initial food recognition model.

[0029] Furthermore, according to a number of pre-processed historical food image data, the initial food recognition model is trained and optimized to obtain a final food recognition model, and model metadata of the final food recognition model is extracted, including the following steps:

[0030] The first loss function of the feature extractor and the second loss function of the domain adversary are weighted together to obtain a joint loss function of the initial food recognition model;

[0031] Using a number of preprocessed historical food image data, pre-training an initial food recognition model to obtain a pre-trained food recognition model;

[0032] Input the preprocessed historical food image data into the pre-trained food recognition model, and use the pre-trained feature extractor to extract the corresponding historical data features;

[0033] Input historical data features into the label predictor to perform meal identification prediction and obtain historical meal identification labels;

[0034] Input the historical data features into the gradient reversal layer of the domain adversary, perform gradient reversal on the historical data features, and obtain the historical data features after gradient reversal;

[0035] Input the historical data features after gradient inversion into the domain classifier of the domain adversary to perform domain classification and obtain the historical domain classification label;

[0036] According to the historical domain classification labels and historical food identification labels, a joint loss function is used to obtain the historical loss value;

[0037] According to the historical loss value, the back propagation algorithm is used to update the model parameters of the pre-trained food recognition model to obtain an updated food recognition model;

[0038] According to a number of pre-processed historical food image data, the updated food recognition model is fine-tuned until the lowest historical loss value is output, thereby obtaining the final food recognition model;

[0039] Extract the model metadata of the final food recognition model.

[0040] Furthermore, based on the user terminal, the meal recognition model is downloaded and updated to obtain a customized meal recognition model, including the following steps:

[0041] Based on the user terminal, download the model metadata of the food recognition model, and reconstruct the model according to the model metadata to obtain a reconstructed food recognition model;

[0042] According to the target meal type set by the user terminal, a number of target historical meal image data are collected and preprocessed to obtain a number of preprocessed target historical meal image data;

[0043] According to a number of pre-processed target historical meal image data, the meal recognition model is trained twice to obtain a customized meal recognition model.

[0044] Furthermore, based on the user terminal, real-time food image data is collected, and according to the real-time food image data, a customized food recognition model is used to perform food recognition to obtain a real-time food recognition result, including the following steps:

[0045] Based on the user terminal, real-time food image data is collected and pre-processed to obtain pre-processed real-time food image data;

[0046] Input the preprocessed real-time food image data into the customized food recognition model, and use the feature extractor to extract the real-time data features of the preprocessed real-time food image data;

[0047] The real-time data features are input into the domain adversary and the label predictor respectively, and the label predictor is used to perform meal identification to obtain the real-time meal identification label;

[0048] Parse the real-time food identification label to obtain the corresponding real-time food identification result;

[0049] Use the gradient reversal layer of the domain adversary to perform gradient reversal on the real-time data features to obtain the real-time data features after gradient reversal;

[0050] Input the real-time data features after gradient inversion into the domain classifier of the domain adversary to perform domain classification and obtain the real-time domain classification label;

[0051] According to the real-time domain classification label and the real-time food identification label, a joint loss function is used to obtain the real-time loss value;

[0052] According to the real-time loss value, the back propagation algorithm is used to update the model parameters of the customized meal recognition model to obtain an updated customized meal recognition model.

[0053] Furthermore, the preprocessed real-time food image data is input into a customized food recognition model, and a feature extractor is used to extract real-time data features of the preprocessed real-time food image data, including the following steps:

[0054] Input the pre-processed real-time food image data into the customized food recognition model;

[0055] Using the first feature channel, extracting real-time high-level data features of the preprocessed real-time food image data;

[0056] Using the second feature channel, extracting real-time low-level data features of the pre-processed real-time food image data;

[0057] According to the preset attention weight value, the real-time high-level data features and the real-time low-level data features are fused to obtain the real-time data features.

[0058] A food identification system based on image recognition is used to implement a food identification method. The system includes a cloud data center and a plurality of user terminals. The cloud data center is respectively connected to the plurality of user terminals for communication.

[0059] The cloud data center includes a data collection unit, an architecture construction unit, an architecture optimization unit, and a training optimization unit which are connected in sequence.

[0060] The beneficial effects of the present invention are:

[0061] The present invention discloses a method and system for identifying food based on image recognition. The method and system utilize the characteristics of transfer learning technology and are based on a model pre-trained on a large data set, thereby reducing the dependence on a large amount of labeled food image data, effectively reducing the cost of data collection and annotation, and achieving high-performance recognition on a small amount of specific food data by fine-tuning model parameters, thereby improving the model's recognition capability for rare food categories. The method adopts an image recognition algorithm that integrates multiple features, thereby improving the generalization capability of the model under different lighting, angles, and background environments by combining high-level and low-level data features, thereby enabling the model to more accurately identify a variety of food images, and maintaining a high recognition accuracy rate even under non-ideal conditions. The method allows users to customize and update the model based on personal preferences or food types in a specific region, thereby improving the adaptability of the model. The combination of a cloud data center and a user terminal realizes rapid iteration and personalization of the model, thereby better meeting the needs of different user groups.

[0062] Other beneficial effects of the present invention will be further described in the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flowchart of the meal identification method based on image recognition in the present invention.

[0064] Figure 2 It is a structural block diagram of the meal recognition system based on image recognition in the present invention. DETAILED DESCRIPTION

[0065] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments.

[0066] Embodiment 1:

[0067] like Figure 1 As shown, this embodiment provides a method for identifying food based on image recognition, comprising the following steps:

[0068] S1: Based on the cloud data center, collect some historical food image data, and use the image recognition and transfer learning fusion algorithm to build a food recognition model based on some historical food image data, including the following steps:

[0069] S1-1: Based on the cloud data center, a number of historical food image data are collected, including food pictures of different angles, lighting conditions, backgrounds and food types, and the number of historical food image data are preprocessed to obtain a number of preprocessed historical food image data;

[0070] Preprocessing includes data cleaning, data enhancement, and size normalization. Data cleaning removes image data of low quality and incorrect labels. Data enhancement increases sample diversity and improves the generalization ability of the model by rotating, scaling, cropping, and color conversion.

[0071] S1-2: Use the Low level feature fusion two-channel CNN (LFTCNN)-Random Forest (RF) algorithm to build the initial food recognition architecture, and use the Domain-Adversarial Neural Network (DANN) algorithm to improve the initial food recognition architecture to obtain an improved food recognition architecture;

[0072] The improved food recognition architecture is the LFTCNN-DANN-RF architecture, that is, the algorithm for constructing the food recognition model is the LFTCNN-DANN-RF algorithm. The food recognition model includes a feature extractor based on the LFTCNN algorithm, a domain adversary based on the DANN algorithm, and a label predictor based on the RF algorithm. The feature extractor is connected to the domain adversary and the label predictor respectively.

[0073] The feature extractor includes a first feature channel and a second feature channel both constructed based on a convolutional neural network (CNN), the domain adversary includes a gradient reversal layer and a domain classification layer, the first feature channel and the second feature channel are respectively connected to the label predictor and the gradient reversal layer, and the gradient reversal layer is connected to the domain classification layer;

[0074] The feature extractor is pre-trained using historical food image data after several pre-processing steps. The first feature channel is used to process the original color image and capture high-level features such as shape and texture. The second feature channel focuses on extracting low-level features such as edges and corners. This can be achieved by using a convolution kernel similar to the Sobel operator or the Laplacian operator. The features of the two channels are fused through the preset attention weight value to obtain the corresponding image data features.

[0075] The gradient reversal layer is located between the feature extractor and the domain classifier. Its function is to reverse the gradient during the back-propagation process, so that the domain classifier can learn the features that distinguish the source domain (original input image) and the target domain (food area) during training, while the feature extractor is trained to generate domain-invariant features, thereby achieving adversarial training;

[0076] The domain classification layer is used to predict whether the feature belongs to the source domain or the target domain based on the inverted features output by the gradient inversion layer. During the training process, the feature extractor tries to generate domain-invariant features, while the domain classifier tries to distinguish which domain these features come from. Through this adversarial training, the feature extractor gradually learns to ignore domain-related information and only retains task-related information.

[0077] Through multiple iterations of training, the model gradually learns to accurately classify in the source domain while extracting features that have good generalization capabilities in the target domain;

[0078] S1-3: Using the Improved Crested Porcupine Optimizer (ICPO) optimization algorithm, the improved food recognition architecture is optimized to obtain an initial food recognition model, including the following steps:

[0079] S1-3-1: Minimizing the prediction error is used as the objective function, and the network parameters of the improved food recognition architecture are encoded as the encoding vector of the ICPO individual in the ICPO optimization algorithm;

[0080] S1-3-2: Use the objective function as the fitness function of the ICPO individual, and set the ICPO population parameters and the maximum number of iterations;

[0081] The formula is:

[0082]

[0083] Where F(x) is the fitness function; R i (x) is the ith edge-cloud collaborative deployment cost of ICPO individual x, including hardware cost, communication cost, monitoring range cost, etc.; i is the cost indicator; W i is the cost weight of the ith item; x is the ICPO individual indicator;

[0084] S1-3-3: Based on the coding vector of the ICPO individual and the ICPO population parameters, the Circle chaotic mapping sequence is used for initialization to generate several initial ICPO individuals of the initial ICPO population;

[0085] The formula is:

[0086]

[0087] In the formula, It is the initial ICPO individual of Circle chaos mapping; is the randomly generated initial ICPO individual; i' is the ICPO individual indicator;

[0088] S1-3-4: According to the fitness function and ICPO population parameters, use the ICPO optimization algorithm to iteratively update the initial ICPO population and retain the best individuals, including the following steps:

[0089] S1-3-4-1: Introduce a cyclic population reduction mechanism to limit the number of individuals in the ICPO population parameters and obtain updated ICPO population parameters for the next iteration;

[0090] The formula is:

[0091]

[0092] In the formula, S t+1 is the number of individuals in the ICPO population parameter of the t+1th iteration; S t is the number of individuals in the ICPO population parameter of the tth iteration; S min is the minimum number of individuals in the ICPO population parameter; a' is the function evaluation parameter; V is the function evaluation cycle parameter; V max is the maximum function evaluation loop parameter; t is the number of iterations indicator;

[0093] S1-3-4-2: Calculate the initial fitness value of the initial ICPO individuals in the initial ICPO population according to the fitness function;

[0094] S1-3-4-3: According to the initial fitness value and the updated ICPO population parameters, the first defense strategy, the second defense strategy, the third defense strategy and the fourth defense strategy are used to update the initial ICPO population to obtain an updated ICPO population;

[0095] The formula for the first defense strategy is:

[0096]

[0097] In the formula, For the updated ICPO individuals within the first defense range; is the initial ICPO individual within the first defense range; τ1 is a random number based on normal distribution; τ2 is a random value in the interval [0,1]; It is the optimal solution within the first defense range; is the vector generated between the true optimal solution within the first defense range and the optimal solution randomly selected from the ICPO population; i' is the ICPO individual indicator; t is the iteration indicator;

[0098] The formula for the second defense strategy is:

[0099]

[0100] In the formula, For the updated ICPO individuals within the second defense range; The initial ICPO individual within the second defense range; is the search upper limit vector of the second defense range; τ3 is a random value in the interval [0,1]; are the r1th and r2th initial ICPO individuals respectively; r1 and r2 are two random integers between [1, S]; is the vector generated between the true optimal solution within the second defense range and the optimal solution randomly selected from the ICPO population;

[0101] The formula for the third defense strategy is:

[0102]

[0103] In the formula, For the updated ICPO individuals within the third defense range; The initial ICPO individual within the third defense range; is the search upper limit vector of the third defense range; are the r2th and r3th initial ICPO individuals respectively; r3 is a random integer between [1, S]; The odor diffusion factor defined for the fitness function; λ t It is a defense factor; Control parameters for search direction;

[0104] The formula for the fourth defense strategy is:

[0105]

[0106] In the formula, For the updated ICPO individuals within the fourth defense range; The initial ICPO individual within the fourth defense range; is the optimal solution within the fourth defense range; τ4 and τ5 are both random values ​​in the interval [0,1]; λ t It is a defense factor; Control parameters for search direction; is the average force affecting the search direction; a' is the convergence speed factor;

[0107] S1-3-4-4: Use the dynamic reverse learning algorithm to perform dynamic reverse learning on the updated ICPO population to generate a dynamic reverse ICPO population;

[0108] The formula is:

[0109]

[0110] In the formula, is the ICPO individual in dynamic reverse direction; γ' is the decreasing inertia coefficient; L max , L min are the maximum and minimum values ​​of the vector space respectively; For the updated ICPO individuals;

[0111] S1-3-4-5: According to the fitness function, calculate the fitness values ​​of all ICPO individuals in the updated ICPO population and the dynamically reversed ICPO population, take the ICPO individual with the minimum fitness value as the optimal individual, and retain the optimal individual;

[0112] S1-3-5: If the number of iterations of iterative optimization reaches the maximum number of iterations or the fitness value of the optimal individual meets the requirements, the optimal solution corresponding to the optimal individual is output;

[0113] S1-3-6: Decode the encoding vector of the ICPO individual of the optimal solution to obtain the optimal network parameters of the improved food recognition architecture;

[0114] S1-3-7: Optimize the improved food recognition architecture according to the optimal network parameters to obtain an initial food recognition model;

[0115] S1-4: Based on a number of pre-processed historical food image data, the initial food recognition model is trained and optimized to obtain a final food recognition model, and model metadata of the final food recognition model is extracted, including the following steps:

[0116] S1-4-1: Weightedly combine the first loss function of the feature extractor (usually the cross entropy loss function of food recognition) and the second loss function of the domain adversary (the loss function of domain classification) to obtain the joint loss function of the initial food recognition model;

[0117] S1-4-2: using a number of pre-processed historical food image data, pre-training an initial food recognition model to obtain a pre-trained food recognition model;

[0118] S1-4-3: Input the preprocessed historical food image data into the pre-trained food recognition model, and use the pre-trained feature extractor to extract the corresponding historical data features;

[0119] S1-4-4: Input the historical data features into the label predictor to perform meal identification prediction and obtain the historical meal identification label;

[0120] S1-4-5: Input the historical data features into the gradient reversal layer of the domain adversary, perform gradient reversal on the historical data features, and obtain the historical data features after gradient reversal;

[0121] S1-4-6: Input the historical data features after gradient inversion into the domain classifier of the domain adversary to perform domain classification and obtain the historical domain classification label;

[0122] S1-4-7: Based on the historical domain classification labels and historical food identification labels, a joint loss function is used to obtain the historical loss value;

[0123] S1-4-8: Based on the historical loss value, use the back propagation algorithm to update the model parameters of the pre-trained food recognition model to obtain an updated food recognition model;

[0124] S1-4-9: fine-tuning the updated food recognition model according to a number of pre-processed historical food image data until the lowest historical loss value is output, thereby obtaining the final food recognition model;

[0125] By minimizing the joint loss function, the model is trained to simultaneously perform the classification task of the source domain and learn domain-invariant features. The model is fine-tuned on the source and target domain data until the domain classifier cannot effectively distinguish the features of the two domains. The performance of the model is evaluated on the target domain to ensure that the model can accurately perform image recognition on the new domain.

[0126] S1-4-9: Extract the model metadata of the final food recognition model. The model metadata may include the model architecture, weights, hyperparameter settings, etc., to facilitate model saving, reconstruction, loading and subsequent analysis;

[0127] S2: Based on the user terminal, download the food recognition model, and update the food recognition model to obtain a customized food recognition model, including the following steps:

[0128] S2-1: based on the user terminal, downloading the model metadata of the food recognition model, and reconstructing the model according to the model metadata to obtain a reconstructed food recognition model;

[0129] S2-2: According to the target meal type set by the user terminal, a number of target historical meal image data are collected and preprocessed to obtain a number of preprocessed target historical meal image data;

[0130] S2-3: Perform secondary model training on the food recognition model according to a number of pre-processed target historical food image data to obtain a customized food recognition model;

[0131] S3: Based on the user terminal, real-time food image data is collected, and a customized food recognition model is used to perform food recognition based on the real-time food image data to obtain a real-time food recognition result, including the following steps:

[0132] S3-1: Based on the user terminal, real-time food image data is collected and pre-processed to obtain pre-processed real-time food image data;

[0133] S3-2: Inputting the preprocessed real-time food image data into the customized food recognition model, and using a feature extractor to extract real-time data features of the preprocessed real-time food image data, including the following steps:

[0134] S3-2-1: Input the pre-processed real-time food image data into the customized food recognition model;

[0135] S3-2-2: using the first feature channel, extracting real-time high-level data features of the pre-processed real-time food image data;

[0136] S3-2-3: Using the second feature channel, extracting real-time low-level data features of the pre-processed real-time food image data;

[0137] S3-2-4: According to the preset attention weight value, the real-time high-level data features and the real-time low-level data features are fused to obtain the real-time data features;

[0138] S3-3: Input the real-time data features into the domain adversary and the label predictor respectively, use the label predictor to perform meal identification, and obtain the real-time meal identification label, including the following steps:

[0139] S3-3-1: Input the real-time data features into the domain adversary and label predictor respectively;

[0140] S3-3-2: Using label predictors

[0141] S3-4: parsing the real-time food identification label to obtain the corresponding real-time food identification result;

[0142] S3-5: Use the gradient reversal layer of the domain adversary to perform gradient reversal on the real-time data features to obtain the real-time data features after gradient reversal;

[0143] S3-6: Input the real-time data features after gradient inversion into the domain classifier of the domain adversary to perform domain classification and obtain the real-time domain classification label;

[0144] S3-7: Based on the real-time domain classification label and the real-time food identification label, a joint loss function is used to obtain a real-time loss value;

[0145] S3-8: According to the real-time loss value, use the back propagation algorithm to update the model parameters of the customized food recognition model to obtain an updated customized food recognition model;

[0146] Maintaining the self-evolution function of the customized food recognition model will make the model's prediction accuracy higher and higher, and it will be better applicable to new data and learn more generalized and robust feature representations.

[0147] Embodiment 2:

[0148] like Figure 2 As shown, this embodiment provides a food identification system based on image recognition, which is used to implement a food identification method. The system includes a cloud data center and a plurality of user terminals. The cloud data center is respectively connected to the plurality of user terminals for communication;

[0149] The user terminal is used to download the food recognition model and update the food recognition model to obtain a customized food recognition model; collect real-time food image data, and use the customized food recognition model to perform food recognition based on the real-time food image data to obtain a real-time food recognition result;

[0150] The cloud data center includes a data acquisition unit, an architecture construction unit, an architecture optimization unit, and a training optimization unit connected in sequence;

[0151] A data acquisition unit, used to acquire a number of historical meal image data, and pre-process the number of historical meal image data to obtain a number of pre-processed historical meal image data;

[0152] An architecture building unit, used to build an initial food recognition architecture using an image recognition algorithm, and to improve the initial food recognition architecture using a transfer learning algorithm to obtain an improved food recognition architecture;

[0153] An architecture optimization unit, used to optimize the improved food recognition architecture using a swarm intelligence optimization algorithm to obtain an initial food recognition model;

[0154] The training optimization unit is used to train and optimize the initial food recognition model according to a number of pre-processed historical food image data to obtain a final food recognition model, and extract model metadata of the final food recognition model.

[0155] The present invention discloses a method and system for identifying food based on image recognition. The method and system utilize the characteristics of transfer learning technology and are based on a model pre-trained on a large data set, thereby reducing the dependence on a large amount of labeled food image data, effectively reducing the cost of data collection and annotation, and achieving high-performance recognition on a small amount of specific food data by fine-tuning model parameters, thereby improving the model's recognition capability for rare food categories. The method adopts an image recognition algorithm that integrates multiple features, thereby improving the generalization capability of the model under different lighting, angles, and background environments by combining high-level and low-level data features, thereby enabling the model to more accurately identify a variety of food images, and maintaining a high recognition accuracy rate even under non-ideal conditions. The method allows users to customize and update the model based on personal preferences or food types in a specific region, thereby improving the adaptability of the model. The combination of a cloud data center and a user terminal realizes rapid iteration and personalization of the model, thereby better meeting the needs of different user groups.

[0156] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0158] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0160] A person of ordinary skill in the art can understand that all or part of the steps in realizing the above-mentioned facts and methods can be completed by instructing the relevant hardware through a program, and the program involved or the program described can be stored in a computer-readable storage medium. When the program is executed, it includes the following steps: At this time, the corresponding method steps are derived, and the storage medium can be ROM / RAM, a disk, an optical disk, etc.

[0161] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying food based on image recognition, characterized in that: The steps include: Based on the cloud data center, some historical food image data are collected, and based on these historical food image data, a food recognition model is constructed using the image recognition and transfer learning fusion algorithm; Based on the user terminal, download the meal recognition model, and update the meal recognition model to obtain a customized meal recognition model; Based on the user terminal, real-time food image data is collected, and based on the real-time food image data, a customized food recognition model is used to perform food recognition to obtain real-time food recognition results.

2. The method for identifying food based on image recognition according to claim 1, characterized in that: Based on the cloud data center, some historical meal image data are collected, and based on the historical meal image data, the image recognition and transfer learning fusion algorithm are used to build a meal recognition model, including the following steps: Based on the cloud data center, a number of historical meal image data are collected, and the number of historical meal image data are preprocessed to obtain a number of preprocessed historical meal image data; Using an image recognition algorithm, an initial food recognition architecture is constructed, and using a transfer learning algorithm, the initial food recognition architecture is improved to obtain an improved food recognition architecture; Use swarm intelligence optimization algorithm to optimize the improved food recognition architecture and obtain the initial food recognition model; According to a number of preprocessed historical food image data, the initial food recognition model is trained and optimized to obtain a final food recognition model, and model metadata of the final food recognition model is extracted.

3. The method for identifying food based on image recognition according to claim 2, characterized in that: The image recognition algorithm is the LFTCNN-RF algorithm, the transfer learning algorithm is the DANN algorithm, and the swarm intelligence optimization algorithm is the ICPO optimization algorithm.

4. The method for identifying food based on image recognition according to claim 3, characterized in that: The food recognition model is constructed based on the LFTCNN-DANN-RF algorithm, and the food recognition model includes a feature extractor constructed based on the LFTCNN algorithm, a domain adversary constructed based on the DANN algorithm, and a label predictor constructed based on the RF algorithm, and the feature extractor is connected to the domain adversary and the label predictor respectively; The feature extractor includes a first feature channel and a second feature channel both constructed based on the CNN algorithm, the domain adversary includes a gradient reversal layer and a domain classification layer, the first feature channel and the second feature channel are respectively connected to the label predictor and the gradient reversal layer, and the gradient reversal layer is connected to the domain classification layer.

5. The method for identifying food based on image recognition according to claim 3, characterized in that: The improved food recognition architecture is optimized using a swarm intelligence optimization algorithm to obtain an initial food recognition model, including the following steps: The objective function is to minimize the prediction error, and the network parameters of the improved food recognition architecture are encoded as the encoding vector of the ICPO individual in the ICPO optimization algorithm; The objective function is used as the fitness function of ICPO individuals, and the ICPO population parameters and the maximum number of iterations are set; Initialization is performed according to the coding vector of the ICPO individual and the ICPO population parameters to generate a number of initial ICPO individuals of the initial ICPO population; According to the fitness function and ICPO population parameters, the ICPO optimization algorithm is used to iteratively update the initial ICPO population and retain the best individuals; If the number of iterations of iterative optimization reaches the maximum number of iterations or the fitness value of the optimal individual meets the requirements, the optimal solution corresponding to the optimal individual is output; Decode the encoding vector of the optimal ICPO individual to obtain the optimal network parameters of the improved food recognition architecture; According to the optimal network parameters, the improved food recognition architecture is optimized to obtain the initial food recognition model.

6. The method for identifying food based on image recognition according to claim 4, characterized in that: According to a number of pre-processed historical food image data, the initial food recognition model is trained and optimized to obtain a final food recognition model, and the model metadata of the final food recognition model is extracted, including the following steps: The first loss function of the feature extractor and the second loss function of the domain adversary are weighted together to obtain a joint loss function of the initial food recognition model; Using a number of preprocessed historical food image data, pre-training an initial food recognition model to obtain a pre-trained food recognition model; Input the preprocessed historical food image data into the pre-trained food recognition model, and use the pre-trained feature extractor to extract the corresponding historical data features; Input historical data features into the label predictor to perform meal identification prediction and obtain historical meal identification labels; Input the historical data features into the gradient reversal layer of the domain adversary, perform gradient reversal on the historical data features, and obtain the historical data features after gradient reversal; Input the historical data features after gradient inversion into the domain classifier of the domain adversary to perform domain classification and obtain the historical domain classification label; According to the historical domain classification labels and historical food identification labels, a joint loss function is used to obtain the historical loss value; According to the historical loss value, the back propagation algorithm is used to update the model parameters of the pre-trained food recognition model to obtain an updated food recognition model; According to a number of pre-processed historical food image data, the updated food recognition model is fine-tuned until the lowest historical loss value is output, thereby obtaining the final food recognition model; Extract the model metadata of the final food recognition model.

7. The method for identifying food based on image recognition according to claim 4, characterized in that: Based on the user terminal, the meal recognition model is downloaded and updated to obtain a customized meal recognition model, including the following steps: Based on the user terminal, download the model metadata of the food recognition model, and reconstruct the model according to the model metadata to obtain a reconstructed food recognition model; According to the target meal type set by the user terminal, a number of target historical meal image data are collected and preprocessed to obtain a number of preprocessed target historical meal image data; According to a number of pre-processed target historical meal image data, the meal recognition model is trained twice to obtain a customized meal recognition model.

8. The method for identifying food based on image recognition according to claim 6, characterized in that: Based on the user terminal, real-time food image data is collected, and based on the real-time food image data, a customized food recognition model is used to perform food recognition to obtain a real-time food recognition result, including the following steps: Based on the user terminal, real-time food image data is collected and pre-processed to obtain pre-processed real-time food image data; Input the preprocessed real-time food image data into the customized food recognition model, and use the feature extractor to extract the real-time data features of the preprocessed real-time food image data; The real-time data features are input into the domain adversary and the label predictor respectively, and the label predictor is used to perform meal identification to obtain the real-time meal identification label; Parse the real-time food identification label to obtain the corresponding real-time food identification result; Use the gradient reversal layer of the domain adversary to perform gradient reversal on the real-time data features to obtain the real-time data features after gradient reversal; Input the real-time data features after gradient inversion into the domain classifier of the domain adversary to perform domain classification and obtain the real-time domain classification label; According to the real-time domain classification label and the real-time food identification label, a joint loss function is used to obtain the real-time loss value; According to the real-time loss value, the back propagation algorithm is used to update the model parameters of the customized meal recognition model to obtain an updated customized meal recognition model.

9. The method for identifying food based on image recognition according to claim 8, characterized in that: The preprocessed real-time food image data is input into the customized food recognition model, and the real-time data features of the preprocessed real-time food image data are extracted using a feature extractor, including the following steps: Input the pre-processed real-time food image data into the customized food recognition model; Using the first feature channel, extracting real-time high-level data features of the preprocessed real-time food image data; Using the second feature channel, extracting real-time low-level data features of the pre-processed real-time food image data; According to the preset attention weight value, the real-time high-level data features and the real-time low-level data features are fused to obtain the real-time data features.

10. A food identification system based on image recognition, used to implement the food identification method according to any one of claims 1 to 9, characterized in that: The system includes a cloud data center and a plurality of user terminals, wherein the cloud data center is respectively connected to the plurality of user terminals for communication; The cloud data center includes a data collection unit, an architecture construction unit, an architecture optimization unit, and a training optimization unit which are connected in sequence.

Images