Food material feature recognition model construction and classification method and system, equipment and medium

Abstract

The invention provides a food material feature recognition model construction and classification method, system and device, and a medium. The food material feature recognition model construction method comprises the steps of obtaining target sample image data of a plurality of sample food materials; wherein the target sample image data comprises visible light image data and infrared image data; constructing a training sample data set based on the target sample image data and food material category data corresponding to the target sample image data; and inputting the training sample data set into a preset model to construct a food material feature recognition model. According to the method, the food material feature recognition model is constructed, so that the feature vectors of the food materials under different environments, different illumination conditions or different angles can be effectively recognized, comprehensive and diversified food material features are extracted, and then the types of the food materials are accurately recognized; the food materials can be accurately classified under different environments, different illumination conditions or different angles, and the universality and coverage of food material classification are improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of food classification, and in particular to a method, system, device and medium for constructing a food feature recognition model and classifying the food. Background Art

[0002] Currently, there are four main types of classification methods for cooking ingredients, namely traditional rule-based classification methods, machine learning methods, deep learning methods, and similarity-based ingredient classification methods.

[0003] Traditional food classification methods are usually based on predefined rules and features, using hand-designed feature extraction algorithms and classification rules. These methods often rely on human experience and expertise and have limited performance when faced with complex food variations and diversity.

[0004] Machine learning methods are widely used in food classification. Common approaches include support vector machines (SVMs), decision trees, and random forests. These methods can achieve a certain degree of food classification accuracy by learning features and classification models from large amounts of sample data. However, traditional machine learning methods still have certain limitations when dealing with the complex characteristics and diversity of food ingredients.

[0005] Deep learning methods have made significant progress in the field of computer vision in recent years and have demonstrated promising performance in food classification. By using multi-layer neural networks for feature learning and classification, deep learning methods can automatically extract high-level feature representations of food ingredients from large amounts of data. Commonly used deep learning models include convolutional neural networks (CNNs) and recurrent neural networks (RNNs).

[0006] Similarity-based methods are also widely used in food classification. These methods calculate the similarity or distance between images to perform classification. Common similarity metrics include Euclidean distance and cosine similarity. Similarity-based methods can better handle the variability and diversity of ingredients in some scenarios.

[0007] Existing food classification methods primarily rely on manually extracted features, requiring experts to analyze and design the characteristics of different ingredients. This approach is labor-intensive and time-consuming, and may not cover all situations when dealing with complex and diverse ingredients. Furthermore, food classification lacks universality: it may perform well in specific scenarios but generalize poorly to other scenarios. Summary of the Invention

[0008] The technical problem to be solved by the present disclosure is to overcome the defects of poor versatility and poor coverage of food classification methods in the prior art, and to provide a food feature recognition model construction and classification method, system, equipment and medium.

[0009] The present disclosure solves the above technical problems through the following technical solutions:

[0010] In a first aspect, a method for constructing a food feature recognition model is provided, the method comprising:

[0011] Acquire target sample image data of a plurality of sample food ingredients;

[0012] Wherein, the target sample image data includes visible light image data and infrared image data;

[0013] Constructing a training sample data set based on the target sample image data and the food category data corresponding to the target sample image data;

[0014] Inputting the training sample data set into a preset model to construct a food feature recognition model;

[0015] The food feature recognition model is used to identify the actual feature vector corresponding to the input actual image data of the food to be recognized.

[0016] Preferably, the method for obtaining target sample image data of a plurality of sample food ingredients includes:

[0017] Obtaining initial sample image data of a number of sample food ingredients;

[0018] Preprocessing the initial sample image data to obtain the target sample image data;

[0019] And / or, the training sample data set includes positive sample data consisting of the same food category, and negative sample data consisting of different food categories.

[0020] Preferably, the preset model includes a Transformer (attention mechanism) model, and the Transformer model includes an input layer, an intermediate layer and an output layer;

[0021] The input layer is used to receive the training sample data set, extract features from the target sample image data in the training sample data set, and output first feature data of several preset scales;

[0022] The intermediate layer is used to receive the first feature data, perform feature fusion on the first feature data, and output second feature data;

[0023] The output layer is used to receive the second feature data, perform a normalization operation on the second feature data, and output a sample feature vector corresponding to the sample food.

[0024] In a second aspect, a method for classifying ingredients is also provided, the method comprising:

[0025] Based on the food feature recognition model, a category correspondence between sample feature vectors of several food ingredients and food categories is pre-constructed;

[0026] The food feature recognition model is constructed based on the above-mentioned method for constructing the food feature recognition model;

[0027] Obtaining actual image data of the target food to be identified;

[0028] Wherein, the actual target image data includes visible light image data and infrared image data;

[0029] Inputting the target actual image data into the food feature recognition model to obtain an actual feature vector corresponding to the target actual image data of the food to be recognized;

[0030] Based on the actual feature vector and the category correspondence, the food category of the food to be identified is obtained.

[0031] Preferably, the step of obtaining the target actual image data of the food to be identified includes:

[0032] Acquiring initial actual image data of the food to be identified;

[0033] Preprocessing the initial actual image data to obtain the target actual image data;

[0034] and / or,

[0035] The step of obtaining the food category of the food to be identified based on the actual feature vector and the category correspondence includes:

[0036] Calculating the similarity between the actual feature vector and each of the sample feature vectors to obtain a number of actual similarity values;

[0037] Selecting the sample feature vector corresponding to the actual similarity value greater than a preset similarity threshold as the target feature vector;

[0038] Based on the category correspondence, the food category corresponding to the target feature vector is used as the food category of the food to be identified.

[0039] Preferably, before the step of obtaining the food category of the food to be identified based on the actual feature vector and the category correspondence, the step further includes:

[0040] When there is a new food that cannot be classified, obtaining a new sample feature vector of the new food;

[0041] The category correspondence is updated based on the new sample feature vector and the new ingredient category of the new ingredient.

[0042] In a third aspect, a system for constructing a food feature recognition model is also provided, the system comprising:

[0043] A sample image acquisition module is used to acquire target sample image data of a plurality of sample food ingredients;

[0044] Wherein, the target sample image data includes visible light image data and infrared image data;

[0045] A data set construction module, which constructs a training sample data set based on the target sample image data and the food category data corresponding to the target sample image data;

[0046] A model building module is used to input the training sample data set into a preset model to build a food feature recognition model;

[0047] The food feature recognition model is used to identify the actual feature vector corresponding to the input actual image data of the food to be recognized.

[0048] Preferably, the sample image acquisition module includes:

[0049] A sample image acquisition unit, configured to acquire initial sample image data of a plurality of sample ingredients;

[0050] a first preprocessing unit, configured to preprocess the initial sample image data to obtain the target sample image data;

[0051] And / or, the training sample data set includes positive sample data consisting of the same food category, and negative sample data consisting of different food categories.

[0052] Preferably, the preset model includes a Transformer model, and the Transformer model includes an input layer, an intermediate layer, and an output layer;

[0053] The input layer is used to receive the training sample data set, extract features from the target sample image data in the training sample data set, and output first feature data of several preset scales;

[0054] The intermediate layer is used to receive the first feature data, perform feature fusion on the first feature data, and output second feature data;

[0055] The output layer is used to receive the second feature data, perform a normalization operation on the second feature data, and output a sample feature vector corresponding to the sample food.

[0056] In a fourth aspect, a food classification system is further provided, the food classification system comprising:

[0057] A correspondence building module is used to pre-build a category correspondence between sample feature vectors of several ingredients and ingredient categories based on an ingredient feature recognition model;

[0058] Wherein, the food feature recognition model is constructed based on the above-mentioned food feature recognition model construction system;

[0059] An actual image acquisition module is used to acquire target actual image data of the food to be identified;

[0060] Wherein, the actual target image data includes visible light image data and infrared image data;

[0061] an actual feature acquisition module, configured to input the target actual image data into the food feature recognition model to obtain an actual feature vector corresponding to the target actual image data of the food to be recognized;

[0062] The food category acquisition module is used to obtain the food category of the food to be identified based on the actual feature vector and the category correspondence.

[0063] Preferably, the actual image acquisition module includes:

[0064] an actual image acquisition unit, configured to acquire initial actual image data of the food to be identified;

[0065] a second preprocessing unit, configured to preprocess the initial actual image data to obtain the target actual image data;

[0066] Preferably, the food category acquisition module includes:

[0067] A similarity obtaining unit, configured to calculate the similarity between the actual feature vector and each of the sample feature vectors to obtain a plurality of actual similarity values;

[0068] a target vector acquisition unit, configured to select the sample feature vector corresponding to the actual similarity value greater than a preset similarity threshold as a target feature vector;

[0069] The food category acquisition unit is configured to use the food category corresponding to the target feature vector as the food category of the food to be identified based on the category correspondence.

[0070] Preferably, the food classification system further includes:

[0071] A new sample acquisition unit, configured to acquire a new sample feature vector of a new food when there is a new food that cannot be classified;

[0072] An ingredient category updating unit is configured to update the category correspondence based on the new sample feature vector and the new ingredient category of the new ingredient.

[0073] In a fifth aspect, an electronic device is also provided, comprising a memory, a processor, and a computer program stored in the memory and for running on the processor. When the processor executes the computer program, it implements the method for constructing a food feature recognition model as described above, or the food classification method as described above.

[0074] In a sixth aspect, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the method for constructing a food feature recognition model as described above, or the method for classifying food as described above is implemented.

[0075] In a seventh aspect, a computer program product is also provided, comprising a computer program, which, when executed by a processor, implements the method for constructing a food feature recognition model as described above, or the method for classifying food as described above.

[0076] On the basis of conforming to the common sense in this field, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.

[0077] The positive progress of this disclosure is:

[0078] The food feature recognition model construction and classification method, system, equipment and medium disclosed in the present invention, by constructing a food feature recognition model, can effectively identify the feature vectors of food in different environments, different lighting conditions or different angles, extract comprehensive and diverse food features, and then accurately identify the category of food, thereby realizing accurate classification of food in different environments, different lighting conditions or different angles, and improving the versatility and coverage of food classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 A first flow chart of the method for constructing a food feature recognition model provided in Example 1 of the present disclosure;

[0080] Figure 2 A second flow chart of the method for constructing a food feature recognition model provided in Example 1 of the present disclosure;

[0081] Figure 3 A schematic diagram of a first process flow of the food classification method provided in Example 2 of the present disclosure;

[0082] Figure 4 A second flow chart of the food classification method provided in Example 2 of the present disclosure;

[0083] Figure 5A third flow chart of the food classification method provided in Example 2 of the present disclosure;

[0084] Figure 6 A fourth flow chart of the food classification method provided in Example 2 of the present disclosure;

[0085] Figure 7 A schematic diagram of the structure of a system for constructing a food feature recognition model provided in Example 3 of the present disclosure;

[0086] Figure 8 A schematic diagram of the structure of the food classification system provided in Example 4 of the present disclosure;

[0087] Figure 9 This is a structural diagram of an electronic device provided in Example 5 of the present disclosure. DETAILED DESCRIPTION

[0088] The present disclosure is further illustrated below by way of examples, but the present disclosure is not limited to the scope of the examples.

[0089] Example 1

[0090] This embodiment provides a method for constructing a food feature recognition model. Figure 1 As shown, the construction method includes:

[0091] S1. Obtain target sample image data of several sample food ingredients.

[0092] The target sample image data includes visible light image data and infrared image data.

[0093] S2. Construct a training sample data set based on the target sample image data and the food category data corresponding to the target sample image data.

[0094] S3. Input the training sample data set into the preset model to construct a food feature recognition model.

[0095] The food feature recognition model is used to identify the actual feature vector corresponding to the input actual image data of the food to be recognized.

[0096] Ingredient category data may also be referred to as ingredient category label data, or simply referred to as ingredient category.

[0097] The target sample image data includes multimodal data consisting of visible light image data and infrared image data. Specifically, for a certain food category, a visible light image of the food can be captured by an ordinary video camera to obtain visible light image data; an infrared image of the food can be captured by an infrared camera, and the infrared image is grayscale processed to obtain an infrared grayscale image, i.e., infrared image data, and finally a training sample data set of the food category under different lighting conditions is obtained.

[0098] Ingredients of the same category can be photographed from multiple angles to obtain visible light image data and infrared image data of the same ingredient at different angles, thereby increasing the richness and diversity of the training sample dataset.

[0099] The training sample data set is input into the preset model to construct an ingredient feature recognition model. The ingredient feature recognition model can extract comprehensive and diverse ingredient features and output the actual feature vector corresponding to the actual image data of the ingredient to be identified.

[0100] The method for constructing a food feature recognition model in this embodiment can effectively identify the feature vectors of food in different environments, different lighting conditions or different angles by constructing a food feature recognition model, and extract comprehensive and diverse food features, laying the foundation for subsequent accurate identification of food categories.

[0101] In an optional embodiment, if Figure 2 As shown, the above step S1 includes:

[0102] S11. Acquire initial sample image data of several sample food ingredients.

[0103] S12: Preprocess the initial sample image data to obtain target sample image data.

[0104] Preprocessing includes but is not limited to resizing, normalization, etc.

[0105] The initial sample image data also includes visible light image data and infrared image data. The target sample image data is obtained after preprocessing the initial sample image data.

[0106] In an optional embodiment, the training sample data set includes positive sample data consisting of the same food category, and negative sample data consisting of different food categories.

[0107] The target sample image data are labeled according to the food category. The target sample image data corresponding to the same food category constitutes the positive sample data, and the target sample image data corresponding to different food categories constitute the negative sample data, and ensure that each food category has a sufficient sample size.

[0108] In the process of building the food feature recognition model, a contrastive loss function is used for the training sample data set composed of positive sample data and negative sample data. This loss function can measure the cosine similarity between feature vectors. For each pair of positive sample data, the cosine similarity between their feature vectors is calculated and maximized. At the same time, for each pair of negative sample data, the cosine similarity between their feature vectors is calculated and minimized. At the same time, backpropagation and the Adam optimization algorithm (an optimization algorithm) are used to update the parameters of the food feature recognition model, so that the feature vectors of samples of the same food category are closer, and the feature vectors of different food categories are farther apart. This maximizes the similarity of food features of the same food category and minimizes the similarity of food features of different food categories, ensuring the accuracy and robustness of the food feature recognition model.

[0109] In an optional embodiment, the preset model includes a Transformer model, and the Transformer model includes an input layer, an intermediate layer, and an output layer;

[0110] The input layer is used to receive a training sample data set, extract features from target sample image data in the training sample data set, and output first feature data of several preset scales;

[0111] The middle layer is used to receive the first feature data, perform feature fusion on the first feature data, and output the second feature data;

[0112] The output layer is used to receive the second feature data, perform a normalization operation on the second feature data, and output a sample feature vector corresponding to the sample food.

[0113] Among them, the intermediate layer can also be called the fusion layer, and the output layer can also be called the fully connected layer.

[0114] The Transformer model performs well in processing sequence data and is suitable for feature extraction of visible light image data and infrared image data. It can fully utilize the advantages of parallel computing to improve the efficiency and real-time performance of subsequent food classification.

[0115] The input layer receives a training sample dataset, concatenates the target sample image data, and then performs multiple scale transformations to extract features, outputting first feature data of several preset scales. For example, the target sample image data is concatenated into 4-channel data, and then subjected to multiple scale transformations to generate data of three preset scales: 128×128, 256×256, and 512×512. A separate Transformer encoder is introduced for feature extraction for each scale input. Each encoder processes the data of the corresponding scale and extracts its local features, obtaining and outputting first feature data of three preset scales.

[0116] The middle layer receives the first feature data output by the input layer, performs feature fusion on the first feature data, and outputs the second feature data. Specifically, the first feature data obtained at each scale is weighted and fused based on its corresponding attention weight, outputting the second feature data. This strategy organically integrates features at different scales to capture more comprehensive and diverse information about food categories.

[0117] The output layer is used to receive the second feature data output by the intermediate layer, perform normalization on the second feature data, and output a sample feature vector corresponding to the sample food.

[0118] Specifically, the second feature data after feature fusion is mapped to a feature vector, which represents the position of the sample in the feature space. Using a normalization operation, such as L2 norm normalization (a type of normalization), the normalized feature vector has unit length, which makes the distance measurement in the feature space more accurate. The cosine similarity between different samples can be directly calculated by their inner product, without being affected by the scaling of the feature vector.

[0119] By fusing the features of visible light image data with those of infrared image data and extracting feature vectors using a deep learning network, this integrated feature representation can more comprehensively and multi-facetedly describe the characteristics of food ingredients, improving the accuracy and robustness of the food feature recognition model.

[0120] The method for constructing a food feature recognition model in this embodiment uses the Transformer model as the basis to construct a food feature recognition model, which can extract comprehensive and diverse food features, improve the accuracy and robustness of the food feature recognition model, and effectively identify the feature vectors of food in different environments, different lighting conditions or different angles, laying the foundation for the subsequent accurate identification of food categories.

[0121] Example 2

[0122] This embodiment provides a method for classifying food materials. Figure 3As shown, the methods for separating ingredients include:

[0123] S4. Based on the food feature recognition model, a category correspondence between sample feature vectors of several food ingredients and food categories is pre-constructed.

[0124] The food feature recognition model is constructed based on the method for constructing the food feature recognition model in Example 1.

[0125] S5. Obtain target actual image data of the food to be identified.

[0126] The actual target image data includes visible light image data and infrared image data.

[0127] S6. Input the target actual image data into the food feature recognition model to obtain the actual feature vector corresponding to the target actual image data of the food to be recognized.

[0128] S7. Based on the actual feature vector and the category correspondence, obtain the category of the food to be identified.

[0129] Based on the food feature recognition model, the category correspondence between the sample feature vectors of several food ingredients and the food category can be constructed in advance.

[0130] Specifically, a category correspondence between the sample feature vectors of several ingredients and the ingredient categories is pre-constructed in the database. For example, if the ingredient category is sweet potato, a category correspondence between sweet potato and its feature vector can be established; if the ingredient category is potato, a category correspondence between potato and its feature vector can be established.

[0131] The food classification method of this embodiment is based on the food feature recognition model of Example 1. Based on the food feature recognition model, a category correspondence between sample feature vectors of several food ingredients and food categories is pre-constructed. By outputting the target actual image data of the food to be identified to the food feature recognition model, the actual feature vector corresponding to the target actual image data of the food to be identified can be obtained. Then, based on the actual feature vector and category correspondence, the food category of the food to be identified can be obtained. The feature vectors of food ingredients in different environments, different lighting conditions, or different angles can be effectively identified, and the category of the food ingredients can be accurately identified. This achieves accurate classification of food ingredients in different environments, different lighting conditions, or different angles, and improves the versatility and coverage of food classification.

[0132] In an optional embodiment, if Figure 4 As shown, the above step S5 includes:

[0133] S51: Acquire initial actual image data of the food to be identified.

[0134] S52: Preprocess the initial actual image data to obtain target actual image data.

[0135] Preprocessing includes but is not limited to resizing, normalization, etc.

[0136] The preprocessing in this embodiment is the same as the preprocessing operation on the initial sample image data in Example 1.

[0137] The initial actual image data also includes visible light image data and infrared image data. The target actual image data is obtained after preprocessing the initial actual image data.

[0138] In an optional embodiment, if Figure 5 As shown, the above step S7 includes:

[0139] S71. Calculate the similarity between the actual feature vector and each sample feature vector to obtain several actual similarity values.

[0140] S72: Select a sample feature vector corresponding to an actual similarity value greater than a preset similarity threshold as a target feature vector.

[0141] S73. Based on the category correspondence, the food category corresponding to the target feature vector is used as the food category of the food to be identified.

[0142] The similarity between the actual feature vector and each sample feature vector is calculated to obtain several actual similarity values. For example, five actual similarity values ​​are obtained, with the first to fifth actual similarity values ​​being 10%, 15%, 20%, 30%, and 90%, respectively. If the preset similarity threshold is 85%, and the fifth actual similarity value of 90% is greater than the preset similarity threshold of 85%, the sample feature vector corresponding to the fifth actual similarity value is used as the target feature vector. If the food category corresponding to the target feature vector is tomato, the food category of the food to be identified is tomato.

[0143] The above values ​​are only exemplary, and those skilled in the art can flexibly set the preset similarity threshold according to actual conditions.

[0144] The food classification method of this embodiment calculates the similarity between the actual feature vector and each sample feature vector, selects the sample feature vector corresponding to the actual similarity value greater than the preset similarity threshold as the target feature vector, and based on the category correspondence, uses the food category corresponding to the target feature vector as the food category of the food to be identified. The food category of the food to be identified can be accurately obtained, and accurate classification of food is achieved in different environments, different lighting conditions or different angles.

[0145] In an optional embodiment, if Figure 6 As shown, before step S7, the above step also includes:

[0146] S8. When there is a new food that cannot be classified, obtain a new sample feature vector of the new food.

[0147] S9. Update the category correspondence based on the new sample feature vector and the new ingredient category of the new ingredient.

[0148] When there are new ingredients that cannot be classified, the new sample feature vector of the new ingredient is obtained through the ingredient feature recognition model in Example 1, and then the category correspondence is updated according to the new sample feature vector and the new ingredient category of the new ingredient; the recognition ability of new ingredients can be continuously improved, so that the overall recognition accuracy is improved.

[0149] The food classification method of this embodiment realizes the update of category correspondence, improves the versatility, coverage and diversity of food classification, can continuously improve the ability to identify new ingredients, and realizes accurate classification of ingredients in different environments, different lighting conditions or different angles.

[0150] Example 3

[0151] This embodiment provides a system for constructing a food feature recognition model. Figure 7 As shown, the build system includes:

[0152] The sample image acquisition module 1 is used to acquire target sample image data of a plurality of sample food ingredients;

[0153] Wherein, the target sample image data includes visible light image data and infrared image data;

[0154] Dataset construction module 2, constructing a training sample dataset based on the target sample image data and the food category data corresponding to the target sample image data;

[0155] Model building module 3, used to input the training sample data set into the preset model to build a food feature recognition model;

[0156] The food feature recognition model is used to identify the actual feature vector corresponding to the input actual image data of the food to be recognized.

[0157] In an optional embodiment, the sample image acquisition module 1 includes:

[0158] The sample image acquisition unit 11 is used to acquire initial sample image data of a plurality of sample ingredients;

[0159] The first pre-processing unit 12 is configured to pre-process the initial sample image data to obtain target sample image data.

[0160] In an optional embodiment, the training sample data set includes positive sample data consisting of the same food category, and negative sample data consisting of different food categories.

[0161] In an optional embodiment, the preset model includes a Transformer model, and the Transformer model includes an input layer, an intermediate layer, and an output layer;

[0162] The input layer is used to receive a training sample data set, extract features from target sample image data in the training sample data set, and output first feature data of several preset scales;

[0163] The middle layer is used to receive the first feature data, perform feature fusion on the first feature data, and output the second feature data;

[0164] The output layer is used to receive the second feature data, perform a normalization operation on the second feature data, and output a sample feature vector corresponding to the sample food.

[0165] The food feature recognition model construction system of this embodiment corresponds to the food feature recognition model construction method in Example 1. The working principle of the construction system is the same as the working principle of the construction method, which will not be repeated here.

[0166] The food feature recognition model construction system of this embodiment can effectively identify the feature vectors of food in different environments, different lighting conditions or different angles by constructing the food feature recognition model, laying the foundation for the subsequent accurate identification of the food category.

[0167] Example 4

[0168] This embodiment provides a food classification system, such as Figure 8 As shown, the food system includes:

[0169] A correspondence building module 4 is used to pre-build a category correspondence between sample feature vectors of a number of ingredients and ingredient categories based on an ingredient feature recognition model;

[0170] The food feature recognition model is constructed based on the food feature recognition model construction system in Example 3;

[0171] The actual image acquisition module 5 is used to acquire the target actual image data of the food to be identified;

[0172] The actual target image data includes visible light image data and infrared image data;

[0173] The actual feature acquisition module 6 is used to input the target actual image data into the food feature recognition model to obtain the actual feature vector corresponding to the target actual image data of the food to be recognized;

[0174] The food category acquisition module 7 is used to obtain the food category of the food to be identified based on the actual feature vector and the category correspondence.

[0175] In an optional embodiment, the actual image acquisition module 5 includes:

[0176] The actual image acquisition unit 51 is used to acquire initial actual image data of the food to be identified;

[0177] A second pre-processing unit 52 is used to pre-process the initial actual image data to obtain target actual image data;

[0178] In an optional embodiment, the food category acquisition module 7 includes:

[0179] A similarity obtaining unit 71 is used to calculate the similarity between the actual feature vector and each sample feature vector to obtain a number of actual similarity values;

[0180] A target vector acquisition unit 72 is configured to select a sample feature vector corresponding to an actual similarity value greater than a preset similarity threshold as a target feature vector;

[0181] The food category acquisition unit 73 is configured to use the food category corresponding to the target feature vector as the food category of the food to be identified based on the category correspondence.

[0182] In an optional embodiment, the food classification system further includes:

[0183] A new sample acquisition unit 8 is used to acquire a new sample feature vector of a new food when there is a new food that cannot be classified;

[0184] The ingredient category updating unit 9 is configured to update the category correspondence based on the new sample feature vector and the new ingredient category of the new ingredient.

[0185] The food classification system of this embodiment corresponds to the food classification method in Example 3. The working principle of the food classification system is the same as that of the food classification method, and will not be repeated here.

[0186] The food classification system of this embodiment is based on the food feature recognition model of Example 3. Based on the food feature recognition model, a category correspondence between sample feature vectors of several food ingredients and food categories is pre-established. By outputting the target actual image data of the food to be identified to the food feature recognition model, the actual feature vector corresponding to the target actual image data of the food to be identified can be obtained. Then, based on the actual feature vector and category correspondence, the food category of the food to be identified can be obtained. This system can effectively identify the feature vectors of food ingredients in different environments, different lighting conditions, or different angles, and thus accurately identify the food category. This improves the versatility and coverage of food classification and enables accurate classification of food ingredients in different environments, different lighting conditions, or different angles.

[0187] Example 5

[0188] This embodiment provides an electronic device, Figure 9 This is a structural diagram of an electronic device provided in this embodiment, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and for running on the processor. When the processor executes the computer program, the method for constructing the food feature recognition model in the above-mentioned embodiment 1, or the food classification method in the above-mentioned embodiment 2 is implemented. Figure 9 The electronic device 70 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present disclosure. Figure 9 As shown, the electronic device 70 may be a general-purpose computing device, such as a server device. Components of the electronic device 70 may include, but are not limited to, the at least one processor 71, the at least one memory 72, and a bus 73 connecting different system components (including the memory 72 and the processor 71).

[0189] The bus 73 includes a data bus, an address bus, and a control bus.

[0190] The memory 72 may include a volatile memory, such as a random access memory (RAM) 721 and / or a cache memory 722 , and may further include a read-only memory (ROM) 723 .

[0191] The memory 72 may also include a program tool 725 (or utility) having a set (at least one) of program modules 724, such program modules 724 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0192] The processor 71 executes various functional applications and data processing by running computer programs stored in the memory 72, such as the method for constructing the food feature recognition model in the above embodiment 1, or the food classification method in the above embodiment 2.

[0193] The electronic device 70 can also communicate with one or more external devices 74. Such communication can be performed through an input / output (I / O) interface 75. In addition, the model generating electronic device 70 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and / or a public network such as the Internet) through a network adapter 76. Figure 9 As shown, the network adapter 76 communicates with other modules of the electronic device 70 via the bus 73. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 70, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.

[0194] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0195] Example 6

[0196] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the method for constructing the food feature recognition model in the above-mentioned embodiment 1, or the food classification method in the above-mentioned embodiment 2.

[0197] The readable storage medium may include, but is not limited to, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0198] In a possible implementation, the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of the method for constructing the food feature recognition model in the above-mentioned embodiment 1, or the steps of the food classification method in the above-mentioned embodiment 2.

[0199] The program code for executing the present disclosure may be written in any combination of one or more programming languages, and may be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on the remote device.

[0200] Example 7

[0201] This embodiment provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the method for constructing the food feature recognition model in the above-mentioned embodiment 1, or the food classification method in the above-mentioned embodiment 2.

[0202] The program code for executing the present disclosure may be written in any combination of one or more programming languages, and may be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on the remote device.

[0203] In some embodiments, a computer program product may take the form of a program, software, a software module, a script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0204] As an implementable manner, a computer program product may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (for example, files storing one or more modules, subroutines, or code portions).

[0205] As an implementation method, the computer program product can be deployed to be executed on one electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed in multiple locations and interconnected by a communication network.

[0206] While specific embodiments of the present disclosure have been described above, those skilled in the art will appreciate that these are merely illustrative and that the scope of protection of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, and such changes and modifications are intended to fall within the scope of protection of the present disclosure.

Claims

1. A method for constructing a food feature recognition model, characterized in that: The construction method comprises: Acquire target sample image data of a plurality of sample food ingredients; Wherein, the target sample image data includes visible light image data and infrared image data; Constructing a training sample data set based on the target sample image data and the food category data corresponding to the target sample image data; Inputting the training sample data set into a preset model to construct a food feature recognition model; The food feature recognition model is used to identify the actual feature vector corresponding to the input actual image data of the food to be recognized.

2. The construction method according to claim 1, characterized in that The method for obtaining target sample image data of a plurality of sample food ingredients comprises: Obtaining initial sample image data of a number of sample food ingredients; Preprocessing the initial sample image data to obtain the target sample image data; And / or, the training sample data set includes positive sample data consisting of the same food category, and negative sample data consisting of different food categories.

3. The construction method according to claim 1, characterized in that The preset model includes a Transformer model, and the Transformer model includes an input layer, an intermediate layer, and an output layer; The input layer is used to receive the training sample data set, extract features from the target sample image data in the training sample data set, and output first feature data of several preset scales; The intermediate layer is used to receive the first feature data, perform feature fusion on the first feature data, and output second feature data; The output layer is used to receive the second feature data, perform a normalization operation on the second feature data, and output a sample feature vector corresponding to the sample food.

4. A food material classification method, characterized in that: The food separation method includes: Based on the food feature recognition model, a category correspondence between sample feature vectors of several food ingredients and food categories is pre-constructed; The food feature recognition model is constructed based on the method for constructing a food feature recognition model according to any one of claims 1 to 3; Obtaining actual image data of the target food to be identified; Wherein, the actual target image data includes visible light image data and infrared image data; Inputting the target actual image data into the food feature recognition model to obtain an actual feature vector corresponding to the target actual image data of the food to be recognized; Based on the actual feature vector and the category correspondence, the food category of the food to be identified is obtained.

5. The food classification method according to claim 4, characterized in that: The step of obtaining the target actual image data of the food to be identified comprises: Acquiring initial actual image data of the food to be identified; Preprocessing the initial actual image data to obtain the target actual image data; and / or, The step of obtaining the food category of the food to be identified based on the actual feature vector and the category correspondence includes: Calculating the similarity between the actual feature vector and each of the sample feature vectors to obtain a number of actual similarity values; Selecting the sample feature vector corresponding to the actual similarity value greater than a preset similarity threshold as the target feature vector; Based on the category correspondence, the food category corresponding to the target feature vector is used as the food category of the food to be identified.

6. The food classification method according to claim 4, characterized in that: Before the step of obtaining the category of the food to be identified based on the actual feature vector and the category correspondence, the step further includes: When there is a new food that cannot be classified, obtaining a new sample feature vector of the new food; The category correspondence is updated based on the new sample feature vector and the new ingredient category of the new ingredient.

7. A system for constructing a food feature recognition model, characterized in that: The build system includes: A sample image acquisition module is used to acquire target sample image data of a plurality of sample food ingredients; Wherein, the target sample image data includes visible light image data and infrared image data; A data set construction module, which constructs a training sample data set based on the target sample image data and the food category data corresponding to the target sample image data; A model building module is used to input the training sample data set into a preset model to build a food feature recognition model; The food feature recognition model is used to identify the actual feature vector corresponding to the input actual image data of the food to be recognized.

8. A food classification system, characterized in that: The food separation system includes: A correspondence building module is used to pre-build a category correspondence between sample feature vectors of several ingredients and ingredient categories based on an ingredient feature recognition model; Wherein, the food feature recognition model is constructed based on the food feature recognition model construction system according to claim 7; An actual image acquisition module is used to acquire target actual image data of the food to be identified; Wherein, the actual target image data includes visible light image data and infrared image data; an actual feature acquisition module, configured to input the target actual image data into the food feature recognition model to obtain an actual feature vector corresponding to the target actual image data of the food to be recognized; The food category acquisition module is used to obtain the food category of the food to be identified based on the actual feature vector and the category correspondence.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and configured to run on the processor, wherein: When the processor executes the computer program, it implements the method for constructing the food feature recognition model according to any one of claims 1 to 3, or the food classification method according to any one of claims 4 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for constructing a food feature recognition model according to any one of claims 1 to 3 or the method for classifying food according to any one of claims 4 to 6 is implemented.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for constructing a food feature recognition model according to any one of claims 1 to 3 or the method for classifying food according to any one of claims 4 to 6 is implemented.

Images