Food maturity identification method and device based on multi-sensor fusion

Abstract

The embodiment of the invention discloses a food maturity identification method and device based on multi-sensor fusion. The method comprises the steps of obtaining first collection data of a sensor for a cooking object, and constructing a feature matrix according to the first collection data; setting a target label representing maturity for each row of the feature matrix, and processing the feature matrix based on a preset feature selection rule to obtain training data; training the initial SVM model based on the training data to obtain a trained SVM model; and when a maturity identification instruction is detected, acquiring second acquisition data of the sensor for the current cooking object, processing the second acquisition data based on the trained SVM model to obtain a classification result representing the target label, and displaying the classification result. According to the embodiment of the invention, the maturity is identified by fusing the data acquired by the plurality of sensors, so that the food maturity is more accurately judged, and the requirement of a user on the maturity is more easily met.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to maturity recognition technology, and more particularly, to a method and device for food maturity recognition based on multi-sensor fusion. Background Art

[0002] As the quality of life improves, users have higher requirements for the quality of cooking products, hoping that the cooked food can be presented at the best maturity to meet the dual needs of health and deliciousness. Traditional cooking methods often rely on the user's experience and intuition, especially when the user's cooking experience is insufficient, which can easily lead to inaccurate judgment of food maturity, resulting in overcooked or unevenly cooked food, and unable to meet the user's maturity requirements. Summary of the invention

[0003] In order to solve the above problems, one or more embodiments of this specification describe a method and device for identifying food maturity based on multi-sensor fusion.

[0004] According to a first aspect, a method for identifying food maturity based on multi-sensor fusion is provided, the method comprising:

[0005] Acquire first collected data of the sensor for the cooking object, and construct a feature matrix according to the first collected data, wherein the collected data includes image data, temperature data and gas data, each row of the feature matrix represents a cooking experiment, and each column of the feature matrix represents a type of collected data;

[0006] Setting a target label representing maturity for each row of the feature matrix, and processing the feature matrix based on a preset feature selection rule to obtain training data, wherein the preset feature selection rule includes at least one of a principal component analysis method, a recursive feature elimination method, and a correlation analysis method;

[0007] Training an initial SVM model based on the training data to obtain a trained SVM model;

[0008] When a maturity recognition instruction is detected, second collected data of the sensor for the current cooking object is obtained, the second collected data is processed based on the trained SVM model, a classification result representing the target label is obtained, and the classification result is displayed.

[0009] Preferably, the image data includes color features, texture features and edge features of the food, the temperature data includes temperature features and temperature change trend features at different time points, and the gas data includes time series features of volatile organic compounds.

[0010] Preferably, before processing the feature matrix based on a preset feature selection rule to obtain training data, the method further includes:

[0011] The feature matrix is ​​preprocessed, and the preprocessing includes normalization, missing value processing and data balancing.

[0012] Preferably, before training the initial SVM model based on the training data to obtain the trained SVM model, the method further includes:

[0013] An initial SVM model is constructed, a kernel function of the initial SVM model is set to a Gaussian radial basis kernel, and an optimal hyperparameter combination of the initial SVM model is determined based on a grid search or a random search.

[0014] Preferably, when the maturity recognition instruction is detected, second collected data of the sensor for the current cooking object is obtained, and the second collected data is processed based on the trained SVM model to obtain a classification result representing the target label, including:

[0015] When a maturity recognition instruction is detected, a cooking method of a current cooking object is determined, and a target SVM model matching the current cooking object and the cooking method is selected from among the trained SVM models;

[0016] Acquire second collected data of the sensor for the current cooking object, process the second collected data based on the target SVM model, and obtain a classification result representing the target label.

[0017] Preferably, the collected data also includes ingredient combination data, volume data and area data, the ingredient combination data includes vector features characterizing the types of ingredient combinations, the area data includes area features of a single ingredient, and the volume data includes volume features of a single ingredient.

[0018] Preferably, the processing of the second collected data based on the trained SVM model to obtain a classification result representing the target label includes:

[0019] Determine a first collection point of the gas data in the first collection data and a second collection point of the gas data in the second collection data;

[0020] Determine the gas escape ratio based on the position change data of the second acquisition point relative to the first acquisition point, and adjust the second acquisition data based on the gas escape ratio to obtain third acquisition data;

[0021] The third collected data is processed based on the trained SVM model to obtain a classification result representing the target label.

[0022] According to a second aspect, a device for identifying food maturity based on multi-sensor fusion is provided, the device comprising:

[0023] A first acquisition module, configured to acquire first collected data of a sensor for a cooking object, and construct a feature matrix according to the first collected data, wherein the collected data includes image data, temperature data, and gas data, wherein each row of the feature matrix represents a cooking experiment, and each column of the feature matrix represents a type of collected data;

[0024] A setting module, used to set a target label representing maturity for each row of the feature matrix, and process the feature matrix based on a preset feature selection rule to obtain training data, wherein the preset feature selection rule includes at least one of a principal component analysis method, a recursive feature elimination method, and a correlation analysis method;

[0025] A training module, used for training an initial SVM model based on the training data to obtain a trained SVM model;

[0026] The second acquisition module is used to acquire the second collected data of the sensor for the current cooking object when a maturity recognition instruction is detected, process the second collected data based on the trained SVM model, obtain a classification result representing the target label, and display the classification result.

[0027] According to a third aspect, there is provided an electronic device, comprising a processor and a memory;

[0028] The processor is connected to the memory;

[0029] The memory is used to store executable program code;

[0030] The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the steps of the method provided in the first aspect or any possible implementation manner of the first aspect.

[0031] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and the computer-readable storage medium stores instructions, which, when the instructions are executed on a computer or a processor, cause the computer or the processor to execute the method provided in the first aspect or any possible implementation of the first aspect.

[0032] The solution provided in the embodiments of this specification can construct a feature matrix based on image data, temperature data and gas data, and train an SVM model with the training data obtained after processing the feature matrix through preset feature selection rules. Finally, the maturity of the current cooking object is intelligently identified based on the trained SVM model, and the maturity is identified by fusing data collected by multiple sensors. The judgment of food maturity is more accurate and it is easier to meet the user's requirements for maturity. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 It is a flowchart of a method for identifying food maturity based on multi-sensor fusion provided in one embodiment of this specification.

[0035] Figure 2 A schematic diagram of the structure of a food maturity recognition device based on multi-sensor fusion provided in one embodiment of the present specification.

[0036] Figure 3 A schematic diagram of the structure of an electronic device provided in one embodiment of this specification. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0038] In the following introduction, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The following introduction provides multiple embodiments of the present application, and different embodiments can be replaced or combined, so the present application can also be considered to include all possible combinations of the same and / or different embodiments recorded. Therefore, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then the present application should also be considered to include embodiments containing one or more of all other possible combinations of A, B, C, and D, even though the embodiment may not be clearly recorded in the following text.

[0039] The following description provides examples and does not limit the scope, applicability or examples set forth in the claims. Changes may be made to the functions and arrangements of the elements described without departing from the scope of the present application. Various processes or components may be appropriately omitted, substituted or added to each example. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted or combined. In addition, features described in some examples may be combined in other examples.

[0040] See also Figure 1 , Figure 1 : is a flow chart of a method for identifying food maturity based on multi-sensor fusion provided in an embodiment of the present application. In an embodiment of the present application, the method includes:

[0041] S101, obtaining first collected data of a sensor for a cooking object, and constructing a feature matrix according to the first collected data.

[0042] The collected data includes image data, temperature data and gas data, each row of the feature matrix represents a cooking experiment, and each column of the feature matrix represents a type of collected data.

[0043] The execution subject of the present application may be a processor, which may be arranged in a cooking device, in an integrated kitchen control system, or in a cloud server, etc.

[0044] In the embodiments of the present specification, the cooking object may be the food material in the cooking device, or the cooking device itself, etc. Different types of sensors, such as image sensors, temperature sensors, gas sensors, etc., may be provided in the cooking device or the integrated kitchen system including the cooking device to respectively collect the corresponding collected data of the cooking object. The first collected data may be the collected data obtained during the experiment for training the SVM model, or the historical collected data. The processor first obtains the first collected data collected by the sensor for the cooking object, and thereby constructs a feature matrix, where each row of the feature matrix represents a cooking experiment, and each column of the feature matrix represents a type of collected data. In addition, the cooking objects are of different types, so the first collected data of each cooking object may be collected separately, and an SVM model may be trained for each cooking object in the subsequent process.

[0045] In one possible implementation, the image data includes color features, texture features, and edge features of the food, the temperature data includes temperature features and temperature change trend features at different time points, and the gas data includes time series features of volatile organic compounds.

[0046] In the embodiments of this specification, image data is data obtained by extracting image features from image information collected by image sensors such as cameras, generally including color features, texture features, edge features, etc. Temperature data is the temperature features at different time points during the food cooking process collected by temperature sensors, and the temperature change trend can also be used as an additional feature. Gas data is the volatile organic compounds (VOCs) released during the food cooking process captured by gas sensors. Different foods will volatilize different types of organic compounds. These data can be represented as feature vectors of time series to extract values ​​at specific time points for representation.

[0047] In other possible implementations, the image data may also include shape features, spectral features, spatial features, positional features, etc. If it is necessary to collect spectral features, it is necessary to set a spectrum generator that can emit light of different wavelengths in the image sensor, or set a spectrum generator separately. Among them, shape features may include the geometric shape of the food, such as round, elliptical, irregular, etc.; spectral features may include reflection or transmission characteristics under light of different wavelengths; spatial features may include the spatial relative positions between multiple food objects; positional features may include the position of a single food object in the cooking device, so as to take into account the different cooking temperatures at different positions of certain cooking devices.

[0048] S102: Setting a target label representing maturity for each row of the feature matrix, and processing the feature matrix based on a preset feature selection rule to obtain training data.

[0049] Wherein, the preset feature selection rule includes at least one of a principal component analysis method, a recursive feature elimination method and a correlation analysis method.

[0050] In the embodiment of this specification, according to the actual results of each test, the staff will also set a target label representing maturity for each row in the feature matrix. The specific content of the target label can be cooked, half-cooked, three-way cooked, over-cooked, raw, etc. Next, considering that a large number of collected data are collected by different types of sensors, so that there are many features in the feature matrix, in order to improve the efficiency and generalization ability of the model, feature selection is also required to screen out more relevant features. At least one feature selection rule will be pre-set in the processor, and the feature matrix will be processed according to the set rules to obtain the processed training data. Among them, the principal component analysis method can retain the main characteristic components of the data by reducing the dimension while removing noise. The recursive feature elimination method can gradually remove the most irrelevant features until the feature subset that contributes the most to the model performance is found. The correlation analysis method can calculate the correlation between the feature and the target label and delete the features with low correlation.

[0051] In one possible implementation, before processing the feature matrix based on a preset feature selection rule to obtain training data, the process further includes:

[0052] The feature matrix is ​​preprocessed, and the preprocessing includes normalization, missing value processing and data balancing.

[0053] In the embodiments of this specification, in order to improve the performance of the model, it is necessary to preprocess the feature data to ensure that the input data has good quality, wherein the preprocessing may include normalization, missing value processing, data balancing, etc. Normalization is because SVM is very sensitive to the scale of the features, and all features need to be normalized, and z-score standardization or minimum-maximum normalization can usually be used. Missing value processing is because there may be some missing data during the data collection process, and these missing values ​​need to be processed by interpolation, mean substitution or deletion of missing samples. Data balancing is because if there are fewer samples at certain maturity levels, SVM training may be affected by unbalanced data, so the data set can be balanced by oversampling or undersampling, and more synthetic samples can be generated using algorithms such as SMOTE.

[0054] S103: training an initial SVM model based on the training data to obtain a trained SVM model.

[0055] In the embodiment of this specification, the SVM model separates data of different categories by finding an optimal decision boundary (i.e., a hyperplane) in a high-dimensional space, and its goal is to find the boundary that makes the boundary point farthest from the hyperplane, that is, to maximize the distance between classes. For the linearly separable case, it can be solved directly according to the set linear formula. For the nonlinearly separable case, SVM will use a kernel function to map the data to a higher dimensional space, and realize linear segmentation in this space. Commonly used kernel functions include linear kernels suitable for linearly separable cases, polynomial kernels suitable for polynomial relationships, Gaussian radial basis kernels (RBF kernels) suitable for complex nonlinear relationships, and sigmoid kernels commonly used to simulate the effects of neural networks. The staff selects the corresponding linear formula and kernel function in advance according to actual needs, constructs the initial SVM model, and trains the initial SVM model according to the processed training data to obtain a trained SVM model. The goal of model training is to find support vectors and decision boundaries. SVM will automatically select support vectors, i.e., data points located near the dividing line between classes.

[0056] In one possible implementation, before training the initial SVM model based on the training data to obtain the trained SVM model, the method further includes:

[0057] An initial SVM model is constructed, a kernel function of the initial SVM model is set to a Gaussian radial basis kernel, and an optimal hyperparameter combination of the initial SVM model is determined based on a grid search or a random search.

[0058] In the embodiments of this specification, the maturity classification problem of food generally has a complex nonlinear relationship, and the Gaussian radial basis kernel is usually selected as the kernel function. In addition, before starting to train the model, it is necessary to set and adjust the hyperparameters of the model. SVM has two main hyperparameters, one is the penalty coefficient C that determines the degree of penalty for misclassification, and the other is y / gammay that controls the influence range of a single training sample. The processor can optimize these hyperparameters by grid search or random search, and find the optimal hyperparameter combination in combination with cross validation.

[0059] S104. When a maturity recognition instruction is detected, obtain second collected data of the sensor for the current cooking object, process the second collected data based on the trained SVM model, obtain a classification result representing the target label, and display the classification result.

[0060] In the embodiments of this specification, after the model is trained, in actual applications, when the user needs to perform intelligent recognition of maturity, the user will send a maturity recognition instruction to the processor through a touch screen or a wirelessly connected control terminal. After receiving the maturity recognition instruction, the processor will continue to obtain the second collected data of the sensor for the current cooking object, and input the second collected data into the trained SVM model to obtain the classification result output by the model. The classification result can characterize the target label corresponding to each second collected data, and then determine the maturity of the food at this time according to the content of the target label, and display the classification result, so that the user can understand the real-time maturity of the ingredients at any time during the cooking process, so that the user can accurately cook food that meets their own maturity preferences.

[0061] In one possible implementation, when the maturity recognition instruction is detected, obtaining second collected data of the sensor for the current cooking object, processing the second collected data based on the trained SVM model, and obtaining a classification result representing the target label includes:

[0062] When a maturity recognition instruction is detected, a cooking method of a current cooking object is determined, and a target SVM model matching the current cooking object and the cooking method is selected from among the trained SVM models;

[0063] Acquire second collected data of the sensor for the current cooking object, process the second collected data based on the target SVM model, and obtain a classification result representing the target label.

[0064] In the embodiments of this specification, for the same food, different cooking methods such as frying, stir-frying, roasting, steaming, etc. will also affect the type and amount of VOCs. For example, generally speaking, high-temperature frying or roasting is more likely to produce more VOCs than low-temperature steaming. Therefore, in order to accurately identify the maturity of food, when training the SVM model, a column can be added to the feature matrix to characterize the classification of cooking methods, or SVM models corresponding to different cooking methods can be trained separately. In addition, depending on the cooking object, SVM models can also be trained separately for each cooking object. After detecting the maturity recognition instruction, the processor will obtain the working information of the currently working cooking equipment to determine the cooking method of the current cooking object, and then select the target SVM model that can match the current cooking object and cooking method at the same time from the trained SVM models, and use this model to process the second collected data to finally obtain the classification result.

[0065] In one possible implementation, the collected data also includes ingredient combination data, volume data and area data, the ingredient combination data includes vector features characterizing the types of ingredient combinations, the area data includes area features of individual ingredients, and the volume data includes volume features of individual ingredients.

[0066] In the embodiment of the present specification, the collected data may also include food combination data, volume data and area data. These three data can be obtained after image recognition based on the image information collected by the image sensor such as the camera, wherein the volume data requires at least two image information collected by at least two image sensors arranged at different positions to be obtained by image recognition. The reason why these three types of collected data are considered additionally is that when cooking, it may not be cooking a single type of food, but cooking a variety of food at the same time, and a variety of food may react to generate different VOCs. In addition, the properties of the processed food (such as silk, slices, blocks) will affect the heating area of ​​the food and the temperature change rate of different positions of the food, which will also affect the maturity speed and maturity state of the food. Therefore, the collected data may also include these three types of data, so that the trained SVM model will also consider the feature data of these three dimensions, and the result of maturity recognition is more accurate. Among them, different numbers can be set for different food ingredients, and the numbers can be combined according to the combination of food ingredients to generate corresponding vector features. According to the area value of a single food ingredient, the feature value is set, and the corresponding area feature can be generated. Similarly, according to the volume value of a single food ingredient, the feature value is set, and the corresponding volume feature can be generated.

[0067] Taking into account that the area feature and the volume feature will jointly affect the ripening speed of the food, in other possible implementations, it is not necessary to consider the area feature or the volume feature separately. Instead, the area feature and the volume feature can be integrated into a new feature by weighting, splicing, etc., and the new feature is used to train the SVM model.

[0068] In one possible implementation, the processing of the second collected data based on the trained SVM model to obtain a classification result representing the target label includes:

[0069] Determine a first collection point of the gas data in the first collection data and a second collection point of the gas data in the second collection data;

[0070] Determine the gas escape ratio based on the position change data of the second acquisition point relative to the first acquisition point, and adjust the second acquisition data based on the gas escape ratio to obtain third acquisition data;

[0071] The third collected data is processed based on the trained SVM model to obtain a classification result representing the target label.

[0072] In the embodiments of the present specification, cooking equipment such as a wok is not completely closed and will come into contact with the air, resulting in different gas data corresponding to VOCs collected at different locations due to the escape of gas in the air, even if the same ingredients are cooked under the same conditions. Different users have different brands and models of kitchen integration systems or purchased cooking equipment, which may lead to differences in the locations where the gas sensors are set, and thus may lead to differences in the detected gas data, affecting the accuracy of maturity recognition. Therefore, each SVM model can also be attached with a first collection point of gas data in the training data of the model. The processor can obtain the actual second collection point of the gas sensor in the user's home through interaction with the gas sensor, and calculate the position change data of the first collection point and the second collection point, and then determine the gas escape ratio, thereby adjusting the gas data in the second collection data to obtain the third collection data, and finally obtain the classification result by processing the third collection data to ensure the accuracy of maturity recognition. Among them, the mapping relationship between the position change data and the gas escape ratio can be set according to the test data after corresponding experiments are carried out in advance, or it can be set directly according to experience. The actual situation will directly determine the gas escape ratio corresponding to the position change data according to the set mapping relationship.

[0073] The following will be combined with the attached Figure 2 , the food maturity recognition device based on multi-sensor fusion provided in the embodiment of the present application is introduced in detail. Figure 2The food maturity recognition device based on multi-sensor fusion shown is used to implement the present application Figure 1 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 1 The embodiment shown.

[0074] See also Figure 2 , Figure 2 Schematic diagram of a food maturity recognition device based on multi-sensor fusion provided in an embodiment of the present application. Figure 2 As shown, the device comprises:

[0075] A first acquisition module 201 is used to acquire first collected data of a sensor for a cooking object, and construct a feature matrix according to the first collected data, wherein the collected data includes image data, temperature data and gas data, each row of the feature matrix represents a cooking experiment, and each column of the feature matrix represents a type of collected data;

[0076] A setting module 202 is used to set a target label representing maturity for each row of the feature matrix, and process the feature matrix based on a preset feature selection rule to obtain training data, wherein the preset feature selection rule includes at least one of a principal component analysis method, a recursive feature elimination method, and a correlation analysis method;

[0077] A training module 203 is used to train an initial SVM model based on the training data to obtain a trained SVM model;

[0078] The second acquisition module 204 is used to acquire the second collected data of the sensor for the current cooking object when a maturity recognition instruction is detected, process the second collected data based on the trained SVM model, obtain a classification result representing the target label, and display the classification result.

[0079] In one possible implementation, the image data includes color features, texture features, and edge features of the food, the temperature data includes temperature features and temperature change trend features at different time points, and the gas data includes time series features of volatile organic compounds.

[0080] In one possible implementation, the setting module 202 is further configured to:

[0081] The feature matrix is ​​preprocessed, and the preprocessing includes normalization, missing value processing and data balancing.

[0082] In one possible implementation, the training module 203 is further configured to:

[0083] An initial SVM model is constructed, a kernel function of the initial SVM model is set to a Gaussian radial basis kernel, and an optimal hyperparameter combination of the initial SVM model is determined based on a grid search or a random search.

[0084] In one possible implementation, the second acquisition module 204 is specifically used for:

[0085] When a maturity recognition instruction is detected, a cooking method of a current cooking object is determined, and a target SVM model matching the current cooking object and the cooking method is selected from among the trained SVM models;

[0086] Acquire second collected data of the sensor for the current cooking object, process the second collected data based on the target SVM model, and obtain a classification result representing the target label.

[0087] In one possible implementation, the collected data also includes ingredient combination data, volume data and area data, the ingredient combination data includes vector features characterizing the types of ingredient combinations, the area data includes area features of individual ingredients, and the volume data includes volume features of individual ingredients.

[0088] In one possible implementation, the second acquisition module 204 is further configured to:

[0089] Determine a first collection point of the gas data in the first collection data and a second collection point of the gas data in the second collection data;

[0090] Determine the gas escape ratio based on the position change data of the second acquisition point relative to the first acquisition point, and adjust the second acquisition data based on the gas escape ratio to obtain third acquisition data;

[0091] The third collected data is processed based on the trained SVM model to obtain a classification result representing the target label.

[0092] Those skilled in the art can clearly understand that the technical solutions of the embodiments of the present application can be implemented with the help of software and / or hardware. The "unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array (Field-Programmable Gate Array, FPGA), an integrated circuit (Integrated Circuit, IC), etc.

[0093] Each processing unit and / or module of the embodiments of the present application may be implemented by an analog circuit that implements the functions described in the embodiments of the present application, or may be implemented by software that executes the functions described in the embodiments of the present application.

[0094] See also Figure 3 , which shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present application, the electronic device can be used to implement Figure 1 The method in the embodiment shown. Figure 3 As shown, the electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .

[0095] The communication bus 302 is used to realize the connection and communication between these components.

[0096] The user interface 303 may include a display screen (Display) and a camera (Camera). The optional user interface 303 may also include a standard wired interface and a wireless interface.

[0097] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0098] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts within the entire electronic device 300, and executes various functions and processes data of the electronic device 300 by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics central processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0099] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 305 may also be optionally at least one storage device located away from the aforementioned processor 301. As Figure 3 As shown, the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and program instructions.

[0100] exist Figure 3 In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the food maturity recognition application based on multi-sensor fusion stored in the memory 305, and specifically perform the following operations:

[0101] Acquire first collected data of the sensor for the cooking object, and construct a feature matrix according to the first collected data, wherein the collected data includes image data, temperature data and gas data, each row of the feature matrix represents a cooking experiment, and each column of the feature matrix represents a type of collected data;

[0102] Setting a target label representing maturity for each row of the feature matrix, and processing the feature matrix based on a preset feature selection rule to obtain training data, wherein the preset feature selection rule includes at least one of a principal component analysis method, a recursive feature elimination method, and a correlation analysis method;

[0103] Training an initial SVM model based on the training data to obtain a trained SVM model;

[0104] When a maturity recognition instruction is detected, second collected data of the sensor for the current cooking object is obtained, the second collected data is processed based on the trained SVM model, a classification result representing the target label is obtained, and the classification result is displayed.

[0105] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0106] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0107] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0108] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0109] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0110] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0111] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program code.

[0112] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by entering a program to instruct the relevant hardware. The program may be stored in a computer-readable memory, and the memory may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0113] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for identifying food maturity based on multi-sensor fusion, characterized in that: The method comprises: Acquire first collected data of the sensor for the cooking object, and construct a feature matrix according to the first collected data, wherein the collected data includes image data, temperature data and gas data, each row of the feature matrix represents a cooking experiment, and each column of the feature matrix represents a type of collected data; Setting a target label representing maturity for each row of the feature matrix, and processing the feature matrix based on a preset feature selection rule to obtain training data, wherein the preset feature selection rule includes at least one of a principal component analysis method, a recursive feature elimination method, and a correlation analysis method; Training an initial SVM model based on the training data to obtain a trained SVM model; When a maturity recognition instruction is detected, second collected data of the sensor for the current cooking object is obtained, the second collected data is processed based on the trained SVM model, a classification result representing the target label is obtained, and the classification result is displayed.

2. The method according to claim 1, characterized in that The image data includes color features, texture features and edge features of the food, the temperature data includes temperature features at different time points and temperature change trend features, and the gas data includes time series features of volatile organic compounds.

3. The method according to claim 1, characterized in that Before the feature matrix is ​​processed based on the preset feature selection rule to obtain training data, the method further includes: The feature matrix is ​​preprocessed, and the preprocessing includes normalization, missing value processing and data balancing.

4. The method according to claim 1, characterized in that: Before the initial SVM model is trained based on the training data to obtain the trained SVM model, the method further includes: An initial SVM model is constructed, a kernel function of the initial SVM model is set to a Gaussian radial basis kernel, and an optimal hyperparameter combination of the initial SVM model is determined based on a grid search or a random search.

5. The method according to claim 1, characterized in that When the maturity recognition instruction is detected, second collected data of the sensor for the current cooking object is obtained, and the second collected data is processed based on the trained SVM model to obtain a classification result representing the target label, including: When a maturity recognition instruction is detected, a cooking method of a current cooking object is determined, and a target SVM model matching the current cooking object and the cooking method is selected from among the trained SVM models; Acquire second collected data of the sensor for the current cooking object, process the second collected data based on the target SVM model, and obtain a classification result representing the target label.

6. The method according to claim 1, characterized in that The collected data also includes food combination data, volume data and area data. The food combination data includes vector features characterizing food combination types, the area data includes area features of a single food, and the volume data includes volume features of a single food.

7. The method according to claim 1, characterized in that The step of processing the second collected data based on the trained SVM model to obtain a classification result representing the target label includes: Determine a first collection point of the gas data in the first collection data and a second collection point of the gas data in the second collection data; Determine the gas escape ratio based on the position change data of the second acquisition point relative to the first acquisition point, and adjust the second acquisition data based on the gas escape ratio to obtain third acquisition data; The third collected data is processed based on the trained SVM model to obtain a classification result representing the target label.

8. A food maturity recognition device based on multi-sensor fusion, characterized in that: The device comprises: A first acquisition module, configured to acquire first collected data of a sensor for a cooking object, and construct a feature matrix according to the first collected data, wherein the collected data includes image data, temperature data, and gas data, wherein each row of the feature matrix represents a cooking experiment, and each column of the feature matrix represents a type of collected data; A setting module, used to set a target label representing maturity for each row of the feature matrix, and process the feature matrix based on a preset feature selection rule to obtain training data, wherein the preset feature selection rule includes at least one of a principal component analysis method, a recursive feature elimination method, and a correlation analysis method; A training module, used for training an initial SVM model based on the training data to obtain a trained SVM model; The second acquisition module is used to acquire the second collected data of the sensor for the current cooking object when a maturity recognition instruction is detected, process the second collected data based on the trained SVM model, obtain a classification result representing the target label, and display the classification result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer-readable storage medium has instructions stored therein, and when the instructions are executed on a computer or a processor, the computer or the processor executes the steps of the method according to any one of claims 1 to 7.

Images