A method and system for food recognition and nutritional analysis based on image drive

Abstract

The present invention proposes a method and system for food recognition and nutritional analysis based on image drive. The method for food recognition and nutritional analysis based on image drive includes: controlling the image acquisition device to acquire the target food and obtain the food image data corresponding to the target food; performing image processing on the food image data corresponding to the target food to obtain the food image data corresponding to the target food after image processing; inputting the food image data corresponding to the target food after image processing into a trained convolutional neural network model, and identifying the food information in the food image data corresponding to the target food through the convolutional neural network model to obtain the food recognition result; performing nutritional analysis on the food information contained in the food recognition result according to the food recognition result to obtain the nutritional analysis result. The system includes modules corresponding to the steps of the method.

Description

Technical Field

[0001] The present invention proposes an image-driven food recognition and nutrition analysis method and system, belonging to the technical field of image recognition. Background Art

[0002] With the improvement of people's living standards and the enhancement of health awareness, the demand for nutritional management of food intake is increasing. However, traditional food nutritional analysis methods often require manual weighing, classification and nutritional calculation of food, which is not only time-consuming and labor-intensive, but also easily affected by human factors, resulting in low accuracy of analysis results.

[0003] In recent years, with the rapid development of computer vision and image processing technology, image-based food recognition technology has gradually emerged. This technology takes pictures of food and uses computers to process and analyze the images, thereby realizing the rapid identification and analysis of food types and nutritional values. However, the existing image-based food recognition technology still has some challenges and limitations. First of all, there are many types of food with different shapes, which makes image recognition very difficult. The image processing method of the image data to be processed in the traditional image recognition method can only meet the needs of individual training and recognition of each food, which not only requires a large amount of data and time, but also makes it difficult to achieve comprehensive coverage. At the same time, it will also affect the accuracy of image recognition. Summary of the invention

[0004] The present invention provides an image-driven food recognition and nutrition analysis method and system to solve the problems existing in the above-mentioned prior art. The technical solutions adopted are as follows:

[0005] A method for food recognition and nutritional analysis based on image driving, the method comprising:

[0006] Controlling the image acquisition device to acquire the target food and obtain the food image data corresponding to the target food;

[0007] Performing image processing on the food image data corresponding to the target food to obtain the food image data corresponding to the target food after image processing; wherein the image processing includes image segmentation and pixel value adjustment;

[0008] Identify food information in the food image data corresponding to the target food using a convolutional neural network model to obtain a food identification result;

[0009] Perform a nutritional analysis on the food information contained in the food identification result according to the food identification result to obtain a nutritional analysis result.

[0010] Further, performing image processing on the food image data corresponding to the target food to obtain the food image data corresponding to the target food after image processing includes:

[0011] Performing noise reduction processing on the food image data corresponding to the target food to obtain the food image data after noise reduction processing;

[0012] The food image data after the noise reduction processing is subjected to image block segmentation and image block pixel value adjustment, and the image block after the pixel value adjustment is obtained by image block stitching.

[0013] Furthermore, the food image data after the noise reduction processing is subjected to image block segmentation and image block pixel value adjustment, and the image block after the pixel value adjustment is obtained by image block stitching, including:

[0014] Performing image block segmentation on the food image data after the noise reduction process to obtain a plurality of image blocks;

[0015] Adjusting the pixel value of each image block corresponding to the food image data after the noise reduction processing to obtain the image block after the pixel value adjustment;

[0016] The image blocks after the pixel values ​​are adjusted are pieced together to obtain food image data corresponding to the target food after the pixel values ​​are adjusted.

[0017] Furthermore, the food image data after the noise reduction process is segmented into image blocks to obtain a plurality of image blocks, including:

[0018] Copying the food image data after the noise reduction process to obtain copied food image data;

[0019] Performing grayscale processing on the reproduced food image data after the noise reduction processing to obtain grayscale processed reproduced food image data;

[0020] Performing image segmentation on the grayscale processed replicated food image data according to a preset size of an initial image block to obtain a plurality of grayscale image blocks; wherein the size of the initial image block is a×a, and a is an odd number;

[0021] Extracting the grayscale value corresponding to each pixel block in the grayscale image block;

[0022] The grayscale evaluation coefficient of each grayscale image block is obtained according to the grayscale value corresponding to each pixel block in the grayscale image block; wherein the grayscale evaluation coefficient is obtained by the following formula:

[0023]

[0024] Where K represents the grayscale evaluation coefficient of each grayscale image block; n represents the number of pixel blocks contained in each grayscale image block except the central pixel block; X wi represents the gray value of the i-th pixel block except the central pixel block; X z Represents the gray value of the central pixel block of the gray image block; k 01 and k 02 represent the first coefficient and the second coefficient respectively; wherein the first coefficient is obtained as follows:

[0025] When the grayscale value of the central pixel block is the maximum grayscale value of the pixel blocks included in the grayscale image block, the first coefficient is obtained by the following formula:

[0026]

[0027] Among them, X p Represents the grayscale average value of the grayscale image block; X min Indicates the minimum grayscale value of the pixel block included in the grayscale image block;

[0028] When the grayscale value of the central pixel block is not the maximum grayscale value of the pixel blocks included in the grayscale image block, the first coefficient is obtained by the following formula:

[0029]

[0030] Among them, X p Represents the grayscale average value of the grayscale image block; X min represents the minimum grayscale value of the pixel block contained in the grayscale image block; X max Indicates the grayscale maximum value of the pixel block included in the grayscale image block;

[0031] And, the second coefficient is obtained by the following formula:

[0032]

[0033] Among them, k 02 represents the second coefficient; X i represents the gray value corresponding to the i-th pixel block in the gray image block; X z Represents the gray value of the central pixel block of the gray image block; X p Represents the grayscale average value of the grayscale image block;

[0034] Determining a constraint condition of a grayscale evaluation coefficient according to the grayscale evaluation coefficient of each grayscale image block;

[0035] Comparing the grayscale evaluation coefficients of each two adjacent grayscale image blocks, and merging the two adjacent grayscale image blocks whose grayscale evaluation coefficients satisfy the constraint condition into one image block, to obtain a plurality of image blocks corresponding to the grayscale processed replicated food image data;

[0036] The original food image data after the noise reduction processing is segmented into image blocks according to the distribution of the multiple image blocks corresponding to the replicated food image data after the grayscale processing, so as to obtain the multiple image blocks corresponding to the food image data after the noise reduction processing.

[0037] Furthermore, determining the constraint condition of the grayscale evaluation coefficient according to the grayscale evaluation coefficient of each grayscale image block includes:

[0038] Extracting a grayscale evaluation coefficient of the grayscale image block;

[0039] Acquire a maximum value and a minimum value of a grayscale evaluation coefficient according to the grayscale evaluation coefficient of the grayscale image block;

[0040] Extracting the central pixel block of the grayscale image block corresponding to the maximum value of the grayscale evaluation coefficient and the minimum value of the grayscale evaluation coefficient;

[0041] Using the distance between the central pixel blocks of the grayscale image block corresponding to the maximum value of the grayscale evaluation coefficient and the minimum value of the grayscale evaluation coefficient as the first distance data;

[0042] According to the grayscale evaluation coefficients of all the grayscale image blocks, obtaining median parameters corresponding to the grayscale evaluation coefficients of all the grayscale image blocks;

[0043] Extracting a preset median parameter offset h, wherein the value range of the median parameter offset h is 3%-10% of the median parameter H;

[0044] The coefficient parameter range is obtained by using the preset median parameter offset h in combination with the median parameter H; wherein the coefficient parameter range is H±h;

[0045] Taking the grayscale image block whose grayscale evaluation coefficient meets the coefficient parameter as the target grayscale image block, and obtaining a plurality of target grayscale image blocks;

[0046] Extracting the distance between the central pixel block of the grayscale image block corresponding to the minimum value of the grayscale evaluation coefficient and the central pixel block of each target grayscale image block as second distance data;

[0047] The first distance data and a plurality of second distance data are used to obtain a first constraint condition compensation amount; wherein the first constraint condition compensation amount is obtained by the following formula:

[0048]

[0049] Among them, ξ 01 represents the compensation of the first constraint condition; S m represents the distance value corresponding to the first distance data; m represents the number of the second distance data; S i represents the distance value corresponding to the i-th second distance data; max(S i ) and min(S i ) represents the maximum distance and the minimum distance contained in the m second distance data; s represents the compensation adjustment coefficient, and the compensation adjustment coefficient is obtained by the following formula:

[0050]

[0051] Among them, S m represents the distance value corresponding to the first distance data; m represents the number of the second distance data; S i represents the distance value corresponding to the i-th second distance data; X m01 represents the grayscale value of the central pixel block of the grayscale image block with the maximum distance value contained in the second distance data; m02 The grayscale value of the central pixel block of the grayscale image block representing the minimum distance value contained in the second distance data;

[0052] The second constraint condition compensation amount is obtained by using the grayscale evaluation coefficients of all the grayscale image blocks; wherein the second constraint condition compensation amount is obtained by the following formula:

[0053]

[0054] Among them, ξ 02 represents the compensation amount of the second constraint condition; r represents the total number of grayscale image blocks; K i represents the grayscale evaluation coefficient corresponding to the i-th grayscale image block; X zi represents the gray value of the central pixel block corresponding to the i-th gray image block; X zp Represents the standard deviation of the grayscale average values ​​corresponding to all grayscale image blocks;

[0055] Extracting preset initial constraints from a database;

[0056] The first constraint condition compensation amount and the second constraint condition compensation amount are combined with the preset initial constraint condition to obtain the constraint condition, wherein the constraint condition is obtained by the following formula:

[0057] X up =(1+ξ 01 +ξ 02 )·X up 0

[0058]

[0059] Among them, X up Indicates the upper limit value of the constraint condition; X up0 represents the upper limit value of the initial constraint; X down Indicates the lower limit of the constraint condition; X down0 Represents the lower limit value of the initial constraint.

[0060] Further, adjusting the pixel value of each image block corresponding to the food image data after the noise reduction processing to obtain the image block after the pixel value adjustment includes:

[0061] Extracting the entire area of ​​the image block of the grayscale processed replicated food image data corresponding to each image block of the food image data after the noise reduction processing;

[0062] Extracting the grayscale value of the pixel blocks contained in the entire area of ​​the image block of the replicated food image data after the grayscale processing corresponding to each image block;

[0063] Obtaining a brightness adjustment coefficient using the grayscale values ​​of the pixel blocks contained in the entire area of ​​the image block;

[0064] Obtaining a pixel value adjustment coefficient using the grayscale evaluation coefficient contained in each image block of the food image data after the noise reduction processing;

[0065] Using the brightness adjustment coefficient and the pixel value adjustment coefficient in combination with a pixel value adjustment model, the pixel value of each image block included in the overall area of ​​each image block is adjusted to obtain an image block after the pixel value is adjusted;

[0066] The pixel value adjustment model structure is as follows:

[0067]

[0068] Among them, R e , G e and B e Respectively represent the enhanced red channel, green channel and blue channel corresponding values ​​of each pixel block in the overall image block area; R, G and B represent the original red channel, green channel and blue channel corresponding values ​​of each pixel block in the overall image block area; t r ,t g and t b Respectively represent the brightness adjustment coefficients corresponding to the red channel, green channel and blue channel; h r 、h g and h b Respectively represent the pixel value adjustment coefficients corresponding to the red channel, green channel, and blue channel.

[0069] Furthermore, the brightness adjustment coefficient and the pixel value adjustment coefficient are obtained by the following formula:

[0070]

[0071] Among them, t r ,t g and t b They represent the brightness adjustment coefficients corresponding to the red channel, green channel, and blue channel respectively; z represents the number of pixel blocks contained in the overall area of ​​the image block; X yi represents the gray value of the i-th pixel block contained in the entire area of ​​the image block; X zpxi represents the grayscale average value of the central pixel block of the image block contained in the overall area of ​​the image block, and when an image block is contained in the overall area of ​​the image block, X zpxi It represents the gray value of the central pixel block of the image block; R p , G p and B p The numerical averages of the red channel, green channel and blue channel of the image block contained in the overall area of ​​the image block respectively; t r0 ,t g0 and t b0 Respectively represent the initial brightness adjustment coefficients corresponding to the preset red channel, green channel and blue channel;

[0072]

[0073] Among them, h r 、h g and h b Respectively represent the pixel value adjustment coefficients corresponding to the red channel, green channel and blue channel; σ k Indicates the pixel value compensation coefficient; h r0 、h g0 and h b0 Respectively represent the initial pixel value adjustment coefficients corresponding to the preset red channel, green channel and blue channel; σ k represents a pixel value compensation coefficient, and the pixel value compensation coefficient is obtained by the following formula:

[0074]

[0075] Where p represents the number of image blocks contained in the overall area of ​​the image block; K xi represents the grayscale evaluation coefficient of the grayscale image block of the replicated food image data corresponding to the i-th image block contained in the entire image block area; K pK represents the average value of the grayscale evaluation coefficients of the grayscale image blocks of the replicated food image data corresponding to the p image blocks contained in the entire image block area; max represents the maximum value of the grayscale evaluation coefficient of the grayscale image block of the replicated food image data corresponding to the p image blocks contained in the overall image block area; at the same time, when the number of image blocks contained in the overall image block area is 1, let σ k =0.

[0076] Furthermore, the food information in the food image data corresponding to the target food is identified by a convolutional neural network model to obtain a food identification result, including:

[0077] Converting the food image data corresponding to the target food after image processing into a data format allowed to be input by the convolutional neural network model;

[0078] Retrieve a trained convolutional neural network model for food recognition from a database;

[0079] Inputting the food image data corresponding to the target food after data format conversion into the trained convolutional neural network model;

[0080] After receiving the food image data corresponding to the target food, the convolutional neural network model identifies the food information in the food image data corresponding to the target food to obtain a food identification result.

[0081] Further, performing a nutritional analysis on the food information contained in the food identification result according to the food identification result to obtain a nutritional analysis result includes:

[0082] Nutrition information corresponding to the food types included in the food recognition results from a database;

[0083] Obtaining each nutrient component and the total amount corresponding to each nutrient component from the nutrient component information;

[0084] Retrieving the first target component and the second target component recorded by the user from the database; wherein the first target component is the nutrient component that the user needs to increase intake; and the second target component is the nutrient component that the user needs to reduce intake;

[0085] The nutritional evaluation value of the food included in the food identification result is obtained according to each nutritional component and the total amount corresponding to each nutritional component; wherein the nutritional evaluation value is obtained by the following formula:

[0086]

[0087] Wherein, Q represents the nutritional evaluation value; x represents the quantity of the first target component contained in the nutritional component information corresponding to the food type; M i represents the content ratio corresponding to the i-th first target component; y represents the amount of the second target component contained in the nutritional component information corresponding to the food type; M j represents the content ratio corresponding to the jth second target component; i and λ j They respectively represent the weight value corresponding to the i-th first target component and the weight value corresponding to the j-th second target component.

[0088] An image-driven food recognition and nutrition analysis system, the image-driven food recognition and nutrition analysis system comprising:

[0089] An image acquisition module, used to control the image acquisition device to acquire the target food and obtain the food image data corresponding to the target food;

[0090] An image processing module, used to perform image processing on the food image data corresponding to the target food, and obtain the food image data corresponding to the target food after image processing;

[0091] A food recognition module, used to recognize food information in the food image data corresponding to the target food through a convolutional neural network model to obtain a food recognition result;

[0092] The nutrition analysis module is used to perform nutrition analysis on the food information contained in the food recognition result according to the food recognition result to obtain the nutrition analysis result.

[0093] Beneficial effects of the present invention:

[0094] The image-driven food recognition and nutrition analysis method and system proposed by the present invention greatly reduces the time and labor intensity of manual participation through automated image acquisition and processing. At the same time, the convolutional neural network model based on deep learning can quickly and accurately identify food types, and realize efficient recognition and analysis of food. By performing preprocessing operations such as image segmentation and pixel value adjustment, the quality of the image and the recognizability of the features are improved, thereby improving the accuracy of recognition. In addition, combined with a pre-established food nutrition database, the nutritional components and calories of the food can be accurately calculated. Users only need to use a smart phone or other device to take food photos to achieve rapid recognition and analysis of food. This method is simple and easy to implement, and is suitable for food nutrition management needs in various scenarios. With the continuous increase in data volume and continuous optimization of the model, the recognition and analysis capabilities of the method will continue to improve. At the same time, by adding new food types and nutritional component information to the database, the application scope of the method can be further expanded. On the other hand, using deep learning models such as convolutional neural networks (CNN) for fruit recognition can automatically learn and extract complex features in images to meet the recognition needs of fruits of different types and shapes. At the same time, the deep learning model has strong generalization ability and can maintain a high recognition accuracy in different scenarios. Due to the use of advanced image processing technology and deep learning models, this technical solution can achieve a high recognition accuracy on image data containing a variety of fruits. The specific accuracy depends on factors such as the model's training data, model structure and parameter settings, but it can generally reach more than 90%. Real-time: This technical solution uses efficient image processing algorithms and deep learning models, which can complete image processing and recognition tasks in a shorter time, meeting real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 is a flow chart of the method of the present invention;

[0096] Figure 2 A system diagram of the system of the present invention. DETAILED DESCRIPTION

[0097] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0098] The embodiment of the present invention proposes a method for food recognition and nutritional analysis based on image drive, such as Figure 1 As shown, the image-driven food recognition and nutrition analysis method includes:

[0099] S1. Controlling an image acquisition device to acquire target food and obtain food image data corresponding to the target food;

[0100] S2, performing image processing on the food image data corresponding to the target food to obtain the food image data corresponding to the target food after image processing; wherein the image processing includes image segmentation and pixel value adjustment;

[0101] S3, identifying food information in the food image data corresponding to the target food through a convolutional neural network model to obtain a food identification result;

[0102] S4. Performing a nutritional analysis on the food information contained in the food identification result according to the food identification result to obtain a nutritional analysis result.

[0103] The working principle of the above technical solution is: first, use an image acquisition device (such as a smart phone, professional camera, etc.) to shoot the target food and obtain high-definition image data of the target food. This method is to obtain sufficiently clear and detailed food images and provide a basis for subsequent processing.

[0104] Then, the collected food image data is preprocessed to improve the accuracy of subsequent recognition. The processing process mainly includes image segmentation and pixel value adjustment. The food part in the image is separated from the background or other irrelevant parts to ensure the accuracy of subsequent recognition. The brightness, contrast and other parameters of the image are adjusted to make the food features in the image more prominent, which is convenient for subsequent feature extraction and recognition.

[0105] Subsequently, the processed food images are identified using a pre-trained convolutional neural network model (CNN). Through multi-layer convolution and pooling operations, the CNN model automatically learns features such as texture, shape, and color in the image to identify the type of food.

[0106] Finally, based on the results of food recognition and the pre-established food nutrition database, the food is analyzed for nutrition. The database contains information on the nutritional composition of various foods, such as protein, fat, carbohydrates, vitamins, etc. By matching the recognition results with the information in the database, the nutritional composition and calorie information of the food can be quickly and accurately obtained.

[0107] The effect of the above technical solution is: through automated image acquisition and processing, the time and labor intensity of manual participation are greatly reduced. At the same time, the convolutional neural network model based on deep learning can quickly and accurately identify food types, and realize efficient recognition and analysis of food. By performing preprocessing operations such as image segmentation and pixel value adjustment, the quality of the image and the recognizability of the features are improved, thereby improving the accuracy of recognition. In addition, combined with the pre-established food nutrition database, the nutritional content and calorie information of the food can be accurately calculated. Users only need to take photos of food using devices such as smartphones to quickly identify and analyze food. This method is simple and easy to use, and is suitable for food nutrition management needs in various scenarios. With the continuous increase in data volume and continuous optimization of the model, the recognition and analysis capabilities of this method will continue to improve. At the same time, by adding new food types and nutritional information to the database, the scope of application of this method can be further expanded.

[0108] In one embodiment of the present invention, image processing is performed on the food image data corresponding to the target food to obtain the food image data corresponding to the target food after image processing, including:

[0109] S201, performing noise reduction processing on the food image data corresponding to the target food to obtain the food image data after noise reduction processing;

[0110] S202, performing image block segmentation and image block pixel value adjustment on the food image data after the noise reduction processing, and obtaining the image blocks after the pixel value adjustment by image block stitching.

[0111] The method of performing image block segmentation and image block pixel value adjustment on the food image data after the noise reduction processing, and obtaining the image block after the pixel value adjustment by image block stitching includes:

[0112] S2021, performing image block segmentation on the food image data after the noise reduction processing to obtain a plurality of image blocks;

[0113] S2022, adjusting the pixel value of each image block corresponding to the food image data after the noise reduction processing, and obtaining the image block after the pixel value adjustment;

[0114] S2023: stitch the image blocks after the pixel values ​​are adjusted to obtain food image data corresponding to the target food after the pixel values ​​are adjusted.

[0115] The working principle of the above technical solution is as follows: First, remove the noise in the food image data, which may come from electronic components, signal interference, light source problems, etc. in the image acquisition process. The principle of noise reduction processing is to analyze the noise in the image through an algorithm and remove it using appropriate methods to improve the quality of the image. Commonly used noise reduction algorithms include mean filtering, median filtering, wavelet denoising, etc. These algorithms can eliminate high-frequency noise in the image and retain the detailed information of the image.

[0116] After the noise reduction process, the food image data is divided into multiple image blocks. This step is to facilitate the subsequent individual pixel value adjustment of each image block. The image block segmentation method can be performed according to the size, shape and characteristics of the image to ensure that each image block contains relatively independent image information. The pixel value of each image block obtained by segmentation is adjusted to improve the image quality and feature recognizability. The pixel value adjustment methods include brightness adjustment, contrast adjustment, etc., which can be selected and adjusted according to specific needs. After the pixel value adjustment, the individual image blocks are reassembled to form a complete image. Compared with the original image, the assembled image has better image quality and clearer feature information, providing a better data basis for subsequent food identification and nutritional analysis.

[0117] The effect of the above technical solution is: through noise reduction processing and pixel value adjustment, the quality of food image data can be significantly improved, noise and interference in the image can be eliminated, and the detailed information of the image can be retained. Image block segmentation and pixel value adjustment can perform separate optimization processing for each image block, thereby enhancing the recognizability of food features in the image and providing more accurate data support for subsequent food identification and nutritional analysis. The food image data after image processing has better quality and clearer feature information, which can improve the accuracy of subsequent food identification and nutritional analysis. This helps users to understand the nutritional content and calories of food more accurately, and provides strong support for healthy eating. Since the image processing process can eliminate noise and interference in the image, the system's requirements for image quality are reduced, thereby enhancing the robustness and stability of the system. Even in the case of poor image quality, the system can maintain a high recognition and analysis accuracy.

[0118] In one embodiment of the present invention, the food image data after the noise reduction process is segmented into image blocks to obtain a plurality of image blocks, including:

[0119] Step 1, copying the food image data after the noise reduction process to obtain copied food image data;

[0120] Step 2, grayscale processing is performed on the replicated food image data after the noise reduction processing to obtain the replicated food image data after the grayscale processing;

[0121] Step 3, performing image segmentation on the grayscale processed replicated food image data according to a preset initial image block size to obtain a plurality of grayscale image blocks; wherein the size of the initial image block is a×a, and a is an odd number;

[0122] Step 4: extract the grayscale value corresponding to each pixel block in the grayscale image block;

[0123] Step 5: Obtain a grayscale evaluation coefficient of each grayscale image block according to the grayscale value corresponding to each pixel block in the grayscale image block; wherein the grayscale evaluation coefficient is obtained by the following formula:

[0124]

[0125] Where K represents the grayscale evaluation coefficient of each grayscale image block; n represents the number of pixel blocks contained in each grayscale image block except the central pixel block; X wi represents the gray value of the i-th pixel block except the central pixel block; X z Represents the gray value of the central pixel block of the gray image block; k 01 and k 02 represent the first coefficient and the second coefficient respectively; wherein the first coefficient is obtained as follows:

[0126] Step 6: When the grayscale value of the central pixel block is the maximum grayscale value of the pixel blocks included in the grayscale image block, the first coefficient is obtained by the following formula:

[0127]

[0128] Among them, X p Represents the grayscale average value of the grayscale image block; X min Indicates the minimum grayscale value of the pixel block included in the grayscale image block;

[0129] Step 7: When the grayscale value of the central pixel block is not the maximum grayscale value of the pixel blocks included in the grayscale image block, the first coefficient is obtained by the following formula:

[0130]

[0131] Among them, X p Represents the grayscale average value of the grayscale image block; X min represents the minimum grayscale value of the pixel block contained in the grayscale image block; X max Indicates the grayscale maximum value of the pixel block included in the grayscale image block;

[0132] And, the second coefficient is obtained by the following formula:

[0133]

[0134] Among them, k 02 represents the second coefficient; X i represents the gray value corresponding to the i-th pixel block in the gray image block; X z Represents the gray value of the central pixel block of the gray image block; X p Represents the grayscale average value of the grayscale image block;

[0135] Step 8: determining the constraint condition of the grayscale evaluation coefficient according to the grayscale evaluation coefficient of each grayscale image block;

[0136] Step 9: compare the grayscale evaluation coefficients of every two adjacent grayscale image blocks, and merge the two adjacent grayscale image blocks whose grayscale evaluation coefficients satisfy the constraint condition into one image block, to obtain a plurality of image blocks corresponding to the grayscale processed replicated food image data;

[0137] Step 10: segment the original food image data after the noise reduction processing into image blocks according to the distribution of the multiple image blocks corresponding to the replicated food image data after the grayscale processing, so as to obtain the multiple image blocks corresponding to the food image data after the noise reduction processing.

[0138] The working principle of the above technical solution is as follows: first, the food image data after noise reduction is copied to ensure that subsequent operations do not change the original data. Then, the copied image is grayscale processed to convert the color image into a grayscale image. Grayscale images only contain brightness information, not color information, which helps to simplify subsequent processing and reduce the amount of calculation.

[0139] The grayscale image is segmented according to the preset initial image block size (a×a, a is an odd number) to obtain multiple grayscale image blocks. The image blocks of odd size can ensure that the central pixel block has a clear location, which is helpful for the subsequent central pixel block analysis.

[0140] The grayscale value of each pixel block in each grayscale image block is extracted. The grayscale evaluation coefficient K is calculated based on the grayscale value of the pixel block. This coefficient takes into account the relationship between the grayscale value of the central pixel block and the grayscale value of the surrounding pixel blocks, as well as the overall grayscale distribution of the grayscale image block. Depending on whether the grayscale value of the central pixel block is the maximum value of the image block, different formulas are used to calculate the first coefficient k01. This reflects the relative brightness of the central pixel block in the image block. The second coefficient k02 is calculated based on the central pixel block, the average grayscale value, and the grayscale value of all pixel blocks, and measures the grayscale changes within the image block.

[0141] Constraints are determined based on the grayscale evaluation coefficients. These conditions may be set based on specific requirements or goals for the image. The grayscale evaluation coefficients of two adjacent grayscale image blocks are compared, and if the constraints are met, the two image blocks are merged. This helps ensure that the image blocks are divided not only based on size, but also based on the similarity and continuity of image content.

[0142] According to the distribution of multiple image blocks corresponding to the grayscale processed replicated food image data, the original food image data after noise reduction is segmented into corresponding image blocks, thus ensuring the consistency and accuracy of the segmentation results.

[0143] The effect of the above technical solution is: through the calculation of the grayscale evaluation coefficient, the content characteristics and grayscale distribution of each grayscale image block can be more accurately evaluated, so as to perform more reasonable image block merging and segmentation. The image block merging and segmentation based on the grayscale value and grayscale evaluation coefficient ensures that the image block not only has a preset size, but also takes into account the similarity and continuity of the image content, and optimizes the image block division results. Through grayscale processing, the color image is converted into a grayscale image, which simplifies the subsequent processing steps and reduces the amount of calculation. The optimized image block segmentation results can provide a more accurate and effective data basis for subsequent food identification and nutritional analysis, thereby improving the efficiency and accuracy of the overall processing. At the same time, the noise reduction processing can effectively remove the noise in the image, improve the signal-to-noise ratio of the image, and provide clear and clean image data for subsequent feature extraction and recognition. Grayscale processing converts the color image into a grayscale image, simplifies the image data, reduces the amount of calculation, and has a positive impact on the extraction of features such as shape and texture. And provide grayscale parameters for the original colored image data; the image block segmentation and merging strategy based on the grayscale evaluation coefficient can optimize the division of image blocks according to the distribution of grayscale values ​​inside the image block and the similarity between adjacent image blocks. This helps to segment different types of fruits into different image blocks and reduce mutual interference. At the same time, by merging adjacent image blocks that meet the constraints, the shape and size of the image blocks can be further optimized to make them more consistent with the actual outline of the fruit, thereby improving the accuracy of recognition. In addition, by extracting features (such as color, shape, texture, etc.) from the segmented image blocks, the unique characteristics of the fruit can be captured, providing rich information for subsequent recognition.

[0144] On the other hand, the grayscale evaluation coefficient provides a quantitative basis for the merging and segmentation of image blocks by comprehensively considering the grayscale relationship between the central pixel block and the surrounding pixel blocks in the grayscale image block and the grayscale distribution of the entire image block. Through the calculation of the grayscale evaluation coefficient, the characteristics of each image block can be more accurately evaluated, thereby optimizing the division of image blocks and improving the accuracy of subsequent fruit recognition. At the same time, the image block segmentation strategy based on the grayscale evaluation coefficient can segment different types of fruits into different image blocks according to the grayscale distribution inside the image block and the similarity between adjacent image blocks. This optimized image block segmentation helps to reduce the interference between different fruits and improve the recognition accuracy. At the same time, through precise image block segmentation and merging, combined with the feature extraction and recognition capabilities of the deep learning model, this technical solution can achieve a high recognition accuracy on image data containing multiple fruits. Since the model can learn the complex features of different fruits, it can maintain a high recognition accuracy even in different scenarios. The recall rate reflects the model's ability to recognize real fruits. Due to the optimization of image block segmentation and merging, the model can more accurately identify and segment the fruits in the image, thereby improving the recall rate.

[0145] In one embodiment of the present invention, determining the constraint condition of the grayscale evaluation coefficient according to the grayscale evaluation coefficient of each grayscale image block includes:

[0146] Step 801, extracting the grayscale evaluation coefficient of the grayscale image block;

[0147] Step 802: Obtain a maximum value and a minimum value of a grayscale evaluation coefficient according to the grayscale evaluation coefficient of the grayscale image block;

[0148] Step 803, extracting the central pixel block of the grayscale image block corresponding to the maximum value of the grayscale evaluation coefficient and the minimum value of the grayscale evaluation coefficient;

[0149] Step 804: using the distance between the central pixel blocks of the grayscale image blocks corresponding to the maximum value of the grayscale evaluation coefficient and the minimum value of the grayscale evaluation coefficient as the first distance data;

[0150] Step 805: acquiring median parameters corresponding to the grayscale evaluation coefficients of all the grayscale image blocks according to the grayscale evaluation coefficients of all the grayscale image blocks;

[0151] Step 806: extracting a preset median parameter offset h, wherein the value range of the median parameter offset h is 3%-10% of the median parameter H;

[0152] Step 807: Obtain a coefficient parameter range by using the preset median parameter offset h in combination with the median parameter H; wherein the coefficient parameter range is H±h;

[0153] Step 808: taking the grayscale image block whose grayscale evaluation coefficient meets the coefficient parameter as the target grayscale image block, and obtaining a plurality of target grayscale image blocks;

[0154] Step 809, extracting the distance between the central pixel block of the grayscale image block corresponding to the minimum value of the grayscale evaluation coefficient and the central pixel block of each target grayscale image block as second distance data;

[0155] Step 8010: Obtain a first constraint condition compensation amount using the first distance data and a plurality of second distance data; wherein the first constraint condition compensation amount is obtained by the following formula:

[0156]

[0157] Among them, ξ 01 represents the compensation of the first constraint condition; S m represents the distance value corresponding to the first distance data; m represents the number of the second distance data; S i represents the distance value corresponding to the i-th second distance data; max(S i ) and min(S i ) represents the maximum distance and the minimum distance contained in the m second distance data; s represents the compensation adjustment coefficient, and the compensation adjustment coefficient is obtained by the following formula:

[0158]

[0159] Among them, S m represents the distance value corresponding to the first distance data; m represents the number of the second distance data; S i represents the distance value corresponding to the i-th second distance data; X m01 represents the grayscale value of the central pixel block of the grayscale image block with the maximum distance value contained in the second distance data; m02 The grayscale value of the central pixel block of the grayscale image block representing the minimum distance value contained in the second distance data;

[0160] Step 8011: Obtain a second constraint condition compensation amount using the grayscale evaluation coefficients of all the grayscale image blocks; wherein the second constraint condition compensation amount is obtained by the following formula:

[0161]

[0162] Among them, ξ 02 represents the compensation amount of the second constraint condition; r represents the total number of grayscale image blocks; K i represents the grayscale evaluation coefficient corresponding to the i-th grayscale image block; X zirepresents the gray value of the central pixel block corresponding to the i-th gray image block; X zp Represents the standard deviation of the grayscale average values ​​corresponding to all grayscale image blocks;

[0163] Step 8012: extracting preset initial constraints from the database;

[0164] Step 8013: Obtain constraint conditions by using the first constraint condition compensation amount and the second constraint condition compensation amount in combination with the preset initial constraint conditions, wherein the constraint conditions are obtained by the following formula:

[0165] X up =(1+ξ 01 +ξ 02 )·X up 0

[0166]

[0167] Among them, X up Indicates the upper limit value of the constraint condition; X up0 represents the upper limit value of the initial constraint; X down Indicates the lower limit of the constraint condition; X down0 Represents the lower limit value of the initial constraint.

[0168] The working principle of the above technical solution is as follows: first, the grayscale evaluation coefficient is extracted from the processed grayscale image block. Then, the maximum and minimum values ​​of the grayscale evaluation coefficient are calculated, and the corresponding central pixel block of the grayscale image block is found. Subsequently, the distance between the central pixel blocks is used as the first distance data (step 804). In addition, based on the grayscale evaluation coefficients of all grayscale image blocks, a median parameter H is obtained, and an offset h based on the median parameter H is set to obtain the coefficient parameter range. Then, the grayscale image block whose grayscale evaluation coefficient is within the coefficient parameter range is used as the target grayscale image block. In addition, the distance between the central pixel block of the grayscale image block corresponding to the minimum value of the grayscale evaluation coefficient and the central pixel block of each target grayscale image block is calculated as the second distance data.

[0169] Finally, the first constraint condition compensation amount is calculated using the first distance data and the plurality of second distance data, and the second constraint condition compensation amount ξ02 is calculated using the grayscale evaluation coefficients of all grayscale image blocks (step 8011). Subsequently, the preset initial constraint condition is extracted from the database. The initial constraint condition is adjusted using the calculated first constraint condition compensation amount and the second constraint condition compensation amount to obtain the final constraint condition.

[0170] The effect of the above technical solution is that by calculating the grayscale evaluation coefficient and the related compensation amount, the constraint conditions for merging or segmenting image blocks can be determined more accurately. This helps to optimize the size and shape of the image blocks while retaining the image information, providing more accurate input for subsequent fruit recognition. The optimization of the constraint conditions can ensure that in the process of merging or segmenting image blocks, image blocks with high similarity are merged together, while image blocks with large differences are separated. This helps to reduce interference between different fruits and improve recognition accuracy. By extracting the initial constraint conditions from the database and dynamically adjusting them in combination with the actual image data, the technical solution can maintain a high recognition accuracy in different scenarios. This enhances the generalization ability of the model, enabling it to adapt to a wider range of application scenarios.

[0171] On the other hand, by calculating the maximum, minimum, median parameters and their offsets of the grayscale evaluation coefficient, as well as the first constraint compensation and the second constraint compensation calculated based on these parameters, the constraint conditions for merging or segmenting the image blocks can be accurately determined. Such precise constraint conditions help ensure that during the image processing process, the division of image blocks can be optimized based on the actual conditions of the image (such as grayscale distribution, similarity, etc.), thereby improving the accuracy of subsequent fruit recognition. At the same time, the precise determination of constraint conditions helps reduce interference between different fruits, ensuring that image blocks with high similarity are merged together, while image blocks with large differences are separated. This processing method helps to improve the accuracy of recognition, allowing the model to more accurately identify the types and quantities of fruits in the image.

[0172] The formula in this technical solution not only takes into account the grayscale evaluation coefficient of the image, but also combines the characteristics of the actual image data (such as grayscale distribution, similarity, etc.) to calculate the constraints. This processing method enables the model to adapt to different application scenarios and image data, thereby improving the generalization ability of the model. At the same time, by extracting the preset initial constraints from the database and dynamically adjusting the constraints in combination with the actual image data, the technical solution can automatically adjust the constraints according to different image data to adapt to different processing requirements. This dynamic adjustability enables the model to respond to various complex situations more flexibly, improving the flexibility and practicality of the model. In addition, although the technical solution involves more calculation steps and formulas, through reasonable algorithm optimization and hardware acceleration, it can ensure that the calculation and adjustment are completed in a shorter time, thereby meeting the real-time requirements. This real-time performance is of great significance for rapid response and timely processing in actual application scenarios.

[0173] In summary, the technical effects of the formula in the above technical solution on performance indicators are mainly reflected in accurately determining constraint conditions, improving recognition accuracy, enhancing generalization ability, dynamic adjustment and real-time performance, etc. These technical effects jointly improve the performance and practicality of the entire technical solution.

[0174] In one embodiment of the present invention, pixel value adjustment is performed on each image block corresponding to the food image data after the noise reduction processing to obtain the image block after the pixel value adjustment, including:

[0175] Step 1, extracting the entire image block area of ​​the grayscale processed replicated food image data corresponding to each image block of the food image data after the noise reduction processing;

[0176] Step 2, extracting the grayscale value of the pixel blocks contained in the entire area of ​​the image block of the replicated food image data after grayscale processing corresponding to each image block;

[0177] Step 3: Obtain a brightness adjustment coefficient using the grayscale value of the pixel block contained in the entire area of ​​the image block;

[0178] Step 4, obtaining a pixel value adjustment coefficient using the grayscale evaluation coefficient contained in each image block of the food image data after the noise reduction process;

[0179] Step 5: adjusting the pixel value of each image block contained in the entire area of ​​each image block by using the brightness adjustment coefficient and the pixel value adjustment coefficient in combination with a pixel value adjustment model to obtain an image block after the pixel value adjustment;

[0180] The pixel value adjustment model structure is as follows:

[0181]

[0182] Among them, R e , G e and B e Respectively represent the enhanced red channel, green channel and blue channel corresponding values ​​of each pixel block in the overall image block area; R, G and B represent the original red channel, green channel and blue channel corresponding values ​​of each pixel block in the overall image block area; t r ,t g and t b Respectively represent the brightness adjustment coefficients corresponding to the red channel, green channel and blue channel; h r 、h g and h b Respectively represent the pixel value adjustment coefficients corresponding to the red channel, green channel, and blue channel.

[0183] The brightness adjustment coefficient and the pixel value adjustment coefficient are obtained by the following formula:

[0184]

[0185] Among them, t r,t g and t b They represent the brightness adjustment coefficients corresponding to the red channel, green channel, and blue channel respectively; z represents the number of pixel blocks contained in the overall area of ​​the image block; X yi represents the gray value of the i-th pixel block contained in the entire area of ​​the image block; X zpxi represents the grayscale average value of the central pixel block of the image block contained in the overall area of ​​the image block, and when an image block is contained in the overall area of ​​the image block, X zpxi It represents the gray value of the central pixel block of the image block; R p , G p and B p The numerical averages of the red channel, green channel and blue channel of the image block contained in the overall area of ​​the image block respectively; t r0 ,t g0 and t b0 Respectively represent the initial brightness adjustment coefficients corresponding to the preset red channel, green channel and blue channel;

[0186]

[0187] Among them, h r 、h g and h b Respectively represent the pixel value adjustment coefficients corresponding to the red channel, green channel and blue channel; σ k Indicates the pixel value compensation coefficient; h r0 、h g0 and h b0 Respectively represent the initial pixel value adjustment coefficients corresponding to the preset red channel, green channel and blue channel; σ k represents a pixel value compensation coefficient, and the pixel value compensation coefficient is obtained by the following formula:

[0188]

[0189] Where p represents the number of image blocks contained in the overall area of ​​the image block; K xi represents the grayscale evaluation coefficient of the grayscale image block of the replicated food image data corresponding to the i-th image block contained in the entire image block area; K p K represents the average value of the grayscale evaluation coefficients of the grayscale image blocks of the replicated food image data corresponding to the p image blocks contained in the entire image block area; max represents the maximum value of the grayscale evaluation coefficient of the grayscale image block of the replicated food image data corresponding to the p image blocks contained in the overall image block area; at the same time, when the number of image blocks contained in the overall image block area is 1, let σ k =0.

[0190] The working principle of the above technical solution is as follows: from the food image data after noise reduction processing, extract the entire image block area of ​​the grayscale processed replicated food image data corresponding to each image block. Extract the grayscale value of each pixel block contained in the above image block overall area. According to the grayscale value of the pixel block, combined with the grayscale average value of the central pixel block, calculate the brightness adjustment coefficient corresponding to the red, green and blue channels. Utilize the grayscale evaluation coefficient of each image block, combined with the preset initial pixel value adjustment coefficient and the pixel value compensation coefficient, calculate the pixel value adjustment coefficient corresponding to the red, green and blue channels. Utilize the brightness adjustment coefficient and the pixel value adjustment coefficient, combined with the pixel value adjustment model, adjust the pixel value of each pixel block in the overall image block area, thereby obtaining the image block after the pixel value adjustment.

[0191] The effect of the above technical solution is: by adjusting the pixel value, the brightness, contrast and color balance of the image can be optimized, thereby improving the overall quality of the image and making the image clearer and brighter. The image block after the pixel value adjustment has a better visual effect, which helps the subsequent feature extraction and recognition algorithm to more accurately identify the type and quantity of food. The technical solution calculates the brightness adjustment coefficient and the pixel value adjustment coefficient based on the gray value and gray evaluation coefficient of the image, so it can adapt to image data with different lighting conditions, different types and quantities of food.

[0192] On the other hand, by comprehensively considering the grayscale value and grayscale evaluation coefficient of the pixel block in the image block, the calculated brightness adjustment coefficient and pixel value adjustment coefficient can more accurately reflect the actual situation of the image, thereby achieving more accurate pixel value adjustment. The brightness adjustment coefficient and pixel value adjustment coefficient of the above technical solution can be dynamically adjusted according to the actual situation of the image data, so as to adapt to the processing requirements of different image data. The calculation process of the above technical solution is relatively simple and the amount of calculation is small. Therefore, it can improve the processing efficiency while ensuring the processing effect and meet the real-time requirements. At the same time, the above technical solution takes into account factors such as the pixel value compensation coefficient, which can reduce the impact of noise and interference on the processing results to a certain extent and enhance the robustness of the processing results.

[0193] In one embodiment of the present invention, food information in food image data corresponding to the target food is identified by a convolutional neural network model to obtain a food identification result, including:

[0194] S301, converting the food image data corresponding to the target food after image processing into a data format allowed to be input by the convolutional neural network model;

[0195] S302, retrieving a trained convolutional neural network model for food recognition from a database;

[0196] S303, inputting the food image data corresponding to the target food after data format conversion into the trained convolutional neural network model;

[0197] S304. After receiving the food image data corresponding to the target food, the convolutional neural network model identifies the food information in the food image data corresponding to the target food to obtain a food identification result.

[0198] The working principle of the above technical solution is as follows: First, before inputting the image data into the convolutional neural network model, the food image data after image processing needs to be converted into a data format allowed by the model. This usually includes adjusting the image to a specific size (such as 224x224 pixels) and color channel (such as RGB) accepted by the model, as well as possible normalization or standardization processing so that the model can correctly process the image data. Then, the trained convolutional neural network model for food recognition is retrieved from the database. This model is usually trained by a large amount of labeled food image data, and has the ability to extract features from images and recognize food. Subsequently, the food image data corresponding to the target food after the data format conversion is input into the loaded convolutional neural network model. This step is to pass the image data to the model so that the model can start processing and analyzing the image. Finally, when the convolutional neural network model receives the food image data corresponding to the target food, it starts processing and analyzing the image data. The model extracts key feature information from the image through a series of operations such as convolutional layers, pooling layers, activation functions, and fully connected layers, and recognizes the food based on these feature information. Ultimately, the model outputs a food recognition result, which is usually a label or probability distribution representing the food category.

[0199] The effect of the above technical solution is that the convolutional neural network model can efficiently process a large amount of image data and quickly give food recognition results. This is very useful for real-time applications or scenarios that require quick response. By using a large amount of labeled food image data for training, the convolutional neural network model can learn the key features of food and achieve a high recognition accuracy on the test data. This is very important for ensuring the reliability of food recognition. The convolutional neural network model has a certain robustness to slight changes in the image (such as lighting, rotation, scale, etc.), and can overcome the impact of these changes on the recognition results to a certain extent. This makes the model more stable and reliable in practical applications. The convolutional neural network model has good scalability, and the recognition performance can be further improved by adding more training data or adjusting the model structure. This enables the model to adapt to different application scenarios and needs.

[0200] In one embodiment of the present invention, performing nutritional analysis on the food information included in the food recognition result according to the food recognition result to obtain the nutritional analysis result includes:

[0201] S401, obtaining nutritional information corresponding to the food type included in the food recognition result from a database;

[0202] S402, obtaining each nutrient component and the total amount corresponding to each nutrient component from the nutrient component information;

[0203] S403, retrieving the first target component and the second target component recorded by the user from the database; wherein the first target component is the nutrient component that the user needs to increase intake; and the second target component is the nutrient component that the user needs to reduce intake;

[0204] S404, obtaining a nutritional evaluation value of the food included in the food identification result according to each nutritional component and the total amount corresponding to each nutritional component; wherein the nutritional evaluation value is obtained by the following formula:

[0205]

[0206] Wherein, Q represents the nutritional evaluation value; x represents the quantity of the first target component contained in the nutritional component information corresponding to the food type; M i represents the content ratio corresponding to the i-th first target component; y represents the amount of the second target component contained in the nutritional component information corresponding to the food type; M j represents the content ratio corresponding to the jth second target component; i and λ j They respectively represent the weight value corresponding to the i-th first target component and the weight value corresponding to the j-th second target component.

[0207] The working principle of the above technical solution is: determine the type of food according to the food recognition result, and retrieve the nutritional information corresponding to these food types from the database. These nutritional information usually include various nutrients contained in the food (such as protein, fat, carbohydrates, vitamins, minerals, etc.) and their corresponding content or total amount. From the retrieved nutritional information, extract each nutrient and its corresponding total amount. This information provides a data basis for subsequent nutritional assessment. Retrieve the first target component and the second target component recorded by the user from the database. The first target component usually refers to the nutrient that the user needs to increase intake, such as protein, vitamins, etc.; the second target component refers to the nutrient that the user needs to reduce intake, such as sugar, saturated fat, etc. The nutritional evaluation value (Q) of the food contained in the food recognition result is calculated using the given formula. The formula comprehensively considers the content ratio of the first target component and the second target component in the food and their corresponding weight values. Through calculation, a value is obtained to quantitatively evaluate the degree of match between the food and the user's nutritional needs.

[0208] The effect of the above technical solution is: the technical solution can perform nutritional assessment of food according to the personalized nutritional needs of the user (the first target component and the second target component). This helps users better understand the nutritional value of the food they consume and whether these foods meet their nutritional needs. By using a formula to calculate the nutritional assessment value, the nutritional assessment results are quantified, so that users can understand the nutritional value of food more intuitively. This quantitative assessment method helps to improve users' understanding and application of nutritional information. The technical solution can provide support for users' healthy diet management. By regularly analyzing the nutritional components and nutritional assessment results of the food they consume, users can adjust their eating habits to ensure adequate intake of nutrients while avoiding excessive intake of unhealthy nutrients. By conducting nutritional analysis in this way, users can pay more attention to the nutritional components and nutritional value of food, thereby improving their nutritional awareness. This is of positive significance for promoting the popularization and development of a healthy diet culture.

[0209] The embodiment of the present invention proposes an image-driven food recognition and nutrition analysis system. Figure 2 As shown, the image-driven food recognition and nutrition analysis system includes:

[0210] An image acquisition module, used to control the image acquisition device to acquire the target food and obtain the food image data corresponding to the target food;

[0211] An image processing module, used to perform image processing on the food image data corresponding to the target food, and obtain the food image data corresponding to the target food after image processing;

[0212] A food recognition module, used to recognize food information in the food image data corresponding to the target food through a convolutional neural network model to obtain a food recognition result;

[0213] The nutrition analysis module is used to perform nutrition analysis on the food information contained in the food recognition result according to the food recognition result to obtain the nutrition analysis result.

[0214] The working principle of the above technical solution is: first, use an image acquisition device (such as a smart phone, professional camera, etc.) to shoot the target food and obtain high-definition image data of the target food. This method is to obtain sufficiently clear and detailed food images and provide a basis for subsequent processing.

[0215] Then, the collected food image data is preprocessed to improve the accuracy of subsequent recognition. The processing process mainly includes image segmentation and pixel value adjustment. The food part in the image is separated from the background or other irrelevant parts to ensure the accuracy of subsequent recognition. The brightness, contrast and other parameters of the image are adjusted to make the food features in the image more prominent, which is convenient for subsequent feature extraction and recognition.

[0216] Subsequently, the processed food images are identified using a pre-trained convolutional neural network model (CNN). Through multi-layer convolution and pooling operations, the CNN model automatically learns features such as texture, shape, and color in the image to identify the type of food.

[0217] Finally, based on the results of food recognition and the pre-established food nutrition database, the food is analyzed for nutrition. The database contains information on the nutritional composition of various foods, such as protein, fat, carbohydrates, vitamins, etc. By matching the recognition results with the information in the database, the nutritional composition and calorie information of the food can be quickly and accurately obtained.

[0218] The effect of the above technical solution is: through automated image acquisition and processing, the time and labor intensity of manual participation are greatly reduced. At the same time, the convolutional neural network model based on deep learning can quickly and accurately identify food types, and realize efficient recognition and analysis of food. By performing preprocessing operations such as image segmentation and pixel value adjustment, the quality of the image and the recognizability of the features are improved, thereby improving the accuracy of recognition. In addition, combined with the pre-established food nutrition database, the nutritional content and calorie information of the food can be accurately calculated. Users only need to take photos of food using devices such as smartphones to quickly identify and analyze food. This method is simple and easy to use, and is suitable for food nutrition management needs in various scenarios. With the continuous increase in data volume and continuous optimization of the model, the recognition and analysis capabilities of this method will continue to improve. At the same time, by adding new food types and nutritional information to the database, the scope of application of this method can be further expanded.

[0219] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for food recognition and nutritional analysis based on image drive, characterized in that: The image-driven food recognition and nutrition analysis method includes: Controlling the image acquisition device to acquire the target food and obtain the food image data corresponding to the target food; Performing image processing on the food image data corresponding to the target food to obtain the food image data corresponding to the target food after image processing; wherein the image processing includes image segmentation and pixel value adjustment; Identify food information in the food image data corresponding to the target food using a convolutional neural network model to obtain a food identification result; Performing nutritional analysis on the food information contained in the food identification result according to the food identification result to obtain a nutritional analysis result; The step of performing image processing on the food image data corresponding to the target food to obtain the food image data corresponding to the target food after image processing includes: Performing noise reduction processing on the food image data corresponding to the target food to obtain the food image data after the noise reduction processing; Performing image block segmentation and image block pixel value adjustment on the food image data after the noise reduction processing, and obtaining the image block after the pixel value adjustment by image block stitching; The method of performing image block segmentation and image block pixel value adjustment on the food image data after the noise reduction processing, and obtaining the image block after the pixel value adjustment by image block stitching includes: Performing image block segmentation on the food image data after the noise reduction process to obtain a plurality of image blocks; Adjusting the pixel value of each image block corresponding to the food image data after the noise reduction processing to obtain the image block after the pixel value adjustment; The image blocks after the pixel values ​​are adjusted are combined to obtain food image data corresponding to the target food after the pixel values ​​are adjusted; The food image data after the noise reduction process is segmented into image blocks to obtain a plurality of image blocks, including: Copying the food image data after the noise reduction process to obtain copied food image data; grayscale processing is performed on the reproduced food image data after the noise reduction processing to obtain the reproduced food image data after the grayscale processing; Performing image segmentation on the grayscale processed replicated food image data according to a preset size of an initial image block to obtain a plurality of grayscale image blocks; wherein the size of the initial image block is a×a, and a is an odd number; Extracting the grayscale value corresponding to each pixel block in the grayscale image block; The grayscale evaluation coefficient of each grayscale image block is obtained according to the grayscale value corresponding to each pixel block in the grayscale image block; wherein the grayscale evaluation coefficient is obtained by the following formula: Where K represents the grayscale evaluation coefficient of each grayscale image block; n represents the number of pixel blocks contained in each grayscale image block except the central pixel block; X wi represents the gray value of the i-th pixel block except the central pixel block; X z Represents the gray value of the central pixel block of the gray image block; k 01 and k 02 represent the first coefficient and the second coefficient respectively; wherein the first coefficient is obtained as follows: When the grayscale value of the central pixel block is the maximum grayscale value of the pixel blocks included in the grayscale image block, the first coefficient is obtained by the following formula: Among them, X p Represents the grayscale average value of the grayscale image block; X min Indicates the minimum grayscale value of the pixel block included in the grayscale image block; When the grayscale value of the central pixel block is not the maximum grayscale value of the pixel blocks included in the grayscale image block, the first coefficient is obtained by the following formula: Among them, X p Represents the grayscale average value of the grayscale image block; X min represents the minimum grayscale value of the pixel block contained in the grayscale image block; X max Indicates the grayscale maximum value of the pixel block included in the grayscale image block; And, the second coefficient is obtained by the following formula: Among them, k 02 represents the second coefficient; X i represents the gray value corresponding to the i-th pixel block in the gray image block; X z Represents the gray value of the central pixel block of the gray image block; X p Represents the grayscale average value of the grayscale image block; Determining a constraint condition of a grayscale evaluation coefficient according to the grayscale evaluation coefficient of each grayscale image block; Comparing the grayscale evaluation coefficients of each two adjacent grayscale image blocks, and merging the two adjacent grayscale image blocks whose grayscale evaluation coefficients satisfy the constraint condition into one image block, to obtain a plurality of image blocks corresponding to the grayscale processed replicated food image data; Performing image block segmentation on the original food image data after the noise reduction processing according to the distribution of the multiple image blocks corresponding to the replicated food image data after the grayscale processing, to obtain the multiple image blocks corresponding to the food image data after the noise reduction processing; Wherein, determining the constraint condition of the grayscale evaluation coefficient according to the grayscale evaluation coefficient of each grayscale image block includes: Extracting a grayscale evaluation coefficient of the grayscale image block; Acquire a maximum value and a minimum value of a grayscale evaluation coefficient according to the grayscale evaluation coefficient of the grayscale image block; Extracting the central pixel block of the grayscale image block corresponding to the maximum value of the grayscale evaluation coefficient and the minimum value of the grayscale evaluation coefficient; Using the distance between the central pixel blocks of the grayscale image block corresponding to the maximum value of the grayscale evaluation coefficient and the minimum value of the grayscale evaluation coefficient as the first distance data; According to the grayscale evaluation coefficients of all the grayscale image blocks, obtaining median parameters corresponding to the grayscale evaluation coefficients of all the grayscale image blocks; Extracting a preset median parameter offset h, wherein the value range of the median parameter offset h is 3%-10% of the median parameter H; The coefficient parameter range is obtained by using the preset median parameter offset h in combination with the median parameter H; wherein the coefficient parameter range is H±h; Taking the grayscale image block whose grayscale evaluation coefficient meets the coefficient parameter as the target grayscale image block, and obtaining a plurality of target grayscale image blocks; Extracting the distance between the central pixel block of the grayscale image block corresponding to the minimum value of the grayscale evaluation coefficient and the central pixel block of each target grayscale image block as second distance data; The first distance data and a plurality of second distance data are used to obtain a first constraint condition compensation amount; wherein the first constraint condition compensation amount is obtained by the following formula: Among them, ξ 01 represents the compensation of the first constraint condition; S m represents the distance value corresponding to the first distance data; m represents the number of the second distance data; S i Indicates the distance value corresponding to the i-th second distance data; max(S i ) and min(S i ) represents the maximum distance and the minimum distance contained in the m second distance data; s represents the compensation adjustment coefficient, and the compensation adjustment coefficient is obtained by the following formula: Among them, S m represents the distance value corresponding to the first distance data; m represents the number of the second distance data; S i represents the distance value corresponding to the i-th second distance data; X m01 represents the grayscale value of the central pixel block of the grayscale image block with the maximum distance value contained in the second distance data; m02 The grayscale value of the central pixel block of the grayscale image block representing the minimum distance value contained in the second distance data; The second constraint condition compensation amount is obtained by using the grayscale evaluation coefficients of all the grayscale image blocks; wherein the second constraint condition compensation amount is obtained by the following formula: Among them, ξ 02 represents the compensation amount of the second constraint condition; r represents the total number of grayscale image blocks; K i represents the grayscale evaluation coefficient corresponding to the i-th grayscale image block; X zi represents the gray value of the central pixel block corresponding to the i-th gray image block; X zp Represents the standard deviation of the grayscale average values ​​corresponding to all grayscale image blocks; Extracting preset initial constraints from a database; The first constraint condition compensation amount and the second constraint condition compensation amount are combined with the preset initial constraint condition to obtain the constraint condition, wherein the constraint condition is obtained by the following formula: Among them, X up Indicates the upper limit value of the constraint condition; X up0 represents the upper limit of the initial constraint; X down Indicates the lower limit of the constraint condition; X down0 Represents the lower limit value of the initial constraint.

2. The image-driven food recognition and nutrition analysis method according to claim 1, characterized in that: Adjusting the pixel value of each image block corresponding to the food image data after the noise reduction processing to obtain the image block after the pixel value adjustment includes: Extracting the entire area of ​​the image block of the grayscale processed replicated food image data corresponding to each image block of the food image data after the noise reduction processing; Extracting the grayscale value of the pixel blocks contained in the entire area of ​​the image block of the replicated food image data after the grayscale processing corresponding to each image block; Obtaining a brightness adjustment coefficient using the grayscale values ​​of the pixel blocks contained in the entire area of ​​the image block; Obtaining a pixel value adjustment coefficient using the grayscale evaluation coefficient contained in each image block of the food image data after the noise reduction processing; Using the brightness adjustment coefficient and the pixel value adjustment coefficient in combination with a pixel value adjustment model, the pixel value of each image block included in the overall area of ​​each image block is adjusted to obtain an image block after the pixel value is adjusted; The pixel value adjustment model structure is as follows: Among them, R e , G e and B e Respectively represent the enhanced red channel, green channel and blue channel corresponding values ​​of each pixel block in the overall image block area; R, G and B represent the original red channel, green channel and blue channel corresponding values ​​of each pixel block in the overall image block area; t r ,t g and t b Respectively represent the brightness adjustment coefficients corresponding to the red channel, green channel and blue channel; h r 、h g and h b Respectively represent the pixel value adjustment coefficients corresponding to the red channel, green channel, and blue channel.

3. The image-driven food recognition and nutrition analysis method according to claim 2, characterized in that: The brightness adjustment coefficient and the pixel value adjustment coefficient are obtained by the following formula: Among them, t r ,t g and t b They represent the brightness adjustment coefficients corresponding to the red channel, green channel, and blue channel respectively; z represents the number of pixel blocks contained in the overall area of ​​the image block; X yi represents the gray value of the i-th pixel block contained in the entire area of ​​the image block; X zpxi represents the grayscale average value of the central pixel block of the image block contained in the overall area of ​​the image block, and when an image block is contained in the overall area of ​​the image block, X zpxi It represents the gray value of the central pixel block of the image block; R p , G p and B p The numerical averages of the red channel, green channel and blue channel of the image block contained in the overall area of ​​the image block respectively; t r0 ,t g0 and t b0 Respectively represent the initial brightness adjustment coefficients corresponding to the preset red channel, green channel and blue channel; Among them, h r 、h g and h b Respectively represent the pixel value adjustment coefficients corresponding to the red channel, green channel and blue channel; h r0 、h g0 and h b0 Respectively represent the initial pixel value adjustment coefficients corresponding to the preset red channel, green channel and blue channel; σ k represents a pixel value compensation coefficient, and the pixel value compensation coefficient is obtained by the following formula: Where p represents the number of image blocks contained in the overall area of ​​the image block; K xi represents the grayscale evaluation coefficient of the grayscale image block of the replicated food image data corresponding to the i-th image block contained in the entire image block area; K p K represents the average value of the grayscale evaluation coefficients of the grayscale image blocks of the replicated food image data corresponding to the p image blocks contained in the entire image block area; max represents the maximum value of the grayscale evaluation coefficient of the grayscale image block of the replicated food image data corresponding to the p image blocks contained in the overall image block area; at the same time, when the number of image blocks contained in the overall image block area is 1, let σ k =0.

4. The image-driven food recognition and nutrition analysis method according to claim 1, characterized in that: The food information in the food image data corresponding to the target food is identified by a convolutional neural network model to obtain a food identification result, including: Converting the food image data corresponding to the target food after image processing into a data format allowed to be input by the convolutional neural network model; Retrieve a trained convolutional neural network model for food recognition from a database; Inputting the food image data corresponding to the target food after data format conversion into the trained convolutional neural network model; After receiving the food image data corresponding to the target food, the convolutional neural network model identifies the food information in the food image data corresponding to the target food to obtain a food identification result.

5. The image-driven food recognition and nutrition analysis method according to claim 1, characterized in that: Performing a nutritional analysis on the food information contained in the food identification result according to the food identification result to obtain a nutritional analysis result includes: Nutrition information corresponding to the food types included in the food recognition results from a database; Obtaining each nutrient component and the total amount corresponding to each nutrient component from the nutrient component information; Retrieving the first target component and the second target component recorded by the user from the database; wherein the first target component is the nutrient component that the user needs to increase intake; and the second target component is the nutrient component that the user needs to reduce intake; The nutritional evaluation value of the food included in the food identification result is obtained according to each nutritional component and the total amount corresponding to each nutritional component; wherein the nutritional evaluation value is obtained by the following formula: Wherein, Q represents the nutritional evaluation value; x represents the quantity of the first target component contained in the nutritional component information corresponding to the food type; M i represents the content ratio corresponding to the i-th first target component; y represents the amount of the second target component contained in the nutritional component information corresponding to the food type; M j represents the content ratio corresponding to the jth second target component; i and λ j They respectively represent the weight value corresponding to the i-th first target component and the weight value corresponding to the j-th second target component.

6. An image-driven food recognition and nutrition analysis system, characterized in that: The image-driven food recognition and nutrition analysis system includes: An image acquisition module, used to control the image acquisition device to acquire the target food and obtain the food image data corresponding to the target food; An image processing module, used to perform image processing on the food image data corresponding to the target food, and obtain the food image data corresponding to the target food after image processing; A food recognition module, used to recognize food information in the food image data corresponding to the target food through a convolutional neural network model to obtain a food recognition result; A nutrition analysis module, used for performing nutrition analysis on the food information contained in the food recognition result according to the food recognition result, and obtaining a nutrition analysis result; The step of performing image processing on the food image data corresponding to the target food to obtain the food image data corresponding to the target food after image processing includes: Performing noise reduction processing on the food image data corresponding to the target food to obtain the food image data after the noise reduction processing; Performing image block segmentation and image block pixel value adjustment on the food image data after the noise reduction processing, and obtaining the image block after the pixel value adjustment by image block stitching; The method of performing image block segmentation and image block pixel value adjustment on the food image data after the noise reduction processing, and obtaining the image block after the pixel value adjustment by image block stitching includes: Performing image block segmentation on the food image data after the noise reduction process to obtain a plurality of image blocks; Adjusting the pixel value of each image block corresponding to the food image data after the noise reduction processing to obtain the image block after the pixel value adjustment; The image blocks after the pixel values ​​are adjusted are combined to obtain food image data corresponding to the target food after the pixel values ​​are adjusted; The food image data after the noise reduction process is segmented into image blocks to obtain a plurality of image blocks, including: Copying the food image data after the noise reduction process to obtain copied food image data; grayscale processing is performed on the reproduced food image data after the noise reduction processing to obtain the reproduced food image data after the grayscale processing; Performing image segmentation on the grayscale processed replicated food image data according to a preset size of an initial image block to obtain a plurality of grayscale image blocks; wherein the size of the initial image block is a×a, and a is an odd number; Extracting the grayscale value corresponding to each pixel block in the grayscale image block; The grayscale evaluation coefficient of each grayscale image block is obtained according to the grayscale value corresponding to each pixel block in the grayscale image block; wherein the grayscale evaluation coefficient is obtained by the following formula: Where K represents the grayscale evaluation coefficient of each grayscale image block; n represents the number of pixel blocks contained in each grayscale image block except the central pixel block; X wi represents the gray value of the i-th pixel block except the central pixel block; X z Represents the gray value of the central pixel block of the gray image block; k 01 and k 02 represent the first coefficient and the second coefficient respectively; wherein the first coefficient is obtained as follows: When the grayscale value of the central pixel block is the maximum grayscale value of the pixel blocks included in the grayscale image block, the first coefficient is obtained by the following formula: Among them, X p Represents the grayscale average value of the grayscale image block; X min Indicates the minimum grayscale value of the pixel block included in the grayscale image block; When the grayscale value of the central pixel block is not the maximum grayscale value of the pixel blocks included in the grayscale image block, the first coefficient is obtained by the following formula: Among them, X p Represents the grayscale average value of the grayscale image block; X min represents the minimum grayscale value of the pixel block contained in the grayscale image block; X max Indicates the grayscale maximum value of the pixel block included in the grayscale image block; And, the second coefficient is obtained by the following formula: Among them, k 02 represents the second coefficient; X i represents the gray value corresponding to the i-th pixel block in the gray image block; X z Represents the gray value of the central pixel block of the gray image block; X p Represents the grayscale average value of the grayscale image block; Determining a constraint condition of a grayscale evaluation coefficient according to the grayscale evaluation coefficient of each grayscale image block; Comparing the grayscale evaluation coefficients of each two adjacent grayscale image blocks, and merging the two adjacent grayscale image blocks whose grayscale evaluation coefficients satisfy the constraint condition into one image block, to obtain a plurality of image blocks corresponding to the grayscale processed replicated food image data; Performing image block segmentation on the original food image data after the noise reduction processing according to the distribution of the multiple image blocks corresponding to the replicated food image data after the grayscale processing, to obtain the multiple image blocks corresponding to the food image data after the noise reduction processing; Wherein, determining the constraint condition of the grayscale evaluation coefficient according to the grayscale evaluation coefficient of each grayscale image block includes: Extracting a grayscale evaluation coefficient of the grayscale image block; Acquire a maximum value and a minimum value of a grayscale evaluation coefficient according to the grayscale evaluation coefficient of the grayscale image block; Extracting the central pixel block of the grayscale image block corresponding to the maximum value of the grayscale evaluation coefficient and the minimum value of the grayscale evaluation coefficient; Using the distance between the central pixel blocks of the grayscale image block corresponding to the maximum value of the grayscale evaluation coefficient and the minimum value of the grayscale evaluation coefficient as the first distance data; According to the grayscale evaluation coefficients of all the grayscale image blocks, obtaining median parameters corresponding to the grayscale evaluation coefficients of all the grayscale image blocks; Extracting a preset median parameter offset h, wherein the value range of the median parameter offset h is 3%-10% of the median parameter H; The coefficient parameter range is obtained by using the preset median parameter offset h in combination with the median parameter H; wherein the coefficient parameter range is H±h; Taking the grayscale image block whose grayscale evaluation coefficient meets the coefficient parameter as the target grayscale image block, and obtaining a plurality of target grayscale image blocks; Extracting the distance between the central pixel block of the grayscale image block corresponding to the minimum value of the grayscale evaluation coefficient and the central pixel block of each target grayscale image block as second distance data; The first distance data and a plurality of second distance data are used to obtain a first constraint condition compensation amount; wherein the first constraint condition compensation amount is obtained by the following formula: Among them, ξ 01 represents the compensation of the first constraint condition; S m represents the distance value corresponding to the first distance data; m represents the number of the second distance data; S i Indicates the distance value corresponding to the i-th second distance data; max(S i ) and min(S i ) represents the maximum distance and the minimum distance contained in the m second distance data; s represents the compensation adjustment coefficient, and the compensation adjustment coefficient is obtained by the following formula: Among them, S m represents the distance value corresponding to the first distance data; m represents the number of the second distance data; S i represents the distance value corresponding to the i-th second distance data; X m01 represents the grayscale value of the central pixel block of the grayscale image block with the maximum distance value contained in the second distance data; m02 The grayscale value of the central pixel block of the grayscale image block representing the minimum distance value contained in the second distance data; The second constraint condition compensation amount is obtained by using the grayscale evaluation coefficients of all the grayscale image blocks; wherein the second constraint condition compensation amount is obtained by the following formula: Among them, ξ 02 represents the compensation amount of the second constraint condition; r represents the total number of grayscale image blocks; K i represents the grayscale evaluation coefficient corresponding to the i-th grayscale image block; X zi represents the gray value of the central pixel block corresponding to the i-th gray image block; X zp Represents the standard deviation of the grayscale average values ​​corresponding to all grayscale image blocks; Extracting preset initial constraints from a database; The first constraint condition compensation amount and the second constraint condition compensation amount are combined with the preset initial constraint condition to obtain the constraint condition, wherein the constraint condition is obtained by the following formula: Among them, X up Indicates the upper limit of the constraint condition; X up0 represents the upper limit value of the initial constraint; X down Indicates the lower limit of the constraint condition; X down0 Represents the lower limit value of the initial constraint.

Images