Image-driven restaurant health assessment method

Abstract

The invention discloses an image-driven restaurant health assessment method, and the method comprises the following steps: collecting restaurant data in a main built-up area range of a research city, including names and cuisines of restaurants, names and pictures of top ten hot sell dishes, and longitude and latitude information of the restaurants; building a scoring platform for health degree evaluation based on hot food selling pictures, and obtaining health degree scores of a part of restaurants through volunteers' sampling and scoring; supplementing restaurant pictures according to dish names of restaurants in the research range; feature recognition of food materials and picture colors is performed based on the hot sell pictures, and cooking mode marking is performed according to the names of cuisines; and by taking the scoring label data of the sample restaurant as dependent variables and taking the characteristic values of the food materials, the cooking modes and the picture colors as independent variables, constructing a random forest model to evaluate the health degree of the whole sample restaurant in the research area. The method for feeding back the catering health level through the picture information is constructed based on machine learning, and a simple and direct restaurant health degree evaluation mode is provided.

Description

Technical Field

[0001] The invention relates to the field of dietary geospatial research, and in particular to an image-driven restaurant health assessment method. Background Art

[0002] Compared with home cooking, dining out is usually more convenient, but it is also accompanied by problems such as high calorie diet and unbalanced nutrition. This unbalanced eating pattern has been considered an important cause of non-communicable chronic diseases such as obesity and cardiovascular disease in people of all ages.

[0003] Healthy diet is a key area in the process of promoting the construction of healthy cities. The concept of healthy cities not only covers traditional aspects such as environmental sanitation, public medical services and health education, but also includes improving the quality of residents' diet and nutritional status. To achieve this goal, it is crucial to fully understand the nutritional quality of food eaten out by residents in urban environments and its impact on the health of urban residents.

[0004] Therefore, in order to fully understand the health level of the living and eating environment of urban residents, understand the nutritional quality of food eaten out and its impact on the city, it is crucial to use appropriate methods and data to evaluate the health level of food provided by restaurants and the choice of restaurants for urban residents in different urban environments. Current traditional evaluation methods include environmental evaluation such as restaurant classification and individual-level dietary evaluation. The former does not take into account individual dietary preferences and behaviors, and the latter is costly and time-consuming in the data collection process, making it difficult to conduct large-scale research.

[0005] To promote dietary health and nutritional improvement in the context of Chinese cities, it is necessary to explore more accurate and effective food nutrition assessment methods in combination with local dietary culture and habits. This will not only help optimize personal dietary choices, but also provide a scientific basis for the formulation of public health policies, thereby promoting the construction of healthy cities and sustainable development. With the advancement of information technology, researchers have begun to use deep learning models to identify food types and calculate food calories. However, due to the complexity of Chinese cuisine and the variety of ingredients, there is currently no complete method to judge the health level of local Chinese food. Summary of the invention

[0006] In view of the above-mentioned problems existing in the prior art, the present invention proposes an image-driven restaurant health assessment method. First, the healthiness of the dishes in the sampled restaurant are scored and assigned healthiness labels. The dish features are extracted based on the food images through a computer vision recognition model and the basic information of the restaurant. Subsequently, the health evaluation of the restaurant is extrapolated from the small sampling sample to a large sample through a random forest machine learning model, which can provide a technical data basis and a new perspective for understanding the regional dietary environment and the fairness of healthy diet.

[0007] The technical solution of the present invention is as follows: An image-driven restaurant health assessment method includes the following steps: S1. Collect spatial vector data of catering spots within the main built-up areas of the research city, define them as restaurant samples, and obtain a set R; the spatial vector data of catering spots includes the cuisine, hot dishes, and longitude and latitude information of each restaurant; combine the crowdsourcing platform data to match pictures of the hot dishes of each restaurant in R, i.e., hot dish pictures; S2. Build a restaurant health evaluation platform SP based on restaurant hot-selling dishes pictures; select a part of restaurants as sample restaurants r through SP, invite volunteers to score the health of sample restaurants r, and calculate the health score H of sample restaurants r based on the scoring data. The set of health scores H is the health label Sr of the sample restaurants; S3, extracting key features of restaurant health evaluation based on restaurant hot-selling dishes pictures, namely restaurant health evaluation features; the restaurant health evaluation features include food features XI, physical features XF and cuisine features XT; the physical features XF include color X col , Complexity X com , brightness change X sf , Brightness X b , Contrast X con ; S4. Based on the health label Sr of the sample restaurant r and the restaurant health evaluation features, a random forest model is constructed, and all restaurants R in the research scope are extrapolated to obtain the health evaluation characteristics of each restaurant R. j Health Score SR j .

[0008] Furthermore, the specific steps of S2 are as follows: S2-1. Perform stratified sampling based on cuisine, and select 20% of R as the survey sample r of SP; the restaurant in r is recorded as r i ; S2-2, volunteers respond to r based on the hot-selling dishes pictures randomly given by SP i The health of the person is scored on a scale of 1, 2, 3, 4, or 5, with 1 being the least healthy and 5 being the healthiest. S2-3, SP randomly selects 50 r from r i Give each volunteer a health score to ensure that each i The number of times s-count is evaluated i ≥8 times; i The scoring sample is s i ; S2-4. Calculate s i The harmonic mean of i Health score H i; Each restaurant r in the sample restaurant r i Health score H i The collection is the healthiness label Sr of the sample restaurant.

[0009] Furthermore, the specific steps of S3 are as follows: S3-1. Use the food image recognition model based on the VireoFood-172 dataset to identify each restaurant r in sample r. i The ingredients of the best-selling dishes are identified by using the pictures, and the results are supplemented by the recognition results of the ClarifaiAPI open interface platform. The ingredient recognition results are deduplicated and used as the restaurant r i Food FeaturesXI i ; S3-2. Calculate restaurant r i The average value of the ratio of red, green, and blue channels on all non-white pixels in all popular dishes images is taken as the restaurant r i Color X col ; S3-3. Calculate restaurant r i The ratio of the number of contour-related pixels of different objects in each popular dish image to the total number of non-white pixels, r i The average complexity of all popular dishes images is taken as the restaurant r i Complexity X com ; S3-4, calculate the radial average of the two-dimensional fast Fourier transform two-dimensional power spectrum of each hot-selling dish picture grayscale image; restaurant r i The average of the radial means of all the popular dishes is taken as the restaurant r i Brightness change X sf ; S3-5, calculate the difference between the average brightness of all non-white pixels in the grayscale image of the hot-selling dishes and the white background; restaurant r i The average difference of all the popular dishes pictures is taken as the restaurant r i Brightness X b ; S3-6, calculate the standard deviation of the brightness of all non-white pixels in the grayscale image of the hot-selling dishes; restaurant r i The average standard deviation of all popular dishes is taken as the restaurant r i Contrast ratio X con .

[0010] Furthermore, the specific steps of S4 are as follows: S4-1, process the data from S1 to S3 and obtain the final health evaluation characteristics. The specific steps are as follows: S4-1-1. Group all the ingredients of the hot-selling dishes in restaurant R into a set to obtain an ingredient list I; encode the ingredient features XI. If restaurant r i If it contains an ingredient in ingredient list I, mark the ingredient as 1, otherwise mark it as 0; S4-1-2. Perform z-score standardization on the physical features XF of all restaurants in R; S4-1-3, perform serial number coding on the cuisine feature XT as a categorical variable; S4-2. Use five classification methods, namely, natural break method, quantile method, standard deviation method, equal interval method and geometric interval method, to perform data-driven classification of the health of R, and divide the restaurants into four categories: healthy, relatively healthy, relatively unhealthy and unhealthy. After traversing different classification methods and classification categories, obtain the classification results of different methods and count the number of restaurants in each category, eliminate the classification method with very few restaurants, and obtain the final restaurant health classification. S4-3, using the restaurant health assessment features obtained in S4-1 as independent variables, the restaurant health classification obtained in S4-2 as dependent variables, 80% of the sample restaurants r as training sets and 20% as test sets, perform SMOTE oversampling random forest modeling; compare the accuracy of the test sets, further eliminate the classification methods with test set accuracy lower than 0.8, and obtain the pre-model M0; perform regularization processing and further feature selection on M0 to obtain the restaurant health assessment model M1; S4-4. Predict the healthiness of R based on the popular dishes in R and the characteristic factors of R obtained by M1, and obtain a healthiness score set SR of all restaurants R within the research scope, wherein SR includes the healthiness score of each restaurant R. j The health evaluation results of SR j .

[0011] The beneficial technical effects of the present invention are: (1) Based on the image data of popular dishes on the crowdsourcing platform, we reflected consumers’ dietary preferences and integrated the restaurant health characteristic factors driven by image data to propose an evaluation system that is suitable for Chinese cuisine; (2) Using a data-driven approach to classify restaurant health categories avoids the need to preset health thresholds to classify different health levels, and can better combine consumer preferences and actual dining environments to distinguish subtle differences in health levels between restaurants; (3) By adopting machine learning methods, we extrapolated the label scoring data and characteristic factors of a small sample to a large sample within the research scope and established a universal image data-driven restaurant big data health assessment method, which filled the gap in the health attributes of catering space data and provided a big data foundation for urban dining environment research. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a comparison chart of the prediction results of the random forest model corresponding to different classification methods. DETAILED DESCRIPTION

[0013] The present invention is described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] The present invention is used to evaluate the healthiness of food in 110,000 restaurants in Shanghai (excluding Chongming Island). The specific steps are as follows: S1. Collect restaurant data within the land area of ​​Shanghai, including restaurant hot-selling dishes, hot-selling dishes pictures, cuisine and address information. Convert geographic information into longitude and latitude information through geocoding and process all restaurants R into shapefile spatial data (coordinate system: WGS1984). And select restaurants R that lack hot-selling dishes pictures. i According to the hot-selling dishes catalog information, call the FuzzyWuzzy library in Python to match the pictures.

[0015] S2. Select survey sample r from the 110,000 restaurant dataset based on the attributes of the cuisine, and use the pictures of the best-selling dishes in r to build a restaurant health rating platform SP based on the best-selling dishes. Call on volunteers to rate the health of the restaurants in r through the platform SP. Each volunteer will randomly select 50 restaurants from r for evaluation and rating. Each time, the restaurant selected is the restaurant with the least number of ratings, and the number of ratings s-count for each restaurant in r is ≥ 8 times, so that each restaurant in r has a health rating of 50. i Can get multiple ratings from different volunteers, r i Rating Collections i The harmonic mean H(s i ) as the number of restaurants in r i The final health score H i . Each restaurant r in the sample restaurant r i Health score H i The collection is the healthiness label Sr of the sample restaurant.

[0016] S3. Extract the key features of restaurant health evaluation based on the restaurant's popular dishes, namely, restaurant health evaluation features. As shown in Table 1, restaurant health evaluation features include food features XI, physical features XF, and cuisine features XT; among which physical features XF include color X col , Complexity X com, brightness change X sf , Brightness X b , Contrast X con The specific steps to extract restaurant health evaluation features are as follows: Table 1 Restaurant health assessment characteristics

[0017] S3-1. To ensure the consistency of subsequent image processing, call the skimage library in Python to use the transform.resize function to resize all images to 256*256, and use the img_as_float function to convert the image to floating point format to ensure that the pixel value is between [0,1]. Return the preprocessed image M1 and use rgb2gray to convert it into a grayscale image M2 for subsequent operations.

[0018] S3-2. Call the image recognition model M1 trained based on the Vireofood-172 image dataset and the Clarifai food image recognition model M2 to identify the ingredients of the hot-selling dishes, respectively, and obtain the ingredient lists I1 and I2. Since M1 is a model based on Chinese cuisine, the ingredient list I1 is used as the standard, and I2 is used as a supplement to obtain the final ingredient list I. The restaurant's ingredient feature factor XI is obtained by encoding the restaurant's ingredients in a 0-1 manner.

[0019] S3-3, perform color feature X on the preprocessed image M1 col Extract, call image.mean in Python to calculate the mean of each channel of the image, and the mean of the result of each popular dish picture of the restaurant is the color X of the restaurant col (Rmean,Gmean,Bmean).

[0020] S3-4. Perform complexity X on the preprocessed image com The canny function of the skimage library is used to extract the edge information in the grayscale image M2. Similarly, the mean number of edges calculated for the restaurant's popular dishes is the complexity X of the restaurant. com .

[0021] S3-5, perform brightness change X on the pre-processed image sf Extract the image frequency domain information in the grayscale image M2, call the fftpack function of the scipy library, measure the spatial frequency change of brightness, calculate the energy of the high-frequency part, and calculate the average value of the value of the restaurant's popular dishes as the brightness change X of the restaurant sf .

[0022] S3-6, perform brightness X on the pre-processed image bExtract the grayscale image M2 and calculate the average pixel value. The average value of the popular dishes is used as the brightness X of the restaurant. b .

[0023] S3-7, perform contrast feature X on the preprocessed image con The grayscale value standard deviation of the grayscale image M2 is calculated to represent the distribution of image brightness. The mean of the restaurant's popular dishes is used as the contrast X of the restaurant. con .

[0024] S4. Define ingredients XI, physical features XF, cuisine XT and per capita price XP as characteristic factors; construct a random forest model based on the health label Sr and characteristic factors of restaurant sample r, and extrapolate the health evaluation SR of each restaurant in R based on the characteristic factors of restaurant R within the land area of ​​Shanghai. The specific steps are as follows; S4-1, process the data from S1 to S3 and obtain the final restaurant health evaluation features. The specific steps are as follows: S4-1-1. Group all the ingredients of the hot-selling dishes in restaurant R into a set to obtain an ingredient list I; encode the ingredient features XI. If restaurant r i If it contains an ingredient in ingredient list I, mark the ingredient as 1, otherwise mark it as 0; S4-1-2. Perform z-score standardization on the physical features XF of all restaurants in R; S4-1-3, perform serial number coding on the cuisine feature XT as a categorical variable; S4-1-4. Perform z-score standardization on the per capita price XP.

[0025] S4-2, five classification methods, namely natural break method, quantile method, standard deviation method, equal interval method and geometric interval method, are used to perform data-driven classification of the healthiness of R; after traversing different classification methods and classification categories, different classification results are obtained and the number of restaurants in each category is counted, such as Figure 1 After eliminating the classification method with very few restaurants, the four-category classification method with equal interval method was finally used to perform data-driven classification of the healthiness of r, and the restaurants were divided into four categories: healthy, relatively healthy, relatively unhealthy, and unhealthy.

[0026] S4-3. Use the restaurant health assessment features obtained in S4-1 as independent variables and the restaurant health classification obtained in S4-2 as the dependent variable to perform random forest modeling.

[0027] S4-4. By calling the SMOTE function of the imblearn library in Python, oversample the category data set with less data in the dependent variable classification to ensure that the number of learning for each category is the same.

[0028] S4-5, using 80% as the training set and 20% as the test set, perform random forest modeling by calling the RandomForestClassifier function of the sklearn library in Python to obtain model MD1. Calculate the feature importance through MD1, and retain the features with importance greater than 0.002, which are recorded as feature F1.

[0029] S4-6. Based on the feature F1 obtained after feature selection, the RandomForestClassifier function of the sklearn library in Python is called again to perform random forest modeling to obtain model MD2, and regularization adjustment is performed based on MD2. GridSearchCV in the sklearn library in Python is called to perform grid search, and hyperparameters such as the depth of the tree in the model, the number of samples required for the minimum node, and the total number of trees are adjusted to determine the hyperparameters of the best performance model without overfitting, and the final model M is trained based on this.

[0030] S4-7, call the predict function in the sklearn library, call the final model M, and make predictions for the remaining restaurants that have not been labeled with healthiness. According to the extrapolation results of the model, the healthiness evaluation results of restaurants within the land area of ​​Shanghai are finally obtained.

[0031] S5. Call the Geopandas and Matplotlib libraries in Python to perform spatial visualization of the health score results of all restaurants within the land area of ​​Shanghai. There are four health levels in total, with level 1 being the least healthy and level 4 being the healthiest.

[0032] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and implementation modes, and can be fully applicable to various fields suitable for the present invention. For those familiar with the art, for those of ordinary skill in the art, various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present invention. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to specific details.

Claims

1. An image-driven restaurant health assessment method, characterized in that: The following steps are involved: S1. Collect spatial vector data of catering spots within the main built-up areas of the research city, define them as restaurant samples, and obtain a set R; the spatial vector data of catering spots includes the cuisine, hot dishes, and longitude and latitude information of each restaurant; combine the crowdsourcing platform data to match pictures of the hot dishes of each restaurant in R, i.e., hot dish pictures; S2, build a restaurant health evaluation platform SP based on restaurant hot-selling dishes pictures; A number of restaurants are selected as sample restaurants r through SP, and volunteers are invited to score the health of sample restaurants r. The health score H of sample restaurants r is calculated based on the scoring data. The set of health scores H is the health label Sr of the sample restaurants. S3, extracting key features of restaurant health evaluation based on restaurant hot-selling dishes pictures, namely restaurant health evaluation features; the restaurant health evaluation features include food features XI, physical features XF and cuisine features XT; the physical features XF include color X col , Complexity X com , brightness change X sf , Brightness X b , Contrast X con ; S4. Based on the health label Sr of the sample restaurant r and the restaurant health evaluation features, a random forest model is constructed, and all restaurants R in the research scope are extrapolated to obtain the health evaluation characteristics of each restaurant R. j Health score SR j .

2. The image-driven restaurant health assessment method according to claim 1, characterized in that: The specific steps of S2 are as follows: S2-1. Perform stratified sampling based on cuisine, and select 20% of R as the survey sample r of SP; the restaurant in r is recorded as r i ; S2-2, volunteers respond to r based on the hot-selling dishes pictures randomly given by SP i The health of the person is scored on a scale of 1, 2, 3, 4, or 5, with 1 being the least healthy and 5 being the healthiest. S2-3, SP randomly selects 50 r from r i Give each volunteer a health score to ensure that each i The number of times s-count is evaluated i ≥8 times; i The scoring sample is s i ; S2-4. Calculate s i The harmonic mean of i Health score H i ; Each restaurant r in the sample restaurant r i Health score H i The collection is the healthiness label Sr of the sample restaurant.

3. The image-driven restaurant health assessment method according to claim 2, characterized in that: The specific steps of S3 are as follows: S3-1. Use the food image recognition model based on the VireoFood-172 dataset to identify each restaurant r in sample r. i The ingredients of the best-selling dishes are identified by using the pictures, and the results are supplemented by the recognition results of the ClarifaiAPI open interface platform. The ingredient recognition results are deduplicated and used as the restaurant r i Food FeaturesXI i ; S3-2. Calculate restaurant r i The average value of the ratio of red, green, and blue channels on all non-white pixels in all popular dishes images is taken as the restaurant r i Color X col ; S3-3. Calculate restaurant r i The ratio of the number of contour-related pixels of different objects in each popular dish image to the total number of non-white pixels, r i The average complexity of all popular dishes images is taken as the restaurant r i Complexity X com ; S3-4, calculate the radial average of the two-dimensional fast Fourier transform two-dimensional power spectrum of each hot-selling dish picture grayscale image; restaurant r i The average of the radial means of all the popular dishes is taken as the restaurant r i Brightness change X sf ; S3-5, calculate the difference between the average brightness of all non-white pixels in the grayscale image of the hot-selling dishes and the white background; restaurant r i The average difference of all the popular dishes pictures is taken as the restaurant r i Brightness X b ; S3-6, calculate the standard deviation of the brightness of all non-white pixels in the grayscale image of the hot-selling dishes; restaurant r i The average standard deviation of all popular dishes images is taken as the restaurant r i Contrast ratio X con .

4. The image-driven restaurant health assessment method according to claim 3, characterized in that: The specific steps of S4 are as follows: S4-1, process the data from S1 to S3 and obtain the final health evaluation characteristics. The specific steps are as follows: S4-1-1. Group all the ingredients of the hot-selling dishes in restaurant R into a set to obtain an ingredient list I; encode the ingredient features XI. If restaurant r i If it contains an ingredient in ingredient list I, mark the ingredient as 1, otherwise mark it as 0; S4-1-2. Perform z-score standardization on the physical features XF of all restaurants in R; S4-1-3, perform serial number coding on the cuisine feature XT as a categorical variable; S4-2. Use five classification methods, namely, natural break method, quantile method, standard deviation method, equal interval method and geometric interval method, to perform data-driven classification of the health of R, and divide the restaurants into four categories: healthy, relatively healthy, relatively unhealthy and unhealthy. After traversing different classification methods and classification categories, obtain the classification results of different methods and count the number of restaurants in each category, eliminate the classification method with very few restaurants, and obtain the final restaurant health classification. S4-3, using the restaurant health assessment features obtained in S4-1 as independent variables, the restaurant health classification obtained in S4-2 as dependent variables, 80% of the sample restaurants r as training sets and 20% as test sets, perform SMOTE oversampling random forest modeling; compare the accuracy of the test sets, further eliminate the classification methods with test set accuracy lower than 0.8, and obtain the pre-model M0; perform regularization processing and further feature selection on M0 to obtain the restaurant health assessment model M1; S4-4. Predict the healthiness of R based on the popular dishes in R and the characteristic factors of R obtained by M1, and obtain a healthiness score set SR of all restaurants R within the research scope, wherein SR includes the healthiness score of each restaurant R. j The health evaluation results of SR j .

Images