Multi-angle tourist destination image perception method based on deep learning technology

Abstract

The invention discloses a multi-angle tourist destination image perception method based on a deep learning technology, belongs to the technical field of machine learning, is used for tourist destination image perception, and comprises the following steps: obtaining and processing photo data generated by tourists; performing scene identification and classification by using the processed photo data; based on a visual aesthetics quality model, predicting photo aesthetics evaluation of tourists; analyzing the emotional experience of the tourist to the tourist destination by using a visual emotional algorithm; and obtaining the perception of the tourist on the image of the tourist destination from a spatial dimension. Compared with the prior art, the method can better capture a construction scene and visual features in a tourism image, integrates visual aesthetics and emotional experience into tourism destination image perception, can more comprehensively understand the love and preference of tourists, depends on tourism resource distribution of destinations, carries out spatial visualization on the tourism image, and improves the tourism experience. And tourism resources of destinations are reasonably allocated for managers, so that suggestions are provided for tourism planning and destination image improvement.

Description

Technical Field

[0001] The invention discloses a method for multi-angle tourist destination image perception based on deep learning technology, belonging to the technical field of machine learning. Background Art

[0002] The image of tourist destinations has always been a hot topic in tourism geography research. Traditional studies on the image of tourist destinations use data in the form of questionnaires and texts, but they have problems such as low statistical efficiency, limited spatial coverage, and inaccurate estimation of tourist size. Previous studies based on tourist photos mainly used manual coding, but this method has problems such as cumbersome coding process, limited sample size, and constraints from uncontrollable factors.

[0003] With the advent of the mobile Internet and big data era, people are increasingly sharing travel photos on social media platforms. The development of deep learning technology has facilitated the efficient processing of massive travel photos, especially the emergence of convolutional neural networks (CNNs), which have made significant progress in the field of image recognition. Compared with traditional methods, deep learning technology can better capture the constructed scenes and visual features in tourism images, but most studies focus on objective tourism scenes and ignore the importance of visual aesthetics and emotional experience. Therefore, incorporating visual aesthetics and emotional experience into the study of tourism destination image perception can more comprehensively understand tourists' preferences and preferences and better promote destination planning and management. Summary of the invention

[0004] The purpose of the present invention is to provide a method for multi-angle tourism destination image perception based on deep learning technology, so as to solve the problems in the prior art that the tourism destination image research coding process is cumbersome, the number of samples is limited, and it is restricted by uncontrollable factors.

[0005] A method for multi-angle tourism destination image perception based on deep learning technology, comprising:

[0006] S1. Acquire and process the photo data generated by tourists;

[0007] S2. Using the processed photo data to perform scene recognition and classification;

[0008] S3. Predict tourists’ aesthetic evaluation of photos based on visual aesthetic quality model;

[0009] S4. Use visual emotion algorithms to analyze tourists’ emotional experience of tourist destinations;

[0010] S5. Obtain tourists’ perception of the image of the tourist destination from the spatial dimension.

[0011] S1 includes obtaining geo-tagged photo data from the Flickr website through the Flickr API, a picture sharing website interface. The geo-tagged photo data includes the photo number Pid, ​​the user number Oid, the shooting time P_date, the longitude Lon, the latitude Lat, and the download link Url, and performing data cleaning to obtain a photo data set for scene recognition.

[0012] S2 includes dividing the processed data set into two parts: a training data set and a test data set. By inputting the training data set into the deep learning model Inception-v4 model, the parameters of the Inception-v4 model are adjusted to minimize the difference between the predicted results and the actual labels.

[0013] S2 includes using the test data set to evaluate the performance of the Inception-v4 model, checking the generalization performance of the Inception-v4 model after adjusting the parameters on data that has never been exposed, and iterating the model multiple times to improve the recognition accuracy.

[0014] S2 includes using the Inception-v4 model to perform scene recognition on tourist photos, screening and retaining scenes with a probability of more than 10% being tourist scenes, and statistically analyzing the probability distribution of each scene in tourist photos. Based on the scene recognition results of the pictures, the photos are divided into nine scene categories, including food, culture, people, architecture, leisure and entertainment, natural scenery, urban life, animals and infrastructure. The kernel density analysis method is used to detect the spatial distribution of tourist scenes.

[0015] S3 includes using the deep learning model NIMA model to evaluate the aesthetic quality of tourists' photos, reflecting tourists' intuitive feelings from the visual content of the photos through the NIMA model, and predicting the probability distribution of the NIMA model's aesthetic evaluation.

[0016] S3 includes the introduction of the global Moran's index, taking into account the relationship between feature locations and attribute values ​​and spatial correlation weights, and the statistical indicator Moran's I statistic for measuring spatial autocorrelation is:

[0017]

[0018] In the formula, Moran's I is the statistical index, z i is the deviation of the attribute of feature i from its mean value, z j is the deviation of the attribute of feature j from its mean value, w i,j is the spatial weight between features i and j, n is the total number of features, and S0 is the aggregation of all spatial weights.

[0019] S3 includes combining hot spot analysis methods to calculate the cold spots and hot spots of photo aesthetic quality in space, and the spatial location of photo aesthetic quality clustering areas for:

[0020]

[0021] Where n1 is the number of research units; x i1 and x j1 is the attribute value of spatial units i1 and j1, w i1j1 is the spatial weight matrix.

[0022] S4 includes using the deep convolutional neural network structure VGGNet model to extract the emotional features in tourists' photos, introducing the emotional keywords in the DeepsentiBank emotional concept classification database, annotating photos with multiple emotional tags, representing the emotional colors of tourists, and obtaining image features F I and visual attention S from image saliency I , by weighting the image features of each region, the loss function of attention is calculated:

[0023] F I =CNN VGGNet (I);

[0024]

[0025] In the formula, I represents the input image, CNN VGGNet (I) indicates that I is input into the deep convolutional neural network for processing to obtain F I , L S is the loss function of attention, n' is the batch size in training, m is the total number of emotion categories, y i' is the emotion category of the image, y i' ∈{1,...,m}, j' is the number of the j'th emotion category, θ l is the parameter matrix of the fully connected layer used to calculate the probability of belonging to class l, T is the maximum number of iterations, is the feature representation of the i'th image in the training batch after weighting by visual attention, x is the number of feature maps in the local area of ​​the image, and f i' is the i'th image in the training batch, and Γ is the indicative function.

[0026] S4 includes using 1km×1km grids as statistical units to spatially grid the data of different emotion types, showing the spatial distribution of tourists' four emotion types towards tourist destinations. The four emotion types include pleasant high arousal, pleasant low arousal, unpleasant low arousal and unpleasant high arousal.

[0027] Compared with the prior art, the present invention has the following beneficial effects: it can better capture the constructed scenes and visual features in the tourism image, integrate visual aesthetics and emotional experience into the research on the image perception of the tourism destination, and more comprehensively understand the likes and preferences of tourists. Relying on the distribution of tourism resources in the destination, the tourism image can be spatially visualized, and managers can reasonably allocate the tourism resources of the destination to provide suggestions for tourism planning and improving the destination image. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of the present invention;

[0029] Figure 2 This is the Inception-v4 model structure diagram. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] A method for multi-angle tourism destination image perception based on deep learning technology, comprising:

[0032] S1. Acquire and process the photo data generated by tourists;

[0033] S2. Using the processed photo data to perform scene recognition and classification;

[0034] S3. Predict tourists’ aesthetic evaluation of photos based on visual aesthetic quality model;

[0035] S4. Use visual emotion algorithms to analyze tourists’ emotional experience of tourist destinations;

[0036] S5. Obtain tourists’ perception of the image of the tourist destination from the spatial dimension.

[0037] S1 includes obtaining geo-tagged photo data from the Flickr website through the Flickr API, a picture sharing website interface. The geo-tagged photo data includes the photo number Pid, ​​the user number Oid, the shooting time P_date, the longitude Lon, the latitude Lat, and the download link Url, and performing data cleaning to obtain a photo data set for scene recognition.

[0038] S2 includes dividing the processed data set into two parts: a training data set and a test data set. By inputting the training data set into the deep learning model Inception-v4 model, the parameters of the Inception-v4 model are adjusted to minimize the difference between the predicted results and the actual labels.

[0039] S2 includes using the test data set to evaluate the performance of the Inception-v4 model, checking the generalization performance of the Inception-v4 model after adjusting the parameters on data that has never been exposed, and iterating the model multiple times to improve the recognition accuracy.

[0040] S2 includes using the Inception-v4 model to perform scene recognition on tourist photos, screening and retaining scenes with a probability of more than 10% being tourist scenes, and statistically analyzing the probability distribution of each scene in tourist photos. Based on the scene recognition results of the pictures, the photos are divided into nine scene categories, including food, culture, people, architecture, leisure and entertainment, natural scenery, urban life, animals and infrastructure. The kernel density analysis method is used to detect the spatial distribution of tourist scenes.

[0041] S3 includes using the deep learning model NIMA model to evaluate the aesthetic quality of tourists' photos, reflecting tourists' intuitive feelings from the visual content of the photos through the NIMA model, and predicting the probability distribution of the NIMA model's aesthetic evaluation.

[0042] S3 includes the introduction of the global Moran's index, taking into account the relationship between feature locations and attribute values ​​and spatial correlation weights, and the statistical indicator Moran's I statistic for measuring spatial autocorrelation is:

[0043]

[0044] In the formula, Moran's I is the statistical index, z i is the deviation of the attribute of feature i from its mean value, z j is the deviation of the attribute of feature j from its mean value, w i,j is the spatial weight between features i and j, n is the total number of features, and S0 is the aggregation of all spatial weights.

[0045] S3 includes combining hot spot analysis methods to calculate the cold spots and hot spots of photo aesthetic quality in space, and the spatial location of photo aesthetic quality clustering areas for:

[0046]

[0047] Where n1 is the number of research units; x i1 and x j1 is the attribute value of spatial units i1 and j1, w i1j1is the spatial weight matrix.

[0048] S4 includes using the deep convolutional neural network structure VGGNet model to extract the emotional features in tourists' photos, introducing the emotional keywords in the DeepsentiBank emotional concept classification database, annotating photos with multiple emotional tags, representing the emotional colors of tourists, and obtaining image features F I and visual attention S from image saliency I , by weighting the image features of each region, the loss function of attention is calculated:

[0049] F I =CNN VGGNet (I);

[0050]

[0051] In the formula, I represents the input image, CNN VGGNet (I) indicates that I is input into the deep convolutional neural network for processing to obtain F I , L S is the loss function of attention, n' is the batch size in training, m is the total number of emotion categories, y i' is the emotion category of the image, y i' ∈{1,...,m}, j' is the number of the j'th emotion category, θ l is the parameter matrix of the fully connected layer used to calculate the probability of belonging to class l, T is the maximum number of iterations, is the feature representation of the i'th image in the training batch after weighting by visual attention, x is the number of feature maps in the local area of ​​the image, and f i' is the i'th image in the training batch, and Γ is the indicative function.

[0052] S4 includes using 1km×1km grids as statistical units to spatially grid the data of different emotion types, showing the spatial distribution of tourists' four emotion types towards tourist destinations. The four emotion types include pleasant high arousal, pleasant low arousal, unpleasant low arousal and unpleasant high arousal.

[0053] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for multi-angle tourism destination image perception based on deep learning technology, characterized in that: include: S1. Acquire and process the photo data generated by tourists; S2. Using the processed photo data to perform scene recognition and classification; S3. Predict tourists’ aesthetic evaluation of photos based on visual aesthetic quality model; S4. Use visual emotion algorithms to analyze tourists’ emotional experience of tourist destinations; S5. Obtain tourists’ perception of the image of the tourist destination from the spatial dimension.

2. According to claim 1, a method for multi-angle tourism destination image perception based on deep learning technology is characterized in that: S1 includes obtaining geo-tagged photo data from the Flickr website through the Flickr API, a picture sharing website interface. The geo-tagged photo data includes the photo number Pid, ​​the user number Oid, the shooting time P_date, the longitude Lon, the latitude Lat, and the download link Url, and performing data cleaning to obtain a photo data set for scene recognition.

3. The method for multi-angle tourism destination image perception based on deep learning technology according to claim 2 is characterized in that: S2 includes dividing the processed data set into two parts: a training data set and a test data set. By inputting the training data set into the deep learning model Inception-v4 model, the parameters of the Inception-v4 model are adjusted to minimize the difference between the predicted results and the actual labels.

4. The method for multi-angle tourism destination image perception based on deep learning technology according to claim 3 is characterized in that: S2 includes using the test data set to evaluate the performance of the Inception-v4 model, checking the generalization performance of the Inception-v4 model after adjusting the parameters on data that has never been exposed, and iterating the model multiple times to improve the recognition accuracy.

5. The method for multi-angle tourism destination image perception based on deep learning technology according to claim 4 is characterized in that: S2 includes using the Inception-v4 model to perform scene recognition on tourist photos, screening and retaining scenes with a probability of more than 10% being tourist scenes, and statistically analyzing the probability distribution of each scene in tourist photos. Based on the scene recognition results of the pictures, the photos are divided into nine scene categories, including food, culture, people, architecture, leisure and entertainment, natural scenery, urban life, animals and infrastructure. The kernel density analysis method is used to detect the spatial distribution of tourist scenes.

6. The method for multi-angle tourism destination image perception based on deep learning technology according to claim 5 is characterized in that: S3 includes using the deep learning model NIMA model to evaluate the aesthetic quality of tourists' photos, reflecting tourists' intuitive feelings from the visual content of the photos through the NIMA model, and predicting the probability distribution of the NIMA model's aesthetic evaluation.

7. The method for multi-angle tourism destination image perception based on deep learning technology according to claim 6 is characterized in that: S3 includes the introduction of the global Moran's index, taking into account the relationship between feature locations and attribute values ​​and spatial correlation weights, and the statistical indicator Moran's I statistic for measuring spatial autocorrelation is: In the formula, Moran's I is the statistical index, z i is the deviation of the attribute of feature i from its mean value, z j is the deviation of the attribute of feature j from its mean value, w i,j is the spatial weight between features i and j, n is the total number of features, and S0 is the aggregation of all spatial weights.

8. The method for multi-angle tourism destination image perception based on deep learning technology according to claim 7 is characterized in that: S3 includes combining hot spot analysis methods to calculate the cold spots and hot spots of photo aesthetic quality in space, and the spatial location of photo aesthetic quality clustering areas for: Where n1 is the number of research units; x i1 and x j1 is the attribute value of spatial units i1 and j1, w i1j1 is the spatial weight matrix.

9. The method for multi-angle tourism destination image perception based on deep learning technology according to claim 8 is characterized in that: S4 includes using the deep convolutional neural network structure VGGNet model to extract the emotional features in tourists' photos, introducing the emotional keywords in the DeepsentiBank emotional concept classification database, annotating photos with multiple emotional tags, representing the emotional colors of tourists, and obtaining image features F I and visual attention S from image saliency I , by weighting the image features of each region, the loss function of attention is calculated: F I =CNN VGGNet (I); In the formula, I represents the input image, CNN VGGNet (I) indicates that I is input into the deep convolutional neural network for processing to obtain F I , L S is the loss function of attention, n' is the batch size in training, m is the total number of emotion categories, y i' is the emotion category of the image, y i' ∈{1,...,m}, j' is the number of the j'th emotion category, θ l is the parameter matrix of the fully connected layer used to calculate the probability of belonging to class l, T is the maximum number of iterations, is the feature representation of the i'th image in the training batch after weighting by visual attention, x is the number of feature maps in the local area of ​​the image, and f i' is the i'th image in the training batch, and Γ is the indicative function.

10. The method for multi-angle tourism destination image perception based on deep learning technology according to claim 9, characterized in that: S4 includes using 1km×1km grids as statistical units to spatially grid the data of different emotion types, showing the spatial distribution of tourists' four emotion types towards tourist destinations. The four emotion types include pleasant high arousal, pleasant low arousal, unpleasant low arousal and unpleasant high arousal.

Images