Method and system for identifying mood and age of tourist in village space environment

Abstract

The invention discloses a method, a system, equipment, a medium and a program for identifying the mood and age of a tourist in a village space environment based on machine learning, and belongs to the technical field of tourism service. The method comprises the steps of preprocessing to-be-identified tourist emotion and age label data to obtain a pre-training data set of the emotion and age label data; and inputting the pre-trained data set of the emotion and age label data into a pre-trained deep learning model, and performing identification to obtain an identification result. According to the invention, the emotion and age of the tourist for the street space environment can be identified in high precision and automatically in real time, and the emotion and age index data result for the space environment can be calculated.

Description

Technical Field

[0001] The present invention relates to the field of tourism service technology, and in particular to a method, system, device, medium and program for identifying tourists' emotions and ages in a village space environment based on machine learning. Background Art

[0002] With the rapid development of tourism, the revival of traditional rural tourism has become an important way to protect historical and cultural heritage and promote rural revitalization. In this process, building a rural architectural environment with regional characteristics is an important link. As an important indicator for evaluating the regional characteristics of space, the emotional perception of tourists in the physical space experience of tourist villages has attracted more and more attention from researchers. Since the 1960s, with the rethinking of modern urban planning dominated by functional zoning, people have paid more and more attention to diverse and vibrant urban spaces. Vibrant cities and streets enhance residents' sense of happiness and social cohesion. Urban street vitality has become an important indicator for evaluating the attractiveness and potential of urban communities for sustainable development.

[0003] At present, there are many methods for investigating the vitality of tourist village streets, the most common of which are field investigation and questionnaire survey.

[0004] During field surveys, photography is usually used to record the distribution of historical buildings in traditional villages. This method of photographing or drawing mainly relies on individual researchers to choose special shooting locations with themselves as observers. This process has disadvantages such as high time cost, large resource investment, subjectivity and bias, limited comprehensiveness and depth of information acquisition, and poor generalizability and comparability of results.

[0005] Questionnaire survey method Questionnaire survey mainly relies on written information, which is difficult to capture the vivid, specific and multi-dimensional actual situation of street vitality. Although the questionnaire survey of street vitality has the advantages of low cost and wide coverage, it also has the disadvantages of limited information acquisition, difficult questionnaire design, recovery rate and efficiency problems, difficult to avoid subjective bias and not applicable to all groups.

[0006] Overall, current methods for identifying tourists’ perception of street spatial environment and further evaluating street vitality are still insufficient. Summary of the invention

[0007] The existing technology usually relies on manual detection methods or dedicated hardware equipment to identify the status information of tourists in street space environment, which makes it difficult to achieve real-time monitoring, and it is impossible to achieve real-time tracking and automatic management of the status of people in the space. The present invention provides a method for identifying the emotions and ages of tourists in the village street space environment based on machine learning, which can accurately and automatically identify the emotions and ages of tourists in the street space environment in real time and calculate the emotional and age index data results of the space environment.

[0008] In order to achieve the above object, the present invention provides the following technical solutions.

[0009] In a first aspect, the present invention provides a method for identifying the emotions and ages of tourists in a village space environment, comprising: Preprocess the emotion and age label data of tourists to be identified to obtain a pre-training data set of emotion and age label data; Input the pre-trained dataset of emotion and age label data into the pre-trained deep learning model to perform recognition and obtain the recognition result; The training method of the pre-trained deep learning model comprises: Classify the collected face data to obtain the face classification label data to be identified; Process the face classification label data to be identified to obtain the emotion and age label data of the tourists to be identified; Input the training set in the pre-training dataset of emotion and age label data into the deep learning model to obtain a trained deep learning model; Input the validation set in the pre-training dataset of emotion and age label data into the trained deep learning model for verification and recognition to determine whether the trained deep learning model meets the set requirements; If the set requirements are met, the image data of the tourist's expression and age to be identified are input into the trained deep learning model to obtain the recognition result.

[0010] As a further improvement of the present invention, the face classification label data to be identified is processed to obtain the emotion and age label data of the tourist to be identified, including: Collect several photos of real people and mark the facial coordinates on the photos to obtain the emotion and age label data of the tourists to be identified; Label and check the tourist emotion and age label data to be identified, select the tourist emotion and age label data to be identified that meet the expected standard quality, and generate a pre-training data set of emotion and age label data; The pre-training dataset of emotion and age label data is divided into training set and validation set.

[0011] As a further improvement of the present invention, the step of inputting the training set in the pre-training data set of emotion and age label data into the deep learning model and performing training optimization to obtain optimal parameters includes: Input the training set in the pre-training dataset of emotion and age label data into the deep learning model for iterative training to generate a feature element layer; The feature element layer is divided into 6x6, 3x3, 2x2, and 1x1 regions, and each region is averagely pooled to obtain feature data; The feature data are integrated and adjusted to the same number of channels to obtain a deep learning model after iterative training.

[0012] As a further improvement of the present invention, the verification set in the pre-training data set of emotion and age label data is input into the trained deep learning model for verification and recognition to determine whether the trained deep learning model meets the set requirements; Input the validation set in the pre-training dataset of emotion and age label data into the trained deep learning model, verify the training parameters, and obtain the validation results; Use the intersection-over-union (mAP) to determine the accuracy of the verification results; When the accuracy rate meets the set value, the iterative training of the deep learning model is completed, and the deep learning model after iterative training is obtained; Use the validation set in the pre-training data set of emotion and age label data to verify the deep learning model after iterative training. If the validation judgment of the deep learning model after iterative training meets the set conditions; If the set requirements are met, the trained deep learning model is obtained.

[0013] As a further improvement of the present invention, the verification set in the pre-training data set using emotion and age label data is used to verify the deep learning model after iterative training. If the verification judgment of the deep learning model after iterative training meets the set conditions, it includes: If the set requirements are not met, the ratio of the training set and the validation set of the emotion and age label data will be readjusted until the deep learning model is verified and judged to meet the set conditions after iterative training.

[0014] As a further improvement of the present invention, if the set requirements are met, the picture data of the tourist's expression and age to be identified are input into the trained deep learning model to obtain emotion and age recognition results, including emotion recognition results and age recognition results.

[0015] In a second aspect, the present invention provides a system for identifying tourists' emotions and ages in a village space environment, comprising: Preprocessing module: used to preprocess the emotion and age label data of tourists to be identified to obtain a pre-training data set of emotion and age label data; Recognition module: used to input the pre-trained dataset of emotion and age label data into the pre-trained deep learning model to obtain recognition results; The training system of the pre-trained deep learning model comprises: To-be-recognized module: used to classify the collected face data and obtain the face classification label data to be recognized; Label data module: used to process the face classification label data to be identified and obtain the emotion and age label data of the tourists to be identified; Training model module: used to input the training set in the pre-training data set of emotion and age label data into the deep learning model to obtain the trained deep learning model; Verification and judgment module: used to input the verification set in the pre-training data set of emotion and age label data into the trained deep learning model for verification and recognition, and to determine whether the trained deep learning model meets the set requirements; Obtain result module: if the set requirements are met, the image data of the tourist's expression and age to be identified is input into the trained deep learning model to obtain the recognition result.

[0016] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of a method for identifying the emotions and ages of tourists in a village space environment when executing the computer program.

[0017] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for identifying the emotions and ages of tourists in a village space environment are implemented.

[0018] In a fifth aspect, the present invention provides a computer program product, characterized in that it includes computer instructions, which, when executed by a processor, implement the steps of a method for identifying the emotions and ages of tourists in a village space environment.

[0019] Compared with the prior art, the present invention has the following beneficial effects: The method for identifying tourists' emotions and ages in village street space environments based on machine learning of the present invention has shown significant advantages and far-reaching positive impacts compared to traditional manual detection or dedicated hardware solutions. This method realizes the automated, real-time, and precise identification of tourists' emotions and ages by efficiently integrating preprocessing technology with advanced deep learning models, greatly improving the level of intelligent management of street space environments. The present invention can not only instantly capture the subtle emotional changes and age distribution characteristics of tourists, but also provide managers with instant feedback, helping them to quickly respond to tourist needs, optimize spatial layout, and improve service quality; at the same time, it also avoids the subjectivity and errors of human judgment, ensuring the objectivity and accuracy of data collection and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present disclosure in any way. In the drawings: Figure 1 It is a flow chart of a method for identifying tourists' emotions and ages in a village space environment according to the present invention; Figure 2 A schematic diagram of a training method for a pre-trained deep learning model of the present invention; Figure 3 It is a specific flow chart of a method for identifying tourists' emotions and ages in a village space environment according to the present invention; Figure 4 The facial expression recognition process of the present invention by adjusting RGB values, flipping, superimposing, etc.; Figure 5 The yolov5 model trained by the present invention shows the emotion recognition results Figure 6 The yolov5 model trained by the present invention shows the age recognition results Figure 7 Precision-Recall curves of different emotion categories of the present invention; Figure 8 The changing trend of the training and testing loss of the model of the present invention with the training rounds; Fig. 9 The performance index changes of the model training and verification stages of the present invention; Fig.10 It is the specific recognition result of tourists in different street space environments of the village in the embodiment of the present invention; Fig.11 It is a structural schematic diagram of a system for identifying tourists' emotions and ages in a village space environment according to the present invention; Fig.12 A schematic diagram of the structure of a training system for a pre-trained deep learning model of the present invention; Fig.13 Schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. The described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present invention.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0023] The existing technology usually relies on manual detection methods or dedicated hardware equipment to identify the state of tourists in street space environments, which makes it difficult to achieve real-time monitoring and cannot achieve real-time tracking and automatic management of the state of people in the space. The present invention provides a method for identifying the emotions and ages of tourists in village street space environments based on machine learning, such as Figure 1 As shown, the method includes: S100: Preprocessing the emotion and age label data of tourists to be identified to obtain a pre-training data set of emotion and age label data; S200: inputting the pre-trained dataset of emotion and age label data into the pre-trained deep learning model to perform recognition to obtain a recognition result; like Figure 2 As shown, the training method of the pre-trained deep learning model includes: S210: Classify the collected face data to obtain face classification label data to be identified; S220: Processing the face classification label data to be identified to obtain the emotion and age label data of the tourist to be identified; S230: inputting a training set in a pre-training data set of emotion and age label data into a deep learning model to obtain a trained deep learning model; S240: Inputting the validation set in the pre-training data set of emotion and age label data into the trained deep learning model for validation and recognition, and determining whether the trained deep learning model meets the set requirements; S250: If the set requirements are met, the image data of the tourist's expression and age to be identified is input into the trained deep learning model to obtain the recognition result.

[0024] In order to achieve the above object, the technical solution adopted by the present invention is as follows: A method and system for identifying tourists' emotions and ages in a village space environment based on machine learning, comprising the following steps: S1: Construct a training dataset containing face position coordinates, emotion and age label data; The pre-training dataset of emotion data contains data on various emotions of tourists in tourist villages.

[0025] Determine the specific environment in which the data is collected to ensure that the dataset reflects the diversity and complexity of real-world applications.

[0026] Choose appropriate scene types, such as village streets, tourist attractions, etc. These scenes have different backgrounds, lighting, and pedestrian density. Consider the light changes during the day and night, different weather conditions (sunny, rainy, etc.), and the flow of people during peak and non-peak hours to ensure the diversity and breadth of the data.

[0027] Use a device with a high-definition camera and a wide-angle lens to collect image or video data of sufficient quantity and quality to meet subsequent model training requirements.

[0028] In order to generate high-quality, structured data sets and provide a basis for subsequent model training, the collected images or videos are preliminarily screened to remove blurry, incomplete or irrelevant samples to the target scene, and the valid images are classified and stored, and classified by time, location or scene type.

[0029] The specific steps of pre-training datasets of various emotions and ages of tourists in tourist villages are as follows: S11: Collect a certain number of photos of real people in different time periods and age groups; S12: Use the labelImg Python script to manually mark the facial coordinates on the image. The facial expressions include four emotional labels: smile, surprise, peace, and negative, as well as four age labels: children, youth, adults, and the elderly for identification and quantification.

[0030] In order to achieve the goal of accurately labeling faces in images, the faces in images are classified by expression and age based on set conditions, and cross-validation is performed to ensure the accuracy and consistency of the labeling.

[0031] Review the annotation results to ensure the consistency and accuracy of the annotations. If necessary, perform multiple rounds of annotation and inspection to ensure that the quality of the dataset meets the expected standards and generate a pre-training dataset; S13: Divide the pre-training dataset into a training set and a validation set.

[0032] S2: Build a deep learning model based on a neural network, input the training data set into the deep learning model to train the deep learning model. After the training is completed, obtain the optimal parameters and save them as the basic model for subsequent recognition, and calculate the number of people and face position coordinates for subsequent analysis; The neural network-based deep learning model includes a backbone network module, a neck network module and a prediction module.

[0033] S21: Use the training set to iteratively train the pre-trained deep learning yolov5s model; The deep learning yolov5s model is pre-trained through the Imagenet (image recognition database) public data set to obtain the pre-trained deep learning yolov5s model.

[0034] S211: Use the YOLOv5 model to perform convolution training on the training set to generate the feature element layer; S212: Use the YOLOv5 model to divide the feature element layer into 6x6, 3x3, 2x2, and 1x1 regions, and then perform average pooling in each region; S213: All feature data obtained by the YOLOv5 model are integrated and adjusted to the same number of channels using the convolution kernel, and finally the prediction result is generated by upsampling.

[0035] S22: To improve recognition accuracy, data augmentation techniques were used during model training, including color changes, image flipping, and multi-image superposition. Slight color adjustments were made to the images, covering changes in brightness, contrast, and saturation to simulate different lighting conditions, thereby enhancing the model's adaptability in various lighting environments; S221: Import the verification set into the deep learning model after iterative training to verify the training parameters and obtain the verification results; S222. Use the intersection-over-union ratio mAP (mean Average Precision, the average accuracy of all categories) to obtain the accuracy of the verification results: like Figure 5 and Figure 6 As shown in the figure, the average precision (mAP@50) is used to obtain the accuracy of the verification results; the calculation steps of the average precision (AP) are as follows: 1) Calculate the intersection over union (IoU): IoU = (A∩B) / (A∪B); In the formula, A is the true value in the verification result, and B is the predicted value in the prediction result.

[0036] 2) Set the threshold: Select an IoU threshold, usually 0.5, as the standard for judging positive and negative samples.

[0037] 3) Calculate the True Positives (TP, true: predicted positive, actual positive), False Positives (FP, false positive: predicted positive, actual negative) and False Negatives (FN, false negative: predicted negative, actual positive) of each sample: -TP: positive samples with IoU ≥ 0.5.

[0038] -FP: negative samples with IoU < 0.5.

[0039] -FN: samples whose true values ​​exist but are not predicted.

[0040] 4) Calculate the precision and recall: Precision=TP / (TP+FP) Recall = TP / (TP + FN) 5) Draw the Precision-Recall curve: Draw the Precision and Recall values ​​into a curve.

[0041] 6) Calculate the average precision (AP): AP = ∑ (Rn-Rn-1) * Pn In the formula, Rn is the value of the nth recall rate; Rn-1 is the value of the n-1th recall rate; Pn is the corresponding Precision value.

[0042] 7) Calculate the average AP (mAP@50) of all samples: mAP@50=1 / n∑AP@50 Where n is the number of all samples, and AP@50 is the average precision of each sample when the IoU threshold is 0.5.

[0043] S223: Use the Cross Entropy Loss function to obtain the training set loss value and the validation set loss value in each training process; S224: Based on the training set loss value and the validation set loss value, use the Cross Entropy Loss function to determine whether the deep learning model after iterative training has converged; If not, execute S221, and then re-perform iterative training until the deep learning model after iterative training converges, and then start executing S23; If yes, execute S23; S23: Adjust the ratio of validation set and training set data; S24: Use the validation set to validate the deep learning model after iterative training to determine whether the deep learning model after iterative training has converged; If not, execute S23 to readjust the ratio of the validation set and the data set until the deep learning model after iterative training converges, and then start executing S25; If yes, execute S25; S25: Obtain the optimal parameters after iterative training and save them to the deep learning model after iterative training to complete the training.

[0044] S3: Use the same model training method as S2 to obtain emotion and age recognition results; S3 classifies the emotion and age of the face image analyzed by S2 S301: Divide the face classification data obtained in S2 into a training set and a validation set; S3011: Iteratively train the pre-trained deep learning yolov5s model using the training set of face classification data; S3012: Importing the verification set of face classification data into the deep learning model after iterative training to verify the training parameters and obtain the verification results; The accuracy of the verification results is judged to ensure that they meet the set conditions; S302: Divide the face classification data obtained in S2 into a training set and a validation set; S3021: Iteratively train the pre-trained deep learning yolov5s model using the training set of face classification data; S3022: Importing the verification set of face classification data into the deep learning model after iterative training to verify the training parameters and obtain the verification results; The accuracy of the verification result is judged, and if the set conditions are met, the verification result is obtained.

[0045] If the set conditions are not met, iterative training is performed again.

[0046] S4: Input the image data of the tourist’s facial expression and age to be identified into the trained deep learning model to obtain the corresponding proportion of each emotion and age recognition results.

[0047] S5: Based on the proportion of various emotions and age recognition results, tourist flow is controlled.

[0048] The method of the present invention first constructs a pre-training data set containing tourists' emotions and age data for subsequent deep learning model training. A deep learning model based on a neural network is constructed, and the pre-training data set is input into the deep learning model to train the deep learning model, thereby improving the automatic recognition accuracy of the deep learning model. After the training is completed, the optimal parameters are obtained and saved. The image data of the tourists' expressions and ages to be identified are input into the trained deep learning model to obtain the corresponding proportions of various emotions and age recognition results. This method can effectively identify the emotions and ages of tourists in different situations, saving a lot of time and cost. On this basis, suggestions for optimizing the street space to enhance tourists' perception are proposed, providing support for improving the sightseeing experience of street space and promoting the sustainable development of traditional villages.

[0049] Furthermore, YOLOv5, a deep learning model based on neural networks, has been widely used and verified, and has proven its stability and reliability in the deep learning community. YOLOv5 was chosen in the study because its algorithm is mature and well documented, which facilitates rapid deployment and experimentation, and is also conducive to methods that can be reproduced in different scenarios. Secondly, the framework of YOLOv5 is relatively flexible, and the code structure adopts a modular design, which means that each part (such as data processing, model training, and reasoning) is relatively independent, allowing fine tuning and customization as needed, increasing the application flexibility in different scenarios. A series of experiments have been conducted on YOLOv5, and the results show that the model can accurately identify the age and expression of pedestrians, which is in line with expectations. Based on these results, it was chosen to continue using YOLOv5 for subsequent research. The YOLOv5 model includes a backbone network module (Backbone), a neck network module (Neck Network) and a prediction module (Prediction). The backbone network module is used to extract common feature representations and serves as a high-performance classifier network. The neck network module is located between the backbone network and the prediction network to enhance the diversity and robustness of the extracted features. The three convolutional networks in the prediction module work together to produce the final output of the object detection results.

[0050] The system of the present invention includes a data set construction module, a model construction module and a model application module. The data set construction module is used to construct a pre-trained data set containing tourists' emotions and age data; the model construction module is used to construct a deep learning model based on a neural network, input the pre-trained data set into the deep learning model to train the deep learning model, and save the trained deep learning model; the model application module is used to input the image data of tourists' emotions and age data to be identified into the trained deep learning model to obtain the corresponding proportion of various emotions and age recognition results. This method can effectively identify tourists' emotions and ages in different situations, saving a lot of time and cost. On this basis, it proposes street space optimization suggestions that enhance tourists' perception, providing support for improving the sightseeing experience of street space and promoting the sustainable development of traditional villages.

[0051] The present invention is further explained below in conjunction with specific embodiments.

[0052] Example: S1: Construct a training dataset containing face position coordinates, emotion and age label data; The pre-training dataset of emotion data contains data on various emotions of tourists in tourist villages.

[0053] The specific steps to construct a training dataset containing face location coordinates as well as emotion and age label data are as follows: S11: Take fixed-point photos at different street locations in the village to collect a certain number of real crowd pictures containing facial information. These pictures contain facial data under different backgrounds, lighting, and pedestrian density scenes. A total of about 800 pictures collected are stored in the picture dataset and saved in the pre-training dataset.

[0054] S12: Use the Python script labelImg to manually mark the face coordinates on the image. The expressions include four emotion labels: smile, surprise, peace, and negative, and four age labels: children, youth, adults, and the elderly. The following steps are included: S121: Use the Create Polygons function in the Labelme tool to label the faces in the image and generate a .json file containing the face coordinate labeling information.

[0055] S122: Use labelme_json_to_dataset.exe in Labelme to batch convert the json format files generated in S121 into png format, and store them in the pre-training dataset as a label dataset.

[0056] S123: using the resize function in Python to convert the file size in the pre-training data set into a uniform size, the present invention converts the pictures in the pre-training data set into a uniform size of 473px high and 473px wide; S13: Divide the pre-training dataset into a training set and a validation set, as follows: The random.sample function in Python is used to divide the dataset into a training set and a validation set, where the ratio of the training set to the validation set is set to 8:2; the final pre-training dataset contains a total of 800 .png labeled images, including 640 training sets and 160 validation sets.

[0057] S2: Construct a deep learning model based on a neural network. The deep learning model based on a neural network includes a backbone network module (Backbone), a neck network module (Neck Network) and a prediction module (Prediction). The backbone network module is used to extract common feature representations and serves as a high-performance classifier network. The neck network module is located between the backbone network and the prediction network to enhance the diversity and robustness of the extracted features. The three convolutional networks in the prediction module work together to produce the final output of the object detection results. Input the pre-trained data set into the deep learning model to train the deep learning model. The training generation is set to 100. After the training is completed, the optimal parameters are obtained and saved in the trained deep learning model. The specific steps are as follows: S21: Use the training set to iteratively train the deep learning yolov5s model that has been built and pre-trained on the Imagenet public dataset. The specific steps are as follows: S211: Use the YOLOv5 model to perform convolution training on the training set to generate the feature element layer; S212: Use the YOLOv5 model to divide the feature element layer into 6x6, 3x3, 2x2, and 1x1 regions, and then perform average pooling in each region; S213: All feature data obtained by the YOLOv5 model are integrated and adjusted to the same number of channels using the convolution kernel, and finally the prediction result is generated by upsampling.

[0058] S22: To improve recognition accuracy, data augmentation techniques were used during model training, including color changes, image flipping, and multi-image superposition. Slight color adjustments were made to the images, covering changes in brightness, contrast, and saturation to simulate different lighting conditions, thereby enhancing the model's adaptability in various lighting environments; like Figure 4As shown in the figure, data augmentation techniques are used to apply data to image datasets, such as adjusting RGB values, horizontal or vertical flipping, image superposition, etc. These techniques are used to expand the dataset and enhance the generalization ability of the model. Figure 4 The process of processing facial images through data augmentation methods is shown. These methods include adjusting RGB values ​​(changing colors), flipping images (changing perspectives), and image superposition (synthesizing new images). These techniques help the model better learn diverse features during training and improve the accuracy of facial expression recognition.

[0059] S221: Import the verification set into the deep learning model after iterative training to verify the training parameters and obtain the verification results; S222: Use the mean average precision (mAP@50) to obtain the accuracy of the verification results; like Figure 7 As shown in the figure, the loss value of each training round (epoch) is recorded during the training and testing process of the model, and a curve is drawn, which shows the trend of the model training and test loss with the training round. The loss curve gradually decreases, indicating that the model gradually converges in continuous learning and optimization.

[0060] like Figure 8 and Fig. 9 Shown is a line graph of IoU values ​​in deep learning model training of the present invention; S223: Use the Cross Entropy Loss function to obtain the training set loss value and the validation set loss value in each training process, such as Figure 8 and Fig. 9 Shown is a line graph of loss values ​​in deep learning model training of the present invention; like Figure 8 Shown: By running the model on the test set, the precision-recall curve for each emotion category is generated.

[0061] like Fig. 9 As shown: The loss and performance indicators of the model in the training and validation stages change with the number of training rounds. Overall, the bounding box loss (box_loss), object detection loss (obj_loss), and classification loss (cls_loss) gradually decrease on the training and validation sets, indicating that the model is gradually learning and converging. The precision (Precision) and recall (Recall) are significantly improved during the training process, and the average precision (mAP_0.5 and mAP_0.5:0.95) on the validation set also increases with the increase in training rounds, showing that the overall performance of the model in detection and classification tasks has improved.

[0062] S224: Based on the training set loss value and the validation set loss value, use the Cross Entropy Loss function to determine whether the deep learning model after iterative training has converged; If not, execute S21, then re-perform iterative training until the deep learning model after iterative training converges, and then start executing S23; If yes, execute S24; S23: Adjust the ratio of validation set and data set data; S24: Use the validation set to validate the deep learning model after iterative training to determine whether the deep learning model after iterative training has converged; If not, execute S23 to readjust the ratio of the validation set and the data set until the deep learning model after iterative training converges, and then start executing S25; If yes, execute S25; S25: Obtain the optimal parameters after iterative training and save them to the deep learning model after iterative training, specifically: The recognition accuracy of the training model on the entire validation set is 0.81, which is at a relatively high level and meets the recognition goal; Extract the model training parameters corresponding to the maximum average AP (mAP@50) value, save the deep learning model, and complete the training.

[0063] S3: Use the same model training method as S2 to obtain emotion and age recognition results; S4: Input the image data of the tourists’ expressions and ages to be identified into the trained deep learning model, and obtain the corresponding proportions of various emotions and age recognition results as shown in Tables 1 and 2. Figure 5 and Figure 6 , which is a schematic diagram of pictures of tourists' expressions and ages to be identified and their annotated pictures.

[0064] like Figure 5 Figure 1 shows the emotion recognition results of the model for the input image. For example, the emotion labels recognized by the model may include "smile", "surprise", "peaceful" or "negative". These results are the direct output of the model based on the classification after image feature extraction.

[0065] like Figure 6 Figure 1 shows the model's age recognition results for the input image. The emotion labels recognized by the model may include "children", "youth", "adults" and "elderly". These results are the direct output of the model based on the classification after image feature extraction.

[0066] Table 1: Figure 5 The percentage of various emotions:

[0067] Table 2: Figure 6 The data of the proportion of each age group:

[0068] like Fig.10 As shown in the figure, the specific recognition results of tourists in different street space environments in the village, among which the total number of expression recognition is 2015, including 170 smiling expressions, 118 laughing expressions, 1646 neutral expressions, and 81 negative emotions. The total number of ages recognized is 2031, including 37 children, 126 elderly people, 1804 adults, and 64 young people.

[0069] like Fig.11 As shown, the second object of the present invention is to provide a system for identifying tourists' emotions and ages in a village space environment, comprising: Preprocessing module 101: used to preprocess the emotion and age label data of tourists to be identified to obtain a pre-training data set of emotion and age label data; Recognition module 201: used to input the pre-trained data set of emotion and age label data into the pre-trained deep learning model to perform recognition to obtain a recognition result; like Fig.12 As shown, the training system of the pre-trained deep learning model includes: The module to be identified 211 is used to classify the collected face data to obtain the face classification label data to be identified; Label data module 221: used to process the face classification label data to be identified, and obtain the emotion and age label data of the tourist to be identified; The training model module 231 is used to input the training set in the pre-training data set of emotion and age label data into the deep learning model to obtain a trained deep learning model; Verification and judgment module 241: used to input the verification set in the pre-training data set of emotion and age label data into the trained deep learning model for verification and recognition, and determine whether the trained deep learning model meets the set requirements; Obtaining result module 251: is used to input the image data of the expression and age of the tourist to be identified into the trained deep learning model to obtain the recognition result if the set requirements are met.

[0070] like Fig.13As shown, the third object of the present invention is to provide an electronic device, the electronic device comprising: a processor 301, a memory 302 and a display screen 303. Among them, the memory 302 and the display screen 303 are connected to the processor 301, such as through a bus 304. Optionally, the electronic device may also include a transceiver 305. It should be noted that in actual applications, the transceiver 305 is not limited to one, and the structure of the electronic device does not constitute a limitation on the embodiments of the present application.

[0071] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0072] The bus 304 may include a path to transmit information between the above components. The bus 304 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 304 may be divided into an address bus, a data bus, a control bus, etc.

[0073] The memory 302 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0074] The memory 302 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 302 to implement the contents shown in the above method embodiment.

[0075] Fig.13 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0076] The fourth object of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program is stored thereon, and when the program is executed by a processor, the aforementioned Figures 1 to 3 The various processes of the method embodiment shown in the figure include, for example, a memory including instructions, and the instructions can be executed by a processor of an electronic device to complete the method.

[0077] A computer-readable storage medium may be a tangible device that holds and stores instructions used by an instruction execution device. A computer-readable storage medium may be, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof. Specifically, a computer-readable storage medium may be a portable computer disk, a hard disk, a USB flash drive, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a lectern random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, an optical disk, a magnetic disk, a mechanical encoding device, and any combination thereof.

[0078] A fifth object of the present invention is to provide a computer program product, comprising computer instructions, which, when executed by a processor, implement the above Figures 1 to 3 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0079] Many embodiments and many applications beyond the examples provided will be apparent to those skilled in the art upon reading the above description. Therefore, the scope of the present teachings should not be determined with reference to the above description, but rather with reference to the preceding claims and the full scope of equivalents to which such claims are entitled. For the purpose of comprehensiveness, all articles and references, including disclosures of patent applications and publications, are incorporated herein by reference. The omission of any aspect of the subject matter disclosed herein in the preceding claims is not intended to be a disclaimer of such subject matter, nor should it be considered that the applicants did not consider such subject matter to be part of the disclosed inventive subject matter.

[0080] The above content is a further detailed description of the present invention. It cannot be determined that the specific implementation methods of the present invention are limited to these. For ordinary technicians in the technical field to which the present invention belongs, without departing from the concept of the present invention, they can also make several simple deductions or substitutions, which should be regarded as belonging to the protection scope of the present invention determined by the submitted claims.

Claims

1. A method for identifying tourists' emotions and ages in a village space environment, characterized in that: include: Preprocess the emotion and age label data of tourists to be identified to obtain a pre-training data set of emotion and age label data; Input the pre-trained dataset of emotion and age label data into the pre-trained deep learning model to perform recognition and obtain the recognition result; The training method of the pre-trained deep learning model comprises: Classify the collected face data to obtain the face classification label data to be identified; Process the face classification label data to be identified to obtain the emotion and age label data of the tourists to be identified; Input the training set in the pre-training dataset of emotion and age label data into the deep learning model to obtain a trained deep learning model; Input the validation set in the pre-training dataset of emotion and age label data into the trained deep learning model for verification and recognition to determine whether the trained deep learning model meets the set requirements; If the set requirements are met, the image data of the tourist's expression and age to be identified are input into the trained deep learning model to obtain the recognition result.

2. The method for identifying tourists' emotions and ages in a village space environment according to claim 1, characterized in that: The face classification label data to be identified is processed to obtain the emotion and age label data of the tourist to be identified, including: Collect several photos of real people and mark the facial coordinates on the photos to obtain the emotion and age label data of the tourists to be identified; Label and check the tourist emotion and age label data to be identified, select the tourist emotion and age label data to be identified that meet the expected standard quality, and generate a pre-training data set of emotion and age label data; The pre-training dataset of emotion and age label data is divided into training set and validation set.

3. The method for identifying tourists' emotions and ages in a village space environment according to claim 1, characterized in that: The step of inputting the training set in the pre-training data set of emotion and age label data into the deep learning model and performing training optimization to obtain the optimal parameters includes: Input the training set in the pre-training dataset of emotion and age label data into the deep learning model for iterative training to generate a feature element layer; The feature element layer is divided into 6x6, 3x3, 2x2, and 1x1 regions, and each region is averagely pooled to obtain feature data; The feature data are integrated and adjusted to the same number of channels to obtain a deep learning model after iterative training.

4. The method for identifying tourists' emotions and ages in a village space environment according to claim 1, characterized in that: The verification set in the pre-training data set of emotion and age label data is input into the trained deep learning model for verification and recognition, and it is determined whether the trained deep learning model meets the set requirements; Input the validation set in the pre-training dataset of emotion and age label data into the trained deep learning model, verify the training parameters, and obtain the validation results; Use the intersection-over-union (mAP) to determine the accuracy of the verification results; When the accuracy rate meets the set value, the iterative training of the deep learning model is completed, and the deep learning model after iterative training is obtained; Use the validation set in the pre-training data set of emotion and age label data to verify the deep learning model after iterative training. If the validation judgment of the deep learning model after iterative training meets the set conditions; If the set requirements are met, the trained deep learning model is obtained.

5. The method for identifying tourists' emotions and ages in a village space environment according to claim 4, characterized in that: The verification set in the pre-training data set of emotion and age label data is used to verify the deep learning model after iterative training. If the verification judgment of the deep learning model after iterative training meets the set conditions, it includes: If the set requirements are not met, the ratio of the training set and the validation set of the emotion and age label data will be readjusted until the deep learning model is verified and judged to meet the set conditions after iterative training.

6. The method for identifying tourists' emotions and ages in a village space environment according to claim 1, characterized in that: If the set requirements are met, the image data of the tourist's expression and age to be identified are input into the trained deep learning model to obtain emotion and age recognition results, including emotion recognition results and age recognition results.

7. A system for identifying tourists' emotions and ages in a village space environment, characterized in that: include: Preprocessing module: used to preprocess the emotion and age label data of tourists to be identified to obtain a pre-training data set of emotion and age label data; Recognition module: used to input the pre-trained dataset of emotion and age label data into the pre-trained deep learning model to obtain recognition results; The training system of the pre-trained deep learning model comprises: To-be-recognized module: used to classify the collected face data and obtain the face classification label data to be recognized; Label data module: used to process the face classification label data to be identified and obtain the emotion and age label data of the tourists to be identified; Training model module: used to input the training set in the pre-training data set of emotion and age label data into the deep learning model to obtain the trained deep learning model; Verification and judgment module: used to input the verification set in the pre-training data set of emotion and age label data into the trained deep learning model for verification and recognition, and to determine whether the trained deep learning model meets the set requirements; Obtain result module: if the set requirements are met, the image data of the tourist's expression and age to be identified is input into the trained deep learning model to obtain the recognition result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of a method for identifying the emotions and ages of tourists in a village space environment as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of a method for identifying the emotions and ages of tourists in a village space environment as described in any one of claims 1 to 6.

10. A computer program product, characterized in that It includes computer instructions, which, when executed by a processor, implement the steps of a method for identifying the emotions and ages of tourists in a village space environment as described in any one of claims 1 to 6.

Images