Intelligent travel planning method and system of Maas service based on 3D face recognition
By analyzing the tiny displacement of user facial feature points and combining adaptive path planning algorithms, the problem of how to balance fatigue state, road conditions and rest sites in 3D face recognition MaaS service is solved, and personalized smart travel planning is realized and improved The safety and comfort of travel.
Patent Information
- Application Number
- CN202411179809.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the smart travel planning of MaaS services for 3D facial recognition, how to balance user fatigue status, real-time road conditions and rest sites during dynamic path planning, ensure the stability and accuracy of fatigue judgment, and dynamically adjust rest sites according to different fatigue levels. The frequency of recommendation has become a technical problem.
By obtaining user facial images, extracting facial feature points, analyzing the tiny displacements of eyelid sagging, lower corners of the mouth and head posture changes, combining preset personalized fatigue judgment thresholds to make fatigue degree grading judgment, and using adaptive path planning algorithm , comprehensively consider transportation, road conditions and rest stations, dynamically optimize travel routes, adjust itinerary schedules, provide personalized navigation services, and adjust the itinerary in real time when fatigue state changes.
It realizes personalized intelligent travel planning based on the user's real-time fatigue status, improves the safety and comfort of long-distance travel, provides personalized navigation suggestions, and enhances the travel experience.
Smart Images

Figure CN119067280A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information technology, and in particular to a smart travel planning method and system for Maas services based on 3D face recognition. Background Art
[0002] In the smart travel planning of MaaS services based on 3D face recognition, it is necessary to explore the relationship between the micro-displacement of facial feature points and the user's fatigue level, and provide personalized travel suggestions based on this. Unlike most studies focusing on fatigue driving, the innovation lies in planning the overall travel plan based on the user's fatigue status. There are several technical contradictions: when performing dynamic path planning, it is necessary to balance multiple factors such as the user's fatigue status, real-time road conditions, and rest stops, which increases the complexity of the algorithm. At the same time, it is also necessary to consider setting fatigue judgment thresholds for user groups and making graded judgments on fatigue levels. How to ensure the stability and accuracy of judgments in a complex and changing environment, and how to dynamically adjust the frequency of rest stop recommendations according to different fatigue levels are technical problems that need to be solved urgently. The existence of these technical contradictions and difficulties makes fatigue judgment and smart travel planning based on facial feature points a challenging research direction. It requires in-depth exploration and innovation in multiple aspects such as image processing, feature extraction, fatigue judgment model, path planning algorithm, real-time monitoring and dynamic adjustment to achieve accurate, efficient and personalized smart travel services. Summary of the invention
[0003] The present invention provides a smart travel planning method for Maas service based on 3D face recognition, which mainly includes: Obtain the user's facial image, pre-process the image, extract facial feature points, obtain a standardized facial image, detect and locate facial feature points including eyes, mouth, nose tip, both ends of eyebrows and chin on the standardized facial image, where both ends of eyebrows and chin are used to determine head posture, and obtain three-dimensional coordinate information of facial feature points; According to the three-dimensional coordinate information of facial feature points, the facial feature point displacement analysis algorithm is used to calculate the tiny displacement of facial fatigue features at different times, including the degree of eyelid droop, the amplitude of the corner of the mouth, and the change of head posture, and obtain the facial fatigue feature displacement sequence; According to the facial fatigue feature displacement sequence, combined with the preset personalized fatigue judgment threshold, the fatigue degree is graded and judged. If the eyelid drooping degree exceeds the user's corresponding fatigue threshold, the user is judged to be in a severe fatigue state. If the mouth corners are turned down more than the threshold, the user is judged to be in a moderate fatigue state. If only the head posture changes more than the threshold, the user is judged to be in a mild fatigue state. According to different fatigue levels, combined with the user's current location information and the user's destination information, the corresponding recommended transportation plan is obtained from the transportation type library, and the recommended route is obtained from the travel route library according to the recommended transportation plan; According to the recommended route and user fatigue level, an adaptive path planning algorithm is used to dynamically optimize the recommended travel route by comprehensively recommending transportation tools, real-time traffic information, and rest stops along the way. The itinerary is adjusted according to the fatigue level, including adding or reducing rest stops, adjusting waypoints, and adjusting the proportion of different transportation modes, to obtain a personalized dynamic optimization travel plan. Based on dynamically optimized travel plans, personalized navigation services are provided to users, including recommended means of transportation, estimated travel time, rest stops and waypoint information. The user's fatigue status is continuously monitored during the journey. When the fatigue status changes, the itinerary is adjusted in real time, including increasing the frequency of rest, changing transportation methods and changing the number of waypoints.
[0004] The present invention provides a smart travel planning system for Maas services based on 3D face recognition, which mainly includes: Image preprocessing module, used to extract facial feature points and standardize facial images; Feature point analysis module, used to calculate the three-dimensional displacement of facial feature points; A fatigue state judgment module is used to judge the fatigue degree according to the displacement sequence and threshold value; A transportation recommendation module is used to recommend transportation based on fatigue level and location information; Path planning module, used to dynamically optimize travel routes and itineraries; The navigation service module is used to provide personalized navigation services and adjust itineraries in real time.
[0005] The technical solution provided by the embodiment of the present invention may have the following beneficial effects: The present invention discloses a smart travel planning method and system for Maas service with 3D face recognition. The method obtains a user's facial image, extracts facial feature points, analyzes fatigue features such as drooping eyelids, downturned corners of the mouth, and changes in head posture, and determines the degree of fatigue in combination with a personalized threshold. According to the degree of fatigue, location, and destination information, a recommended plan is obtained from a transportation tool library and a route library. An adaptive path planning algorithm is used to comprehensively consider transportation tools, road conditions, and rest stops, dynamically optimize travel routes, and adjust itinerary arrangements. During the navigation process, the fatigue state is continuously monitored, and the itinerary is adjusted in real time, including increasing the frequency of rest, changing the mode of transportation, etc. The present invention realizes personalized intelligent travel planning based on the user's real-time fatigue state, improves the safety and comfort of long-distance travel, and has important practical value. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] Figure 1 Flow chart of the smart travel planning method for the Maas service of 3D face recognition of the present invention.
[0007] Figure 2 Schematic diagram of the smart travel planning method and system for the Maas service of 3D face recognition of the present invention.
[0008] Figure 3 Another schematic diagram of the smart travel planning method and system for the Maas service of 3D face recognition of the present invention. DETAILED DESCRIPTION
[0009] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.
[0010] like Figure 1-3 The smart travel planning method and system of the Maas service of 3D face recognition in this embodiment may specifically include: Step S101, obtain the user's facial image, pre-process the image, extract facial feature points, obtain a standardized facial image, detect and locate facial feature points including eyes, mouth, nose tip, both ends of eyebrows and chin on the standardized facial image, where both ends of eyebrows and chin are used to judge the head posture, and obtain the three-dimensional coordinate information of facial feature points.
[0011] Acquire a user's facial image, perform denoising and illumination compensation on the facial image to obtain a first image; adjust the facial area to a standard position and size through affine transformation based on the first image to obtain a standardized facial image; use a Haar feature classifier to roughly locate the standardized facial image to determine the approximate areas of the eyes, nose and mouth; use an active shape model to accurately locate key feature points of the corners of the eyes, the tip of the nose and the corners of the mouth within the approximate areas to obtain two-dimensional feature point coordinates; obtain camera calibration parameters, and calculate the three-dimensional spatial coordinates of the feature points based on the two-dimensional feature point coordinates and the camera calibration parameters; use a PnP algorithm to match the three-dimensional spatial coordinates with the corresponding two-dimensional feature point coordinates, solve the camera posture, and obtain a head posture estimation result; and calculate the relative positions and size ratios of facial organs such as eyes, nose and mouth based on the three-dimensional spatial coordinates for facial expression analysis and identity recognition.
[0012] Specifically, after obtaining the user's facial image, the image is denoised, and histogram equalization is used for illumination compensation to eliminate external environmental interference. Gaussian filtering is used to smooth the image to improve image quality. The facial area is adjusted to a standard position and size through affine transformation to obtain a standardized facial image. On the standardized facial image, the Haar feature classifier is used to locate the approximate areas of the eyes, nose and mouth. The active shape model is then used to accurately locate key feature points such as the corners of the eyes, the tip of the nose, and the corners of the mouth in each area. The active shape model learns the statistical distribution of facial features through a pre-labeled training set, and iteratively adjusts the position of the feature points during the positioning process to best match the target image. At the same time, the two ends of the eyebrows and the chin contour points are detected, and the two-dimensional pixel coordinates of all feature points are recorded. According to the acquired two-dimensional feature point coordinates, combined with the camera calibration parameters, the depth information of the feature points is calculated through the perspective projection principle. The camera calibration process uses a checkerboard calibration plate, shoots multiple calibration plate images at different angles, and uses the Zhang method to solve the camera intrinsic parameter matrix and distortion coefficient. The two-dimensional coordinates are converted into three-dimensional space coordinates to form a three-dimensional representation model of facial feature points. The facial plane is constructed using the 3D coordinates of the feature points at both ends of the eyebrows and the chin. The angle between the normal vector of the plane and the optical axis of the camera is calculated to obtain the pitch angle and left and right rotation angle of the head. The PnP algorithm is used to match the 3D feature points with their corresponding 2D projection points to solve the camera posture, thereby obtaining a more accurate head posture estimation result. According to the coordinates of the 3D feature points, the relative positions and size ratios of facial organs such as eyes, nose, and mouth are calculated for subsequent facial expression analysis and identity recognition.
[0013] After obtaining the user's facial image, median filtering is used for denoising, and the filter window size is set to 3×3 pixels. Histogram equalization is used for illumination compensation to stretch the image grayscale value distribution to the range of 0-255. A Gaussian filter is used for smoothing, with a kernel size of 5×5 and a standard deviation σ set to 1.5. Affine transformation adjusts the facial area to a standard size, such as 256×256 pixels. On the standardized image, the Haar feature classifier uses a pre-trained cascade classifier to locate the approximate areas of the eyes, nose, and mouth. The active shape model describes facial features through 68 key points, and the gradient descent algorithm is used to iteratively optimize the feature point positions. The maximum number of iterations is set to 50 times, and the convergence threshold is 0.01 pixels. The two-dimensional pixel coordinates of all feature points are recorded to form a 68×2 matrix. The camera calibration uses a 9×6 checkerboard calibration plate and takes 20 calibration images at different angles. The Zhang method is used to solve the camera intrinsic parameter matrix and distortion coefficient. The focal lengths fx and fy are approximately 800 pixels, and the principal point coordinates cx and cy are approximately the center of the image. The perspective projection principle is used to convert the two-dimensional coordinates into three-dimensional space coordinates in combination with the camera parameters. When constructing the facial plane, three points, the outer corner of the left eyebrow, the outer corner of the right eyebrow and the bottom of the chin, are selected to calculate the plane normal vector. Through the vector dot multiplication operation, the angle between the normal vector and the camera optical axis is obtained, and the head pitch angle and left and right rotation angle are obtained. The PnP algorithm uses the RANSAC iterative method, sets the internal point threshold to 2 pixels, and the maximum number of iterations is 100 times to solve the camera extrinsic parameter matrix. According to the coordinates of the three-dimensional feature points, the ratio of the distance between the eyes to the width of the face, the ratio of the length of the nose to the height of the face, and other features are calculated for subsequent facial expression analysis and identity recognition. These ratio values are usually between 0.3 and 0.5, which can be used as important indicators of individual characteristics.
[0014] Step S102, based on the three-dimensional coordinate information of the facial feature points, a facial feature point displacement analysis algorithm is used to analyze and calculate the minute displacements of facial fatigue features at different times, including the degree of eyelid droop, the amplitude of the downward turn of the mouth corners, and the change in head posture, to obtain a facial fatigue feature displacement sequence.
[0015] The three-dimensional coordinate information of the facial feature points is obtained, and the displacement of the corresponding feature points between adjacent frames is calculated according to the three-dimensional coordinate information to obtain an original displacement sequence; the original displacement sequence is smoothed by using a sliding window mean filter to obtain a smoothed displacement sequence; the smoothed displacement sequence is predicted and corrected by using a Kalman filter, and the position estimation of each feature point is iteratively updated to obtain a more accurate displacement sequence; the cumulative displacement of each feature point in the displacement sequence in multiple consecutive frames is calculated to obtain basic data for feature extraction; three fatigue features, namely, the degree of eyelid ptosis, the amplitude of the corner of the mouth downward and the change in head posture, are extracted from the basic data, wherein the degree of eyelid ptosis is the change in the distance between the midpoints of the upper and lower eyelids, the amplitude of the corner of the mouth downward is the vertical displacement of the corner of the mouth, and the change in head posture is the change in the plane normal vector composed of the nose tip point and the outer point of the eyebrow; the change in the three fatigue features between adjacent time windows is calculated to generate a facial fatigue feature displacement sequence.
[0016] Specifically, the acquisition frequency of facial feature points is set to 30 frames per second to ensure that the time interval between two adjacent frames is 33.33 milliseconds. According to the three-dimensional coordinate information of facial feature points, the displacement of corresponding feature points between adjacent frames is calculated by Euclidean distance. The feature points related to eyelids, including the midpoint of upper and lower eyelids, mouth corners, including left and right mouth corners, and head posture, including the nose tip and the outer eyebrow points, are processed separately to obtain the original displacement sequence. The original displacement sequence is smoothed by sliding window mean filtering, and the window size is set to 5 time points, that is, a time span of 166.65 milliseconds, to eliminate the noise caused by detection errors and obtain the smoothed displacement sequence. The smoothed displacement sequence is predicted and corrected by Kalman filtering, and the state transfer matrix is set to the unit matrix and the observation matrix is [1,0;0,1]. Assuming that the displacement remains constant in a short time, the position estimate of each feature point is iteratively updated to obtain a more accurate displacement sequence. The filtered data is analyzed in time series, and the cumulative displacement of each feature point in 10 consecutive frames of 333.3 milliseconds is calculated as the basic data for feature extraction. The three fatigue features of eyelid ptosis, which corresponds to the change in the distance between the upper and lower eyelid midpoints, the downward angle of the mouth corners, which corresponds to the vertical displacement of the mouth corners, and the change in head posture, which corresponds to the change in the plane normal vector composed of the nose tip point and the outer eyebrow points, are extracted from the processed displacement sequence. The change of each feature between adjacent time windows is calculated to generate a facial fatigue feature displacement sequence. The facial feature point acquisition system captures data at a frequency of 30 frames per second to ensure that the interval between adjacent frames is 33.33 milliseconds. For each frame of captured facial feature points, the key points extracted are the midpoints of the upper and lower eyelids (y1, y2), the left and right mouth corners (m1, m2), the nose tip (n), and the left and right eyebrow outer points (b1, b2). The displacement of these points between adjacent frames is calculated using the Euclidean distance formula, such as eyelid ptosis deye=|y2-y1|. The sliding window mean filter uses a 5-frame 166.65 millisecond window to smooth the original displacement sequence. For example, for eyelid droop, the smoothed value is deyesmooth=(deyet+deyet-1+...+deyet-4) / 5. deyet represents the eyelid droop value at time t, deyet-1, deyet-2, deyet-3, deyet-4 represent the eyelid droop values at time t-1, t-2, t-3, and t-4 respectively. The Kalman filter sets the state transfer matrix to a 2x2 identity matrix and the observation matrix to [1,0;0,1], assuming that the displacement remains constant in a short time. Apply the filter to each feature point, such as the predicted value of eyelid droop deyepred=deyesmootht-1, the observed value deyesmootht, and the optimized estimate deyeopt is obtained through iterative update.Time series analysis calculates the cumulative displacement within 10 frames (333.3 milliseconds), such as the cumulative eyelid droop Deye = Σdeyeopt. Finally, the fatigue features are calculated as the eyelid droop degree Feye=(Deyet-Deyet-1) / 333.3, the mouth corner downturn amplitude Fmouth=(m1yt+m2yt-m1yt-1-m2yt-1) / 333.3, and the head posture change Fhead=arccos(nt·nt-1), where Deyet is the eyelid droop value at the current time t, Deyet-1 is the eyelid droop value at the time t-1, m1yt is the y coordinate of the left mouth corner at the current time t, m2yt is the y coordinate of the right mouth corner at the current time t, m1yt-1 is the y coordinate of the left mouth corner at the previous time t-1, m2yt-1 is the y coordinate of the right mouth corner at the previous time t-1, n is the normal vector of the plane formed by the nose tip point and the outer eyebrow point, and nt-1 is the normal vector of the head posture plane at the previous time t-1. These eigenvalues constitute the facial fatigue feature displacement sequence for subsequent fatigue state analysis.
[0017] Step S103, based on the facial fatigue feature displacement sequence and the preset personalized fatigue judgment threshold, the fatigue degree is graded and judged. If the degree of eyelid droop exceeds the fatigue threshold corresponding to the user, the user is judged to be in a severe fatigue state; if the downward angle of the mouth corners exceeds the threshold, the user is judged to be in a moderate fatigue state; if only the head posture change exceeds the threshold, the user is judged to be in a mild fatigue state.
[0018] A facial fatigue feature displacement sequence is obtained, wherein the facial fatigue feature displacement sequence includes the degree of eyelid drooping, the amplitude of the corner of the mouth drooping, and the change of the head posture; according to the facial fatigue feature displacement sequence, a sliding time window method is used to calculate the average values of the degree of eyelid drooping, the amplitude of the corner of the mouth drooping, and the change of the head posture in the window to obtain a feature average value sequence; a preset personalized fatigue judgment threshold is obtained from a user database, wherein the personalized fatigue judgment threshold includes the eyelid drooping threshold, the corner of the mouth drooping threshold, and the head posture change threshold; the personalized fatigue judgment threshold is dynamically adjusted by a weighted average method to obtain an adjusted threshold; for each data point in the feature average value sequence, a decision tree algorithm is applied to perform multi-level judgment, and the splitting rule of the decision tree algorithm is based on the adjusted threshold; a fatigue quantification index is used to represent the user's fatigue state, wherein the fatigue quantification index is obtained by weighted summation of the degree of eyelid drooping, the amplitude of the corner of the mouth drooping, and the change of the head posture; a state transition rule is set according to the fatigue quantification index to judge the user's fatigue level.
[0019] Specifically, according to the facial fatigue feature displacement sequence, the sliding time window technology is used, the window size is set to 10 seconds, and it slides once per second. The average values of eyelid ptosis, mouth corner downturned amplitude and head posture change in the window are calculated to obtain the feature average value sequence. The preset personalized fatigue judgment thresholds are obtained from the user database, including the eyelid ptosis threshold Teye, mouth corner downturned threshold Tmouth and head posture change threshold Thead, and the weighted average method is used to dynamically adjust the threshold Tadjusted=w1*Tbase+w2*agefactor+w3*occupationfactor+w4*historyfactor by combining the user age agefactor, occupation type occupationfactor and historical fatigue data historyfactor, where w1 to w4 are the corresponding weight coefficients and Tbase is the basic threshold. The decision tree algorithm is applied to each data point in the feature average value sequence for multi-level judgment, and the eyelid ptosis, mouth corner downturned amplitude and head posture change are used as input features to output the fatigue degree classification result. The splitting rule of the decision tree is based on the adjusted threshold Tadjusted, which comprehensively considers the combined effect of multiple features. The fatigue quantification index F is used to represent the user's fatigue state. The F value range is 0 to 100. It is calculated by weighted summation, F=a*eyefactor+b*mouthfactor+c*headfactor, where a, b, and c are weight coefficients. According to the F value, the state transition rule is set as F<30 for normal state, 30≤F<60 for mild fatigue, 60≤F<80 for moderate fatigue, and F≥80 for severe fatigue. The change trend of the fatigue quantification index F value within 5 consecutive minutes is recorded. If it shows a continuous upward trend, the fatigue level is automatically increased. Based on user feedback and actual fatigue judgment results, the personalized fatigue model parameters are periodically updated, including the decision tree structure, the weight coefficient of the threshold adjustment formula, and the weight coefficient of the fatigue quantification index, to achieve adaptive optimization of the model. The facial fatigue detection system collects user facial images at a frequency of 30 frames per second and extracts the feature point displacement sequence. A 10-second sliding window is used, updated once per second, and the feature average value in the window is calculated. For example, for the degree of eyelid droop, the average value of 300 frames of data is calculated. The basic thresholds obtained from the user database include Teyebase=5mm, Tmouthbase=3mm, and Theadbase=10°. The thresholds are dynamically adjusted using the weighted average method, with the weights set to w1=0.6, w2=0.1, w3=0.2, and w4=0.1. Taking a 25-year-old office worker as an example, agefactor=1.0, occupationfactor=1.2, and historyfactor=0.9, the adjusted eyelid ptosis threshold Teyeadjusted=5.25mm is obtained.The decision tree algorithm uses the CART (Classification and Regression Tree) method, with the adjusted threshold as the split point. The tree depth is set to 3, and the minimum number of leaf node samples is 100. The weight coefficients of the fatigue quantification index F are set to a=0.5, b=0.3, and c=0.2. If the user's eyelid droops 5.5mm, the corner of the mouth is downturned 2.8mm, and the head posture changes by 8°, then F=0.5*(5.5 / 5.25)+0.3*(2.8 / 3.15)+0.2*(8 / 10.5)=1.09 is calculated, which is judged as a severe fatigue state. The personalized model is updated every hour, and the decision tree structure and weight coefficients are adjusted according to user feedback. For example, if the user frequently reports misjudgment, the decision tree depth will be increased to 4, and the weights in the F value calculation will be adjusted to a=0.4, b=0.4, and c=0.2 to increase the influence of the downturned corner of the mouth factor.
[0020] Step S104, according to different fatigue levels, combined with the user's current location information and the user's destination information, obtain the corresponding recommended transportation plan from the transportation type library, and obtain the recommended route from the travel route library according to the recommended transportation plan.
[0021] The user's fatigue level, current location and destination information are obtained, and the type of transportation is screened using a multi-factor weighted scoring method based on the information, wherein the scoring criteria include safety factor, comfort, time efficiency and destination feature adaptability; the road network distance between the user's current location and the destination is obtained, and the estimated travel time and route of different transportation tools are calculated using the Dijkstra algorithm in combination with real-time traffic information and weather conditions; the candidate transportation tools are comprehensively evaluated using the Mamdani fuzzy reasoning method, and the input variables include fatigue level, travel distance, adjusted travel time, weather conditions and road information; according to the recommended transportation tool plan, the corresponding route information is extracted from the travel route library, and the route is optimized; the user's historical travel data and feedback information are obtained, and according to the historical travel data and feedback information, the recommendation results are personalized and optimized using a collaborative filtering algorithm to obtain the final recommended transportation tool and route plan.
[0022] Specifically, the multi-factor weighted scoring method is used to screen the types of transportation according to the user's fatigue level, current location and destination information. The scoring criteria include safety factor, comfort, time efficiency and destination feature adaptability. The weight setting adopts the hierarchical analysis method, considering the fatigue level, travel distance and destination type, such as business district and scenic spot. The three with the highest scores are selected from the transportation type library as candidates. The road network distance between the user's current location and the destination is obtained by using the geographic information system. Combined with real-time traffic information and weather conditions, the estimated travel time and route of different transportation tools are calculated by the Dijkstra algorithm. The random forest regression algorithm is used to input fatigue level, travel distance, weather conditions and historical data to predict the adjustment ratio of travel time. The Mamdani fuzzy inference system is used to comprehensively evaluate the candidate transportation tools. The input variables include fatigue level, travel distance, adjusted travel time, weather conditions and road conditions. Each input variable is fuzzified, such as dividing the fatigue level into three fuzzy sets of "mild", "moderate" and "severe". Set inference rules, such as "If the fatigue level is severe and the travel distance is long, the suitability of the transportation tool is low". Defuzzification is performed through the centroid method to obtain the suitability score of the transportation tool and determine the final recommended transportation tool plan. Based on the recommended transportation tool plan, the corresponding route information is extracted from the travel route library, and the route is optimized using the improved A* algorithm, considering the distribution of transportation hubs, the location of rest stops, and the fatigue recovery time. Generate a segmented route with rest points, and use a fuzzy controller to dynamically adjust the travel time and rest time of each section according to the characteristics of different transportation tools and the user's fatigue level. Combined with the user's historical travel data and feedback information, the collaborative filtering algorithm is used to personalize and optimize the recommendation results to improve the accuracy of the recommendation and user satisfaction. First, the user's current fatigue level is 75 moderate fatigue, the current location is the downtown business district, and the destination is a suburban resort. The weights calculated using the hierarchical analysis method are safety factor 0.4, comfort 0.3, time efficiency 0.2, and destination suitability 0.1. Score the candidate transportation tools, private cars, taxis, and high-speed rail, with the highest score of private cars being 8.5 points. The GIS calculated the road network distance to be 150 km, and the real-time traffic conditions showed that the highway was slightly congested and the weather was clear. The modified Dijkstra algorithm estimated the private car travel time to be 100 minutes. The random forest regression algorithm, based on 1000 decision trees, predicted the travel time adjustment ratio to be 1.15, and the adjusted travel time to be 115 minutes. The Mamdani fuzzy inference system mapped the fatigue level of 75 to the "moderate" fuzzy set, and the travel distance of 150 km to the "moderate" fuzzy set. The fuzzy rule "If the fatigue level is moderate and the travel distance is moderate, then the vehicle suitability is moderate" was applied, and the private car suitability score of 0.7 was obtained by defuzzification through the centroid method.The A* algorithm was used to optimize the route, and a rest stop was set at 50 km and 100 km from the starting point, with each rest stop lasting 15 minutes. The fuzzy controller dynamically adjusted the driving time according to the user's fatigue, and a 45-minute driving time was recommended for the first leg. The collaborative filtering algorithm analyzed the user's historical 100 travel data and found that the user preferred to drive a private car. The recommendation results were fine-tuned and finally confirmed that private cars were recommended as a means of transportation. The total travel time was expected to be 145 minutes, including two rest stops.
[0023] A weighted calculation is performed based on the user's fatigue level, travel distance and transportation availability to generate a comprehensive score for each available transportation option, and the transportation option with the highest score is used as the recommended transportation option, where the transportation option includes one or more transportation combinations.
[0024] The user's fatigue level and travel distance information are obtained, wherein the fatigue level is divided into three fuzzy sets of low, medium and high, and the travel distance is divided into three fuzzy sets of short, medium and long; according to the fatigue level and travel distance, the weight coefficients of the three factors of safety factor, comfort and time efficiency are dynamically adjusted by using a Mamdani fuzzy logic controller; based on the weight coefficients, a transportation tool combination scheme is constructed by using a non-dominated sorting genetic algorithm, and the transportation tool combination scheme includes two types: a single transportation tool and a combination of multiple transportation tools; for the transportation tool combination scheme, the safety factor, comfort and time efficiency scores of each feasible scheme are calculated, wherein the safety factor is based on historical accident rate data and real-time road condition information, the comfort is based on user feedback scores and vehicle types, and the time efficiency is based on estimated travel time and real-time weather factors; a hierarchical analysis method is used to combine weighted scores, the number of transfers and transportation costs to obtain the final score of each transportation tool scheme, wherein the hierarchical analysis method includes constructing a hierarchical structure, constructing a judgment matrix, and calculating a characteristic vector to obtain the weight of each criterion; it is judged whether there are other schemes with a score difference less than a threshold value for the scheme with the highest final score, if so, multiple alternative schemes are recommended at the same time, and if not, the scheme with the highest score is selected as the recommended transportation tool scheme.
[0025] Specifically, the Mamdani fuzzy logic controller is used to dynamically adjust the weight coefficient according to the user's fatigue level and travel distance. The mapping relationship between fatigue level, travel distance and three factors, safety factor, comfort level and time efficiency, is set, and the weight coefficient of each factor is obtained through fuzzy reasoning. The input variable fatigue level is divided into three fuzzy sets of low, medium and high, and the travel distance is divided into three fuzzy sets of short, medium and long. The output variable is the weight of each factor, and the centroid method is used for defuzzification. The non-dominated sorting genetic algorithm is used to construct the transportation tool combination scheme, considering two cases of a single transportation tool and a combination of multiple transportation tools. The set constraints include travel distance, transportation tool availability and the upper limit of the number of transfers, among which the transportation tool availability is obtained by real-time query of vehicle scheduling and reservation. All feasible solutions that meet the conditions are generated, and a unique identifier is assigned to each solution. For each feasible transportation tool solution, its safety factor, comfort level and time efficiency score are calculated. The safety factor is based on historical accident rate data and real-time road condition information, the comfort level is based on user feedback scores and vehicle type, and the time efficiency is based on estimated travel time and real-time weather factors. The min-max normalization process is used to make the scores of each item between 0 and 1, and then multiplied by the weight coefficient to obtain the weighted score. The analytic hierarchy process is used to comprehensively consider the weighted score, the number of transfers and the transportation cost to calculate the final score of each transportation plan. A three-layer hierarchy is constructed, with the target layer being the optimal transportation plan, the criterion layer being safety, comfort, efficiency, convenience and economy, and the plan layer being each feasible plan. The judgment matrix is constructed by the 9-level scaling method, and the eigenvector is calculated to obtain the weight of each criterion, and finally the comprehensive score of each plan is obtained. The plan with the highest score is selected as the recommended transportation plan. If there are multiple plans with similar scores and the score difference is less than 5%, multiple alternative plans are recommended for users to choose from. The intelligent travel recommendation system first obtains the user's fatigue level of 70 points of moderate fatigue and the travel distance of 200 kilometers. The Mamdani fuzzy logic controller maps the fatigue level to the "medium" fuzzy set with a membership of 0.8 and the travel distance to the "long" fuzzy set with a membership of 0.9. By setting fuzzy rules such as "if fatigue is medium and the distance is long, the safety weight is high", the weight coefficients are defuzzified by the centroid method as safety 0.4, comfort 0.3, and time 0.3. The non-dominated sorting genetic algorithm sets the population size to 100, the number of iterations to 50, the crossover probability to 0.8, and the mutation probability to 0.1, and generates 20 feasible solutions that meet the constraints, including "private car for the entire journey" and "high-speed rail + taxi". Real-time query shows that there are sufficient high-speed rail tickets and the waiting time for taxis is within 5 minutes. For the "high-speed rail + taxi" solution, the safety factor is calculated based on historical data to be 0.95, the user rating is 4.2 for comfort, the full score is 5, and the total estimated time is 150 minutes. Considering the current light rainfall, the time efficiency is adjusted to 0.9. After min-max normalization and weighting, the comprehensive score is 0.85.The analytic hierarchy process constructs a 5×5 judgment matrix and calculates the criterion layer weight vector [0.3, 0.25, 0.2, 0.15, 0.1]. The scores of each plan under the five criteria are multiplied by the weights. The final score of the "high-speed rail + taxi" plan is 0.88, 0.05 points ahead of the second place "private car for the whole journey". Therefore, the system recommends "high-speed rail + taxi" as the best plan and provides "private car for the whole journey" as an alternative.
[0026] Step S105, based on the recommended route and the user's fatigue level, an adaptive path planning algorithm is used to comprehensively recommend transportation tools, real-time traffic information, and rest stops along the way to dynamically optimize the recommended travel route. The itinerary is adjusted according to the fatigue level, including adding and reducing rest stops, adjusting waypoints, and adjusting the proportion of different transportation modes, to obtain a personalized dynamically optimized travel plan.
[0027] Initial path planning is performed according to the user's fatigue level and recommended routes, and the road network is abstracted into a weighted graph, in which nodes represent waypoints and rest stops, and edge weights are calculated by combining distance, time and fatigue recovery factors; the entire journey is divided into multiple time periods using a sliding time window method, and the local path is optimized using an ant colony optimization algorithm within the time period; the user's real-time fatigue status is obtained by fusion of facial recognition and physiological sensor data, and the fatigue index is quantified; a multi-objective optimization model is constructed, and the objective function of the multi-objective optimization model includes total travel time, fatigue level change and transportation mode switching cost, and the NSGA-II algorithm is used to solve the Pareto optimal solution set of the multi-objective optimization model; a fuzzy comprehensive evaluation is performed on the candidate solutions in the Pareto optimal solution set, and an evaluation index system is established, which includes time efficiency, comfort, safety and economy, and the index weights are determined by the hierarchical analysis method, and the comprehensive score of each solution is calculated, and the solution with the highest score is selected as the final personalized dynamic optimization travel solution.
[0028] Specifically, according to the user's fatigue level and recommended routes, the adaptive A* algorithm is used for initial path planning. The road network is abstracted into a weighted graph, with nodes representing waypoints and rest stops. The edge weights are calculated by comprehensively considering distance, time, and fatigue recovery factors. The heuristic function is dynamically adjusted according to the user's current fatigue level. The higher the fatigue level, the more attractive the rest stop is. Adaptive search is achieved by modifying the node evaluation function f(n)=g(n)+w(t)*h(n), where w(t) is the dynamic weight related to the fatigue level. f(n) is the evaluation function value of node nn, which determines how the search algorithm prioritizes different nodes. g(n) is the actual cost from the starting node to node n. h(n) is the estimated cost from node n to the target node (heuristic function). Using the sliding time window technology, the entire journey is divided into multiple 30-minute time periods, which are updated every 10 minutes. The ant colony optimization algorithm is used to optimize the local path in each time period. Considering real-time traffic information and the user's real-time fatigue status, the number and location of rest stops in the path are dynamically adjusted. The real-time fatigue status of users is obtained through the fusion of facial recognition and physiological sensor data, and quantified as a fatigue index of 0-100. The pheromone volatilization rate and local update rules in the ant colony algorithm are adjusted according to the degree of traffic congestion. A multi-objective optimization model is constructed, and the objective function includes the total travel time, fatigue level change and transportation mode switching cost. The constraints include the maximum continuous driving time, the minimum rest time and the fixed timetable of public transportation. The NSGA-II algorithm is used to solve the Pareto optimal solution set, and the characteristics of different transportation modes are considered in the crossover and mutation operations, such as the flexibility of private cars and the punctuality of public transportation. The proportion of different transportation modes is dynamically adjusted, and the transfer point selection is optimized by introducing a transfer penalty factor, and the real-time availability of various transportation modes is considered. The candidate solutions are ranked using the fuzzy comprehensive evaluation method, and an evaluation index system is established, including time efficiency, comfort, safety and economy. The index weights are determined by the hierarchical analysis method, combined with the user personalized preference model, which is built based on the user's historical behavior data through the collaborative filtering algorithm, and the comprehensive score of each solution is calculated. The solution with the highest score is selected as the final personalized dynamic optimization travel solution. The intelligent travel system obtains the user's current fatigue level of 75 (severe fatigue) and plans a 300-kilometer journey. The adaptive A* algorithm initializes the road network diagram, which contains 200 nodes and 500 edges. The edge weight calculation considers 60% of the distance, 30% of the time and 10% of the fatigue recovery. The dynamic weight of the heuristic function w(t) increases from 1.0 to 1.5 as the fatigue level increases. The system updates the path every 30 minutes, using 10 ants for local optimization, and the pheromone volatility rate is 0.1. The real-time traffic conditions show that the highway in the direction of the destination is moderately congested, and the system increases the pheromone volatility rate of this section to 0.2. The user's fatigue index is obtained by integrating facial recognition, whose weight is 0.7, and heart rate data, whose weight is 0.3.The multi-objective optimization model sets the maximum continuous driving time to 2 hours and the minimum rest time to 15 minutes. The NSGA-II algorithm uses a population size of 100 and iterates 50 times to obtain 20 Pareto optimal solutions. Dynamic adjustment shows that the proportion of private car use should be reduced by 10% and the proportion of high-speed rail use should be increased by 15%. The transfer penalty factor is set to 1.2, and after optimization, the transfer is selected in a city 150 kilometers away from the starting point. The fuzzy comprehensive evaluation uses 5 evaluation levels, and the indicator weights obtained by the hierarchical analysis method are time efficiency 0.3, comfort 0.25, safety 0.3, and economy 0.15. User historical data analysis shows that comfortable travel is preferred, and the comfort weight is adjusted to 0.35. The final recommended plan is to drive 1.5 hours to the high-speed rail station, rest for 20 minutes, take a 2-hour high-speed rail, and take a taxi for 30 minutes after leaving the station to reach the destination, with a total travel time of 4 hours and 20 minutes.
[0029] For users with severe fatigue, increase the number of rest stops, prompt them to take a rest every preset time, and give priority to places with comfortable rest facilities; for users with moderate fatigue, increase the number of rest stops and adjust some waypoints to shorten the single trip time; for users with mild fatigue, choose waypoints that can provide short rest.
[0030] According to the quantitative value of the user's fatigue level, the route is preliminarily planned to obtain an initial route plan including rest stops and waypoints; the initial route is optimized using a dynamic programming algorithm, in which the state transfer equation uses the number of rest stops, single travel time and total travel time as constraints; a multi-attribute utility function is constructed, and the comfort of rest facilities, location convenience and fatigue recovery efficiency are used as evaluation indicators to obtain the score of rest facilities, facility type and user feedback; the weighted summation method is used to calculate the comprehensive score of each candidate rest point, and the rest point combination with the highest score is selected to form an optimized rest plan; when the fatigue level exceeds the preset threshold or the continuous driving time exceeds the preset percentage of the plan, the route is re-planned, and the ant colony algorithm is used to locally optimize the remaining sections. By adjusting the pheromone update rules and the heuristic information calculation method, the location and frequency of the rest points are dynamically adjusted in combination with the real-time road conditions and the distribution of rest points.
[0031] Specifically, according to the user's fatigue level, the decision tree algorithm is used to make a preliminary route planning. The fatigue level is quantified into 0-100 points by integrating physiological indicators such as facial expression recognition, blinking frequency, heart rate variability, etc., combined with driving behavior characteristics such as steering wheel operation frequency and lane deviation times. The severe fatigue threshold is set to 80 points, the moderate fatigue threshold is set to 50 points, and less than 50 points is mild fatigue. Corresponding to different decision paths, an initial route plan containing rest stations and waypoints is generated. The dynamic programming algorithm is used to optimize the initial route. The state transfer equation considers the number of rest stations, single travel time and total travel time as constraints. Combined with real-time traffic information and the actual distribution of rest points in the POI database, the rest interval is dynamically adjusted. For severely fatigued users, rest points are set within 60-90 minutes; moderately fatigued users set rest points within 90-120 minutes; and mildly fatigued users set short rest points within 120-150 minutes. A multi-attribute utility function is constructed, and the comfort of rest facilities, location convenience and fatigue recovery efficiency are used as evaluation indicators. The score, facility type and user feedback of rest facilities are obtained by calling the API interface of Amap, and the comprehensive score of each candidate rest point is calculated by weighted summation method. The genetic algorithm is used to select the rest point combination with the highest score to form an optimized rest plan. The sliding time window technology is used to update the user fatigue status every 5 minutes. When the fatigue level exceeds the preset threshold or the continuous driving time exceeds 120% of the plan, the route replanning mechanism is triggered. The ant colony algorithm is used to locally optimize the remaining sections. By adjusting the pheromone update rules and heuristic information calculation methods, the location and frequency of rest points are dynamically adjusted in combination with real-time road conditions and rest point distribution. At the same time, the K-means clustering algorithm is used to regroup the waypoints to shorten the single trip time and generate an updated personalized itinerary plan. In the intelligent itinerary planning, the user fatigue level is detected to be 85 points, which is a severe fatigue state. The initial route is generated by the decision tree algorithm, which includes 5 rest stops and 3 waypoints. The dynamic programming algorithm considers a total journey of 300 kilometers and sets the state transition equation f(i,j)=min{f(i-1,k)+t(k,j)}, where i is the rest stop number, j is the current location, and t(k,j) is the travel time. f(i,j) is the minimum cost from the starting point to the i-th rest stop and then to the j-th destination, such as time or distance. k is the index of the intermediate rest stop, which is used to traverse all possible paths from the previous rest stop i−1 to the current rest stop i and then to the destination j. f(i−1,k) is the minimum cost from the starting point to the i−1-th rest stop and then to the k-th rest stop. t(k,j) is the cost from the k-th rest stop directly to the destination j, such as time or distance. The real-time traffic conditions show that there is a construction slowdown 50 kilometers ahead, so the first rest stop is moved forward to 75 minutes.The service area information along the way was obtained through the Amap API. The comfort rating of service area A was 4.5 stars, the location convenience was 0.9, and the fatigue recovery efficiency was 0.8; the indicators of service area B were 4 stars, 0.95, and 0.75 respectively. The multi-attribute utility function U=0.4 comfort+0.3 convenience+0.3*recovery efficiency, and the score of service area A was calculated to be 0.875, and the score of service area B was 0.845. Service area A was selected as the first rest point. The genetic algorithm set the population size to 50, iterated 100 times, and selected the optimal rest point combination. The sliding time window updated the user status every 5 minutes. After driving for 90 minutes, it was detected that the user fatigue rose to 92 points, triggering re-planning. The ant colony algorithm set 10 ants, the pheromone importance factor α=1, the heuristic factor β=2, and the next rest point was advanced by 15 minutes. At the same time, the K-means algorithm, where K=2, re-clustered the remaining waypoints, merged the two waypoints originally 30 kilometers apart into one, and shortened the single trip time. The final personalized plan included 6 rest stops, with an average rest stop every 70 minutes, and the total travel time was shortened from the originally planned 5 hours to 4 hours and 45 minutes.
[0032] Step S106, based on the dynamic optimization of the travel plan, provide the user with personalized navigation services, including recommended transportation, estimated travel time, rest stops and waypoint information. During the journey, the user's fatigue status is continuously monitored, and when the fatigue status changes, the itinerary is adjusted in real time, including increasing the frequency of rest, changing the mode of transportation and changing the number of waypoints.
[0033] Receive the user's historical travel data, current fatigue status and destination type code, the historical travel data is collected and uploaded by the user device; according to the historical travel data, current fatigue status and destination type code, a four-layer multi-layer perceptron neural network is used to calculate the personalized weight coefficient, the personalized weight coefficient includes distance weight, time weight, switching cost weight and preference coefficient weight; obtain the road network data, and calculate the weight of each edge in the road network according to the personalized weight coefficient, the edge weight is obtained by weighting the distance, time, switching cost and preference coefficient; the edge weight calculation formula is W=α distance+β time+γ switching cost+δ*preference coefficient, where α, β, γ, δ are weight coefficients of distance, time, switching cost, and preference coefficient; the A algorithm is used to plan the path of the weighted road network to obtain the initial optimal path; the user's physiological indicators and driving behavior data are received, and the physiological indicators and driving behavior data are collected and uploaded by the user's device; based on the physiological indicators and driving behavior data, the long short-term memory network is used to predict the user's fatigue state; if the user's fatigue state exceeds a preset threshold, a dynamic programming algorithm is used to adjust the itinerary in real time to obtain the adjusted optimal path; for the adjusted optimal path, it is determined whether the mode of transportation needs to be changed, and if the mode of transportation needs to be changed, the best alternative is selected.
[0034] Specifically, according to the dynamic optimization travel plan, a four-layer multi-layer perceptron neural network is used to model user preferences. The input layer contains 50 neurons, which receive historical travel data, current fatigue status and destination type encoding. The destination type is converted into a numerical feature through One-Hot encoding. The two hidden layers contain 30 and 20 neurons respectively, using the ReLU activation function. The output layer contains 5 neurons, corresponding to the personalized weight coefficients of different factors. The back propagation algorithm and Adam optimizer are used for training, and the learning rate is set to 0.001. The A algorithm is used for path planning, and the road network is abstracted into a weighted graph. The edge weight calculation formula is W=α distance+β time+γ switching cost+δ*preference coefficient, where α, β, γ, and δ are the weight coefficients output by the multi-layer perceptron. The road condition information is obtained in real time through the Amap API, and the edge weight is updated dynamically. The transportation tool switching cost is calculated based on the conversion time and convenience between different transportation modes. A fatigue state prediction model based on long short-term memory network (LSTM) was constructed. The network contains one LSTM layer with 64 units and two fully connected layers with 32 and 1 neurons. The input sequence is the physiological indicators such as facial expressions, blinking frequency, heart rate, etc. collected once a minute in the past 60 minutes, as well as driving behavior data such as steering wheel operation and vehicle speed change. The sliding window method is used to generate training samples with a window size of 60 and a step size of 1. The mean square error is used as the loss function and the Adam optimizer is used for training. The dynamic programming algorithm is used for real-time itinerary adjustment. The state is defined as S(i,t,f), which represents the optimal decision at position i, time t, and fatigue level f. The decision variables include whether to rest, whether to change the mode of transportation, and the next waypoint selection. The state transition equation is S(i,t,f)=max{R(a)+S(i',t',f')}, where R(a) is the immediate benefit of taking action a, and (i',t',f') is the new state after taking action a. The optimal decision sequence is updated every 15 minutes through the rolling optimization method. For the decision of changing the mode of transportation, a decision tree model is constructed. The input features include the current fatigue level, the remaining distance, the types of available transportation tools and the real-time traffic conditions. The decision tree performs feature splitting by minimizing the Gini coefficient, with the maximum depth set to 5 and the number of leaf node samples not less than 100. According to the output of the decision tree, it is determined whether the mode of transportation needs to be changed and the best alternative is selected. Smart travel can plan a 300-kilometer long-distance trip for users. First, the multi-layer perceptron neural network generates personalized weight coefficients of 0.2 for distance, 0.3 for time, 0.3 for switching cost and 0.2 for comfort based on the user's past 20 travel records, the current fatigue level of 80 points (severe fatigue) and the type of business destination (unique hot encoding [0,1,0,0,0]). The A* algorithm abstracts the road network into a weighted graph containing 500 nodes, obtains real-time traffic conditions through the Amap API, and finds that the highway in the direction of the destination is moderately congested.The algorithm calculates that the optimal route is to drive 100 kilometers to the high-speed rail station, take the high-speed rail for 150 kilometers, and then take a taxi for 50 kilometers to the destination. The LSTM fatigue prediction model receives physiological and driving data from the past 60 minutes, including an average of 30 blinks per minute, a heart rate of 75 beats per minute, and a steering wheel adjustment frequency of 2.5 times per minute. The model predicts that the fatigue level will reach 90 points in 30 minutes. The dynamic programming algorithm calculates the optimal decision based on the current state S (100km, 90min, 80) and the predicted state: rest for 20 minutes at the high-speed rail station and then transfer to the high-speed rail. The input features of the decision tree model include fatigue level 80, remaining distance 200km, available transportation [private car, high-speed rail, taxi], and highway congestion level 2. The decision tree output recommends changing to high-speed rail travel. Based on the results, the updated personalized navigation plan pushed to the user is recommended to rest for 20 minutes at the high-speed rail station at 100 kilometers and then take the high-speed rail to continue the journey. The total estimated time is 4 hours, which is 30 minutes shorter than the original plan.
[0035] The above embodiment is only one of the preferred implementation modes of the present invention and should not be used to limit the protection scope of the present invention. Any changes or modifications that are made to the main design concept and spirit of the present invention and have no substantive significance, and the technical problems they solve are still consistent with the present invention, should be included in the protection scope of the present invention.
Claims
1. The intelligent travel planning method of Maas service based on 3D face recognition is characterized by: The method comprises: Obtain the user's facial image, pre-process the image, extract facial feature points, obtain a standardized facial image, detect and locate facial feature points including eyes, mouth, nose tip, both ends of eyebrows and chin on the standardized facial image, where both ends of eyebrows and chin are used to determine head posture, and obtain three-dimensional coordinate information of facial feature points; According to the three-dimensional coordinate information of facial feature points, the facial feature point displacement analysis algorithm is used to calculate the tiny displacement of facial fatigue features at different times, including the degree of eyelid droop, the amplitude of the corner of the mouth, and the change of head posture, and obtain the facial fatigue feature displacement sequence; According to the facial fatigue feature displacement sequence, combined with the preset personalized fatigue judgment threshold, the fatigue degree is graded and judged. If the eyelid drooping degree exceeds the user's corresponding fatigue threshold, the user is judged to be in a severe fatigue state. If the mouth corners are turned down more than the threshold, the user is judged to be in a moderate fatigue state. If only the head posture changes more than the threshold, the user is judged to be in a mild fatigue state. According to different fatigue levels, combined with the user's current location information and the user's destination information, the corresponding recommended transportation plan is obtained from the transportation type library, and the recommended route is obtained from the travel route library according to the recommended transportation plan; According to the recommended route and user fatigue level, an adaptive path planning algorithm is used to dynamically optimize the recommended travel route by comprehensively recommending transportation tools, real-time traffic information, and rest stops along the way. The itinerary is adjusted according to the fatigue level, including adding or reducing rest stops, adjusting waypoints, and adjusting the proportion of different transportation modes, to obtain a personalized dynamic optimization travel plan. Based on dynamically optimized travel plans, personalized navigation services are provided to users, including recommended means of transportation, estimated travel time, rest stops and waypoint information. The user's fatigue status is continuously monitored during the journey. When the fatigue status changes, the itinerary is adjusted in real time, including increasing the frequency of rest, changing transportation methods and changing the number of waypoints.
2. The method according to claim 1, characterized in that The method of obtaining a user's facial image, preprocessing the image, extracting facial feature points, obtaining a standardized facial image, detecting and locating facial feature points including eyes, mouth, nose tip, both ends of eyebrows and chin on the standardized facial image, wherein both ends of eyebrows and chin are used to determine head posture, and obtaining three-dimensional coordinate information of facial feature points, includes: Acquire a user's facial image, and perform denoising and illumination compensation on the facial image to obtain a first image; According to the first image, adjusting the facial region to a standard position and size through affine transformation to obtain a standardized facial image; Using a Haar feature classifier to roughly locate the standardized facial image to determine the approximate areas of the eyes, nose, and mouth; Using active shape model to accurately locate key feature points of eye corners, nose tip and mouth corners in the approximate area, and obtain two-dimensional feature point coordinates; Obtain camera calibration parameters, and calculate the three-dimensional spatial coordinates of the feature points according to the two-dimensional feature point coordinates and the camera calibration parameters; Using the PnP algorithm, the three-dimensional space coordinates are matched with the corresponding two-dimensional feature point coordinates to solve the camera posture and obtain the head posture estimation result; According to the three-dimensional spatial coordinates, the relative positions and size ratios of facial organs such as eyes, nose, and mouth are calculated for facial expression analysis and identity recognition.
3. The method according to claim 1, characterized in that The facial feature point displacement analysis algorithm is used to analyze the three-dimensional coordinate information of the facial feature points, calculate the small displacements of facial fatigue features including eyelid drooping degree, mouth corner downturn amplitude, and head posture change at different times, and obtain the facial fatigue feature displacement sequence, including: Acquire the three-dimensional coordinate information of the facial feature points, calculate the displacement of the corresponding feature points between adjacent frames according to the three-dimensional coordinate information, and obtain an original displacement sequence; Smoothing the original displacement sequence using a sliding window mean filter to obtain a smoothed displacement sequence; The smoothed displacement sequence is predicted and corrected by using Kalman filtering, and the position estimation of each feature point is iteratively updated to obtain a more accurate displacement sequence; Calculating the cumulative displacement of each feature point in the displacement sequence within multiple consecutive frames to obtain basic data for feature extraction; Extract three fatigue features from the basic data: eyelid ptosis, mouth corner downturned amplitude, and head posture change, wherein the eyelid ptosis is the change in the distance between the upper and lower eyelid midpoints, the mouth corner downturned amplitude is the vertical displacement of the mouth corner point, and the head posture change is the change in the plane normal vector composed of the nose tip point and the outer eyebrow point; The variation of the three fatigue features between adjacent time windows is calculated to generate a facial fatigue feature displacement sequence.
4. The method according to claim 1, characterized in that: The fatigue degree is graded and judged according to the facial fatigue feature displacement sequence and the preset personalized fatigue judgment threshold. If the eyelid drooping degree exceeds the fatigue threshold corresponding to the user, the user is judged to be in a severe fatigue state. If the mouth corners are turned down by more than the threshold, the user is judged to be in a moderate fatigue state. If only the head posture changes by more than the threshold, the user is judged to be in a mild fatigue state, including: Acquire a facial fatigue feature displacement sequence, wherein the facial fatigue feature displacement sequence includes eyelid drooping degree, mouth corner downward angle, and head posture change; According to the facial fatigue feature displacement sequence, a sliding time window method is used to calculate the average values of eyelid drooping degree, mouth corner downturn amplitude and head posture change within the window to obtain a feature average value sequence; Acquire a preset personalized fatigue judgment threshold from a user database, wherein the personalized fatigue judgment threshold includes an eyelid ptosis threshold, a mouth corner downturn threshold, and a head posture change threshold; Dynamically adjusting the personalized fatigue judgment threshold by using a weighted average method to obtain an adjusted threshold; For each data point in the feature mean value sequence, a decision tree algorithm is applied to perform multi-level judgment, wherein the splitting rule of the decision tree algorithm is based on the adjusted threshold value; A fatigue quantification index is used to represent the fatigue state of the user, wherein the fatigue quantification index is calculated by weighted summing the degree of eyelid drooping, the amplitude of the corner of the mouth being turned down, and the change of the head posture; A state transition rule is set according to the fatigue quantification index to determine the user's fatigue level.
5. The method according to claim 1, characterized in that The method of obtaining a corresponding recommended transportation scheme from a transportation type library according to different fatigue levels, combined with the user's current location information and the user's destination information, and obtaining a recommended route from a travel route library according to the recommended transportation scheme includes: Obtaining user fatigue level, current location and destination information, and screening the type of transportation using a multi-factor weighted scoring method based on the information, wherein the scoring criteria include safety factor, comfort, time efficiency and destination feature suitability; Obtain the road network distance between the user's current location and the destination, combine real-time traffic information and weather conditions, and use the Dijkstra algorithm to calculate the estimated travel time and route of different means of transportation; A Mamdani fuzzy inference method is used to comprehensively evaluate the candidate transportation means, wherein the input variables include fatigue level, travel distance, adjusted travel time, weather conditions and road condition information; According to the recommended transportation scheme, corresponding route information is extracted from the travel route library, and the route is optimized; Obtaining the user's historical travel data and feedback information, and using a collaborative filtering algorithm to personalize and optimize the recommendation results based on the historical travel data and feedback information to obtain the final recommended transportation and route plan; It also includes: performing weighted calculations based on the user's fatigue level, travel distance, and transportation availability, generating a comprehensive score for each available transportation option, and selecting the transportation option with the highest score as the recommended transportation option, wherein the transportation option includes one or more transportation combinations.
6. The method according to claim 5, characterized in that The method performs weighted calculation based on the user's fatigue level, travel distance and transportation availability, generates a comprehensive score for each available transportation solution, and uses the transportation solution with the highest score as the recommended transportation solution, wherein the transportation solution includes one or more transportation combinations, including: Acquire information on the user's fatigue level and travel distance, wherein the fatigue level is divided into three fuzzy sets of low, medium and high, and the travel distance is divided into three fuzzy sets of short, medium and long; According to the fatigue level and travel distance, the weight coefficients of the three factors of safety factor, comfort level and time efficiency are dynamically adjusted by using the Mamdani fuzzy logic controller; Based on the weight coefficients, a transportation tool combination scheme is constructed using a non-dominated sorting genetic algorithm, wherein the transportation tool combination scheme includes a single transportation tool and a combination of multiple transportation tools; For the transportation combination scheme, calculate the safety factor, comfort level and time efficiency score of each feasible scheme, the safety factor is based on historical accident rate data and real-time traffic information, the comfort level is based on user feedback scores and vehicle type, and the time efficiency is based on estimated travel time and real-time weather factors; The final score of each transportation scheme is obtained by combining the weighted score, the number of transfers and the transportation cost using the hierarchical analysis method, wherein the hierarchical analysis method includes constructing a hierarchical structure, constructing a judgment matrix, and calculating a characteristic vector to obtain the weight of each criterion; Determine whether there are other options with a score difference less than a threshold for the option with the highest final score. If so, recommend multiple alternative options at the same time. If not, select the option with the highest score as the recommended transportation option.
7. The method according to claim 1, characterized in that The method adopts an adaptive path planning algorithm based on the recommended route and the user's fatigue level, comprehensively recommends transportation tools, real-time traffic information, and rest stops along the way to dynamically optimize the recommended travel route, and adjusts the itinerary according to the fatigue level, including adding and reducing rest stops, adjusting waypoints, and adjusting the proportion of different transportation modes, to obtain a personalized dynamic optimization travel plan, including: Initial path planning is performed based on the user's fatigue level and recommended routes, and the road network is abstracted into a weighted graph, where nodes represent waypoints and rest stops, and edge weights are calculated by combining distance, time, and fatigue recovery factors; Using the sliding time window method, the whole process is divided into multiple time periods, and the ant colony optimization algorithm is used to optimize the local path within the time period; Obtain the user's real-time fatigue status, which is obtained through facial recognition and physiological sensor data fusion to quantify the fatigue index; Constructing a multi-objective optimization model, wherein the objective function of the multi-objective optimization model includes total travel time, fatigue level change, and transportation mode switching cost, and using the NSGA-II algorithm to solve the Pareto optimal solution set of the multi-objective optimization model; Conduct fuzzy comprehensive evaluation on the candidate solutions in the Pareto optimal solution set, establish an evaluation index system, the evaluation index system includes time efficiency, comfort, safety and economy, determine the index weights through hierarchical analysis method, calculate the comprehensive score of each solution, and select the solution with the highest score as the final personalized dynamic optimization travel solution; It also includes: for severely fatigued users, increasing the number of rest stops, prompting users to take a rest every preset time, and giving priority to locations with comfortable rest facilities; for moderately fatigued users, increasing the number of rest stops and adjusting some waypoints to shorten the single trip time; for mildly fatigued users, selecting waypoints that can provide short rest.
8. The method according to claim 7, characterized in that For severely fatigued users, the number of rest stops is increased, and a rest reminder is given every preset time, and places with comfortable rest facilities are preferred; for moderately fatigued users, rest stops are increased, and some waypoints are adjusted to shorten the single trip time; for slightly fatigued users, waypoints that can provide short rest are selected, including: According to the quantified value of the user's fatigue level, the route is preliminarily planned to obtain an initial route plan including rest stops and waypoints; The initial route is optimized using a dynamic programming algorithm, wherein the state transition equation uses the number of rest stops, single travel time and total travel time as constraints; A multi-attribute utility function was constructed, taking the comfort, location convenience and fatigue recovery efficiency of rest facilities as evaluation indicators to obtain the scores, facility types and user feedback of rest facilities. The weighted summation method is used to calculate the comprehensive score of each candidate rest point, and the rest point combination with the highest score is selected to form an optimized rest plan; When the fatigue level exceeds the preset threshold or the continuous driving time exceeds the preset percentage of the plan, the route is replanned and the ant colony algorithm is used to locally optimize the remaining sections. By adjusting the pheromone update rules and heuristic information calculation methods, combined with real-time road conditions and rest point distribution, the location and frequency of rest points are dynamically adjusted.
9. The method according to claim 1, characterized in that: Based on the dynamic optimization of travel plans, the user is provided with personalized navigation services, including recommended transportation, estimated travel time, rest stops and waypoint information; the user's fatigue status is continuously monitored during the journey, and when the fatigue status changes, the itinerary is adjusted in real time, including increasing the frequency of rest, changing the mode of transportation and changing the number of waypoints, including: Receiving historical travel data of the user, current fatigue status and destination type code, wherein the historical travel data is collected and uploaded by the user device; According to the historical travel data, the current fatigue state and the destination type code, a four-layer multi-layer perceptron neural network is used to calculate the personalized weight coefficient, wherein the personalized weight coefficient includes a distance weight, a time weight, a switching cost weight and a preference coefficient weight; Acquire road network data, and calculate the weight of each edge in the road network according to the personalized weight coefficient, wherein the edge weight is obtained by weighting the distance, time, switching cost and preference coefficient; The edge weight calculation formula is W = α distance + β time + γ switching cost + δ * preference coefficient, where α, β, γ, δ are the weight coefficients of distance, time, switching cost and preference coefficient; Use A algorithm to plan the path of the weighted road network and obtain the initial optimal path; Receiving physiological indicators and driving behavior data of a user, wherein the physiological indicators and driving behavior data are collected and uploaded by a user device; According to the physiological indicators and driving behavior data, a long short-term memory network is used to predict the user's fatigue state; If the user's fatigue status exceeds the preset threshold, a dynamic programming algorithm is used to adjust the itinerary in real time to obtain the optimal route after adjustment; For the adjusted optimal path, determine whether it is necessary to change the mode of transportation. If it is necessary to change the mode of transportation, select the best alternative. 10.3D face recognition Maas service intelligent travel planning system, characterized by: The system comprises: Image preprocessing module, used to extract facial feature points and standardize facial images; Feature point analysis module, used to calculate the three-dimensional displacement of facial feature points; A fatigue state judgment module is used to judge the fatigue degree according to the displacement sequence and threshold value; A transportation recommendation module is used to recommend transportation based on fatigue level and location information; Path planning module, used to dynamically optimize travel routes and itineraries; The navigation service module is used to provide personalized navigation services and adjust itineraries in real time.
Citation Information
Patent Citations
Fixed path service system of intelligent network connection traffic system
CN110969833A
Travel management method, related device and system
CN115880892A
Public travel service system based on face recognition mode
CN117495642A
Personalized tourism planning system based on artificial intelligence
CN118210973A
Path dynamic planning method and device, terminal equipment and storage medium
CN118228906A
