Method and System for Rail Transit Passenger Energy Information Service under Multi-Source Data Fusion

Abstract

The present application relates to a method and system for rail transit passenger energy information service under multi-source data fusion, which solves the problem that in information display and communication, the form is single, lacking an effective guiding mechanism, it is difficult to attract the attention of passengers, and the communication effect of energy-saving information is poor. The method includes: inputting the extracted energy consumption data of the target line in the current period and the historical selection preferences of the corresponding consumption group into a pre-trained personalized energy-saving recommendation generation model to output personalized energy-saving recommendations; on the external display of rail transit, display the target line and station information concerned by passengers in the center of the screen, and use dynamic effects to guide passengers to pay attention to the recommended energy-saving information. The present application has the following effects: accurately capture the information concerned by passengers, generate adapted energy-saving recommendations, enhance passenger interaction, optimize operation energy efficiency, and improve service and energy utilization levels.

Description

Technical Field

[0001] The present invention relates to the field of rail transit operation management, and particularly to a rail transit passenger energy information service method and system under multi-source data fusion. Background Art

[0002] In the field of rail transit, with the increasing attention paid to energy issues, how to provide energy information services for passengers and guide passengers to participate in energy-saving actions has become an important topic for improving the operation efficiency and sustainability of rail transit. Effective energy information services can not only help passengers better plan their trips, reduce energy consumption, but also improve the overall operation efficiency of the rail transit system.

[0003] Currently, some rail transit systems have begun to try to provide energy information to passengers. Common methods include displaying overall line information on station display screens and simply informing some basic energy-saving tips through broadcasts.

[0004] Existing rail transit energy information service technologies have many drawbacks. In terms of information acquisition, it is impossible to accurately capture the specific attention information of passengers, such as the target line and station, and it is difficult to meet the personalized needs of passengers. In terms of generating energy-saving suggestions, there is a lack of in-depth analysis of passengers' behavior patterns, and it is impossible to provide targeted energy-saving solutions according to the characteristics of different passengers, resulting in low practicality and effectiveness of energy-saving suggestions. In terms of information display and communication, the form is single, lacking an effective guidance mechanism, and it is difficult to attract the attention of passengers, resulting in poor communication effects of energy-saving information. Summary of the Invention

[0005] In order to accurately capture passengers' attention information, generate adapted energy-saving suggestions, enhance passenger interaction, optimize operation energy efficiency, and improve service and energy utilization levels, the present application provides a rail transit passenger energy information service method and system under multi-source data fusion.

[0006] In the first aspect, the present application provides a rail transit passenger energy information service method under multi-source data fusion, adopting the following technical solutions:

[0007] A rail transit passenger energy information service method under multi-source data fusion includes:

[0008] Analyze whether a passenger enters the preset distance range of the rail transit external display;

[0009] If not, the rail transit external display only shows the overall line;

[0010] If so, monitor and obtain the behavior data of the passenger, and identify and obtain the attention information of the passenger according to the behavior data of the passenger, where the attention information includes the target line and station;

[0011] Extract the energy consumption data of the target line during the current period from the energy consumption database according to the passengers' attention information;

[0012] Obtain the passenger behavior model data, use the clustering algorithm to divide the consumption groups with similar corresponding passenger behavior patterns, and search for the historical selection preferences of the corresponding consumption groups in the passenger behavior model library. Among them, the passenger behavior model data includes travel time data, travel route data, consumption data, and historical energy-saving suggestion adoption data;

[0013] Input the extracted energy consumption data of the target line during the current period and the historical selection preferences of the corresponding consumption groups into the pre-trained personalized energy-saving suggestion generation model, and output personalized energy-saving suggestions. Among them, the personalized energy-saving suggestions include recommended energy-saving carriages, transfer paths, and key stations;

[0014] On the external display of the rail transit, display the target line and station information concerned by the passengers in the center of the screen, and use dynamic effects to guide the passengers to pay attention to the recommended energy-saving information.

[0015] By adopting the above technical solutions, it is possible to display content targeted according to whether the passengers are close to the display; provide personalized attention information according to the passengers' behavior data; generate personalized energy-saving suggestions through energy consumption data and historical selection preferences, provide guidance such as energy-saving carriages and transfer paths for passengers; use dynamic effects to guide attention to energy-saving information, enhance the passengers' energy-saving awareness, promote the rational use of rail transit energy, and also optimize the passengers' travel experience.

[0016] Optionally, monitor and obtain the passengers' behavior data, and identify and obtain the passengers' attention information according to the passengers' behavior data, including:

[0017] Start the camera device to capture the passenger image data within the preset distance range;

[0018] Based on the passenger image data, use the human pose recognition algorithm to analyze the passenger's body pose in real time, locate each key point of the human body, and calculate the standing orientation and head rotation angle of the passenger;

[0019] Analyze whether the calculated standing orientation of the passenger is within the effective judgment area of the preset standing orientation;

[0020] If not, it is determined that the passenger is not interested in the display information;

[0021] If so, conduct a quantitative analysis of the passenger's head rotation according to the head rotation quantitative analysis method, and obtain various quantitative indicators of the passenger's head rotation;

[0022] According to the various quantitative indicators of the standing orientation and head rotation, and according to the preset weighted algorithm, calculate the comprehensive score;

[0023] When and only when the comprehensive score exceeds a pre - set threshold, it is determined that the corresponding passenger is interested in the display information;

[0024] After determining that the corresponding passenger is interested in the display information, combined with the gaze estimation algorithm based on facial key points, capture the passenger's facial feature points through the camera, calculate the passenger's gaze direction and the landing position on the screen, judge the specific information area where the landing point is located, and record in detail the area type and precise coordinates;

[0025] Use the time - series analysis method to continuously monitor and obtain the stay trajectory and stay time of the passenger's gaze in the corresponding area;

[0026] Arrange the stay trajectory and stay time of the gaze in each area of the screen, as well as the passenger's standing orientation and head rotation angle in chronological order to form a series of behavior data samples, and each sample contains various behavior feature values at different times;

[0027] Extract key features from the behavior data samples and convert them into feature vectors, and input the converted feature vectors into the pre - constructed and trained passenger attention information analysis model to output the passenger's attention information.

[0028] By adopting the above - mentioned technical solution, through the camera device and the pose recognition algorithm, it can accurately judge the passenger's interest in the display information and effectively reduce invalid analysis. Combining the gaze estimation algorithm and time - series analysis, it records in detail the gaze trajectory and stay time to form behavior data samples. Converting them into feature vectors and inputting them into the analysis model can output accurate passenger attention information, laying a foundation for providing personalized services later and greatly improving the pertinence and effectiveness of services.

[0029] Optionally, the training and construction of the passenger attention information analysis model include:

[0030] Obtain the behavior data samples with completed annotations and the corresponding passenger attention information for each sample;

[0031] Divide the labeled samples into a training set and a validation set;

[0032] Input the feature data and the corresponding attention information labels in the training set into the Naive Bayes classification model, and the Naive Bayes model starts to calculate the conditional probability of each feature under different attention information categories and the prior probability of each attention information category;

[0033] Based on the conditional probability of each feature under different attention information categories and the prior probability of each attention information category, the Naive Bayes classification model completes the preliminary construction of the probability relationship between features and attention information;

[0034] Use the validation set to evaluate the model during training and calculate the accuracy rate;

[0035] If the accuracy rates for consecutive preset numbers of times all exceed the preset accuracy rate, it is determined that the passenger attention information analysis model has completed training;

[0036] Otherwise, based on the stage where the accuracy rate is located, adjust the smoothing parameter of the Naive Bayes model, and retrain the passenger attention information analysis model.

[0037] By adopting the above technical solution, by dividing the training set and the validation set, the model performance can be effectively evaluated. Using the Naive Bayes classification model to construct the probability relationship between features and attention information, the logic is clear and the calculation is efficient. Using the validation set to evaluate and calculate the accuracy rate can ensure the model effect. When the accuracy rate does not meet the standard, adjusting the smoothing parameter and retraining can improve the generalization ability and accuracy of the model, so as to accurately output the passenger attention information and provide strong support for rail transit information services.

[0038] Optionally, it further includes the steps after starting the camera device to capture the passenger image data falling within the preset distance range, specifically as follows:

[0039] Analyze whether there are multiple passengers according to the passenger image data;

[0040] If the answer is no, maintain the original settings and continue with the subsequent steps;

[0041] If the answer is yes, assign a unique identifier to each passenger in the order of their entry into the frame;

[0042] According to the image data of each passenger, calculate the standing orientation and head rotation angle of each passenger respectively, analyze various quantization indexes of their head rotation, as well as the gaze direction and the landing position on the screen;

[0043] Organize the stay trajectories and stay times of each passenger's gaze in each area of the screen, as well as the standing orientation and head rotation angle in chronological order to form independent behavior data samples for each passenger, extract key features from these samples, and transform them into feature vectors;

[0044] Parallelly input the feature vectors of each passenger into the pre-constructed and trained passenger attention information analysis model, and use distributed computing resources to perform calculations simultaneously to output the attention information of each passenger.

[0045] By adopting the above technical solutions, when multiple passengers are detected, a unique identifier is assigned to each of them, enabling accurate individual differentiation. The behavior data of each passenger is calculated and sorted separately, and key features are extracted and transformed into vectors to achieve a detailed analysis of the behaviors of multiple passengers. The feature vectors are input into the analysis model in parallel, and distributed computing resources are used for simultaneous calculation, greatly improving the processing efficiency. The attention information of each passenger can be quickly and accurately output, enhancing the comprehensiveness and timeliness of rail transit information services.

[0046] Optionally, it further includes steps after outputting the attention information of each passenger, specifically as follows:

[0047] According to the attention information of each passenger, the energy consumption data of the target line during the current period is independently extracted from the energy consumption database. At the same time, based on the behavior model data of each passenger, a clustering algorithm is used to divide consumption groups with similar behavior patterns for the corresponding passengers, and the historical selection preferences of the consumption group are searched.

[0048] The energy consumption data of the target line corresponding to each passenger and the historical selection preferences of the consumption group are respectively input into a pre-trained personalized energy-saving advice generation model to generate independent personalized energy-saving advice for each passenger.

[0049] By adopting the above technical solutions, energy consumption data is extracted according to the attention information of each passenger, consumption groups are divided in combination with the behavior model, and historical preferences are found, which can deeply explore individual characteristics. These are respectively input into the personalized energy-saving advice generation model to realize the independent generation of energy-saving advice for each passenger. This not only fully meets the personalized needs of different passengers but also accurately guides passengers to save energy during travel, helping rail transit achieve efficient energy conservation and high-quality services.

[0050] Optionally, it further includes steps after generating independent personalized energy-saving advice for each passenger, specifically as follows:

[0051] According to the number of passengers, the screen is divided into corresponding grid areas, and different main colors are set for the split-screen areas of each passenger.

[0052] The target line and station information that each passenger is concerned about are displayed in the center of the split-screen area, and dynamic effects are used to guide the corresponding passengers to pay attention to the recommended energy-saving information.

[0053] By adopting the above technical solutions, the screen grid area is divided according to the number of passengers and differentiated with different main colors, which is convenient for passengers to quickly locate their exclusive information. Placing the target line and station information in the center of the split screen and combining dynamic effects to guide attention to energy-saving advice can enhance the pertinence and attractiveness of information display, provide passengers with a personalized and immersive interactive experience, help passengers efficiently obtain the required content, and improve the quality and efficiency of rail transit information services.

[0054] Optionally, it further includes the steps after presenting the target line and station information that each passenger is concerned about in the center of the split screen area and using dynamic effects to guide the corresponding passengers to pay attention to the recommended energy-saving information, specifically as follows:

[0055] Start the directional microphone array to capture the conversation voices of passengers and their companions, and quickly analyze the language types of the voices through a speech recognition engine and natural language processing technology;

[0056] According to the analyzed language types, the system automatically matches the corresponding language voice packs;

[0057] Announce the screen positions of the recommended energy-saving information for the corresponding passengers in sequence according to the passenger identification, and the volume is automatically adjusted according to the station environmental noise, ensuring that it is higher than the background noise by a preset decibel.

[0058] By adopting the above technical solutions, the directional microphone array captures voices, analyzes the language types through speech recognition and natural language processing, and automatically matches the voice packs, which can solve the communication barriers of passengers with different languages. Announcing the screen positions of the energy-saving information in sequence according to the passenger identification and automatically adjusting the volume according to the environmental noise ensure that each passenger can clearly obtain the information, improving the inclusiveness and humanization of the service and making the rail transit information service more considerate and comprehensive.

[0059] In a second aspect, the present application provides a rail transit passenger energy information service system under multi-source data fusion, adopting the following technical solutions:

[0060] A rail transit passenger energy information service system under multi-source data fusion includes a memory, a processor, and a program stored on the memory and executable on the processor. When the program is loaded and executed by the processor, it can implement the rail transit passenger energy information service method under multi-source data fusion as described in the first aspect. Description of the Drawings

[0061] Figure 1 is the overall flowchart of a rail transit passenger energy information service method according to an embodiment of the present application under multi-source data fusion.

[0062] Figure 2 is the flowchart of monitoring and obtaining the behavior data of passengers and identifying and obtaining the concerned information of passengers according to the behavior data in another embodiment of the present application.

[0063] Figure 3 is the flowchart of the training and construction of a passenger concerned information analysis model in another embodiment of the present application.

[0064] Figure 4It is a flowchart diagram of the steps after starting the camera device to capture the image data of passengers falling within a preset distance range in another embodiment of the present application. DETAILED DESCRIPTION

[0065] The present application is further described in detail below in conjunction with the accompanying drawings.

[0066] Reference Figure 1 , is a rail transit passenger energy information service method under multi-source data fusion disclosed in this application, comprising:

[0067] Step S100, analyzing whether the passenger enters the preset distance range of the rail transit external display. If not, execute step S200; if yes, execute step S300.

[0068] The preset distance range is a specific distance area around the external display of rail transit, for example, a circular area with a radius of 2 meters with the display as the center. The system determines whether the passenger enters this range to determine the subsequent display information.

[0069] The method of obtaining the information is as follows: ultrasonic sensors are installed around the display. This sensor can emit ultrasonic waves, receive the reflected ultrasonic waves, and calculate the distance between the passenger and the display based on the round-trip time of the ultrasonic waves.

[0070] The general process is as follows: the sensor continuously emits ultrasonic waves. When a passenger enters the detection range of the sensor, the reflected ultrasonic waves are received by the sensor, and the sensor transmits the distance data to the system. The system compares the received distance with the preset distance. If the distance is less than or equal to the preset distance, it determines that the passenger has entered; otherwise, it is determined that the passenger has not entered. For example, at a busy transfer station, when passenger B approaches the display, the sensor immediately senses and calculates the distance. The system determines whether passenger B has entered the preset range based on the calculation results, and decides whether to display personalized information.

[0071] Step S200: The rail transit external display only displays the entire route.

[0072] The overall route is a complete information display of all stations and line directions included in the rail transit, covering each operating line, station name and the connection relationship between stations, helping passengers understand the overall picture of rail transit.

[0073] The general process is as follows: When the system determines that no passengers have entered the preset distance range, the display's microprocessor will trigger a read instruction to obtain the overall route data from the database through the communication line. Subsequently, the data is decoded and parsed, and the overall route map is presented on the display according to the established display format and layout. For example, during non-peak hours on weekdays, there are fewer passengers waiting on the platform and no one approaches the display. At this time, the display continues to display the overall route, which is convenient for passengers who occasionally pass by and want to understand the route overview.

[0074] Step S300, monitoring and acquiring passenger behavior data, and identifying and acquiring passenger concern information based on the passenger behavior data, the concern information including target routes and stations.

[0075] Among them, behavioral data refers to data that can reflect the behavioral characteristics of passengers, such as body posture, head rotation, gaze focus, etc., which can be used to analyze passenger interests.

[0076] Focus information: Content that passengers are interested in on rail transit, such as target routes and stations, can help passengers plan their trips.

[0077] The acquisition method is as follows: use the cameras around the display to capture the passenger image, use the sensor to collect body posture and head rotation data, and use the eye tracking device to obtain the gaze point information.

[0078] The general process is as follows: the camera continuously shoots, and the sensor collects data in real time. The system first performs human posture recognition on the image, analyzes body posture and head rotation; then determines the gaze point through line of sight tracking. Based on these data, combined with the preset algorithm, it is determined which information the passenger is interested in, and thus the information of interest is identified. For example, at the transfer station, passenger Xiao Li approaches the display, and the camera captures him leaning forward, turning his head, and staring at a specific area of ​​the screen for a long time. After analysis, the system determines that he is paying attention to a certain transfer line and station.

[0079] Step S400: extracting energy consumption data of the target line in the current period from the energy consumption database according to the passenger's concern information.

[0080] Target lines: rail transit lines where passengers are concerned about and want to know energy consumption information, such as Metro Line 1.

[0081] Energy consumption data: records the energy consumption of the target line in the current period, such as power consumption, traction energy consumption, etc.

[0082] The acquisition method is as follows: the energy consumption data is stored in the energy consumption database, which collects the energy consumption data of each line through the data acquisition system and is centrally managed by the server. The system establishes a connection with the server through the database interface to obtain the energy consumption data.

[0083] The general process is as follows: After the system identifies the passenger's concerned information, it determines the target line. Subsequently, it searches for the energy consumption data of the current period in the database according to the line identification. After finding the data, it extracts and transmits it to the subsequent processing module in the specified format. For example, when passenger Zhang is concerned about Line 2, after the system confirms, it extracts the energy consumption data of Line 2 in the current period from the database, providing a basis for generating personalized energy-saving suggestions later.

[0084] Step S500: Obtain the passenger behavior model data, use the clustering algorithm to divide the consumption groups with similar corresponding passenger behavior patterns, and search for the historical selection preferences of the corresponding consumption groups in the passenger behavior model library. Among them, the passenger behavior model data includes travel time data, travel route data, consumption data, and historical energy-saving suggestion adoption data.

[0085] Passenger behavior model data: Covers multi-dimensional data such as the passenger's travel time, route, consumption, and historical energy-saving suggestion adoption situation, used to depict the passenger's travel behavior characteristics.

[0086] Consumption group: A group formed by dividing passengers with similar behavior patterns according to the clustering algorithm, facilitating the analysis of the common behaviors and preferences of the group.

[0087] Historical selection preference: The tendency shown by the consumption group in the past in aspects such as travel choices and energy-saving measure adoption.

[0088] The acquisition method is as follows: The passenger behavior model data is stored in the passenger behavior model library and is collected and sorted by the rail transit operation system in the long term. The system obtains the data through the model library interface.

[0089] The general process is as follows: The system obtains the behavior model data of a certain passenger from the passenger behavior model library, uses the clustering algorithm to classify this passenger and other passengers according to behavior similarity, and divides the consumption group. Then, it searches for and extracts the historical selection preference data of this consumption group in the model library. For example, passenger Wang often commutes in the morning and has chosen the energy-saving carriage many times. After clustering, the system classifies him into a certain consumption group, and then extracts the historical preferences of this group in aspects such as energy-saving carriage selection and transfer routes, helping to generate personalized suggestions.

[0090] Step S600: Input the extracted energy consumption data of the target line in the current period and the historical selection preferences of the corresponding consumption group into the pre-trained personalized energy-saving suggestion generation model, and output personalized energy-saving suggestions. Among them, the personalized energy-saving suggestions include recommended energy-saving carriages, transfer paths, and key stations.

[0091] Personalized energy-saving suggestions: Energy-saving travel suggestions customized for passengers based on the energy consumption of the line concerned by the passengers and the historical preferences of the consumption group to which they belong, including recommended energy-saving carriages, transfer paths, and key stations.

[0092] Personalized energy-saving advice generation model: An artificial intelligence model trained with a large amount of data that can analyze the input data and output targeted energy-saving advice.

[0093] The acquisition methods are as follows: The personalized energy-saving advice generation model is pre-trained and stored in the server, and the system uses it by calling the model interface. The energy consumption data is obtained from the energy consumption database, and the historical selection preferences are obtained from the passenger behavior model library.

[0094] The general process is as follows: The system inputs the current period energy consumption data of the target line extracted from the database and the historical selection preferences of the corresponding consumption group into the personalized energy-saving advice generation model interface. The internal algorithm of the model analyzes the data correlation and outputs personalized energy-saving advice according to the learned pattern. For example, for passenger Xiao Zhao who is concerned about Line 3, the model combines the energy consumption of Line 3 and the preferences of his consumption group to output suggestions such as the recommended energy-saving carriage numbers and energy-saving transfer paths of Line 3.

[0095] Step S700, on the external display of the rail transit, display the target line and station information that the passengers are concerned about in the center of the screen, and use dynamic effects to guide the passengers to pay attention to the recommended energy-saving information.

[0096] Dynamic effects: Use visual change effects such as animations, flashes, and movements to attract the passengers' attention and highlight the displayed content.

[0097] Center of the screen: The visual core position of the display, where important information is placed for passengers to quickly obtain.

[0098] The acquisition methods are as follows: The displayed target line, station information, and recommended energy-saving information are obtained from the recognition results of the passengers' concerned information and the personalized energy-saving advice generation module respectively. The dynamic effects are realized through a pre-written display control program.

[0099] The general process is as follows: After the system obtains the target line, station information, and personalized energy-saving advice that the passengers are concerned about, it presents the target line and station information in a prominent manner in the center of the screen, such as enlarging the font and using bright colors. At the same time, call the display control program to add dynamic effects to the energy-saving information area, such as flashing prompts and arrow guides. For example, at a certain station, passenger Liu is concerned about a certain station on Line 4. The system displays the relevant information in the center of the screen and uses a flashing effect to guide him to pay attention to the recommended energy-saving carriages and transfer path information of Line 4.

[0100] Refer to Figure 2 , monitor and obtain the passengers' behavior data, and identify and obtain the passengers' concerned information based on the passengers' behavior data, including:

[0101] Step S310, start the camera device to capture the passenger image data within the preset distance range.

[0102] Camera device: A device installed near the external display of rail transit for taking pictures of passengers, such as a high-definition camera.

[0103] Preset distance range: A specific area defined with the display as the center. Only passengers within this range will be photographed, such as a circular area with a radius of 2 meters.

[0104] Passenger image data: An image file containing information such as the appearance and posture of passengers captured by the camera device.

[0105] The acquisition method is as follows: When the system detects that a passenger enters the preset distance range, it will send a start command to the camera device. The camera device starts to work and transmits the captured images to the system in the form of digital signals.

[0106] The general process is as follows: Once the distance monitoring module in the system identifies that a passenger enters the preset range, it immediately triggers the camera device to start. The camera device continuously takes pictures of the passengers at a set frame rate, such as 30 frames per second, and transmits these image data to the data processing unit in real time for subsequent analysis. For example, at a busy transfer station, when passenger Zhang enters within 2 meters of the display, the camera device automatically starts and takes a series of pictures of Zhang, providing basic data for subsequent analysis of his behavior.

[0107] Step S320: Based on the passenger image data, use a human pose recognition algorithm to analyze the body posture of the passenger in real time, locate each key point of the human body, and calculate the standing orientation and head rotation angle of the passenger.

[0108] Human pose recognition algorithm: An artificial intelligence algorithm that can identify the positions and postures of various parts of the human body from images, such as the OpenPose algorithm.

[0109] Body posture: The posture states of the passenger in the image, such as standing, walking, sitting posture, etc.

[0110] Key point: Representative parts of the human body, such as shoulders, elbows, wrists, hips, knees, ankles, etc., used to determine the body posture.

[0111] Standing orientation: The direction pointed by the front of the passenger's body when standing.

[0112] Head rotation angle: The angle by which the passenger's head rotates relative to the body's central axis, used to judge the direction of attention.

[0113] The acquisition method is as follows: The algorithm is pre-stored in the algorithm library of the system and is called through a program; the passenger image data is obtained from the shooting in step S310.

[0114] The general process is as follows: The system calls the human body pose recognition algorithm and inputs the passenger image data obtained in step S310 into it. The algorithm analyzes the image, identifies the key points of the human body, and determines the body pose based on the relative positions of the key points. Based on the body pose, mathematical methods such as trigonometric functions are used to calculate the standing orientation and the head rotation angle. For example, at the station, the image of passenger Xiao Wang is input into the algorithm. The algorithm identifies the key points such as his shoulders and hips, and calculates that Xiao Wang's standing orientation is 45° to the left of the display, and his head has rotated 30° towards the display direction.

[0115] Step S330: Analyze whether the calculated standing orientation of the passenger is within the effective judgment area of the pre-set standing orientation. If not, execute step S340; if so, execute step S350.

[0116] Standing orientation: The direction pointed by the front of the passenger's body when standing. Centered on the display, it can be represented by an angle. Effective judgment area: A specific angular range set around the display. Within this range, it indicates that the passenger may be paying attention to the display information. For example, based on the front of the display as a reference, the range is 30° to the left and right.

[0117] The acquisition method is as follows: The standing orientation data is calculated from step S320. The range of the effective judgment area is pre-set during system initialization and stored in the configuration file. The system obtains it by reading the configuration file.

[0118] The general process is as follows: The system reads the range of the effective judgment area and compares the calculated standing orientation angle of the passenger with it. If the standing orientation angle is within the effective judgment area, it indicates that the passenger may be paying attention to the display information; otherwise, it is less likely. For example, on the platform, the standing orientation angle of passenger Li is 20° to the right of the front of the display. After the system compares, it finds that this angle is within the effective judgment area of 30° to the left and right, and determines that his standing orientation meets the conditions for further analysis.

[0119] Step S340: Determine that the passenger is not interested in the display information.

[0120] Step S350: Conduct a quantitative analysis of the passenger's head rotation according to the head rotation quantitative analysis method, and obtain various quantitative indicators of the passenger's head rotation.

[0121] Head rotation quantitative analysis method: It is a technical means based on mathematical models and algorithms to accurately evaluate the passenger's head rotation. It analyzes multi-dimensional information such as the amplitude, frequency, and start and end angles of the head rotation, and converts the head rotation behavior into specific numerical values, so as to objectively and accurately measure the degree of the passenger's attention to the display information. For example, a quantitative model constructed based on trigonometric functions and time series analysis can extract key features from the dynamic head rotation data.

[0122] Quantitative indicators: A series of specific values obtained through quantitative analysis of head rotation, used to precisely describe the characteristics of head rotation. Common quantitative indicators include the rotation angle, which intuitively reflects the amplitude of head rotation; the rotation speed, that is, the angular change of head rotation per unit time, reflecting the speed of head rotation; and the rotation acceleration, used to measure the rate of change of rotation speed and revealing the dynamic trend of head rotation. These quantitative indicators are interrelated and jointly constitute a comprehensive description of head rotation behavior.

[0123] Obtaining method: The head rotation quantitative analysis method is a professional algorithm built into the system and stored in a dedicated algorithm library. When the system needs to perform head rotation quantitative analysis, the algorithm is called through a specific program interface. The head rotation angle data directly comes from step S320. In step S320, through the human body pose recognition algorithm to analyze the passenger image, the head rotation angle of the passenger has been accurately calculated.

[0124] The general process is as follows: When the system enters step S350, it first obtains the head rotation angle data of the passenger from step S320. These data are usually a series of angle values that change over time, recording the head positions of the passenger at different times. Then, the system calls the head rotation quantitative analysis method pre-stored in the algorithm library and inputs these angle data into the algorithm. The algorithm deeply processes the input angle data based on the preset mathematical model and logic. For example, by calculating the difference in angles between adjacent moments and dividing by the time interval, the head rotation speed is obtained; and then by analyzing the change in rotation speed, the rotation acceleration is calculated. After this series of calculations and analyses, finally, various quantitative indicators that can comprehensively reflect the head rotation characteristics of the passenger are obtained. These quantitative indicators will be used as important bases for comprehensive judgment and analysis in subsequent steps.

[0125] Step S360, according to the standing orientation and various quantitative indicators of head rotation, and according to the preset weighted algorithm, calculate the comprehensive score.

[0126] Standing orientation: Refers to the direction pointed by the front of the passenger's body when standing, which is one of the important factors for judging whether the passenger is paying attention to the display information.

[0127] Head rotation quantitative indicators: Data obtained through quantitative analysis of the passenger's head rotation, such as rotation angle, speed, etc., can reflect the degree of attention of the passenger.

[0128] Weighted algorithm: A calculation method that assigns weights according to the influence degree of different factors on the result, used to synthesize multiple factors to obtain the final result.

[0129] Comprehensive score: The score calculated according to the weighted algorithm from the quantified indicators of standing orientation and head rotation, which is used to determine the degree of interest of passengers in the information on the display.

[0130] The acquisition method is as follows: The quantified indicators of standing orientation and head rotation are obtained from the previous steps (S320, S350). The weighted algorithm is preset and stored in the program by the system, and the system calls this algorithm for calculation.

[0131] The general process is as follows: The system first extracts the passenger's standing orientation data and the quantified indicators of head rotation from the previous steps. Then, according to the preset weighted algorithm, different weights are assigned to the standing orientation and each quantified indicator of head rotation. For example, it is considered that the standing orientation has a greater impact on judging interest, and a weight of 0.6 is assigned, the weight of the head rotation speed is 0.2, and the weight of the rotation angle is 0.2. Next, each item of data is multiplied by the corresponding weight and then added together to obtain the comprehensive score. For example, passenger Xiaosun's standing orientation meets the attention condition and gets 80 points, the corresponding score of the head rotation speed is 70 points, and the corresponding score of the rotation angle is 85 points. After weighted calculation: 80×0.6 + 70×0.2 + 85×0.2 = 79 points, and the comprehensive score of 79 points is obtained.

[0132] In step S370, when and only when the comprehensive score exceeds the preset threshold, it is determined that the corresponding passenger is interested in the information on the display.

[0133] Comprehensive score: In step S360, the value obtained by comprehensively calculating the data such as the quantified indicators of standing orientation and head rotation through the weighted algorithm, which is used to measure the degree of attention of passengers to the information on the display.

[0134] Preset threshold: A fixed value set in advance by the system, which is used as the standard for judging whether passengers are interested in the information on the display. When the comprehensive score is compared with it, if it is higher than it, it is determined to be interested, otherwise it is not interested.

[0135] In step S380, when it is determined that the corresponding passenger is interested in the information on the display, combined with the gaze estimation algorithm based on facial key points, the camera captures the facial feature points of the passenger, calculates the gaze direction of the passenger and the landing position on the screen, judges the specific information area where the landing point is located, and records the area type and accurate coordinates in detail.

[0136] Gaze estimation algorithm based on facial key points: An algorithm that estimates the gaze direction of passengers by identifying key facial feature points, such as the corners of the eyes, pupils, etc.

[0137] Facial feature points: The landmark positions on the face, such as the key points of parts like eyes, nose, mouth, etc., which are used to determine the facial orientation and gaze direction.

[0138] Gaze direction: The direction that the passenger's eyes are looking at, which can reflect the content they are concerned about.

[0139] Landing point: The specific location where the gaze is projected on the screen, expressed by coordinates.

[0140] Information area: Different functional areas on the screen, such as route map area, site introduction area, etc.

[0141] The acquisition method is as follows: the sight line estimation algorithm based on facial key points is stored in the algorithm library of the system and called by the program. The passenger facial image data captured by the camera comes from step S310, and the facial feature points are extracted from the facial image by a special facial recognition algorithm.

[0142] The general process is as follows: After the system determines that the passenger is interested in the information on the display, it calls the line of sight estimation algorithm based on facial key points. The algorithm extracts facial feature points from the facial image captured by the camera, and uses these feature points, combined with geometric models and mathematical calculation methods, to estimate the direction of the passenger's gaze. According to the gaze direction, combined with the spatial position and imaging principle of the display, the position of the gaze on the screen is calculated. Then, based on the information area pre-divided on the screen, the specific area type where the landing point is located is determined, and the area type and the precise coordinates of the landing point are recorded in detail. For example, passenger Xiao Zhang is interested in the information on the display. The system extracts his facial feature points through an algorithm, calculates the gaze direction, determines that his gaze falls on the site introduction area on the screen, and records the area type and the landing point coordinates (x, y).

[0143] Step S390, using a time series analysis method, continuously monitors and obtains the trajectory and duration of passengers' gazes in the corresponding area.

[0144] Time series analysis method: a method of analyzing data based on time sequence, which can reveal the laws and trends of data changes over time.

[0145] Gaze trajectory: The path formed by the passenger's gaze moving across different areas of the screen, reflecting the switching of their attention content.

[0146] Dwell time: the length of time the eyes remain fixed on a certain area of ​​the screen, reflecting the degree of attention paid to the content in that area.

[0147] The general process is as follows: the system continuously obtains the passenger's gaze location information from step S380, and attaches a timestamp to each acquired location data. Based on these timestamped location information, the system calls the time series analysis method to connect the location of adjacent moments in chronological order, thereby drawing the gaze trajectory. At the same time, by calculating the time interval between changes in adjacent location positions, the duration of gaze stay in each area is determined. For example, when passenger Xiao Li is looking at the display, the system obtains his gaze location every 0.1 seconds, and connects these location points over time to obtain his gaze trajectory. By calculating the time interval between each position change, it is known that he stayed in the station introduction area for 3 seconds and in the transfer information area for 2 seconds.

[0148] Step S3A0, sorting the gaze trajectory and dwell time in each area of ​​the screen, as well as the passenger's standing direction and head rotation angle in chronological order to form a series of behavior data samples, each sample containing various behavior feature values ​​at different times.

[0149] Dwell trajectory: The path formed by the passenger's eyes moving across different areas of the screen, showing the process of shifting focus.

[0150] Dwell time: The length of time that eyes remain fixed on a specific area of ​​the screen, reflecting the degree of attention.

[0151] Standing direction: The direction in which the passenger's body points when standing, used to determine their initial attention tendency.

[0152] Head rotation angle: The angle at which the passenger's head rotates relative to the central axis of the body, which assists in analyzing changes in attention.

[0153] Behavioral data sample: A data set formed by integrating the above-mentioned behavioral feature values ​​and arranging them in chronological order for subsequent analysis of passenger behavior patterns.

[0154] The general process is as follows: the system first extracts the trajectory of the passenger's gaze in each area of ​​the screen, the dwell time, as well as the standing direction and head rotation angle data from the corresponding steps. Then, in chronological order, all kinds of data at the same time are summarized to form behavioral data samples. Each sample is like a "behavioral snapshot", recording the various behavioral characteristics of the passenger at that moment. For example, in the time period of 10:00:00-10:00:05, the standing direction of passenger Xiao Wang was 15° to the left of the front of the display, the head rotation angle was 20° to the right, the eyes stayed in the station query area for 3 seconds, and the trajectory showed that it moved from the upper left corner of the area to the lower right corner. The system organizes these data into a behavioral data sample in chronological order.

[0155] Step S3B0, extract key features from the behavior data sample and convert them into feature vectors, input the converted feature vectors into a pre-built and trained passenger attention information analysis model, and output the passenger's attention information.

[0156] Key features: behavioral features extracted from behavioral data samples that are of great value in determining the information that passengers pay attention to, such as areas where the gaze stays for a long time, specific changes in standing orientation, etc.

[0157] Feature vector: An ordered vector that represents key features numerically to facilitate computer processing and analysis.

[0158] Passenger concern information analysis model: The artificial intelligence model, which has been trained with a large amount of data, can analyze and output passengers' concern information based on the input feature vector.

[0159] The general process is as follows: the system obtains the behavior data sample from step S3A0, and selects key features from it, such as the long stay of the eyes in a certain route map area. These key features are then converted into feature vectors, such as digitizing the information such as the stay time and regional coordinates to form a vector. Finally, the feature vector is input into the passenger attention information analysis model, and the model outputs the passenger's attention information, such as the target line and station through internal algorithm analysis. For example, the behavior data sample of passenger Xiao Zhang shows that his eyes stay for a long time on the route map of Line 2 and a certain station location. The system extracts key features and converts them into vectors for input into the model, and concludes that Xiao Zhang pays attention to Line 2 and the station.

[0160] Reference Figure 3 ,The training and construction of the passenger attention information analysis model includes:

[0161] Step S3B1, obtaining the labeled behavior data samples and the passenger attention information corresponding to each sample.

[0162] Completely labeled behavioral data samples: Based on the original behavioral data samples, manually or through a specific algorithm, mark the data set of passenger attention information corresponding to the sample. For example, mark the concerned routes and stations corresponding to the areas where passengers' gazes stay for a long time in a certain sample.

[0163] Passenger focus information: The content that passengers actually focus on when viewing rail transit displays, such as specific lines, station names, transfer information, etc.

[0164] The acquisition method is as follows: Behavior data samples can be obtained from a large number of past passenger behavior monitoring records, which are stored in the database. The labeling work can be done manually by professionals or with the help of some semi-automatic labeling tools. For example, using image recognition auxiliary tools, quickly mark the screen area information corresponding to the gaze point.

[0165] The general process is as follows: First, retrieve all relevant behavior data samples from the database. These samples contain information such as the standing orientation of passengers, head rotation angles, gaze fixation trajectories, and time. Then, organize professionals or use semi-automatic annotation tools to determine and annotate the information that passengers are concerned about based on the behavior data in the samples. For example, for a behavior data sample that records a passenger's gaze staying in a specific area on the screen for a long time, the annotator labels the information that the passenger is concerned about, such as the first and last bus times of a certain route, according to the content displayed in that area.

[0166] Step S3B2: Divide the annotated samples into a training set and a validation set.

[0167] Training set: A part of the data divided from the annotated behavior data samples, used to train the passenger attention information analysis model to enable the model to learn the relationship between features and attention information.

[0168] Validation set: Another part of the data divided from the annotated samples, used to evaluate the performance of the model during the training process and test the adaptability of the model to new data.

[0169] Obtaining method: Both the training set and the validation set are sourced from the behavior data samples obtained and annotated in step S3B1. When dividing, it is usually in a certain proportion, such as the common 70% as the training set and 30% as the validation set.

[0170] The general process is as follows: First, determine the division ratio. For example, set the training set to account for 70% and the validation set to account for 30%. Then, use methods such as random sampling or stratified sampling to divide the annotated behavior data samples. Random sampling is to randomly select a certain number of data from the samples as the training set, and the remaining as the validation set; stratified sampling considers the distribution of different features in the samples and extracts data from each layer according to the proportion to ensure that the feature distributions of the training set and the validation set are similar. For example, for 1000 annotated behavior data samples, using random sampling, randomly select 700 as the training set, and the remaining 300 as the validation set.

[0171] Step S3B3: Input the feature data and the corresponding attention information labels in the training set into the Naive Bayes classification model, and the Naive Bayes model starts to calculate the conditional probability of each feature under different attention information categories and the prior probability of each attention information category.

[0172] Feature data: Representative information extracted from behavior data samples, such as the standing orientation of passengers, head rotation angles, gaze fixation times, and trajectories, etc. These data can help the model identify the patterns of passengers' attention information.

[0173] Attention information label: The actual content that passengers are concerned about corresponding to the feature data, such as the concerned lines, stations, transfer information, etc., which is used to guide the model learning.

[0174] Naive Bayes classification model: A classification method based on Bayes' theorem and the assumption of feature conditional independence. By calculating the probabilities of features appearing under different categories, it predicts the category to which new data belongs.

[0175] Conditional probability: The probability of a certain feature appearing under a certain attention information category, indicating the degree of association between the feature and the category.

[0176] Prior probability: The probability of a certain attention information category appearing without any new information, reflecting the basic distribution of the category in the overall data.

[0177] The general process is as follows: After organizing the feature data and the corresponding attention information labels in the training set into a format acceptable to the model, they are input into the naive Bayes classification model. The model starts to traverse the data in the training set. For each attention information category, it counts the number of times each feature appears under this category, and then calculates the conditional probability of each feature under different attention information categories. At the same time, it counts the number of times each attention information category appears in the training set and calculates the prior probability of each attention information category. For example, in the training set, there are sample data about passengers' concern for line A and line B. The model will count the number of times the passengers' standing orientation is towards the left side of the display when they are concerned about line A, and the total number of times line A is concerned in all samples, so as to calculate the corresponding conditional probability and prior probability.

[0178] Step S3B4, based on the conditional probability of each feature under different attention information categories and the prior probability of each attention information category, the naive Bayes classification model completes the preliminary construction of the probability relationship between the features and the attention information.

[0179] The general process is as follows: After obtaining the conditional probability of each feature under different attention information categories and the prior probability of each attention information category, the naive Bayes classification model integrates these probabilities according to Bayes' theorem. For the newly input feature data, the model uses the calculated probabilities to calculate the posterior probabilities of each attention information category under this feature data. For example, when there is new passenger behavior data, including features such as standing orientation towards the right side of the display and gaze staying in a certain area of the screen for 5 seconds, the model calculates the posterior probabilities of the passenger's concern for line C and station D according to the previously calculated conditional probabilities and prior probabilities, so as to establish the probability relationship between the features and the attention information. Through a large number of such calculations, the model completes the preliminary construction of the probability relationship between the features and the attention information, laying a foundation for subsequent prediction and analysis.

[0180] Step S3B5: Use the validation set to evaluate the model during training and calculate the accuracy rate.

[0181] Accuracy rate: The proportion of the number of samples correctly predicted by the model to the total number of samples, which is an important indicator to measure the performance of the model. In this scenario, it refers to the proportion of the number of samples in which the model accurately determines the information of interest to passengers to the total number of samples in the validation set.

[0182] Obtaining method: The validation set data used to evaluate the model comes from the validation set divided in Step S3B2. After the model completes preliminary training in Step S3B4, this validation set is used to evaluate the model.

[0183] General process: Input the feature data in the validation set into the Naive Bayes classification model that has been preliminarily trained in Step S3B4. The model predicts the information of interest to passengers corresponding to each sample according to the constructed probability relationship. Then, compare the prediction results with the actual information labels of interest corresponding to the samples in the validation set, and count the number of samples correctly predicted. Finally, divide the number of samples correctly predicted by the total number of samples in the validation set to obtain the accuracy rate of the model on the validation set. For example, if there are 100 samples in the validation set and the model correctly predicts the information of interest to passengers for 80 of them, then the accuracy rate is 80÷100 = 80%. Through this accuracy rate, the adaptability of the model to new data and the prediction accuracy can be judged, providing a basis for the subsequent judgment of whether the training is completed.

[0184] Step S3B6: If the accuracy rates for consecutive preset times all exceed the preset accuracy rate, it is determined that the model for analyzing the information of interest to passengers is trained.

[0185] General process is as follows: After obtaining the accuracy rate by evaluating the model in Step S3B5, the system compares this accuracy rate with the preset accuracy rate. At the same time, record the number of consecutive times reaching the preset accuracy rate. If the accuracy rates for consecutive preset times all exceed the preset accuracy rate, it is determined that the model for analyzing the information of interest to passengers is trained. For example, the preset accuracy rate is 85%, the consecutive preset times is 3 times. In the first evaluation, the accuracy rate of the model is 86%, the second is 88%, and the third is 87%. Since it exceeds 85% for 3 consecutive times, the system then determines that the model training is completed. If the condition that the accuracy rates for consecutive preset times all exceed the preset accuracy rate is not met, go to Step S3B7 to adjust and retrain the model.

[0186] Step S3B7: Otherwise, based on the stage where the accuracy rate is located, adjust the smoothing parameter of the Naive Bayes model and retrain the model for analyzing the information of interest to passengers.

[0187] Smoothing parameter: In the Naive Bayes model, a parameter used to solve the zero-probability problem. When a certain feature does not appear under a certain category in the training data, it will cause the conditional probability of this feature under this category to be 0. The smoothing parameter can avoid this situation and ensure the stability of the model.

[0188] Obtaining method: The current passenger attention information analysis model (Naive Bayes model) in training directly obtains from the process that is currently being trained. The accuracy data comes from the evaluation result of step S3B5. The smoothing parameter is given a default value during model initialization, stored in the configuration parameters of the model, and can be read and modified during adjustment.

[0189] The general process is as follows: When it is judged in step S3B6 that the model training is not completed, that is, when the accuracy rate for a continuous preset number of times does not exceed the preset accuracy rate, the system will adjust the smoothing parameter according to the current stage of the accuracy rate and the pre-set rules. For example, if the accuracy rate is low, appropriately increase the smoothing parameter to enhance the generalization ability of the model to the data; if the accuracy rate is close to the preset value, slightly adjust the smoothing parameter. After adjustment, use the training set data in step S3B2 to retrain the model, and repeat the process from step S3B3 to S3B6 again, continuously optimizing the model until the model meets the training completion standard. For example, the preset accuracy rate is 80%, and the accuracy rates of the model's three consecutive evaluations are 75%, 76%, and 77% respectively, not meeting the standard. The system judges that the accuracy rate is low, increases the smoothing parameter from the default 0.1 to 0.3, and then retrains the model with the training set data.

[0190] Refer to Figure 4 , a method for providing rail transit passenger energy information service under multi-source data fusion further includes steps after starting the camera device to capture passenger image data within a preset distance range, specifically as follows:

[0191] Step SA00: Analyze whether there are multiple passengers based on the passenger image data. If the answer is no, execute step SB00; if the answer is yes, execute step SC00.

[0192] Passenger image data: The passenger image information captured by starting the camera device within a preset distance range, including the appearance, posture, etc. of the passengers.

[0193] Obtaining method: The passenger image data comes from the started camera device. After the shooting is completed, these data are stored in the system's temporary storage area in the form of digital image files, waiting for subsequent analysis and processing.

[0194] The general process is as follows: The system calls the image analysis algorithm to process the passenger image data in the storage area. The algorithm counts the number of passengers in the image by identifying information such as human body contours and facial features in the image. If the analysis result shows that there is only one passenger in the image, the system executes step SB00; if multiple passengers are detected, the system executes step SC00. For example, in a certain surveillance video at a station, the camera device captures image data. After the system analyzes it, it finds that there are 3 passengers in the video, determines that there are multiple passengers, and then executes step SC00.

[0195] Step SB00: Maintain the original settings and continue with the subsequent steps.

[0196] Original settings: Refer to the various parameters and operating modes preset by the system for the single-passenger scenario before judging the number of passengers, including but not limited to the basic configuration of the image analysis algorithm, the established rules of the data processing flow, etc.

[0197] Step SC00: Assign a unique identifier to each passenger in the order they enter the frame.

[0198] Unique identifier: A unique identity label assigned to each passenger, used to accurately identify and distinguish the data and behavior information of different passengers in the subsequent process, generally composed of numbers, letters, or a combination of both.

[0199] Obtaining method: The system creates unique identifiers based on a preset identifier generation rule. This rule is usually stored in the system configuration file. For example, it can generate identifiers by combining a timestamp and a random number.

[0200] The general process is as follows: When the system determines that there are multiple passengers in step SA00, it then executes step SC00. The system assigns unique identifiers to each passenger in the order they enter the camera frame, according to the preset identifier generation rule. First, the system records the time point when each passenger enters the frame, and then sorts them according to the time sequence. Then, using the identifier generation rule, it generates the first unique identifier for the passenger who enters the frame first, and so on. For example, in the surveillance video of a large transfer station, three passengers enter one after another.

[0201] The system first records their entry times. In order, it generates the identifier "P001_20250209100001" for the first passenger to enter (where "P001" is a fixed prefix representing the passenger, and the following numbers are a combination of timestamp and random number), "P002_20250209100005" for the second passenger, and "P003_20250209100010" for the third passenger, which facilitates the subsequent separate processing and analysis of the behavior data of different passengers.

[0202] Step SD00: According to the image data of each passenger, calculate the standing orientation and head rotation angle of each passenger respectively, analyze various quantitative indicators of head rotation, as well as the gaze direction and the landing position on the screen.

[0203] The acquisition method is as follows: The image data of each passenger comes from the data stored correspondingly after being captured by the imaging device and judged in step SA00 and marked with identification in step SC00. Various algorithms required for analysis, such as the human pose recognition algorithm for calculating the standing orientation and head rotation angle, and the gaze estimation algorithm for determining the gaze direction and landing position, are all stored in the system algorithm library and can be called at any time.

[0204] The general process is as follows: The system first obtains the image data of each passenger with a unique identifier from the storage area. Then, call the human pose recognition algorithm to analyze the body pose of the passengers in the image and calculate the standing orientation and head rotation angle. Next, based on the change of the head rotation angle over time, use mathematical calculation methods to obtain various quantitative indicators of head rotation. For example, calculate the difference in head rotation angles at adjacent times divided by the time interval to get the rotation speed. After that, call the gaze estimation algorithm, identify facial key points such as the corners of the eyes and pupils, and estimate the gaze direction in combination with a geometric model, and calculate the landing position on the screen according to the spatial position relationship between the gaze direction and the screen. For example, in a station, the system obtains the image data of passenger A. After algorithm analysis, it is found that his standing orientation is 30° to the left of the screen, his head rotates 45° to the right, the rotation speed is 15° per second, the gaze direction points to the route map area on the screen, and the landing position coordinates are (x1, y1).

[0205] Step SE00: Organize the stay trajectories and stay times of each passenger's gaze in each area of the screen, as well as the standing orientation and head rotation angle in chronological order to form independent behavior data samples, extract key features from these samples, and transform them into feature vectors.

[0206] The general process is as follows: the system first summarizes the gaze positions and corresponding timestamps of each passenger at different times, generates gaze trajectory by analyzing the changes in the gaze positions, and calculates the dwell time in each area. Then, these dwell trajectories, dwell time, standing direction, and head rotation angle data obtained in step SD00 are integrated in chronological order to form independent behavior data samples. Next, the algorithm is used to filter out key features from the behavior data samples. For example, if a passenger's gaze stays in a certain station information area for much longer than in other areas, this long stay can be used as a key feature. Finally, these key features are converted into numerical values ​​to form feature vectors. For example, the behavior data sample of passenger B shows that his gaze has a complex dwell trajectory in the screen route map area, stays at a certain station information for 5 seconds, stands facing the screen, and the head turns 20° to the left. The system extracts the key feature of gaze stay for 5 seconds and converts it into the value "5". After the other key features are digitized, they form a feature vector [5,0,20] (assuming that "0" represents the numerical representation of standing facing the screen).

[0207] In step SF00, the feature vector of each passenger is input in parallel into a pre-built and trained passenger attention information analysis model, and distributed computing resources are used to perform calculations simultaneously to output the attention information of each passenger.

[0208] Feature vector: An ordered vector composed of the numerical representation of key features in the passenger behavior data sample, including key information such as standing direction, head rotation angle, gaze dwell time, etc., which is used to input the analysis model.

[0209] Passenger information analysis model: This artificial intelligence model, which has been trained with a large amount of data, can analyze and output the information that passengers are actually concerned about, such as the routes, stations, transfer information, etc., based on the input feature vector.

[0210] Distributed computing resources: A collection of computing resources consisting of multiple computing nodes, which work together through network connections and can process multiple tasks at the same time to improve computing efficiency. In this step, it is used to parallelize the feature vector analysis tasks of multiple passengers.

[0211] The acquisition method is as follows: the feature vector of each passenger comes from the calculation result of step SE00. The passenger attention information analysis model is pre-built and trained, stored in the server, and used by the system by calling the model interface. Distributed computing resources are provided by the computing cluster where the system is located, and are allocated and managed through the resource scheduling system.

[0212] The general process is as follows: the system inputs the feature vector of each passenger generated in step SE00 into the pre-built and trained passenger attention information analysis model in parallel. Using distributed computing resources, multiple computing nodes simultaneously process and analyze the feature vectors of different passengers. The model calculates and analyzes each feature vector based on the relationship between the internally learned features and attention information. For example, for the feature vector [3,15,4] of passenger C (assuming that they represent the numerical representation of the gaze dwell time, the head turning 15° to the right, and the standing direction being a certain angle to the left), the model determines that the passenger is concerned about the first and last bus times of a certain line based on the rules obtained through training. Finally, the model outputs the attention information of each passenger, laying the foundation for providing personalized services to passengers in the future.

[0213] A rail transit passenger energy information service method under multi-source data fusion also includes steps after outputting each passenger's attention information, which are as follows:

[0214] Step SG00, based on the attention information of each passenger, independently extract the energy consumption data of the target line in the current period from the energy consumption database. At the same time, based on the behavioral model data of each passenger, a clustering algorithm is used to divide the corresponding passengers into consumer groups with similar behavior patterns, and find the historical selection preferences of the consumer group.

[0215] The general process is as follows: the system first determines the target line of interest based on the interest information of each passenger output in step SF00. Then, the energy consumption data of the target line in the current period is extracted from the energy consumption database. For example, passenger D is interested in Line 2, and the system queries the power consumption, equipment operation energy consumption and other data of Line 2 in the current period from the energy consumption database. At the same time, the system calls the clustering algorithm, takes the behavior model data of each passenger obtained in step SE00 as input, and divides the passengers into different consumer groups according to the similarity of the behavior patterns. After the division is completed, for each consumer group, its historical selection preferences are searched from the historical data record database. For example, through analysis, it is found that a consumer group has chosen to travel during non-peak hours many times in the past to save energy. This is the historical selection preference of the consumer group.

[0216] Step SH00, input the target route energy consumption data and consumer group historical selection preferences corresponding to each passenger into the pre-trained personalized energy-saving suggestion generation model, and generate independent personalized energy-saving suggestions for each passenger.

[0217] The general process is as follows: the system organizes the target line energy consumption data and consumer group historical selection preferences corresponding to each passenger obtained in step SG00 into a format acceptable to the model, and then inputs it into the personalized energy-saving suggestion generation model that has been trained in advance. The model uses internal algorithms and learned patterns to analyze and process the input data. For example, for passenger E, the corresponding target line Line 3 has high energy consumption, and the consumer group has a historical preference to avoid peak travel. The model combines this information to generate personalized energy-saving suggestions such as "It is recommended that you take Line 3 during non-peak hours, when the line energy consumption is lower and travel is more energy-efficient." The model generates corresponding energy-saving suggestions for each passenger independently to meet the energy-saving needs of different passengers.

[0218] A rail transit passenger energy information service method based on multi-source data fusion also includes steps after generating independent personalized energy-saving suggestions for each passenger, specifically as follows:

[0219] Step SI00, divide the screen into corresponding grid areas according to the number of passengers, and set a different main color tone for each passenger's split-screen area.

[0220] Grid area: The screen is divided into multiple small areas according to certain rules. Each area can be used to display exclusive information of different passengers, making it convenient for passengers to quickly locate their own information area.

[0221] Main color: The main color set for each passenger split-screen area to distinguish the information display areas of different passengers and enhance visual recognition.

[0222] The acquisition method is as follows: the number of passengers comes from the result obtained by the camera device and the analysis and statistics in the previous steps. The rules of screen division and color setting are stored in the system configuration file, and the system calls the corresponding rules to operate according to the number of passengers.

[0223] The general process is as follows: the system first obtains the number of passengers in the current scene. For example, through preliminary analysis, it is known that there are 5 passengers. Then, according to the preset screen division rules, the screen is divided into a corresponding number of grid areas. Assuming that the screen is a rectangle, according to the rule of uniform division, the screen is divided into an appropriate number of parts horizontally and vertically to form 5 grid areas of similar size. After that, according to another set of preset color allocation rules, a different main color is set for each grid area. For example, the area of ​​the first passenger is set to blue as the main color, the second is set to green, the third is set to yellow, etc., to ensure that the split-screen area of ​​each passenger is visually distinguished, so that passengers can quickly find their own information display area.

[0224] Step SJ00: Display the target line and station information that each passenger is concerned about in the center of the split screen area, and use dynamic effects to guide the corresponding passengers to pay attention to the recommended energy-saving information.

[0225] The general process is as follows: First, the system extracts the target line and station information that each passenger is concerned about from Step SF00, and then displays this information in the center position of the split screen area corresponding to the passenger divided in Step SI00. For example, if passenger F is concerned about a certain station on Line 5, the system will clearly display the line map of Line 5 and the detailed information of this station, such as transfer information around the station, the first and last train times, etc., in the center of the split screen area divided for passenger F. After that, the system uses front-end display technology and animation libraries to add dynamic effects to the recommended energy-saving information within the split screen area. For example, make the text of the energy-saving information appear in a flashing manner, or add an arrow animation that moves from the edge of the screen to the area of the energy-saving information, so as to guide the corresponding passengers to pay attention to the recommended energy-saving information and improve the transmission effect of the energy-saving information.

[0226] A rail transit passenger energy information service method under multi-source data fusion further includes steps after displaying the target line and station information that each passenger is concerned about in the center of the split screen area and using dynamic effects to guide the corresponding passengers to pay attention to the recommended energy-saving information, specifically as follows:

[0227] Step SK00: Activate the directional microphone array to capture the communication voices of passengers and their companions, and quickly analyze the language type of the voices through a speech recognition engine and natural language processing technology.

[0228] Directional microphone array: A device composed of multiple microphones that can collect sounds in a specific direction with high sensitivity, and is used to capture the communication voices of passengers and their companions in this step.

[0229] Speech recognition engine: A technical tool that converts speech signals into text information, can recognize the speech of different languages, and convert it into a form of text that can be processed by a computer.

[0230] Natural language processing technology: A series of technologies for analyzing, understanding, and generating human languages, and is used to analyze the recognized text and extract key information, such as the language type, in this step.

[0231] Language type: The language used by passengers for communication, such as Mandarin, English, Cantonese, etc.

[0232] The acquisition method is as follows: The directional microphone array is a part of the system hardware device and is directly integrated into the information collection system of the station. The speech recognition engine and natural language processing technology are usually integrated in the system background in the form of software libraries and can be called at any time.

[0233] The general process is as follows: The system activates the directional microphone array, which starts capturing the conversation voices of passengers and their companions within the station. The collected voice signals are first transmitted to the speech recognition engine, which converts the voice signals into text content. For example, converting a Mandarin conversation voice into the corresponding text "Which line should we take to reach the transfer station". Then, the converted text is input into the natural language processing technology module, which analyzes features such as the vocabulary and grammar structure of the text to determine the language type of the voice. If it is analyzed that the above text is Mandarin, the system records this language type information to prepare for matching the corresponding voice package later.

[0234] Step SM00, according to the analyzed language type, the system automatically matches the corresponding language voice package.

[0235] Language type: The language used by passengers for communication, such as Mandarin, English, Japanese, etc., sourced from the analysis result of step SK00.

[0236] Voice package: A collection of pre-recorded voice files containing various information, with each language corresponding to a voice package, used to broadcast relevant information to passengers, such as energy-saving suggestions, screen position prompts, etc.

[0237] Obtaining method: The analyzed language type is obtained from step SK00. The voice packages are stored in the system's voice resource database, which is classified and stored according to language types for convenient and rapid retrieval.

[0238] The general process is as follows: After the system obtains the language type used by passengers for communication in step SK00, based on this language type information, it retrieves in the voice resource database. The database stores voice packages in different languages, and each voice package has a corresponding language identifier. For example, when the system determines that the language used by passengers is English, it finds the corresponding English voice package by querying the language identifier of the voice packages in the database. This English voice package contains various preset voice contents, such as the English expression of energy-saving suggestions, English prompts to guide passengers to pay attention to screen information, etc., for subsequent playback according to requirements to ensure that information can be conveyed in the language familiar to passengers.

[0239] Step SN00, sequentially broadcast the screen positions where the energy-saving information recommended for the corresponding passengers is located according to the passenger identifier order, and the volume is automatically adjusted according to the station environmental noise, ensuring that it is higher than the background noise by a preset decibel.

[0240] Passenger identifier: A unique identity label assigned to each passenger in step SC00, used to accurately identify different passengers so as to broadcast relevant information targeted at them.

[0241] Energy-saving information: The personalized energy-saving suggestions generated in step SH00, as well as relevant energy information prompts, such as target line energy consumption data, etc.

[0242] Screen position: In the split-screen areas divided in step SI00, the specific position for displaying the corresponding passenger energy-saving information, and passengers are guided to pay attention through voice.

[0243] Preset decibel: A volume difference standard preset by the system to ensure that the broadcast voice volume is higher than the background noise, so as to ensure that passengers can hear clearly, such as set to 10 decibels.

[0244] Obtaining method: The passenger identification comes from the allocation result of step SC00, the energy-saving information is obtained from step SH00, and the screen position information is determined according to the screen division result of step SI00. The preset decibel value is set and stored in the system configuration file during system initialization. The station ambient noise is obtained by real-time collection through an ambient noise sensor, and the sensor data is transmitted to the system for volume adjustment calculation.

[0245] General process: The system sequentially extracts the energy-saving information voice content corresponding to the passenger from the voice package according to the order of passenger identification. For example, first extract the energy-saving information voice of passenger A. At the same time, combine the split-screen area position divided for passenger A in step SI00 to generate a voice prompt about the screen position where the energy-saving information is located. Then, obtain the ambient noise value of the current station through the ambient noise sensor, and calculate the appropriate broadcast volume according to the preset decibel value. Suppose the ambient noise is 50 decibels and the preset decibel is 10 decibels, then the broadcast volume is set to 60 decibels. Finally, through the station's broadcast system, broadcast the passenger's energy-saving information and its position on the screen at the calculated volume to ensure that each passenger can clearly receive the exclusive energy-saving information.

[0246] Furthermore, for the step in step S600 of "inputting the extracted current-period energy consumption data of the target line and the historical selection preferences of the corresponding consumption group into a pre-trained personalized energy-saving suggestion generation model and outputting personalized energy-saving suggestions", the following settings can be made, specifically as follows:

[0247] Step Sa00, use a convolutional neural network to extract features from the facial images captured by the camera, combine deep learning algorithms to analyze facial muscle movements in real time to judge the passenger's emotional state; and use the Gaussian mixture model in voiceprint recognition technology to extract features and match the voice collected by the microphone, identify the passenger's identity and associate its historical behavior data, and parse the voice content through natural language processing technology to mine the passenger's preference tendencies in aspects such as time, energy consumption, and crowding degree.

[0248] Step Sb00, deeply integrate the personalized energy-saving recommendation generation model with the path optimization model based on reinforcement learning to build a fusion model framework. In the reinforcement learning model, define the state space as information such as the real-time emotions, preferences, current location, and target route of passengers; define the action space as different path planning strategies, such as choosing different routes and transfer stations. Design a reward function to give reward feedback according to the actual effect, giving positive rewards for meeting passengers' time requirements and negative rewards for excessive energy consumption or overcrowding.

[0249] Based on a large amount of historical data and machine learning algorithms training, construct a real-time preference-parameter mapping table. According to the emotion and preference information obtained from micro-expression and voiceprint recognition, query the mapping table to determine the adjustment range of the parameters related to time, energy consumption, and crowding degree in the model. Adopt an adaptive learning rate strategy to dynamically adjust the step size of parameter update according to the training effect and feedback of the model.

[0250] For the energy consumption data of the target route in the current period, use data smoothing algorithms such as wavelet transform to remove noise, and combine with the Kalman filter algorithm to accurately predict the energy consumption trend in the short term in the future based on multi-source information such as the real-time train operation status, line conditions, and weather. For the historical selection preference data of the consumer group, use feature engineering methods such as principal component analysis to extract key features and transform them into numerical vectors, and perform one-hot encoding. Use association rule mining algorithms such as Apriori to find the potential connections between different preferences, and combine with the quantified emotion and preference information obtained from micro-expression and voiceprint recognition, and use the weighted fusion algorithm to dynamically weight and adjust the historical selection preference data.

[0251] Step Sc00, input the preprocessed and enhanced data, as well as the adjusted model parameters, into the fused personalized energy-saving recommendation generation model. The model adopts the Transformer architecture and uses its powerful attention mechanism to fully consider the associations between various factors, and combines with the Pareto optimal path calculation result to generate personalized energy-saving recommendations including recommended energy-saving carriages, transfer paths, and energy-saving measures at key stations.

[0252] In addition, considering the situation of passengers' dissatisfaction, there can be subsequent settings after step Sc00, which are as follows:

[0253] Step Sd00, use multi-modal perception technology to deploy multiple cameras to capture the micro-expression changes and body movements of passengers in all directions, and analyze the behavior intentions of passengers through pose estimation algorithms. Use directional microphones and speech recognition technology to collect passengers' voice feedback. Integrate touch sensors and pressure sensors on the display device to sense the interaction behavior between passengers and the display device. Use sentiment analysis algorithms, combined with recurrent neural networks and attention mechanisms in deep learning, to deeply analyze these feedbacks to judge passengers' satisfaction, doubts, and interested parts about the recommendations.

[0254] Step Se00, if it is analyzed that the passenger has doubts or is not interested in some suggestions, combined with the content of Pareto optimal path dynamic programming, re-evaluate the path and suggestions.

[0255] Using the Monte Carlo tree search algorithm, when recalculating the Pareto optimal path, consider more potential path combinations and factors. The specific application process of the Monte Carlo tree search algorithm is as follows:

[0256] Initialize the search tree: Take the current passenger's location and destination as the root node, and use possible path selections (such as different lines, transfer stations) as child nodes to construct the initial structure of the Monte Carlo tree. Each node contains relevant information about the path, such as estimated energy consumption, required time, estimated crowding degree, etc.

[0257] Simulation and evaluation: Starting from the root node, randomly select a path for simulation. During the simulation process, consider the impact of temporary traffic control on line operation. For example, if a certain line does not stop at some stations due to control, then the path passing through this station needs to be re-planned to bypass this station during the simulation; for sudden passenger flow changes, if there is a sudden increase in passenger flow at a certain station, the transfer time and crowding degree evaluation of this station will be increased during the simulation. After each simulation ends, evaluate the path according to the simulation results. The evaluation indicators include energy consumption, time, crowding degree, and the matching degree with the passenger's preferences. For example, if the passenger is sensitive to time, then the weight of the time indicator in the evaluation will be increased accordingly.

[0258] Expansion and update of the tree: According to the simulation results, select the path nodes with better performance for expansion, and add more possible path selections as child nodes. At the same time, update the evaluation value and access times of each node. The nodes with higher evaluation values and more access times have a greater probability of being selected in subsequent searches. Through multiple iterations of simulation and tree expansion and update, gradually find a better path combination.

[0259] Determine the new path: After a certain number of iterations, select the path with the optimal evaluation value as the recalculated Pareto optimal path. This path fully considers factors such as temporary traffic control, sudden passenger flow changes, and the real-time preferences of passengers.

[0260] Based on the same inventive concept, an embodiment of the present invention provides a rail transit passenger energy information service system under multi-source data fusion, including a memory and a processor. The memory stores a program that can be run on the processor to implement any Figures 1 to 4 of the methods.

[0261] The embodiments of this specific implementation manner are all preferred embodiments of this application, and do not limit the protection scope of this application. Therefore, all equivalent changes made according to the structure, shape, and principle of this application shall be covered within the protection scope of this application.

Claims

1. A rail transit passenger energy information service method based on multi-source data fusion, characterized in that: include: Analyze whether the passenger enters the preset distance range of the rail transit external display; If not, the rail transit external display will only show the overall route; If yes, monitor and obtain the passenger's behavior data, and identify and obtain the passenger's attention information based on the passenger's behavior data, the attention information including the target route and station; According to the passengers' attention information, the energy consumption data of the target line in the current period is extracted from the energy consumption database; Obtain passenger behavior model data, and use clustering algorithms to divide the corresponding passenger behavior patterns into consumer groups with similar patterns, and search for the historical selection preferences of the corresponding consumer groups in the passenger behavior model library, where the passenger behavior model data includes travel time data, travel route data, consumption data, and historical energy-saving suggestion adoption data; The extracted energy consumption data of the target route during the current period and the historical selection preferences of the corresponding consumer groups are input into the pre-trained personalized energy-saving suggestion generation model to output personalized energy-saving suggestions, where the personalized energy-saving suggestions include recommended energy-saving carriages, transfer routes and key stations; On the external display of rail transit, the target lines and station information that passengers are concerned about are displayed in the center of the screen, and dynamic effects are used to guide passengers to pay attention to recommended energy-saving information.

2. According to the rail transit passenger energy information service method under multi-source data fusion according to claim 1, it is characterized in that: Monitoring and obtaining passenger behavior data, and identifying and obtaining passenger information of interest based on passenger behavior data include: Starting the camera device to capture image data of passengers within a preset distance range; Based on passenger image data, a human posture recognition algorithm is used to analyze the passenger's body posture in real time, locate the key points of the human body, and calculate the passenger's standing direction and head rotation angle; Analyze and calculate whether the passenger's standing orientation obtained is within a pre-set effective judgment area for the standing orientation; If not, it is determined that the passenger is not interested in the display information; If yes, then a head rotation quantitative analysis is performed on the passenger according to the head rotation quantitative analysis method, and various quantitative indicators of the passenger's head rotation are obtained; Based on the quantitative indicators of standing orientation and head rotation, and according to the preset weighted algorithm, a comprehensive score is calculated; If and only if the comprehensive score exceeds a preset threshold, it is determined that the corresponding passenger is interested in the display information; When it is determined that the corresponding passenger is interested in the information on the display, the camera is used to capture the passenger's facial feature points in combination with the gaze estimation algorithm based on facial key points, calculate the passenger's gaze direction and the location of the gaze on the screen, determine the specific information area where the gaze is located, and record the area type and precise coordinates in detail; Use time series analysis methods to continuously monitor and obtain the trajectory and duration of passengers' gazes in the corresponding areas; The gaze trajectory and duration in each area of ​​the screen, as well as the passenger's standing direction and head rotation angle are sorted in chronological order to form a series of behavior data samples, each of which contains various behavior feature values ​​at different times; Key features are extracted from the behavioral data samples and converted into feature vectors. The converted feature vectors are input into a pre-built and trained passenger attention information analysis model to output the passenger attention information.

3. According to the rail transit passenger energy information service method under multi-source data fusion according to claim 2, it is characterized in that: The training and construction of the passenger attention information analysis model includes: Obtain the labeled behavior data samples and the passenger attention information corresponding to each sample; Divide the labeled samples into training set and validation set; The feature data and the corresponding attention information labels in the training set are input into the Naive Bayes classification model. The Naive Bayes model starts to calculate the conditional probability of each feature under different attention information categories, as well as the prior probability of each attention information category. Based on the conditional probability of each feature under different categories of attention information and the prior probability of each category of attention information, the naive Bayes classification model completes the preliminary construction of the probabilistic relationship between the feature and the attention information; Use the validation set to evaluate the model in training and calculate the accuracy; If the accuracy rate exceeds the preset accuracy rate for consecutive preset times, it is determined that the passenger attention information analysis model has completed training; Otherwise, the smoothing parameters of the naive Bayes model are adjusted based on the accuracy stage, and the passenger attention information analysis model is retrained.

4. According to the rail transit passenger energy information service method under multi-source data fusion according to claim 2, it is characterized in that: The method further includes the following steps after starting the camera device to capture the image data of the passenger within the preset distance range: Analyzing whether there are multiple passengers based on the passenger image data; If not, maintain the original settings and continue with the subsequent steps; If yes, a unique identifier is assigned to each passenger in the order in which they enter the screen; Based on the image data of each passenger, the standing direction and head rotation angle of each passenger are calculated, and the quantitative indicators of head rotation, as well as the gaze direction and the landing point position on the screen are analyzed; The gaze trajectory, dwell time, standing direction, and head rotation angle of each passenger in each area of ​​the screen are sorted in chronological order to form independent behavioral data samples. Key features are extracted from these samples and converted into feature vectors. The feature vector of each passenger is input in parallel into a pre-built and trained passenger attention information analysis model, and distributed computing resources are used to perform calculations simultaneously to output the attention information of each passenger.

5. The rail transit passenger energy information service method under multi-source data fusion according to claim 4 is characterized in that: The following steps are included after outputting the attention information of each passenger: According to each passenger's attention information, the energy consumption data of the target line in the current period is extracted independently from the energy consumption database. At the same time, according to each passenger's behavior model data, a clustering algorithm is used to divide the corresponding passenger into consumer groups with similar behavior patterns, and the historical choice preferences of the consumer groups are found; The target route energy consumption data and consumer group historical selection preferences corresponding to each passenger are respectively input into the pre-trained personalized energy-saving suggestion generation model to generate independent personalized energy-saving suggestions for each passenger.

6. The rail transit passenger energy information service method under multi-source data fusion according to claim 5 is characterized in that: It also includes the steps after generating individual personalized energy saving recommendations for each passenger, as follows: Divide the screen into corresponding grid areas according to the number of passengers, and set a different main color tone for each passenger's split-screen area; The target routes and station information of each passenger's interest are displayed in the center of the split-screen area, and dynamic effects are used to guide the corresponding passengers to pay attention to the recommended energy-saving information.

7. The rail transit passenger energy information service method under multi-source data fusion according to claim 6 is characterized in that: The method further includes steps after displaying the target route and station information that each passenger is concerned about in the center of the split-screen area and using dynamic effects to guide the corresponding passengers to pay attention to the recommended energy-saving information, which are as follows: The directional microphone array is activated to capture the voices of passengers and their companions, and the language type of the voices is quickly analyzed through the speech recognition engine and natural language processing technology; According to the analyzed language type, the system automatically matches the voice package of the corresponding language; The screen location of the energy-saving information recommended for the corresponding passengers will be announced in the order of passenger identification, and the volume will be automatically adjusted according to the ambient noise of the station to ensure that it is higher than the preset decibel level of background noise.

8. A rail transit passenger energy information service system based on multi-source data fusion, characterized in that: It includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the program can be loaded and executed by the processor to implement a rail transit passenger energy information service method under multi-source data fusion as described in any one of claims 1 to 7.

Images