Rail transit passenger energy information service method and system under multi-source data fusion

Abstract

The invention relates to a rail transit passenger energy information service method and system under multi-source data fusion, and solves the problems that in information display and transmission, the form is single, an effective guiding mechanism is lacked, the attention of passengers is difficult to attract, and the transmission effect of energy-saving information is poor. The method comprises the following steps: inputting extracted energy consumption data of a target line in a current time period and historical selection preference of a corresponding consumption group into a pre-trained personalized energy-saving suggestion generation model, and outputting a personalized energy-saving suggestion; and on a rail transit external display, displaying target line and station information concerned by the passengers in the center of a screen, and guiding the passengers to concern the recommended energy-saving information by applying a dynamic effect. The method has the advantages that passenger attention information is accurately captured, adaptive energy-saving suggestions are generated, passenger interaction is enhanced, operation energy efficiency is optimized, and the service and energy utilization level is improved.

Description

Technical Field

[0001] The present invention relates to the field of rail transit operation management, and in particular to a rail transit passenger energy information service method and system under multi-source data fusion. Background Art

[0002] In the field of rail transit, as energy issues are increasingly concerned, how to provide energy information services to passengers and guide them to participate in energy-saving actions has become an important issue in improving the efficiency and sustainability of rail transit operations. Effective energy information services can not only help passengers better plan their trips and reduce energy consumption, but also improve the overall operational efficiency of the rail transit system.

[0003] At present, some rail transit systems have begun to try to provide energy information to passengers. Common methods include displaying overall line information on station displays and simply informing passengers of some basic energy-saving tips through broadcasts.

[0004] There are many drawbacks in the existing rail transit energy information service technology. In terms of information acquisition, it is impossible to accurately capture the specific information of passengers, such as target lines and stations, and it is difficult to meet the personalized needs of passengers. In terms of energy-saving suggestions, there is a lack of in-depth analysis of passenger behavior patterns, and it is impossible to provide targeted energy-saving solutions based on the characteristics of different passengers, resulting in low practicality and effectiveness of energy-saving suggestions. In terms of information display and communication, the form is single and there is a lack of effective guidance mechanism, which makes it difficult to attract the attention of passengers, resulting in poor communication of energy-saving information. Summary of the invention

[0005] In order to accurately capture information of passenger concern, generate adaptive energy-saving suggestions, enhance passenger interaction, optimize operational energy efficiency, and improve service and energy utilization levels, the present application provides a rail transit passenger energy information service method and system under multi-source data fusion.

[0006] In the first aspect, the present application provides a rail transit passenger energy information service method under multi-source data fusion, which adopts the following technical solution: A rail transit passenger energy information service method based on multi-source data fusion, comprising: Analyze whether the passenger enters the preset distance range of the rail transit external display; If not, the rail transit external display will only show the overall route; If yes, monitor and obtain the passenger's behavior data, and identify and obtain the passenger's attention information based on the passenger's behavior data, the attention information including the target route and station; According to the passengers' attention information, the energy consumption data of the target line in the current period is extracted from the energy consumption database; Obtain passenger behavior model data, and use clustering algorithms to divide the corresponding passenger behavior patterns into consumer groups with similar patterns, and search for the historical selection preferences of the corresponding consumer groups in the passenger behavior model library, where the passenger behavior model data includes travel time data, travel route data, consumption data, and historical energy-saving suggestion adoption data; The extracted energy consumption data of the target route during the current period and the historical selection preferences of the corresponding consumer groups are input into the pre-trained personalized energy-saving suggestion generation model to output personalized energy-saving suggestions, where the personalized energy-saving suggestions include recommended energy-saving carriages, transfer routes and key stations; On the rail transit external display, the target lines and station information that passengers are concerned about are displayed in the center of the screen, and dynamic effects are used to guide passengers to pay attention to recommended energy-saving information.

[0007] By adopting the above technical solutions, content can be displayed in a targeted manner based on whether the passenger is close to the display; personalized attention information can be provided based on passenger behavior data; personalized energy-saving suggestions can be generated through energy consumption data and historical selection preferences, and passengers can be provided with guidance on energy-saving carriages, transfer routes, etc.; dynamic effects can be used to guide attention to energy-saving information, enhance passengers' energy-saving awareness, promote the rational use of rail transit energy, and optimize passengers' travel experience.

[0008] Optionally, monitoring and obtaining passenger behavior data, and identifying and obtaining passenger concern information based on the passenger behavior data includes: Starting the camera device to capture image data of passengers within a preset distance range; Based on passenger image data, a human posture recognition algorithm is used to analyze the passenger's body posture in real time, locate the key points of the human body, and calculate the passenger's standing direction and head rotation angle; Analyze and calculate whether the passenger's standing orientation obtained is within a pre-set effective judgment area for the standing orientation; If not, it is determined that the passenger is not interested in the display information; If yes, then a head rotation quantitative analysis is performed on the passenger according to the head rotation quantitative analysis method, and various quantitative indicators of the passenger's head rotation are obtained; Based on the quantitative indicators of standing orientation and head rotation, and according to the preset weighted algorithm, a comprehensive score is calculated; If and only if the comprehensive score exceeds a preset threshold, it is determined that the corresponding passenger is interested in the display information; When it is determined that the corresponding passenger is interested in the information on the display, the camera is used to capture the passenger's facial feature points in combination with the gaze estimation algorithm based on facial key points, calculate the passenger's gaze direction and the location of the gaze on the screen, determine the specific information area where the gaze is located, and record the area type and precise coordinates in detail; Use time series analysis methods to continuously monitor and obtain the trajectory and duration of passengers' gazes in the corresponding areas; The gaze trajectory and duration in each area of ​​the screen, as well as the passenger's standing direction and head rotation angle are sorted in chronological order to form a series of behavior data samples, each of which contains various behavior feature values ​​at different times; Key features are extracted from the behavioral data samples and converted into feature vectors. The converted feature vectors are input into a pre-built and trained passenger attention information analysis model to output the passenger attention information.

[0009] By adopting the above technical solution, the camera device and gesture recognition algorithm can accurately judge the passenger's interest in the display information, which can effectively reduce invalid analysis. Combined with the sight estimation algorithm and time series analysis, the gaze trajectory and dwell time are recorded in detail to form a behavioral data sample. It is converted into a feature vector input analysis model, which can output accurate passenger attention information, lay the foundation for the subsequent provision of personalized services, and greatly improve the pertinence and effectiveness of services.

[0010] Optionally, the training and construction of the passenger attention information analysis model includes: Obtain the labeled behavior data samples and the passenger attention information corresponding to each sample; Divide the labeled samples into training set and validation set; The feature data and the corresponding attention information labels in the training set are input into the Naive Bayes classification model. The Naive Bayes model starts to calculate the conditional probability of each feature under different attention information categories, as well as the prior probability of each attention information category. Based on the conditional probability of each feature under different categories of attention information and the prior probability of each category of attention information, the naive Bayes classification model completes the preliminary construction of the probabilistic relationship between the feature and the attention information; Use the validation set to evaluate the model in training and calculate the accuracy; If the accuracy rate exceeds the preset accuracy rate for consecutive preset times, it is determined that the passenger attention information analysis model has completed training; Otherwise, the smoothing parameters of the naive Bayes model are adjusted based on the accuracy stage, and the passenger attention information analysis model is retrained.

[0011] By adopting the above technical solution, the model performance can be effectively evaluated by dividing the training set and the validation set. The naive Bayes classification model is used to construct the probability relationship between features and information of interest, which has clear logic and efficient calculation. The validation set is used to evaluate and calculate the accuracy to ensure the model effect. When the accuracy rate does not meet the standard, the smoothing parameters are adjusted and retrained to improve the generalization ability and accuracy of the model, so as to accurately output the information of passengers' interest and provide strong support for rail transit information services.

[0012] Optionally, the method further includes a step after starting the camera device to capture and obtain image data of passengers within a preset distance range, which is specifically as follows: Analyzing whether there are multiple passengers based on the passenger image data; If not, maintain the original settings and continue with the subsequent steps; If yes, a unique identifier is assigned to each passenger in the order in which they enter the screen; Based on the image data of each passenger, the standing direction and head rotation angle of each passenger are calculated, and the quantitative indicators of head rotation, as well as the gaze direction and the landing point position on the screen are analyzed; The gaze trajectory, dwell time, standing direction, and head rotation angle of each passenger in each area of ​​the screen are sorted in chronological order to form independent behavioral data samples. Key features are extracted from these samples and converted into feature vectors. The feature vector of each passenger is input in parallel into a pre-built and trained passenger attention information analysis model, and distributed computing resources are used to perform calculations simultaneously to output the attention information of each passenger.

[0013] By adopting the above technical solution, when multiple passengers are detected, they are assigned unique identifiers to accurately distinguish individuals. The behavior data of each passenger is calculated and sorted separately, and the key features are extracted and converted into vectors to achieve detailed analysis of the behavior of multiple passengers. The feature vectors are input into the analysis model in parallel, and the distributed computing resources are used for simultaneous calculation, which greatly improves the processing efficiency, can quickly and accurately output the information of concern to each passenger, and improve the comprehensiveness and timeliness of rail transit information services.

[0014] Optionally, the method further includes steps after outputting the concerned information of each passenger, which are as follows: According to each passenger's attention information, the energy consumption data of the target line in the current period is extracted independently from the energy consumption database. At the same time, according to each passenger's behavior model data, a clustering algorithm is used to divide the corresponding passenger into consumer groups with similar behavior patterns, and the historical choice preferences of the consumer groups are found; The target route energy consumption data and consumer group historical selection preferences corresponding to each passenger are respectively input into the pre-trained personalized energy-saving suggestion generation model to generate independent personalized energy-saving suggestions for each passenger.

[0015] By adopting the above technical solutions, energy consumption data can be extracted according to the information that each passenger is concerned about, and consumer groups can be divided into behavioral models and historical preferences can be found to deeply explore individual characteristics. These are input into the personalized energy-saving suggestion generation model to generate energy-saving suggestions for each passenger independently. Not only can it fully meet the personalized needs of different passengers, but it can also accurately guide passengers to save energy during travel, helping rail transit to achieve efficient energy saving and high-quality services.

[0016] Optionally, further steps are included after generating individual personalized energy saving recommendations for each passenger, as follows: Divide the screen into corresponding grid areas according to the number of passengers, and set a different main color tone for each passenger's split-screen area; The target routes and station information of each passenger's interest are displayed in the center of the split-screen area, and dynamic effects are used to guide the corresponding passengers to pay attention to the recommended energy-saving information.

[0017] By adopting the above technical solution, the screen grid area is divided according to the number of passengers, and different main colors are used to distinguish them, so that passengers can quickly locate exclusive information. Placing the target line and station information in the center of the split screen, combined with dynamic effects to guide attention to energy-saving suggestions, can enhance the pertinence and attractiveness of information display, provide passengers with a personalized and immersive interactive experience, help passengers efficiently obtain the required content, and improve the quality and efficiency of rail transit information services.

[0018] Optionally, the method further includes displaying the target route and station information that each passenger is concerned about in the center of the split-screen area, and using dynamic effects to guide the corresponding passengers to pay attention to the recommended energy-saving information, which is specifically as follows: The directional microphone array is activated to capture the voices of passengers and their companions, and the language type of the voices is quickly analyzed through the speech recognition engine and natural language processing technology; According to the analyzed language type, the system automatically matches the voice package of the corresponding language; The screen location of the energy-saving information recommended for the corresponding passengers will be announced in the order of passenger identification, and the volume will be automatically adjusted according to the ambient noise of the station to ensure that it is higher than the preset decibel level of background noise.

[0019] By adopting the above technical solutions, the directional microphone array captures the voice, analyzes the language type through voice recognition and natural language processing, and automatically matches the voice package, which can solve the communication barriers of passengers with different languages. The energy-saving information screen position is broadcast in the order of passenger identification, and the volume is automatically adjusted according to the ambient noise to ensure that every passenger can clearly obtain the information, improve the inclusiveness and humanity of the service, and make the rail transit information service more intimate and comprehensive.

[0020] In the second aspect, the present application provides a rail transit passenger energy information service system under multi-source data fusion, which adopts the following technical solutions: A rail transit passenger energy information service system under multi-source data fusion, comprising a memory, a processor and a program stored in the memory and executable on the processor, wherein the program can be loaded and executed by the processor to implement the rail transit passenger energy information service method under multi-source data fusion as described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the overall process of a rail transit passenger energy information service method under multi-source data fusion in an embodiment of the present application.

[0022] Figure 2 It is a flowchart of another embodiment of the present application for monitoring and obtaining passenger behavior data, and identifying and obtaining passenger's concerned information based on the passenger's behavior data.

[0023] Figure 3 It is a flowchart of the training and construction of a passenger attention information analysis model according to another embodiment of the present application.

[0024] Figure 4 It is a flowchart diagram of the steps after starting the camera device to capture the image data of passengers falling within a preset distance range in another embodiment of the present application. DETAILED DESCRIPTION

[0025] The present application is further described in detail below in conjunction with the accompanying drawings.

[0026] Reference Figure 1 , is a rail transit passenger energy information service method under multi-source data fusion disclosed in this application, comprising: Step S100, analyzing whether the passenger enters the preset distance range of the rail transit external display. If not, execute step S200; if yes, execute step S300.

[0027] The preset distance range is a specific distance area around the external display of rail transit, for example, a circular area with a radius of 2 meters with the display as the center. The system determines whether the passenger enters this range to determine the subsequent display information.

[0028] The method of obtaining the information is as follows: ultrasonic sensors are installed around the display. This sensor can emit ultrasonic waves, receive the reflected ultrasonic waves, and calculate the distance between the passenger and the display based on the round-trip time of the ultrasonic waves.

[0029] The general process is as follows: the sensor continuously emits ultrasonic waves. When a passenger enters the detection range of the sensor, the reflected ultrasonic waves are received by the sensor, and the sensor transmits the distance data to the system. The system compares the received distance with the preset distance. If the distance is less than or equal to the preset distance, it determines that the passenger has entered; otherwise, it is determined that the passenger has not entered. For example, at a busy transfer station, when passenger B approaches the display, the sensor immediately senses and calculates the distance. The system determines whether passenger B has entered the preset range based on the calculation results, and decides whether to display personalized information.

[0030] Step S200: The rail transit external display only displays the entire route.

[0031] The overall route is a complete information display of all stations and line directions included in the rail transit, covering each operating line, station name and the connection relationship between stations, helping passengers understand the overall picture of rail transit.

[0032] The general process is as follows: When the system determines that no passengers have entered the preset distance range, the display's microprocessor will trigger a read instruction to obtain the overall route data from the database through the communication line. Subsequently, the data is decoded and parsed, and the overall route map is presented on the display according to the established display format and layout. For example, during non-peak hours on weekdays, there are fewer passengers waiting on the platform and no one approaches the display. At this time, the display continues to display the overall route, which is convenient for passengers who occasionally pass by and want to understand the route overview.

[0033] Step S300, monitoring and acquiring passenger behavior data, and identifying and acquiring passenger concern information based on the passenger behavior data, the concern information including target routes and stations.

[0034] Among them, behavioral data refers to data that can reflect the behavioral characteristics of passengers, such as body posture, head rotation, gaze focus, etc., which can be used to analyze passenger interests.

[0035] Focus information: Content that passengers are interested in on rail transit, such as target routes and stations, can help passengers plan their trips.

[0036] The acquisition method is as follows: use the cameras around the display to capture the passenger image, use the sensor to collect body posture and head rotation data, and use the eye tracking device to obtain the gaze point information.

[0037] The general process is as follows: the camera continuously shoots, and the sensor collects data in real time. The system first performs human posture recognition on the image, analyzes body posture and head rotation; then determines the gaze point through line of sight tracking. Based on these data, combined with the preset algorithm, it is determined which information the passenger is interested in, and thus the information of interest is identified. For example, at the transfer station, passenger Xiao Li approaches the display, and the camera captures him leaning forward, turning his head, and staring at a specific area of ​​the screen for a long time. After analysis, the system determines that he is paying attention to a certain transfer line and station.

[0038] Step S400: extracting energy consumption data of the target line in the current period from the energy consumption database according to the passenger's concern information.

[0039] Target lines: rail transit lines where passengers are concerned about and want to know energy consumption information, such as Metro Line 1.

[0040] Energy consumption data: records the energy consumption of the target line in the current period, such as power consumption, traction energy consumption, etc.

[0041] The acquisition method is as follows: the energy consumption data is stored in the energy consumption database, which collects the energy consumption data of each line through the data acquisition system and is centrally managed by the server. The system establishes a connection with the server through the database interface to obtain the energy consumption data.

[0042] The general process is as follows: After the system identifies the passenger's concern information, it determines the target line. Then, it searches the database for energy consumption data for the current period according to the line identification. After finding the data, it extracts it in the prescribed format and transmits it to the subsequent processing module. For example, if passenger Xiao Zhang is concerned about Line 2, the system will extract the energy consumption data of Line 2 for the current period from the database after confirmation, providing a basis for the subsequent generation of personalized energy-saving suggestions.

[0043] Step S500, obtain passenger behavior model data, and use a clustering algorithm to divide the corresponding passenger behavior patterns into consumer groups with similar patterns, and search the passenger behavior model library for the historical selection preferences of the corresponding consumer groups, wherein the passenger behavior model data includes travel time data, travel route data, consumption data, and historical energy-saving suggestion adoption data.

[0044] Passenger behavior model data: covers multi-dimensional data such as passenger travel time, route, consumption and adoption of historical energy-saving suggestions, which is used to characterize passenger travel behavior characteristics.

[0045] Consumer groups: Based on clustering algorithms, passengers with similar behavior patterns are divided into groups to facilitate analysis of group common behaviors and preferences.

[0046] Historical choice preferences: the tendencies that consumer groups have shown in the past in terms of travel choices, adoption of energy-saving measures, etc.

[0047] The acquisition method is as follows: the passenger behavior model data is stored in the passenger behavior model library, which is collected and sorted by the rail transit operation system for a long time. The system obtains data through the model library interface.

[0048] The general process is as follows: the system obtains the behavior model data of a certain passenger from the passenger behavior model library, and uses the clustering algorithm to classify the passenger and other passengers according to the similarity of behavior, and divides them into consumer groups. Then, the historical selection preference data of the consumer group is searched and extracted in the model library. For example, passenger Xiao Wang often commutes in the morning and chooses energy-saving carriages many times. After clustering, the system divides him into a certain consumer group, and then extracts the historical preferences of the group in terms of energy-saving carriage selection, transfer routes, etc., to help generate personalized suggestions.

[0049] Step S600, input the extracted energy consumption data of the target route during the current period and the historical selection preferences of the corresponding consumer group into a pre-trained personalized energy-saving suggestion generation model, and output personalized energy-saving suggestions, wherein the personalized energy-saving suggestions include recommended energy-saving carriages, transfer routes and key stations.

[0050] Personalized energy-saving suggestions: Based on the energy consumption of the routes that passengers are concerned about and the historical preferences of their consumer groups, customized energy-saving travel suggestions are provided to passengers, including recommended energy-saving carriages, transfer routes and key stations.

[0051] Personalized energy-saving suggestion generation model: The artificial intelligence model, which has been trained with a large amount of data, can analyze the input data and output targeted energy-saving suggestions.

[0052] The acquisition method is as follows: the personalized energy-saving suggestion generation model is pre-trained and stored in the server, and the system uses it by calling the model interface. The energy consumption data is obtained from the energy consumption database, and the historical selection preferences are obtained from the passenger behavior model library.

[0053] The general process is as follows: the system extracts the energy consumption data of the target line during the current period from the database, as well as the historical selection preferences of the corresponding consumer group, and inputs them into the personalized energy-saving suggestion generation model interface. The model's internal algorithm analyzes data associations and outputs personalized energy-saving suggestions according to the trained and learned patterns. For example, for passenger Xiao Zhao who is concerned about Line 3, the model combines the energy consumption of Line 3 and the preferences of his consumer group to output recommendations such as the energy-saving carriage number for Line 3 and the energy-saving transfer route.

[0054] Step S700: Display the target line and station information that passengers are concerned about in the center of the screen on the rail transit external display, and use dynamic effects to guide passengers to pay attention to recommended energy-saving information.

[0055] Dynamic effects: Use animation, flashing, movement and other visual effects to attract passengers' attention and highlight the display content.

[0056] Center of the screen: The visual core of the display, where important information is placed for passengers to quickly access.

[0057] The acquisition method is as follows: the displayed target route, station information and recommended energy-saving information are obtained from the passenger attention information recognition result and personalized energy-saving suggestion generation module respectively. The dynamic effect is achieved through the pre-written display control program.

[0058] The general process is as follows: After the system obtains the target line, station information and personalized energy-saving suggestions that the passenger is concerned about, the target line and station information are presented in a striking manner in the center of the screen, such as enlarging the font and using bright colors. At the same time, the display control program is called to add dynamic effects in the energy-saving information area, such as flashing prompts and arrow guidance. For example, at a certain station, passenger Xiao Liu is concerned about a station on Line 4. The system displays the relevant information in the center of the screen and uses a flashing effect to guide him to pay attention to the recommended energy-saving carriages and transfer path information on Line 4.

[0059] Reference Figure 2 , monitor and obtain the passenger's behavior data, and identify and obtain the passenger's attention information based on the passenger's behavior data, including: Step S310: Start the camera device to capture image data of passengers within a preset distance range.

[0060] Camera device: A device installed near the external display of rail transit and used to capture passenger images, such as a high-definition camera.

[0061] Preset distance range: A specific area defined with the display as the center. Only passengers within this range will be photographed, such as a circular area with a radius of 2 meters.

[0062] Passenger image data: image file captured by the camera containing information such as the passenger's appearance and posture.

[0063] The acquisition method is as follows: when the system detects that a passenger enters the preset distance range, it will send a start command to the camera device, and the camera device will start working and transmit the captured image to the system in the form of a digital signal.

[0064] The general process is as follows: once the distance monitoring module in the system recognizes that a passenger has entered the preset range, the camera device is immediately triggered to start. The camera device continuously captures passenger images at a set frame rate, such as 30 frames per second, and transmits these image data to the data processing unit in real time for subsequent analysis. For example, at a busy transfer station, when passenger Xiao Zhang enters within 2 meters of the display, the camera device automatically starts and captures a series of images of Xiao Zhang, providing basic data for subsequent analysis of his behavior.

[0065] Step S320, based on the passenger image data, a human posture recognition algorithm is used to analyze the passenger's body posture in real time, locate various key points of the human body, and calculate the passenger's standing direction and head rotation angle.

[0066] Human posture recognition algorithm: an artificial intelligence algorithm that can identify the position and posture of various parts of the human body from an image, such as the OpenPose algorithm.

[0067] Body posture: The passenger’s standing, walking, sitting, and other postures in the image.

[0068] Key points: Representative parts of the human body, such as shoulders, elbows, wrists, hips, knees, ankles, etc., used to determine body posture.

[0069] Standing direction: The direction in which the passenger's body points when standing.

[0070] Head rotation angle: The angle at which the passenger's head rotates relative to the central axis of the body, used to determine the direction of attention.

[0071] The acquisition method is as follows: the algorithm is pre-stored in the algorithm library of the system and called through the program; the passenger image data is acquired from the shooting in step S310.

[0072] The general process is as follows: the system calls the human posture recognition algorithm and inputs the passenger image data obtained in step S310. The algorithm analyzes the image, identifies the key points of the human body, and determines the body posture by the relative position of the key points. Based on the body posture, mathematical methods such as trigonometric functions are used to calculate the standing direction and head rotation angle. For example, at the station, the image of passenger Xiao Wang is input into the algorithm, and the algorithm identifies his key points such as shoulders and hips, and calculates that Xiao Wang's standing direction is 45° to the left of the display, and his head is turned 30° toward the display.

[0073] Step S330, analyzing and calculating whether the passenger standing direction is within the effective judgment area of ​​the preset standing direction. If not, executing step S340; if yes, executing step S350.

[0074] Standing direction: the direction in front of the passenger's body when standing, with the display as the center, which can be expressed by angle. Effective judgment area: a specific angle range set around the display, indicating that the passenger may pay attention to the display information within this range, such as 30° to the left and right of the display directly in front of the display.

[0075] The acquisition method is as follows: the standing orientation data is calculated from step S320, and the effective judgment area range is pre-set when the system is initialized and stored in the configuration file, and the system obtains it by reading the configuration file.

[0076] The general process is as follows: the system reads the effective judgment area range and compares the passenger standing direction angle calculated in step S320 with it. If the standing direction angle is within the effective judgment area, it means that the passenger is likely to pay attention to the display information; otherwise, it is unlikely to pay attention. For example, at the platform, the passenger Xiao Li's standing direction angle is 20° to the right of the front of the display. After comparison, the system finds that the angle is within the effective judgment area of ​​30° on the left and right, and determines that his standing direction meets the conditions for further analysis.

[0077] Step S340, determining that the passenger is not interested in the display information.

[0078] Step S350, performing a quantitative analysis on the passenger's head rotation according to the head rotation quantitative analysis method, and obtaining various quantitative indicators of the passenger's head rotation.

[0079] Quantitative analysis of head rotation: It is a technical means to accurately evaluate the head rotation of passengers based on mathematical models and algorithms. It converts the head rotation behavior into specific values ​​by analyzing multi-dimensional information such as the amplitude, frequency, and start and end angles of the head rotation, thereby objectively and accurately measuring the degree of attention paid by passengers to the information on the display. For example, the quantitative model built based on trigonometric functions and time series analysis can extract key features from dynamic head rotation data.

[0080] Quantitative indicators: a series of specific values ​​obtained through quantitative analysis of head rotation, which are used to accurately describe the characteristics of head rotation. Common quantitative indicators include rotation angle, which intuitively reflects the magnitude of head rotation; rotation speed, that is, the change in the angle of head rotation per unit time, which reflects the speed of head rotation; and rotation acceleration, which is used to measure the rate of change of rotation speed and can reveal the dynamic trend of head rotation. These quantitative indicators are interrelated and together constitute a comprehensive description of head rotation behavior.

[0081] Acquisition method: The head rotation quantitative analysis method is a professional algorithm built into the system and is stored in a special algorithm library. When the system needs to perform head rotation quantitative analysis, the algorithm is called through a specific program interface. The head rotation angle data comes directly from step S320. In step S320, the passenger image is analyzed through the human posture recognition algorithm, and the passenger's head rotation angle has been accurately calculated.

[0082] The general process is as follows: When the system enters step S350, it first obtains the passenger's head rotation angle data from step S320. These data are usually a series of angle values ​​that change over time, recording the passenger's head position at different times. Then, the system calls the head rotation quantitative analysis method pre-stored in the algorithm library and inputs these angle data into the algorithm. The algorithm performs in-depth processing on the input angle data based on the preset mathematical model and logic. For example, by calculating the difference in angles at adjacent moments and dividing it by the time interval, the head rotation speed is obtained; then the change in rotation speed is analyzed to calculate the rotation acceleration. After this series of calculations and analyses, various quantitative indicators that can fully reflect the passenger's head rotation characteristics are finally obtained. These quantitative indicators will serve as an important basis for comprehensive judgment and analysis in subsequent steps.

[0083] Step S360, calculating a comprehensive score based on the various quantitative indicators of standing orientation and head rotation and according to a preset weighting algorithm.

[0084] Standing direction: refers to the direction in front of the passenger's body when standing. It is one of the important factors to determine whether the passenger pays attention to the information on the display.

[0085] Quantitative indicators of head rotation: Data obtained through quantitative analysis of passengers' head rotation, such as rotation angle, speed, etc., can reflect the passengers' attention level.

[0086] Weighted algorithm: A calculation method that assigns weights to different factors based on their impact on the result, used to combine multiple factors to arrive at the final result.

[0087] Comprehensive score: The standing orientation and head rotation quantitative indicators are combined into a score calculated using a weighted algorithm to determine the passenger's interest in the information on the display.

[0088] The acquisition method is as follows: the standing orientation and head rotation quantitative indicators are obtained from the previous steps (S320, S350). The weighting algorithm is pre-set by the system and stored in the program, and the system calls the algorithm for calculation.

[0089] The general process is as follows: the system first extracts the passenger standing orientation data and head rotation quantitative indicators from the previous steps. Then, according to the preset weighted algorithm, different weights are assigned to the standing orientation and each head rotation quantitative indicator. For example, it is believed that the standing orientation has a greater impact on the judgment of interest, and a weight of 0.6 is assigned, the head rotation speed weight is 0.2, and the rotation angle weight is 0.2. Then, multiply each data by the corresponding weight and add them together to obtain a comprehensive score. For example, passenger Xiao Sun's standing orientation meets the attention conditions and gets 80 points, the head rotation speed corresponds to a score of 70 points, and the rotation angle corresponds to a score of 85 points. After weighted calculation: 80×0.6+70×0.2+85×0.2=79 points, and the comprehensive score is 79 points.

[0090] Step S370: If and only if the comprehensive score exceeds a preset threshold, it is determined that the corresponding passenger is interested in the display information.

[0091] Comprehensive score: In step S360, a value is obtained by comprehensively calculating data such as standing orientation and head rotation quantitative indicators through a weighted algorithm to measure the passenger's attention to the display information.

[0092] Pre-set threshold: A fixed value set by the system in advance as a criterion for judging whether a passenger is interested in the information on the display. When the comprehensive score is compared with it, if it is higher than it, it is judged to be interested, otherwise it is not interested.

[0093] Step S380, when it is determined that the corresponding passenger is interested in the display information, the facial feature points of the passenger are captured by the camera in combination with the line of sight estimation algorithm based on facial key points, the passenger's gaze direction and the landing point position on the screen are calculated, the specific information area where the landing point is located is determined, and the area type and precise coordinates are recorded in detail.

[0094] Gaze estimation algorithm based on facial key points: an algorithm that estimates the passenger's gaze direction by identifying key facial feature points, such as the corners of the eyes, pupils, etc.

[0095] Facial landmarks: iconic locations on the face, such as eyes, nose, mouth, etc., which are used to determine facial orientation and line of sight.

[0096] Gaze direction: The direction in which a passenger's eyes are looking can reflect what they are paying attention to.

[0097] Landing point: The specific location where the gaze is projected on the screen, expressed by coordinates.

[0098] Information area: Different functional areas on the screen, such as route map area, site introduction area, etc.

[0099] The acquisition method is as follows: the sight line estimation algorithm based on facial key points is stored in the algorithm library of the system and called by the program. The passenger facial image data captured by the camera comes from step S310, and the facial feature points are extracted from the facial image by a special facial recognition algorithm.

[0100] The general process is as follows: After the system determines that the passenger is interested in the information on the display, it calls the line of sight estimation algorithm based on facial key points. The algorithm extracts facial feature points from the facial image captured by the camera, and uses these feature points, combined with geometric models and mathematical calculation methods, to estimate the direction of the passenger's gaze. According to the gaze direction, combined with the spatial position and imaging principle of the display, the position of the gaze on the screen is calculated. Then, based on the information area pre-divided on the screen, the specific area type where the landing point is located is determined, and the area type and the precise coordinates of the landing point are recorded in detail. For example, passenger Xiao Zhang is interested in the information on the display. The system extracts his facial feature points through an algorithm, calculates the gaze direction, determines that his gaze falls on the site introduction area on the screen, and records the area type and the landing point coordinates (x, y).

[0101] Step S390, using a time series analysis method, continuously monitors and obtains the trajectory and duration of passengers' gazes in the corresponding area.

[0102] Time series analysis method: a method of analyzing data based on time sequence, which can reveal the laws and trends of data changes over time.

[0103] Gaze trajectory: The path formed by the passenger's gaze moving across different areas of the screen, reflecting the switching of their attention content.

[0104] Dwell time: the length of time the eyes remain fixed on a certain area of ​​the screen, reflecting the degree of attention paid to the content in that area.

[0105] The general process is as follows: the system continuously obtains the passenger's gaze location information from step S380, and attaches a timestamp to each acquired location data. Based on these timestamped location information, the system calls the time series analysis method to connect the location of adjacent moments in chronological order, thereby drawing the gaze trajectory. At the same time, by calculating the time interval between changes in adjacent location positions, the duration of gaze stay in each area is determined. For example, when passenger Xiao Li is looking at the display, the system obtains his gaze location every 0.1 seconds, and connects these location points over time to obtain his gaze trajectory. By calculating the time interval between each position change, it is known that he stayed in the station introduction area for 3 seconds and in the transfer information area for 2 seconds.

[0106] Step S3A0, sorting the gaze trajectory and dwell time in each area of ​​the screen, as well as the passenger's standing direction and head rotation angle in chronological order to form a series of behavior data samples, each sample containing various behavior feature values ​​at different times.

[0107] Dwell trajectory: The path formed by the passenger's eyes moving across different areas of the screen, showing the process of shifting focus.

[0108] Dwell time: The length of time that eyes remain fixed on a specific area of ​​the screen, reflecting the degree of attention.

[0109] Standing direction: The direction in which the passenger's body points when standing, used to determine their initial attention tendency.

[0110] Head rotation angle: The angle at which the passenger's head rotates relative to the central axis of the body, which assists in analyzing changes in attention.

[0111] Behavioral data sample: A data set formed by integrating the above-mentioned behavioral feature values ​​and arranging them in chronological order for subsequent analysis of passenger behavior patterns.

[0112] The general process is as follows: the system first extracts the trajectory of the passenger's gaze in each area of ​​the screen, the dwell time, as well as the standing direction and head rotation angle data from the corresponding steps. Then, in chronological order, all kinds of data at the same time are summarized to form behavioral data samples. Each sample is like a "behavioral snapshot", recording the various behavioral characteristics of the passenger at that moment. For example, in the time period of 10:00:00-10:00:05, the standing direction of passenger Xiao Wang was 15° to the left of the front of the display, the head rotation angle was 20° to the right, the eyes stayed in the station query area for 3 seconds, and the trajectory showed that it moved from the upper left corner of the area to the lower right corner. The system organizes these data into a behavioral data sample in chronological order.

[0113] Step S3B0, extract key features from the behavior data sample and convert them into feature vectors, input the converted feature vectors into a pre-built and trained passenger attention information analysis model, and output the passenger attention information.

[0114] Key features: behavioral features extracted from behavioral data samples that are of great value in determining the information that passengers pay attention to, such as areas where the gaze stays for a long time, specific changes in standing orientation, etc.

[0115] Feature vector: An ordered vector that represents key features numerically to facilitate computer processing and analysis.

[0116] Passenger concern information analysis model: The artificial intelligence model, which has been trained with a large amount of data, can analyze and output passengers' concern information based on the input feature vector.

[0117] The general process is as follows: the system obtains the behavior data sample from step S3A0, and selects key features from it, such as the long stay of the eyes in a certain route map area. These key features are then converted into feature vectors, such as digitizing the information such as the stay time and regional coordinates to form a vector. Finally, the feature vector is input into the passenger attention information analysis model, and the model outputs the passenger's attention information, such as the target line and station through internal algorithm analysis. For example, the behavior data sample of passenger Xiao Zhang shows that his eyes stay for a long time on the route map of Line 2 and a certain station location. The system extracts key features and converts them into vectors for input into the model, and concludes that Xiao Zhang pays attention to Line 2 and the station.

[0118] Reference Figure 3 ,The training and construction of the passenger attention information analysis model includes: Step S3B1, obtaining the labeled behavior data samples and the passenger attention information corresponding to each sample.

[0119] Completed labeled behavior data samples: Based on the original behavior data samples, manually or through a specific algorithm, mark the data set of passenger attention information corresponding to the sample. For example, mark the concerned routes and stations corresponding to the areas where passengers' gazes stay for a long time in a certain sample.

[0120] Passenger focus information: The content that passengers actually focus on when viewing rail transit displays, such as specific lines, station names, transfer information, etc.

[0121] The acquisition method is as follows: Behavior data samples can be obtained from a large number of past passenger behavior monitoring records, which are stored in the database. The labeling work can be done manually by professionals or with the help of some semi-automatic labeling tools. For example, image recognition auxiliary tools can be used to quickly mark the screen area information corresponding to the gaze point.

[0122] The general process is as follows: First, all relevant behavioral data samples are retrieved from the database. These samples contain information such as the passenger's standing direction, head rotation angle, gaze trajectory and time. Then, professionals are organized or semi-automatic annotation tools are used to determine the passenger's attention information based on the behavioral data in the sample and annotate it. For example, for a behavioral data sample that records a passenger's gaze staying in a specific area on the screen for a long time, the annotator will mark the passenger's attention to the first and last bus times of a certain line based on the content displayed in the area.

[0123] Step S3B2, divide the labeled samples into a training set and a validation set.

[0124] Training set: A part of the data divided from the labeled behavior data samples is used to train the passenger attention information analysis model to allow the model to learn the relationship between features and attention information.

[0125] Validation set: Another part of the data divided from the labeled samples is used to evaluate the performance of the model during training and test the model's ability to adapt to new data.

[0126] Acquisition method: Both the training set and the validation set are derived from the behavioral data samples acquired and labeled in step S3B1. The data is usually divided according to a certain ratio, such as 70% as the training set and 30% as the validation set.

[0127] The general process is as follows: First, determine the division ratio, for example, set the training set to 70% and the validation set to 30%. Then use random sampling or stratified sampling to divide the labeled behavior data samples. Random sampling is to randomly extract a certain amount of data from the sample as the training set, and the rest as the validation set; stratified sampling considers the distribution of different features in the sample, and extracts data from each layer in proportion to ensure that the feature distribution of the training set and the validation set is similar. For example, there are 1,000 labeled behavior data samples, and random sampling is used to randomly extract 700 as the training set, and the remaining 300 as the validation set.

[0128] Step S3B3, input the feature data in the training set and the corresponding attention information labels into the Naive Bayes classification model, and the Naive Bayes model starts to calculate the conditional probability of each feature under different attention information categories, as well as the prior probability of each attention information category.

[0129] Feature data: Representative information extracted from behavioral data samples, such as the passenger’s standing direction, head rotation angle, gaze dwell time and trajectory, etc. This data can help the model identify the patterns of passengers’ attention.

[0130] Information tags of interest: The content that passengers actually pay attention to corresponding to the feature data, such as the routes, stations, transfer information, etc., which are used to guide model learning.

[0131] Naive Bayes classification model: A classification method based on Bayes’ theorem and the conditional independence assumption of features. It predicts the category to which new data belongs by calculating the probability of features appearing in different categories.

[0132] Conditional probability: The probability of a feature appearing under a certain category of information of interest, indicating the degree of association between the feature and the category.

[0133] Prior probability: The probability of a certain information category appearing in the absence of any new information, reflecting the basic distribution of the category in the overall data.

[0134] The general process is as follows: After the feature data and the corresponding attention information labels in the training set are organized into a format acceptable to the model, they are input into the naive Bayes classification model. The model begins to traverse the data in the training set. For each attention information category, the number of times each feature appears in the category is counted, and then the conditional probability of each feature under different attention information categories is calculated. At the same time, the number of times each attention information category appears in the training set is counted, and the prior probability of each attention information category is calculated. For example, in the training set, there are sample data about passengers paying attention to lines A and B. The model will count the number of times the passenger stands facing the left side of the display when paying attention to line A, as well as the total number of times the passenger pays attention to line A in all samples, thereby calculating the corresponding conditional probability and prior probability.

[0135] Step S3B4, based on the conditional probability of each feature under different attention information categories and the prior probability of each attention information category, the naive Bayes classification model completes the preliminary construction of the probabilistic relationship between the feature and the attention information.

[0136] The general process is as follows: After obtaining the conditional probability of each feature under different categories of information of interest and the prior probability of each category of information of interest, the naive Bayes classification model integrates these probabilities according to the Bayesian theorem. For newly input feature data, the model uses the calculated probabilities to calculate the posterior probabilities of each category of information of interest under the feature data. For example, when there is new passenger behavior data, including features such as standing toward the right side of the display and the eyes staying in a certain area of ​​the screen for 5 seconds, the model calculates the posterior probability of the passenger paying attention to line C and station D based on the previously calculated conditional probabilities and prior probabilities, thereby establishing a probabilistic relationship between features and information of interest. Through a large number of such calculations, the model completes the preliminary construction of the probabilistic relationship between features and information of interest, laying the foundation for subsequent predictions and analysis.

[0137] Step S3B5, use the validation set to evaluate the model in training and calculate the accuracy.

[0138] Accuracy: The ratio of the number of samples correctly predicted by the model to the total number of samples, which is an important indicator to measure the performance of the model. In this scenario, it refers to the ratio of the number of samples that the model accurately judges the information that passengers care about to the total number of samples in the validation set.

[0139] Acquisition method: The validation set data used to evaluate the model comes from the validation set divided in step S3B2. After the model completes the preliminary training in step S3B4, the validation set is used to evaluate the model.

[0140] The general process is as follows: the feature data in the validation set is input into the naive Bayes classification model preliminarily trained in step S3B4. The model predicts the passenger attention information corresponding to each sample based on the constructed probability relationship. Then the prediction results are compared with the attention information labels actually corresponding to the samples in the validation set, and the number of samples predicted correctly is counted. Finally, the number of samples predicted correctly is divided by the total number of samples in the validation set to obtain the accuracy of the model on the validation set. For example, the validation set has 100 samples, and the model correctly predicts the passenger attention information of 80 of them, so the accuracy is 80÷100=80%. This accuracy can be used to judge the model's adaptability to new data and prediction accuracy, providing a basis for judging whether the training is completed in the future.

[0141] Step S3B6: If the accuracy rate exceeds the preset accuracy rate for consecutive preset times, it is determined that the passenger attention information analysis model has completed training.

[0142] The general process is as follows: After completing the model evaluation in step S3B5 and obtaining the accuracy, the system compares the accuracy with the preset accuracy. At the same time, the number of consecutive times the preset accuracy is recorded. If the accuracy of the consecutive preset times exceeds the preset accuracy, it is determined that the passenger attention information analysis model has completed training. For example, the preset accuracy is 85%, the consecutive preset number of times is 3 times, the model accuracy is 86% in the first evaluation, 88% in the second evaluation, and 87% in the third evaluation. It exceeds 85% for 3 consecutive times. At this time, the system determines that the model training is completed. If the condition that the accuracy of the consecutive preset times exceeds the preset accuracy is not met, enter step S3B7 to adjust and retrain the model.

[0143] Step S3B7, on the contrary, the smoothing parameters of the naive Bayes model are adjusted based on the stage of accuracy, and the passenger attention information analysis model is retrained.

[0144] Smoothing parameter: In the Naive Bayes model, a parameter used to solve the zero probability problem. When a feature does not appear in a certain category in the training data, the conditional probability of the feature in that category will be 0. The smoothing parameter can avoid this situation and ensure the stability of the model.

[0145] Acquisition method: The passenger attention information analysis model (Naive Bayes model) currently in training is directly obtained from the process of ongoing training. The accuracy data comes from the evaluation results of step S3B5. The smoothing parameters are given default values ​​when the model is initialized and stored in the configuration parameters of the model, which can be read and modified during adjustment.

[0146] The general process is as follows: when it is judged in step S3B6 that the model training is not completed, that is, the accuracy of the preset number of consecutive times does not exceed the preset accuracy, the system will adjust the smoothing parameter according to the current stage of the accuracy according to the pre-set rules. For example, if the accuracy is low, the smoothing parameter is appropriately increased to increase the generalization ability of the model to the data; if the accuracy is close to the preset value, the smoothing parameter is slightly adjusted. After adjustment, the model is retrained using the training set data in step S3B2, and the process from steps S3B3 to S3B6 is repeated again, and the model is continuously optimized until the model reaches the standard for training completion. For example, the preset accuracy is 80%, and the model is evaluated for 3 consecutive times with an accuracy of 75%, 76%, and 77%, respectively, which does not meet the standard. The system determines that the accuracy is low, increases the smoothing parameter from the default 0.1 to 0.3, and then retrains the model with the training set data.

[0147] Reference Figure 4 A rail transit passenger energy information service method under multi-source data fusion also includes steps after starting a camera device to capture and obtain passenger image data falling within a preset distance range, specifically as follows: Step SA00, analyzing whether there are multiple passengers based on the passenger image data. If not, execute step SB00; if yes, execute step SC00.

[0148] Passenger image data: Passenger image information within a preset distance range obtained by activating a camera device, including the passenger's appearance, posture, etc.

[0149] Acquisition method: Passenger image data comes from the activated camera device. After shooting, these data are stored in the system temporary storage area in the form of digital image files, waiting for subsequent analysis and processing.

[0150] The general process is as follows: the system calls the image analysis algorithm to process the passenger image data in the storage area. The algorithm counts the number of passengers in the image by identifying information such as human body contours and facial features in the image. If the analysis result shows that there is only one passenger in the image, the system executes step SB00; if multiple passengers are detected, the system executes step SC00. For example, in a monitoring screen of a station, the camera device captures the image data. After analysis, the system finds that there are 3 passengers in the picture, so it determines that there are multiple passengers and then executes step SC00.

[0151] Step SB00, maintain the original settings and continue with subsequent steps.

[0152] Original settings: refers to the parameters and operating modes preset by the system for a single-passenger scenario before determining the number of passengers, including but not limited to the basic configuration of the image analysis algorithm, established rules of the data processing flow, etc.

[0153] Step SC00, assigning a unique identifier to each passenger in turn according to the order in which the passengers enter the screen.

[0154] Unique ID: A unique identification tag assigned to each passenger, which is used to accurately identify and distinguish the data and behavioral information of different passengers in subsequent processes. It is generally composed of numbers, letters, or a combination of the two.

[0155] Acquisition method: The system will create a unique identifier based on a preset identifier generation rule. This rule is usually stored in the system configuration file. For example, the identifier can be generated by combining a timestamp with a random number.

[0156] The general process is as follows: When the system determines that there are multiple passengers in step SA00, it executes step SC00. The system assigns a unique identifier to each passenger in the order in which the passengers enter the camera screen according to the preset identifier generation rules. First, the system records the time point when each passenger enters the screen, and then sorts them according to the time sequence. Then, using the identifier generation rules, the first unique identifier is generated for the first passenger to enter the screen, and so on. For example, in the monitoring screen of a large transfer station, three passengers enter in succession.

[0157] The system first records their entry time, and then generates an identifier "P001_20250209100001" for the first passenger to enter in order (where "P001" is a fixed prefix representing the passenger, and the following numbers are a combination of a timestamp and a random number), generates "P002_20250209100005" for the second passenger, and generates "P003_20250209100010" for the third passenger, so as to facilitate the subsequent processing and analysis of the behavioral data of different passengers.

[0158] Step SD00, based on the image data of each passenger, calculate the standing direction and head rotation angle of each passenger, analyze the various quantitative indicators of the head rotation, as well as the gaze direction and the landing position on the screen.

[0159] The acquisition method is as follows: the image data of each passenger is taken by the camera device and stored accordingly after the judgment in step SA00 and the assignment of the SC00 mark. The various algorithms required for analysis, such as the human posture recognition algorithm used to calculate the standing direction and head rotation angle, and the sight estimation algorithm used to determine the direction of the gaze and the location of the landing point, are stored in the system algorithm library and can be called at any time.

[0160] The general process is as follows: the system first obtains the image data of each passenger with a unique identifier from the storage area. Then, the human posture recognition algorithm is called to analyze the passenger's body posture in the image and calculate the standing direction and head rotation angle. Next, based on the change of the head rotation angle over time, the various quantitative indicators of head rotation are obtained by mathematical calculation methods, such as calculating the difference between the head rotation angles at adjacent moments and dividing it by the time interval to obtain the rotation speed. After that, the line of sight estimation algorithm is called to estimate the gaze direction by identifying facial key points, such as the corners of the eyes, pupils, etc., combined with the geometric model, and the landing point position on the screen is calculated based on the relationship between the gaze direction and the spatial position of the screen. For example, at the station, the system obtains the image data of passenger A. After algorithm analysis, it is concluded that his standing direction is 30° to the left of the screen, his head is turned 45° to the right, the rotation speed is 15° per second, the gaze direction points to the route map area on the screen, and the landing point position coordinates are (x1, y1).

[0161] Step SE00, sort the gaze trajectory, dwell time, standing direction, and head rotation angle of each passenger in each area of ​​the screen in chronological order to form independent behavioral data samples, extract key features from these samples, and convert them into feature vectors.

[0162] The general process is as follows: the system first summarizes the gaze positions and corresponding timestamps of each passenger at different times, generates gaze trajectory by analyzing the changes in the gaze positions, and calculates the dwell time in each area. Then, these dwell trajectories, dwell time, standing direction, and head rotation angle data obtained in step SD00 are integrated in chronological order to form independent behavior data samples. Next, the algorithm is used to filter out key features from the behavior data samples. For example, if a passenger's gaze stays in a certain station information area for much longer than in other areas, this long stay can be used as a key feature. Finally, these key features are converted into numerical values ​​to form feature vectors. For example, the behavior data sample of passenger B shows that his gaze has a complex dwell trajectory in the screen route map area, stays at a certain station information for 5 seconds, stands facing the screen, and the head turns 20° to the left. The system extracts the key feature of gaze stay for 5 seconds and converts it into the value "5". After the other key features are digitized, they form a feature vector [5,0,20] (assuming that "0" represents the numerical representation of standing facing the screen).

[0163] In step SF00, the feature vector of each passenger is input in parallel into a pre-built and trained passenger attention information analysis model, and distributed computing resources are used to perform calculations simultaneously to output the attention information of each passenger.

[0164] Feature vector: An ordered vector composed of the numerical representation of key features in the passenger behavior data sample, including key information such as standing direction, head rotation angle, gaze dwell time, etc., which is used to input the analysis model.

[0165] Passenger information analysis model: This artificial intelligence model, which has been trained with a large amount of data, can analyze and output the information that passengers are actually concerned about, such as the routes, stations, transfer information, etc., based on the input feature vector.

[0166] Distributed computing resources: A collection of computing resources consisting of multiple computing nodes, which work together through network connections and can process multiple tasks at the same time to improve computing efficiency. In this step, it is used to parallelize the feature vector analysis tasks of multiple passengers.

[0167] The acquisition method is as follows: the feature vector of each passenger comes from the calculation result of step SE00. The passenger attention information analysis model is pre-built and trained, stored in the server, and used by the system by calling the model interface. Distributed computing resources are provided by the computing cluster where the system is located, and are allocated and managed through the resource scheduling system.

[0168] The general process is as follows: the system inputs the feature vector of each passenger generated in step SE00 into the pre-built and trained passenger attention information analysis model in parallel. Using distributed computing resources, multiple computing nodes simultaneously process and analyze the feature vectors of different passengers. The model calculates and analyzes each feature vector based on the relationship between the internally learned features and attention information. For example, for the feature vector [3,15,4] of passenger C (assuming that they represent the numerical representation of the gaze dwell time, the head turning 15° to the right, and the standing direction being a certain angle to the left), the model determines that the passenger is concerned about the first and last bus times of a certain line based on the rules obtained through training. Finally, the model outputs the attention information of each passenger, laying the foundation for providing personalized services to passengers in the future.

[0169] A rail transit passenger energy information service method under multi-source data fusion also includes steps after outputting each passenger's attention information, which are as follows: Step SG00, based on the attention information of each passenger, independently extract the energy consumption data of the target line in the current period from the energy consumption database. At the same time, based on the behavioral model data of each passenger, a clustering algorithm is used to divide the corresponding passengers into consumer groups with similar behavior patterns, and find the historical selection preferences of the consumer group.

[0170] The general process is as follows: the system first determines the target line of interest based on the interest information of each passenger output in step SF00. Then, the energy consumption data of the target line in the current period is extracted from the energy consumption database. For example, passenger D is interested in Line 2, and the system queries the power consumption, equipment operation energy consumption and other data of Line 2 in the current period from the energy consumption database. At the same time, the system calls the clustering algorithm, takes the behavior model data of each passenger obtained in step SE00 as input, and divides the passengers into different consumer groups according to the similarity of the behavior patterns. After the division is completed, for each consumer group, its historical selection preferences are searched from the historical data record database. For example, through analysis, it is found that a consumer group has chosen to travel during non-peak hours many times in the past to save energy. This is the historical selection preference of the consumer group.

[0171] Step SH00, input the target route energy consumption data and consumer group historical selection preferences corresponding to each passenger into the pre-trained personalized energy-saving suggestion generation model, and generate independent personalized energy-saving suggestions for each passenger.

[0172] The general process is as follows: the system organizes the target line energy consumption data and consumer group historical selection preferences corresponding to each passenger obtained in step SG00 into a format acceptable to the model, and then inputs it into the personalized energy-saving suggestion generation model that has been trained in advance. The model uses internal algorithms and learned patterns to analyze and process the input data. For example, for passenger E, the corresponding target line Line 3 has high energy consumption, and the consumer group has a historical preference to avoid peak travel. The model combines this information to generate personalized energy-saving suggestions such as "It is recommended that you take Line 3 during non-peak hours, when the line energy consumption is lower and travel is more energy-efficient." The model generates corresponding energy-saving suggestions for each passenger independently to meet the energy-saving needs of different passengers.

[0173] A rail transit passenger energy information service method based on multi-source data fusion also includes steps after generating independent personalized energy-saving suggestions for each passenger, specifically as follows: Step SI00, divide the screen into corresponding grid areas according to the number of passengers, and set a different main color tone for each passenger's split-screen area.

[0174] Grid area: The screen is divided into multiple small areas according to certain rules. Each area can be used to display exclusive information of different passengers, making it convenient for passengers to quickly locate their own information area.

[0175] Main color: The main color set for each passenger split-screen area to distinguish the information display areas of different passengers and enhance visual recognition.

[0176] The acquisition method is as follows: the number of passengers comes from the result obtained by the camera device and the analysis and statistics in the previous steps. The rules of screen division and color setting are stored in the system configuration file, and the system calls the corresponding rules to operate according to the number of passengers.

[0177] The general process is as follows: the system first obtains the number of passengers in the current scene. For example, through preliminary analysis, it is known that there are 5 passengers. Then, according to the preset screen division rules, the screen is divided into a corresponding number of grid areas. Assuming that the screen is a rectangle, according to the rule of uniform division, the screen is divided into an appropriate number of parts horizontally and vertically to form 5 grid areas of similar size. After that, according to another set of preset color allocation rules, a different main color is set for each grid area. For example, the area of ​​the first passenger is set to blue as the main color, the second is set to green, the third is set to yellow, etc., to ensure that the split-screen area of ​​each passenger is visually distinguished, so that passengers can quickly find their own information display area.

[0178] Step SJ00, display the target route and station information of each passenger in the center of the split-screen area, and use dynamic effects to guide the corresponding passengers to pay attention to the recommended energy-saving information.

[0179] The general process is as follows: the system first extracts the target line and station information that each passenger is concerned about from step SF00, and then displays this information in the center of the split-screen area for the corresponding passenger divided in step SI00. For example, passenger F is concerned about a station on Line 5, and the system will clearly display the route map of Line 5 and detailed information of the station, such as transfer information around the station, first and last bus times, etc., in the center of the split-screen area divided for passenger F. Afterwards, the system uses front-end display technology and animation library to add dynamic effects to the recommended energy-saving information in the split-screen area. For example, the text of the energy-saving information is presented in a flashing manner, or an arrow animation is added that moves from the edge of the screen to the energy-saving information area, so as to guide the corresponding passengers to pay attention to the recommended energy-saving information and improve the communication effect of the energy-saving information.

[0180] A rail transit passenger energy information service method under multi-source data fusion also includes steps after displaying target line and station information of each passenger's concern in the center of the split-screen area and using dynamic effects to guide the corresponding passengers to pay attention to the recommended energy-saving information, which are as follows: Step SK00, start the directional microphone array to capture the communication voice between passengers and their companions, and quickly analyze the language type of the voice through the voice recognition engine and natural language processing technology.

[0181] Directional microphone array: A device composed of multiple microphones that can collect sounds in a specific direction with high sensitivity. In this step, it is used to capture the voice communication between passengers and their companions.

[0182] Speech recognition engine: A technical tool that converts speech signals into text information. It can recognize speech in different languages ​​and convert it into text form that can be processed by computers.

[0183] Natural language processing technology: a series of technologies for analyzing, understanding and generating human language, which are used in this step to analyze the recognized text and extract key information, such as language type.

[0184] Language: The language used by passengers to communicate, such as Mandarin, English, Cantonese, etc.

[0185] The acquisition method is as follows: the directional microphone array is part of the system hardware equipment and is directly integrated into the station's information collection system. The speech recognition engine and natural language processing technology are usually integrated into the system background in the form of a software library and can be called at any time.

[0186] The general process is as follows: the system starts the directional microphone array, which begins to capture the voice communication between passengers and their companions in the station. The collected voice signal is first transmitted to the speech recognition engine, which converts the voice signal into text content. For example, a piece of Mandarin communication voice is converted into the corresponding text "Let's take Line 7 to get to the transfer station." Then, the converted text is input into the natural language processing technology module, which determines the language type of the speech by analyzing the vocabulary, grammatical structure and other features of the text. If the above text is analyzed to be Mandarin, the system will record the language type information to prepare for the subsequent matching of the corresponding voice package.

[0187] Step SM00, based on the analyzed language type, the system automatically matches the voice package of the corresponding language.

[0188] Language type: The language used by passengers to communicate, such as Mandarin, English, Japanese, etc., which comes from the analysis results of step SK00.

[0189] Voice Pack: A collection of pre-recorded voice files containing various information, one for each language, used to broadcast relevant information to passengers, such as energy-saving advice, screen location prompts, etc.

[0190] Acquisition method: The analyzed language type is obtained from step SK00. The voice package is stored in the voice resource database of the system, which is classified and stored according to the language type for easy and fast retrieval.

[0191] The general process is as follows: After the system obtains the language used by the passenger in step SK00, it searches the voice resource database based on the language information. The database stores voice packages in different languages, and each voice package has a corresponding language identifier. For example, when the system determines that the language used by the passenger is English, it finds the corresponding English voice package by querying the language identifier of the voice package in the database. This English voice package contains various preset voice contents, such as English expressions of energy-saving suggestions, English prompts to guide passengers to pay attention to screen information, etc., so that they can be played later according to needs to ensure that the information can be conveyed in a language familiar to passengers.

[0192] Step SN00, broadcast the screen position of the energy-saving information recommended for the corresponding passengers in the order of passenger identification, and the volume is automatically adjusted according to the station ambient noise to ensure that it is higher than the preset decibels than the background noise.

[0193] Passenger identification: A unique identity tag assigned to each passenger in step SC00, used to accurately identify different passengers so that information related to them can be broadcasted in a targeted manner.

[0194] Energy-saving information: personalized energy-saving suggestions generated in step SH00, and related energy information prompts, such as target line energy consumption data, etc.

[0195] Screen position: In the split-screen area divided in step SI00, the specific location of the corresponding passenger energy-saving information is displayed, and the passenger's attention is guided by voice.

[0196] Preset decibel: A volume difference standard preset by the system to ensure that the broadcast voice volume is higher than the background noise and that passengers can hear it clearly, such as when it is set to 10 decibels.

[0197] Acquisition method: The passenger identification is obtained from the allocation result of step SC00, the energy saving information is obtained from step SH00, and the screen position information is determined according to the screen division result of step SI00. The preset decibel value is set when the system is initialized and stored in the system configuration file. The station environmental noise is collected in real time by the environmental noise sensor, and the sensor data is transmitted to the system for volume adjustment calculation.

[0198] General process: The system extracts the energy-saving information voice content corresponding to the passenger from the voice package in the order of the passenger identification. For example, the energy-saving information voice of passenger A is extracted first. At the same time, combined with the position of the split-screen area divided for passenger A in step SI00, a voice prompt about the screen position where the energy-saving information is located is generated. Then, the ambient noise value of the current station is obtained through the ambient noise sensor, and the appropriate broadcast volume is calculated according to the preset decibel value. Assuming that the ambient noise is 50 decibels and the preset decibel is 10 decibels, the broadcast volume is set to 60 decibels. Finally, the energy-saving information of the passenger and its position on the screen are broadcast through the station's broadcasting system at the calculated volume to ensure that each passenger can clearly receive exclusive energy-saving information.

[0199] Further, for the step S600 of "inputting the extracted energy consumption data of the target line in the current period and the historical selection preferences of the corresponding consumer group into the pre-trained personalized energy-saving suggestion generation model to output the personalized energy-saving suggestion", the following settings can be made, specifically as follows: Step Sa00, uses a convolutional neural network to extract features from facial images captured by the camera, combines deep learning algorithms to analyze facial muscle movements in real time, and determines the emotional state of the passenger; and uses the Gaussian mixture model in voiceprint recognition technology to extract and match features from the voice collected by the microphone, identify the passenger's identity and associate it with his or her historical behavior data, and analyze the voice content through natural language processing technology to explore the passenger's preferences for time, energy consumption, congestion, etc.

[0200] Step Sb00, deeply integrate the personalized energy-saving suggestion generation model with the reinforcement learning-based path optimization model to build a fusion model framework. In the reinforcement learning model, the state space is defined as the passenger's real-time emotions, preferences, current location, target route and other information; the action space is defined as different path planning strategies, such as choosing different routes, transfer stations, etc. Design a reward function, give reward feedback based on the actual effect, give positive rewards for meeting the passenger's time requirements, and give negative rewards for excessive energy consumption or overcrowding.

[0201] Based on a large amount of historical data and machine learning algorithm training, a real-time preference-parameter mapping table is constructed. Based on the emotion and preference information obtained from micro-expression and voiceprint recognition, the mapping table is queried to determine the adjustment range of parameters related to time, energy consumption, and congestion in the model. An adaptive learning rate strategy is adopted to dynamically adjust the step size of parameter updates based on the training effect and feedback of the model.

[0202] For the energy consumption data of the target line in the current period, we use data smoothing algorithms such as wavelet transform to remove noise, and combine it with the Kalman filter algorithm to accurately predict the future short-term energy consumption trend based on multi-source information such as real-time train operation status, line conditions, and weather. For the historical choice preference data of consumer groups, we use feature engineering methods such as principal component analysis to extract key features and convert them into numerical vectors for unique hot encoding. We use association rule mining algorithms such as Apriori to find the potential connections between different preferences, combine the emotions and preference information quantified by micro-expressions and voiceprint recognition, and use weighted fusion algorithms to dynamically weight and adjust the historical choice preference data.

[0203] In step Sc00, the preprocessed and enhanced data and the adjusted model parameters are input into the fused personalized energy-saving recommendation generation model. The model adopts the Transformer architecture and uses its powerful attention mechanism to fully consider the relationship between various factors, and combines the Pareto optimal path calculation results to generate personalized energy-saving recommendations including recommended energy-saving carriages, transfer routes, and energy-saving measures for key stations.

[0204] In addition, considering the situation that the passenger is dissatisfied, there may be subsequent settings after step Sc00, as follows: Step Sd00: Use multimodal sensing technology to deploy multiple cameras to capture passengers' micro-expression changes and body movements in all directions, and analyze passengers' behavioral intentions through posture estimation algorithms. Use directional microphones and voice recognition technology to collect passengers' voice feedback. Integrate touch sensors and pressure sensors on the display device to sense the interaction between passengers and the display device. Use sentiment analysis algorithms, combined with recurrent neural networks and attention mechanisms in deep learning, to deeply analyze these feedbacks and determine passengers' satisfaction with the suggestions, points of doubt, and areas of interest.

[0205] In step Se00, if the analysis finds that the passenger has doubts about or is not interested in some suggestions, the path and suggestions are re-evaluated in combination with the Pareto optimal path dynamic planning content.

[0206] Using the Monte Carlo tree search algorithm, more potential path combinations and factors are considered when recalculating the Pareto optimal path. The specific application process of the Monte Carlo tree search algorithm is as follows: Initialize the search tree: Take the current passenger's location and destination as the root node, and possible path options (such as different routes, transfer stations) as child nodes to build the initial structure of the Monte Carlo tree. Each node contains relevant information about the path, such as expected energy consumption, required time, expected congestion, etc.

[0207] Simulation and evaluation: Starting from the root node, a path is randomly selected for simulation. During the simulation, the impact of temporary traffic control on line operation is taken into account. For example, if some stations on a line do not stop due to control, the path simulation passing through the station needs to be replanned to bypass the station; for sudden changes in passenger flow, if a station has a surge in passenger flow, the transfer time and congestion assessment of the station will be increased during the simulation. After each simulation, the path is evaluated based on the simulation results. The evaluation indicators include energy consumption, time, congestion, and the degree of match with passenger preferences. For example, if the passenger is sensitive to time, the weight of the time indicator in the evaluation will increase accordingly.

[0208] Tree expansion and update: According to the simulation results, select the path nodes with better performance for expansion, and add more possible path selections as child nodes. At the same time, update the evaluation value and visit times of each node. The nodes with higher evaluation values ​​and more visits have a greater probability of being selected in subsequent searches. Through multiple iterative simulations and tree expansion and update, gradually find a better path combination.

[0209] Determine the new path: After a certain number of iterations, the path with the best evaluation value is selected as the recalculated Pareto optimal path. This path fully considers factors such as temporary traffic control, sudden changes in passenger flow, and the real-time preferences of passengers.

[0210] Based on the same inventive concept, an embodiment of the present invention provides a rail transit passenger energy information service system under multi-source data fusion, including a memory and a processor, wherein the memory stores data that can be executed on the processor to implement Figures 1 to 4 Procedure for either method.

[0211] The embodiments of this specific implementation method are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Therefore, all equivalent changes made based on the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A rail transit passenger energy information service method based on multi-source data fusion, characterized in that: include: Analyze whether the passenger enters the preset distance range of the rail transit external display; If not, the rail transit external display will only show the overall route; If yes, monitor and obtain the passenger's behavior data, and identify and obtain the passenger's attention information based on the passenger's behavior data, the attention information including the target route and station; According to the passengers' attention information, the energy consumption data of the target line in the current period is extracted from the energy consumption database; Obtain passenger behavior model data, and use clustering algorithms to divide the corresponding passenger behavior patterns into consumer groups with similar patterns, and search for the historical selection preferences of the corresponding consumer groups in the passenger behavior model library, where the passenger behavior model data includes travel time data, travel route data, consumption data, and historical energy-saving suggestion adoption data; The extracted energy consumption data of the target route during the current period and the historical selection preferences of the corresponding consumer groups are input into the pre-trained personalized energy-saving suggestion generation model to output personalized energy-saving suggestions, where the personalized energy-saving suggestions include recommended energy-saving carriages, transfer routes and key stations; On the rail transit external display, the target lines and station information that passengers are concerned about are displayed in the center of the screen, and dynamic effects are used to guide passengers to pay attention to recommended energy-saving information.

2. According to the rail transit passenger energy information service method under multi-source data fusion according to claim 1, it is characterized in that: Monitoring and obtaining passenger behavior data, and identifying and obtaining passenger information of interest based on passenger behavior data include: Starting the camera device to capture image data of passengers within a preset distance range; Based on passenger image data, a human posture recognition algorithm is used to analyze the passenger's body posture in real time, locate the key points of the human body, and calculate the passenger's standing direction and head rotation angle; Analyze and calculate whether the passenger's standing orientation obtained is within a pre-set effective judgment area for the standing orientation; If not, it is determined that the passenger is not interested in the display information; If yes, then a head rotation quantitative analysis is performed on the passenger according to the head rotation quantitative analysis method, and various quantitative indicators of the passenger's head rotation are obtained; Based on the quantitative indicators of standing orientation and head rotation, and according to the preset weighted algorithm, a comprehensive score is calculated; If and only if the comprehensive score exceeds a preset threshold, it is determined that the corresponding passenger is interested in the display information; When it is determined that the corresponding passenger is interested in the information on the display, the camera is used to capture the passenger's facial feature points in combination with the gaze estimation algorithm based on facial key points, calculate the passenger's gaze direction and the location of the gaze on the screen, determine the specific information area where the gaze is located, and record the area type and precise coordinates in detail; Use time series analysis methods to continuously monitor and obtain the trajectory and duration of passengers' gazes in the corresponding areas; The gaze trajectory and duration in each area of ​​the screen, as well as the passenger's standing direction and head rotation angle are sorted in chronological order to form a series of behavior data samples, each of which contains various behavior feature values ​​at different times; Key features are extracted from the behavioral data samples and converted into feature vectors. The converted feature vectors are input into a pre-built and trained passenger attention information analysis model to output the passenger attention information.

3. According to the rail transit passenger energy information service method under multi-source data fusion according to claim 2, it is characterized in that: The training and construction of the passenger attention information analysis model includes: Obtain the labeled behavior data samples and the passenger attention information corresponding to each sample; Divide the labeled samples into training set and validation set; The feature data and the corresponding attention information labels in the training set are input into the Naive Bayes classification model. The Naive Bayes model starts to calculate the conditional probability of each feature under different attention information categories, as well as the prior probability of each attention information category. Based on the conditional probability of each feature under different categories of attention information and the prior probability of each category of attention information, the naive Bayes classification model completes the preliminary construction of the probabilistic relationship between the feature and the attention information; Use the validation set to evaluate the model in training and calculate the accuracy; If the accuracy rate exceeds the preset accuracy rate for consecutive preset times, it is determined that the passenger attention information analysis model has completed training; Otherwise, the smoothing parameters of the naive Bayes model are adjusted based on the accuracy stage, and the passenger attention information analysis model is retrained.

4. According to the rail transit passenger energy information service method under multi-source data fusion according to claim 2, it is characterized in that: The method further includes the following steps after starting the camera device to capture the image data of the passenger within the preset distance range: Analyzing whether there are multiple passengers based on the passenger image data; If not, maintain the original settings and continue with the subsequent steps; If yes, a unique identifier is assigned to each passenger in the order in which they enter the screen; Based on the image data of each passenger, the standing direction and head rotation angle of each passenger are calculated, and the quantitative indicators of head rotation, as well as the gaze direction and the landing point position on the screen are analyzed; The gaze trajectory, dwell time, standing direction, and head rotation angle of each passenger in each area of ​​the screen are sorted in chronological order to form independent behavioral data samples. Key features are extracted from these samples and converted into feature vectors. The feature vector of each passenger is input in parallel into a pre-built and trained passenger attention information analysis model, and distributed computing resources are used to perform calculations simultaneously to output the attention information of each passenger.

5. The rail transit passenger energy information service method under multi-source data fusion according to claim 4 is characterized in that: The following steps are included after outputting the attention information of each passenger: According to each passenger's attention information, the energy consumption data of the target line in the current period is extracted independently from the energy consumption database. At the same time, according to each passenger's behavior model data, a clustering algorithm is used to divide the corresponding passenger into consumer groups with similar behavior patterns, and the historical choice preferences of the consumer groups are found; The target route energy consumption data and consumer group historical selection preferences corresponding to each passenger are respectively input into the pre-trained personalized energy-saving suggestion generation model to generate independent personalized energy-saving suggestions for each passenger.

6. The rail transit passenger energy information service method under multi-source data fusion according to claim 5 is characterized in that: It also includes the steps after generating individual personalized energy saving recommendations for each passenger, as follows: Divide the screen into corresponding grid areas according to the number of passengers, and set a different main color tone for each passenger's split-screen area; The target routes and station information of each passenger's interest are displayed in the center of the split-screen area, and dynamic effects are used to guide the corresponding passengers to pay attention to the recommended energy-saving information.

7. The rail transit passenger energy information service method under multi-source data fusion according to claim 6 is characterized in that: The method further includes steps after displaying the target route and station information that each passenger is concerned about in the center of the split-screen area and using dynamic effects to guide the corresponding passengers to pay attention to the recommended energy-saving information, which are as follows: The directional microphone array is activated to capture the voices of passengers and their companions, and the language type of the voices is quickly analyzed through the speech recognition engine and natural language processing technology; According to the analyzed language type, the system automatically matches the voice package of the corresponding language; The screen location of the energy-saving information recommended for the corresponding passengers will be announced in the order of passenger identification, and the volume will be automatically adjusted according to the ambient noise of the station to ensure that it is higher than the preset decibel level of background noise.

8. A rail transit passenger energy information service system based on multi-source data fusion, characterized in that: It includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the program can be loaded and executed by the processor to implement a rail transit passenger energy information service method under multi-source data fusion as described in any one of claims 1 to 7.

Images