Intelligent rail transit data management and analysis method, system and equipment and medium

Abstract

The embodiment of the invention provides an intelligent rail transit data management and analysis method and system. The method comprises the following steps: receiving and importing professional rail transit data to a data lake; performing full-process quality monitoring on the professional rail transit data and generating a data quality index; calculating a rail transit index based on the data quality index; providing a user interaction interface to access, analyze and display the rail transit indexes; wherein in the process of executing the step of monitoring the quality of the whole process and generating the data quality index and the step of calculating the rail transit index, task coordination and resource scheduling are continuously carried out, so that automatic operation according to a preset strategy and priority is ensured. According to the embodiment of the invention, the real-time performance and quality of data processing and the expansibility of the system are remarkably improved, the fusion of cross-professional data and the dynamic allocation of computing resources are enhanced, the whole-process data management is realized, the overall operation efficiency and service quality are improved, and the travel experience of passengers is improved.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to an intelligent rail transit data management and analysis method, an intelligent rail transit data management and analysis system, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the steady development of large cities in China, rail transit, as an important part of urban transportation, has a direct impact on the development of cities and the daily lives of residents due to its operational efficiency and service quality. Its operational strategy is usually formulated based on the analysis of complex rail transit data and the evaluation of indicators. The rail transit data system is a complex and large system involving twelve professional sub-fields such as driving, passenger flow, energy consumption, and finance. In order to improve the operational efficiency and service quality of rail transit, it is necessary to conduct comprehensive analysis and effective management of data in these fields.

[0003] The intelligent rail transit data platform uses artificial intelligence (AI) big models and Internet of Things technologies to manage, analyze and optimize the data of the rail transit system. Through the various operating indicators calculated by the system, it can timely reflect the road network situation and help subway operation decision-makers to formulate better operation plans to improve traffic operation efficiency, service quality and passenger experience. With the acceleration of urbanization, the rail transit system plays an increasingly important role, so the efficient management and intelligent use of rail transit data is particularly urgent.

[0004] Existing rail transit index calculation systems mostly use traditional database technology and data warehouse technology, which have certain limitations in data access, processing and analysis. For example, traditional databases are limited in performance when processing large-scale, high-concurrency data. Although data warehouse technology can integrate and analyze data, it lacks real-time and flexibility. At the same time, it is actually very difficult to verify the correctness of the index calculation results, which can only be estimated by experienced engineers.

[0005] In the existing technology, some rail transit index calculation systems use data integration platforms, but there are the following technical problems:

[0006] 1. Data real-time problem: The traditional ETL (the process of extracting, transforming and loading data) process is usually batch processing, which leads to delays in data updates and cannot meet the needs of real-time monitoring and decision-making.

[0007] 2. Insufficient data quality control: Existing systems may lack effective data quality governance mechanisms, resulting in errors or inconsistencies in data during the ETL process, affecting the accuracy of indicator calculations.

[0008] 3. System scalability issues: With the expansion of the rail transit network, the amount and complexity of data continue to increase, and traditional systems may have bottlenecks in scalability.

[0009] 4. Difficulty in data fusion in professional fields: Due to the different data formats and standards in different professional fields, existing systems may have difficulties in data fusion, affecting the depth and breadth of comprehensive analysis.

[0010] 5. Uneven distribution of computing resources: When calculating data in multiple professional fields, the existing system may not be able to dynamically adjust computing resources according to actual needs, resulting in resource waste or computing bottlenecks.

[0011] 6. Lack of full-process data governance: Existing systems may only focus on data storage and analysis, but ignore the full-process data governance, including data collection, transmission, processing, storage and use. Summary of the invention

[0012] In view of the above problems, embodiments of the present invention are proposed to provide an intelligent rail transit data management and analysis method, an intelligent rail transit data management and analysis system, an electronic device and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0013] In order to solve the above problems, an embodiment of the present invention discloses an intelligent rail transit data management and analysis method, which includes: receiving and importing professional rail transit data into a data lake; performing full-process quality monitoring on the professional rail transit data and generating data quality indicators; calculating rail transit indicators based on the data quality indicators; providing a user interaction interface to access, analyze and display the rail transit indicators; wherein, in the process of executing the steps of performing full-process quality monitoring and generating data quality indicators and the step of calculating rail transit indicators, task coordination and resource scheduling are continuously performed to ensure automated operation according to predetermined strategies and priorities.

[0014] Optionally, the step of receiving and importing professional rail transit data into the data lake includes: capturing the professional rail transit data of the data source through a data access proxy mode; and / or, listening to a specific source server port through a monitoring mode to access the professional rail transit data; performing format conversion and decryption processing on the professional rail transit data; and storing the professional rail transit data after format conversion and decryption processing in the data lake.

[0015] Optionally, the step of generating data quality indicators includes: detecting the timeliness and relevance of code table data; checking the field integrity, real-time, cross-professional association consistency and update cycle consistency of the master data; and generating the data quality indicators of the code table data and the master data according to the available thresholds of indicators set by the user.

[0016] Optionally, the step of calculating the rail transit indicator based on the data quality indicator includes: for the single data source indicator in the rail transit indicator, using the data quality indicator after passing the whole process quality monitoring as the single data source indicator; for the composite data source indicator in the rail transit indicator, checking and ensuring the consistency of the pulling cycle of multi-source data; calculating the composite data source indicator according to a predefined calculation logic.

[0017] Optionally, the step of providing a user interactive interface to access, analyze and display the rail transit indicators includes: integrating industry big models to understand user query intentions; guiding users to provide time periods, qualifiers, statistical granularity and atomic indicators of query indicators; building a complete derived indicator query based on the information provided by the user; executing the query and presenting the rail transit indicators in a visual manner.

[0018] Optionally, the following steps are also included: classifying the received professional rail transit data into real-time indicator data and offline indicator data; establishing a fast processing channel for the real-time indicator data, and batch processing the offline indicator data.

[0019] Optionally, the step of continuously coordinating tasks and scheduling resources includes: setting a timed execution strategy according to the time characteristics of the professional rail transit data; building a task execution network based on the dependencies between the rail transit indicators; dynamically adjusting task priorities and resource allocation; monitoring task execution status and handling abnormal situations.

[0020] An embodiment of the present invention also discloses an intelligent rail transit data management and analysis system, which includes: a data receiving and importing module, which is used to receive and import professional rail transit data into a data lake; a quality monitoring module, which is used to perform full-process quality monitoring on the professional rail transit data and generate data quality indicators; a traffic indicator calculation module, which is used to calculate rail transit indicators based on the data quality indicators; a traffic indicator processing module, which is used to provide a user interaction interface to access, analyze and display the rail transit indicators; a coordination and scheduling module, which is used to continuously perform task coordination and resource scheduling during the process of the quality monitoring module performing the steps of performing the full-process quality monitoring and generating data quality indicators and the traffic indicator calculation module calculating the rail transit indicators, so as to ensure automated operation according to predetermined strategies and priorities.

[0021] Optionally, the data receiving and importing module includes: an agent capture module, which is used to capture the professional rail transit data of the data source through a data access agent mode; and / or a monitoring access module, which is used to monitor a specific source server port through a monitoring mode to access the professional rail transit data; a conversion and decryption module, which is used to perform format conversion and decryption processing on the professional rail transit data; and a data storage module, which is used to store the professional rail transit data after format conversion and decryption processing in the data lake.

[0022] Optionally, the quality monitoring module includes: a code table data detection module, used to detect the timeliness and relevance of the code table data; a master data detection module, used to check the field integrity, real-time, cross-professional association consistency and update cycle consistency of the master data; a data quality indicator generation module, used to generate the data quality indicators of the code table data and the master data according to the indicator available threshold set by the user.

[0023] Optionally, the traffic indicator calculation module includes: a single data source indicator determination module, which is used to use the data quality indicator after passing the full process quality monitoring as the single data source indicator for the single data source indicator in the rail transit indicator; a periodic consistency check module, which is used to check and ensure the consistency of the multi-source data pulling cycle for the composite data source indicator in the rail transit indicator; and a composite data source indicator calculation module, which is used to calculate the composite data source indicator according to a predefined calculation logic.

[0024] Optionally, the traffic indicator processing module includes: a large model integration module, which is used to integrate industry large models to understand user query intentions; a user guidance module, which is used to guide users to provide time periods, qualifiers, statistical granularity and atomic indicators of query indicators; a derived indicator query construction module, which is used to construct a complete derived indicator query based on the information provided by the user; and a rail transit indicator presentation module, which is used to execute queries and present the rail transit indicators in a visual manner.

[0025] Optionally, it also includes: a rail transit data classification module, which is used to classify the received professional rail transit data into real-time indicator data and offline indicator data; a fast and batch processing module, which is used to establish a fast processing channel for the real-time indicator data and perform batch processing on the offline indicator data.

[0026] Optionally, the coordination and scheduling module includes: an execution strategy setting module, which is used to set a timing execution strategy according to the time characteristics of the professional rail transit data; an execution network construction module, which is used to construct a task execution network based on the dependency relationship between the rail transit indicators; a priority and allocation adjustment module, which is used to dynamically adjust task priority and resource allocation; a status and exception monitoring module, which is used to monitor the task execution status and handle abnormal situations.

[0027] An embodiment of the present invention also discloses an electronic device, comprising: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, enables the electronic device to execute the intelligent rail transit data management and analysis method as described above.

[0028] The embodiment of the present invention further discloses a computer-readable storage medium, the computer program stored in the storage medium enables a processor to execute the intelligent rail transit data management and analysis method as described above.

[0029] The embodiments of the present invention include the following advantages:

[0030] The intelligent rail transit data management and analysis solution provided by the embodiment of the present invention receives and imports professional rail transit data into a data lake; performs full-process quality monitoring on professional rail transit data and generates data quality indicators; calculates rail transit indicators based on data quality indicators; and provides a user interaction interface to access, analyze and display rail transit indicators; wherein, in the process of performing the steps of performing full-process quality monitoring and generating data quality indicators and the step of calculating rail transit indicators, task coordination and resource scheduling are continuously performed to ensure automated operation according to predetermined strategies and priorities.

[0031] The embodiment of the present invention can realize real-time access and processing of data through data lake technology, solve the problem of data delay in the traditional ETL process, and meet the needs of real-time monitoring and decision-making. Implementing a full-process quality monitoring mechanism and generating data quality indicators effectively solves the errors or inconsistencies that may occur in the ETL process, and improves the accuracy of rail transit indicator calculation. The application of data lake and artificial intelligence technology enables the system to better cope with the increase in data volume and complexity caused by the expansion of the rail transit network, and improves the scalability and flexibility of the system. Through the intelligent rail transit data platform, data in different professional fields can be more easily standardized and integrated, which improves the depth and breadth of comprehensive analysis. When calculating data in multiple professional fields, through continuous task coordination and resource scheduling, computing resources can be dynamically adjusted according to actual needs, avoiding resource waste or computing bottlenecks. The full-process data governance from data collection, transmission, processing, storage to use is realized, ensuring the high quality and consistency of data, which helps to improve the reliability and efficiency of the overall system. A user interaction interface is provided to facilitate users to access, analyze and display rail transit indicators, improving user experience and system usability. The operational indicators obtained through systematic calculation and analysis can timely reflect the road network situation, help subway operation decision-makers to formulate better operation plans, and improve the operational efficiency and service quality of rail transit.

[0032] In summary, the intelligent rail transit data management and analysis solution has significantly improved the real-time, quality and system scalability of data processing through technological innovation and system optimization compared to the traditional rail transit indicator calculation system. At the same time, it has strengthened the integration of cross-professional data and the dynamic allocation of computing resources, realized full-process data governance, and provided strong support for rail transit operation decisions, thereby improving overall operational efficiency and service quality, and improving passengers' travel experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flowchart of the steps of an intelligent rail transit data management and analysis method according to an embodiment of the present invention;

[0034] Figure 2 It is a structural schematic diagram of a comprehensive management and analysis solution for intelligent rail transit data based on a data lake and AI according to an embodiment of the present invention;

[0035] Figure 3 It is a schematic diagram of the principle of data access using a data access proxy mode according to an embodiment of the present invention;

[0036] Figure 4 It is a schematic diagram of entity relationship (ER for short) of a small part of energy consumption indicators in an embodiment of the present invention;

[0037] Figure 5 is a flow chart of a data processing and analysis platform according to an embodiment of the present invention;

[0038] Figure 6 It is a structural block diagram of an intelligent rail transit data management and analysis system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] The intelligent rail transit data management and analysis solution proposed in the embodiment of the present invention realizes efficient management, comprehensive analysis and optimization of rail transit system data by introducing artificial intelligence big models and Internet of Things technologies. The solution receives and imports professional rail transit data into the data lake, performs full-process quality monitoring and generates data quality indicators, and calculates rail transit operation indicators based on these indicators. Compared with traditional systems, this solution significantly improves the real-time performance, quality and scalability of data processing, while strengthening the integration of cross-professional data and the dynamic allocation of computing resources. In addition, through the user interaction interface, users can easily access, analyze and display various operation indicators, support subway operation decision-makers to formulate better operation plans, thereby improving the operation efficiency and service quality of rail transit and improving passengers' travel experience.

[0041] Reference Figure 1 , shows a flowchart of the steps of an intelligent rail transit data management and analysis method according to an embodiment of the present invention. The intelligent rail transit data management and analysis method can be applied to a rail transit system, referred to as a system. The intelligent rail transit data management and analysis method can specifically include the following steps:

[0042] Step 101, receiving and importing professional rail transit data into the data lake.

[0043] In the embodiment of the present invention, in the rail transit system, there are many types of data sources, involving multiple fields such as driving, passenger flow, energy consumption, finance, etc. The goal of step 101 is to uniformly receive and import these professional rail transit data into the data lake to provide a basis for subsequent data analysis and processing. The data lake is a storage system that can store massive structured and unstructured data, which has high flexibility and scalability.

[0044] In actual application, data needs to be collected from various data sources. These data sources may include sensors, ticketing systems, energy consumption monitoring equipment, etc. For example, the driving data obtained from sensors may include the real-time position, speed, acceleration, etc. of the train; the passenger flow data obtained from the ticketing system may include the time and location of passengers entering and leaving the station; the energy consumption data obtained from the energy consumption monitoring equipment may include the power and other resources consumed during the operation of the train. The formats and types of the collected data may vary, and the system needs to standardize these data. The standardized data is imported into the data lake. In this process, the system will perform preliminary cleaning and conversion of the data to ensure the integrity and consistency of the data. For example, the driving data may have missing values ​​or outliers, and the system will use interpolation or other algorithms to process them; the passenger flow data may have duplicate records, and the system will perform deduplication processing. During the data conversion process, format conversion will also be performed, such as converting timestamps of different formats into the standard UTC format. After the data is imported into the data lake, the system will classify and store the data. According to different data types and sources, the system will create corresponding storage structures, such as tables, documents, object storage, etc. The data lake is also highly scalable and can automatically expand storage space as the amount of data increases to ensure the stable operation of the system.

[0045] For example, suppose a city's rail transit system needs to process driving data, passenger flow data, and energy consumption data from 50 stations every day. The system collects about 10GB of data from these stations every day, including millions of records. Through data lake technology, the system can import this data into the data lake after standardization in a short period of time, and store and manage it efficiently, providing a reliable data foundation for subsequent data analysis and processing.

[0046] Step 102, conduct full-process quality monitoring on professional rail transit data and generate data quality indicators.

[0047] In an embodiment of the present invention, in a rail transit system, the quality of data directly affects the accuracy of data analysis and decision-making. Therefore, the goal of step 102 is to perform full-process quality monitoring of the professional rail transit data imported into the data lake and generate corresponding data quality indicators to ensure the accuracy, consistency and integrity of the data. Full-process quality monitoring includes data collection, transmission, processing, storage and use. In the data collection stage, the system will perform preliminary verification and cleaning of the collected data. For example, the system will check whether the speed and acceleration in the driving data are within a reasonable range, whether the passenger entry and exit time in the passenger flow data is reasonable, and whether the power consumption in the energy consumption data is consistent with expectations. For data that does not meet the requirements, the system will mark it and trigger the corresponding processing flow. In the data transmission stage, the system will monitor the data transmission process to ensure the integrity and reliability of the data. For example, the system will detect the loss rate and transmission delay of the data packet. If a problem is found, it will automatically retransmit or take other error correction measures. In the data processing and storage stage, the system will monitor the data conversion and storage process to ensure the correctness and consistency of the data. For example, the system will check whether the format and type of the converted data are correct and whether the stored data is complete. During the data usage phase, the system will monitor the data usage process to ensure the security and privacy of the data. For example, the system will control the access rights to the data to ensure that only authorized users can access and use the data. At the same time, the system will also record and audit the use of the data to ensure that the use of the data complies with relevant regulations and standards. Through quality monitoring throughout the entire process, the system can generate a series of data quality indicators. These indicators can include data integrity, accuracy, consistency, timeliness, etc. For example, the system can generate a data integrity indicator to indicate the proportion of missing values ​​in the data; generate a data accuracy indicator to indicate the proportion of erroneous values ​​in the data; generate a data consistency indicator to indicate the proportion of inconsistent values ​​in the data; generate a data timeliness indicator to indicate the speed at which the data is updated.

[0048] For example: Assume that the system needs to process driving data, passenger flow data and energy consumption data from 50 stations every day. Through full-process quality monitoring, the system found that 1% of the driving data records had missing values, 2% of the passenger flow data records had duplicate values, and 3% of the energy consumption data records had abnormal values. Through automatic cleaning and processing, the system improves the data integrity index to 99.9%, the accuracy index to 99.8%, the consistency index to 99.7%, and the timeliness index to 95%. These data quality indicators provide a reliable data basis for the subsequent calculation of rail transit indicators.

[0049] Step 103, calculating rail transit indicators based on data quality indicators.

[0050] In an embodiment of the present invention, in a rail transit system, rail transit indicators are an important basis for measuring operational efficiency and service quality. Therefore, the goal of step 103 is to calculate various rail transit indicators based on data quality indicators to provide a reliable reference for subway operation decisions. The calculation of rail transit indicators needs to consider multiple factors, including driving, passenger flow, energy consumption and other data, and is calculated by the system.

[0051] In the actual application process, the system will screen and process the original data based on the data quality indicators generated in step 102. High-quality data will be used for the calculation of rail transit indicators, while low-quality data will be filtered or further processed. For example, the system can eliminate data records with error values ​​exceeding a certain threshold based on the accuracy index of the data; and complete data records with fewer missing values ​​based on the completeness index of the data. Then, the system will calculate various rail transit indicators according to different needs. These indicators may include operating efficiency indicators, service quality indicators, energy consumption indicators, etc. For example, operating efficiency indicators may include train punctuality, average driving interval, train running speed, etc.; service quality indicators may include passenger flow, passenger satisfaction, passenger complaint rate, etc.; energy consumption indicators may include unit energy consumption, energy utilization rate, etc. The system will calculate the screened high-quality data according to the predetermined algorithm and model to obtain the specific values ​​of each indicator. During the calculation process, the system will use various data analysis and modeling techniques, such as statistical analysis, machine learning, deep learning, etc. Statistical analysis can be used to calculate basic statistical indicators, such as mean, median, standard deviation, etc.; machine learning can be used for prediction and classification, such as predicting passenger flow trends and classifying passenger behavior patterns; deep learning can be used for complex data analysis and modeling, such as establishing energy consumption prediction models and optimizing traffic scheduling plans.

[0052] For example: suppose the system needs to calculate the operating efficiency index of a subway line. By analyzing the driving data, the system can calculate the train punctuality rate (e.g. 98%), average train interval (e.g. 5 minutes), train speed (e.g. 50 kilometers per hour), etc. of the line. These indicators can help subway operation decision makers understand the operation of the line, find potential problems, and formulate corresponding improvement measures. For example, if the train punctuality rate is low, the operation decision makers can adjust the driving scheduling plan, optimize the train operation schedule, and improve the punctuality rate.

[0053] Step 104 , providing a user interaction interface to access, analyze and display rail transit indicators.

[0054] In the embodiment of the present invention, in the rail transit system, the user interaction interface is an important tool for data analysis and display, which provides users with a convenient access path to help users understand and use rail transit indicators. The goal of step 104 is to design and implement an efficient and friendly user interaction interface that enables users to easily access, analyze and display various rail transit indicators.

[0055] In actual application, the user interaction interface needs to have good usability and ease of use. The system should adopt an intuitive interface design to ensure that users can easily get started and operate. For example, the interface should use a clear navigation structure and icons to help users quickly find the required functions and information; the interface should use a concise and clear layout and color scheme to reduce the user's cognitive burden; the interface should provide detailed help documents and operation guides to help users solve problems in use. The user interaction interface needs to provide a variety of data display and analysis tools. The system should support a variety of data visualization methods, such as line charts, bar charts, pie charts, heat maps, etc., to help users intuitively understand and analyze rail transit indicators. For example, users can view the changing trend of train punctuality through a line chart, compare passenger flow on different lines through a bar chart, and analyze energy consumption distribution through a heat map. The system should also support a variety of data analysis tools, such as filtering, sorting, aggregation, drilling, etc., to help users deeply analyze and mine data. For example, users can filter data for a specific time period through the filtering function, find the highest and lowest indicators through the sorting function, calculate the total amount and average through the aggregation function, and view detailed sub-indicators through the drilling function. In addition, the user interface needs to have flexible customization and expansion capabilities. The system should allow users to customize the data display and analysis interface according to their needs. For example, users can select and arrange different indicator modules according to their focus to create a personalized dashboard; users can select and set different chart types and styles according to their preferences to create personalized data reports. The system should also support integration and expansion with other systems, such as integration with subway operation management systems, customer service systems, energy management systems, etc., to achieve data sharing and linkage.

[0056] For example: suppose a subway operating company needs to monitor and analyze the operation of multiple lines. Through the user interface, operational decision makers can easily access and view the operating efficiency indicators, service quality indicators, and energy consumption indicators of each line. Through the line chart, they can view the trend of train punctuality of each line and find out which lines have low punctuality problems; through the bar chart, they can compare the passenger flow of different lines and understand the peak hours and main lines of passenger travel; through the heat map, they can analyze the energy consumption distribution of each line and find out the lines and stations with high energy consumption. Through these data display and analysis tools, operational decision makers can fully understand the operation of the subway, find problems in time, and formulate corresponding improvement measures.

[0057] In the process of executing the whole process quality monitoring and generating data quality indicators, i.e., step 102, and calculating the rail transit indicators, i.e., step 103, task coordination and resource scheduling are continuously performed to ensure automated operation according to predetermined strategies and priorities.

[0058] In the process of performing full-process quality monitoring and calculating rail transit indicators, the system needs to carry out continuous task coordination and resource scheduling to ensure that various tasks can be carried out smoothly and computing resources can be reasonably allocated. For example:

[0059] Task coordination: The system needs to manage and coordinate different data processing and analysis tasks to ensure that they proceed as planned. For example, the system needs to determine the order of data collection, transmission, processing, storage, and analysis to avoid task conflicts and delays.

[0060] Resource Scheduling: The system needs to dynamically adjust the allocation of computing resources to ensure that each task can be executed efficiently. For example, when a task requires a large amount of computing resources, the system will prioritize the allocation of resources to that task; when some tasks are completed, the system will allocate the released resources to other tasks.

[0061] The system needs to coordinate tasks and schedule resources according to predetermined strategies and priorities. The predetermined strategy refers to the rules and processes that the system follows when processing and analyzing data. For example:

[0062] Data quality monitoring strategy: The system may prioritize data from key data sources to ensure the high quality of this data.

[0063] Data analysis strategy: The system may prioritize the calculation of certain important rail transit indicators, such as train punctuality and passenger flow.

[0064] Priority refers to the relative importance of different tasks. The system will allocate resources based on the priority of the task to ensure that important tasks can get sufficient computing resources. For example:

[0065] High-priority tasks: such as real-time data monitoring and key indicator calculation, the system will prioritize resource allocation to ensure that these tasks can be completed in a timely manner.

[0066] Low priority tasks: such as historical data analysis and auxiliary indicator calculations, the system will process these tasks when there are sufficient resources.

[0067] Through continuous task coordination and resource scheduling, the system can ensure that full-process quality monitoring and rail transit indicator calculations are automatically operated according to predetermined strategies and priorities, improve the data processing and analysis efficiency of the rail transit system, and ensure the accuracy and timeliness of operational decisions.

[0068] The intelligent rail transit data management and analysis solution provided by the embodiment of the present invention receives and imports professional rail transit data into a data lake; performs full-process quality monitoring on professional rail transit data and generates data quality indicators; calculates rail transit indicators based on data quality indicators; and provides a user interaction interface to access, analyze and display rail transit indicators; wherein, in the process of performing the steps of performing full-process quality monitoring and generating data quality indicators and the step of calculating rail transit indicators, task coordination and resource scheduling are continuously performed to ensure automated operation according to predetermined strategies and priorities.

[0069] The embodiment of the present invention can realize real-time access and processing of data through data lake technology, solve the problem of data delay in the traditional ETL process, and meet the needs of real-time monitoring and decision-making. Implementing a full-process quality monitoring mechanism and generating data quality indicators effectively solves the errors or inconsistencies that may occur in the ETL process, and improves the accuracy of rail transit indicator calculation. The application of data lake and artificial intelligence technology enables the system to better cope with the increase in data volume and complexity caused by the expansion of the rail transit network, and improves the scalability and flexibility of the system. Through the intelligent rail transit data platform, data in different professional fields can be more easily standardized and integrated, which improves the depth and breadth of comprehensive analysis. When calculating data in multiple professional fields, through continuous task coordination and resource scheduling, computing resources can be dynamically adjusted according to actual needs, avoiding resource waste or computing bottlenecks. The full-process data governance from data collection, transmission, processing, storage to use is realized, ensuring the high quality and consistency of data, which helps to improve the reliability and efficiency of the overall system. A user interaction interface is provided to facilitate users to access, analyze and display rail transit indicators, improving user experience and system usability. The operational indicators obtained through systematic calculation and analysis can timely reflect the road network situation, help subway operation decision-makers to formulate better operation plans, and improve the operational efficiency and service quality of rail transit.

[0070] In summary, the intelligent rail transit data management and analysis solution has significantly improved the real-time, quality and system scalability of data processing through technological innovation and system optimization compared to the traditional rail transit indicator calculation system. At the same time, it has strengthened the integration of cross-professional data and the dynamic allocation of computing resources, realized full-process data governance, and provided strong support for rail transit operation decisions, thereby improving overall operational efficiency and service quality, and improving passengers' travel experience.

[0071] In an exemplary embodiment of the present invention, an implementation method of receiving and importing professional rail transit data into a data lake is to capture the professional rail transit data of a data source through a data access proxy mode; and / or, to access the professional rail transit data by listening to a specific source server port through a listening mode; to perform format conversion and decryption processing on the professional rail transit data; and to store the professional rail transit data after format conversion and decryption processing in the data lake.

[0072] The data access proxy mode is a way to capture data from a data source through a proxy server. The proxy server acts as a middleman in the data flow and receives data from different data sources. For example, various sensors, monitoring equipment, ticketing systems, etc. in the rail transit system can send data to the data lake through the proxy server. The proxy server can perform preliminary processing on the data, such as data verification and format conversion, to ensure the consistency and integrity of the data. For example, suppose that passenger flow sensors are installed at multiple stations on a subway line. These sensors monitor the entry and exit of passengers in real time. Through the proxy server, the data from these sensors is captured and aggregated on the proxy server, which converts the format of the data and transmits it to the data lake.

[0073] The monitoring mode directly receives data sent by the data source by monitoring a specific server port. In this mode, the system starts monitoring on the predetermined server port and waits for the data source to actively send data. This method is suitable for scenarios where real-time data reception is required, such as train operation status monitoring and real-time energy consumption monitoring. The monitoring mode can ensure the timeliness and accuracy of data and reduce data latency. For example, suppose that the energy consumption monitoring equipment in the rail transit system monitors the energy consumption of the train in real time. These devices send data to the system's monitoring port through the network. After the system monitors the data, it processes it immediately and stores it in the data lake.

[0074] Whether the data is accessed through the data access proxy mode or the monitoring mode, it may have different formats and encryption methods. In order to ensure the consistency and security of the data in the data lake, the data needs to be format converted and decrypted. Format conversion is to convert the original format of the data source into a unified standard format of the system, and decryption is to decrypt the encrypted data to ensure the readability and security of the data. For example, some data sources use the Extensible Markup Language (XML) format to transmit data, while the system uses the JavaScript Object Notation (JSON) format for storage and analysis. The system needs to convert the received XML data into JSON format. In addition, some data sources will encrypt the data for transmission for security reasons. The system needs to decrypt the data after receiving it to ensure the integrity and security of the data.

[0075] The data that has been format converted and decrypted needs to be stored in the data lake. A data lake is a storage system that can store massive amounts of structured and unstructured data. The advantage of a data lake is its high scalability and flexibility. It can store large amounts of data from different data sources and support subsequent data analysis and processing. For example, the system stores train operation data, energy consumption data, passenger flow data, etc. that have been format converted and decrypted in the data lake. The data in the data lake can be analyzed and calculated by the system at any time to generate various operational indicators and reports, helping operational decision makers to develop optimization plans.

[0076] The embodiments of the present invention use data access proxy mode and monitoring mode, so that the system can flexibly respond to the access requirements of different data sources and data types, ensuring the comprehensiveness and real-time nature of the data. Through format conversion and decryption processing, the system can ensure the consistency and security of access data, and ensure the unified management and secure storage of data in the data lake. The use of the data lake enables the system to efficiently store and manage massive amounts of data, and supports subsequent data analysis and processing, improving the overall performance and expansion capabilities of the system. Through efficient data access and processing, the system can generate accurate and timely operational indicators, help operational decision makers formulate scientific decision-making plans, and improve the operational efficiency and service quality of the rail transit system.

[0077] In an exemplary embodiment of the present invention, one implementation method of generating data quality indicators is to detect the timeliness and relevance of code table data; check the field integrity, real-time, cross-professional association consistency and update cycle consistency of master data; generate the data quality indicators of the code table data and the master data according to the available threshold of the indicator set by the user. Code table data refers to auxiliary data used in the system to describe and define other data, such as dictionaries, coding tables, etc. Detecting the timeliness and relevance of code table data is the first step in generating data quality indicators. Timeliness refers to whether the code table data is updated in a timely manner and whether it reflects the latest business needs and actual conditions; relevance refers to whether the association between code table data and business data is correct, and whether there is a missing or wrong association. For example, the station code table (code table data) used in a subway system needs to be updated regularly to reflect newly built or closed stations. The system needs to detect the update date of this code table to ensure its timeliness. In addition, the system also needs to check the association between the station code table and the actual operation data to ensure that the code of each station has a corresponding record in the operation data.

[0078] Master data is the most important business data in the system, involving multiple professional fields such as operations, passenger flow, energy consumption, etc. The quality check of master data includes the following aspects:

[0079] Field integrity: Ensure that each field in each data record has the correct value and is not missing. For example, each record in the train operation data should contain all necessary fields such as train number, operation time, station information, etc.

[0080] Real-time: Ensure that data can be updated in real time and reflect the latest situation. For example, passenger flow data should be able to record passengers entering and leaving the station in real time to avoid data delays.

[0081] Cross-disciplinary association consistency: Ensure that the associations between data in different professional fields are correct and consistent. For example, the association between energy consumption data and train operation data should be accurate, and the energy consumption data for a certain period of time should match the corresponding train operation data.

[0082] Update cycle consistency: Ensure that the update cycle of the same type of data is consistent, and that data from different sources can be updated in a timely manner within the same time range. For example, passenger flow data for all stations should be updated at the same time interval to avoid data lag at some stations.

[0083] For example, a subway operation system needs to ensure the consistency between train operation data, station passenger flow data and energy consumption data. The system needs to check the field integrity of these data to ensure that all records have necessary fields; check the real-time nature of the data to ensure real-time updates; check the consistency of associations between different data to ensure that the associations are correct; and check the update cycle of the data to ensure that the data is updated within the same time range.

[0084] The system also needs to generate specific data quality indicators based on the indicator availability threshold set by the user. The indicator availability threshold is a standard set by the user based on business needs and actual conditions to evaluate data quality. For example, the user can set the data integrity threshold to 99%, the real-time threshold to 90%, the cross-professional association consistency threshold to 95%, etc. The system will evaluate the code table data and master data based on these thresholds and generate data quality indicators. For example, a subway operating company sets a real-time threshold of 90%. The system will detect the update of all real-time data. If more than 10% of the data fails to be updated within the specified time, the data real-time is considered unqualified. Similarly, the field integrity threshold set by the user is 99%. The system will check the field integrity of all data records. If more than 1% of the records have missing fields, the data integrity is considered unqualified.

[0085] Through comprehensive detection of code table data and master data, the embodiment of the present invention can accurately evaluate the timeliness, relevance, field integrity, real-time, cross-professional association consistency and update cycle consistency of the data to ensure high data quality. Through real-time monitoring and regular inspections, the system can promptly discover problems in the data, such as data missing, association errors, update delays, etc., and take corresponding corrective measures to improve the accuracy and reliability of the data. Users can set indicator availability thresholds according to actual business needs, and the system will generate specific data quality indicators based on these thresholds to ensure that data quality assessments meet business needs and improve the flexibility and adaptability of the system. High-quality data is the basis for accurate operational decisions. By ensuring the high quality of data, the system can provide accurate and reliable data support for operational decisions and improve the accuracy and timeliness of operational decisions.

[0086] In an exemplary embodiment of the present invention, an implementation method for calculating rail transit indicators based on data quality indicators is as follows: for single data source indicators in rail transit indicators, data quality indicators that have passed full-process quality monitoring are used as single data source indicators; for composite data source indicators in rail transit indicators, the consistency of the pulling cycle of multi-source data is checked and ensured; and composite data source indicators are calculated according to predefined calculation logic.

[0087] For the single data source indicators in rail transit indicators, the system must first ensure the quality of the data. Single data source indicators refer to data indicators extracted from an independent data source, such as the passenger flow of a single station, the energy consumption of a certain train, etc. The data quality indicators that have passed the full process quality monitoring will be used as single data source indicators. This means that only data that has undergone strict data quality testing and meets the standards of timeliness, integrity, and relevance will be used to calculate single data source indicators. For example, suppose a subway system needs to calculate the average daily passenger flow of each station. The system will first conduct full process quality monitoring of the passenger flow data of all stations to ensure the timeliness and integrity of the data. If the data of a station is missing or delayed, the system will process and supplement it accordingly, and only qualified data will be used to calculate the average daily passenger flow indicator.

[0088] For composite data source indicators in rail transit indicators, the system needs to process data from multiple data sources and ensure the consistency of the pulling cycle of these data. Composite data source indicators refer to indicators calculated by combining data from multiple different data sources, such as comprehensive energy consumption in a certain period of time, operating efficiency of the entire line, etc. In order to ensure the accuracy of the calculation results, the system needs to check and ensure the consistency of the pulling cycle of multi-source data, that is, data from different data sources are pulled synchronously within the same time range. For example, suppose the system needs to calculate the comprehensive energy consumption and operating efficiency within a certain period of time, which requires combining the operating data and energy consumption data of the train. The system must first ensure that these data are pulled in the same time period and processed synchronously. If the time range of energy consumption data and operating data is inconsistent, the system will make corresponding adjustments to ensure data synchronization. Then, the system will combine these data to calculate the comprehensive energy consumption and operating efficiency according to the predefined calculation logic.

[0089] Predefined calculation logic refers to the calculation rules and formulas that have been defined in the system design phase, which are used to calculate composite data source indicators. These calculation logics can be customized and optimized according to business needs and actual conditions. For example, the system can define different calculation formulas to calculate comprehensive energy consumption, operating efficiency, passenger flow and other indicators to ensure the accuracy and reliability of the calculation results. For example, assuming that the system needs to calculate the operating efficiency within a certain period of time, the predefined calculation logic may include the following steps: first calculate the operating time and energy consumption of each train, then summarize the data of all trains, and calculate the comprehensive operating efficiency of the entire line. The system will automatically calculate according to these predefined logics to generate the final rail transit indicators.

[0090] Through strict full-process quality monitoring and the use of qualified data quality indicators, the embodiment of the present invention can ensure the data quality of rail transit indicators and improve the reliability and accuracy of the data. By checking and ensuring the consistency of the pulling cycle of multi-source data, the system can effectively process composite data source indicators, ensure that data from different data sources are pulled synchronously within the same time range, and improve the accuracy of the calculation results. The predefined calculation logic standardizes and automates the calculation process of the system, reduces human intervention, and improves calculation efficiency and consistency. The predefined calculation logic can be customized and optimized according to business needs to ensure that the system can flexibly adapt to different business scenarios and needs, and improve the adaptability and flexibility of the system.

[0091] In an exemplary embodiment of the present invention, an implementation method of providing a user interactive interface to access, analyze and display rail transit indicators is to integrate the industry big model to understand the user's query intention; guide the user to provide the time period, qualifiers, statistical granularity and atomic indicators of the query indicator; build a complete derived indicator query based on the information provided by the user; execute the query and present the rail transit indicators in a visual manner.

[0092] In actual application, the system integrates industry big models, which are trained to accurately understand the user's query intent. The industry big model not only includes natural language processing capabilities, but also combines professional knowledge in the field of rail transit to accurately parse the user's query needs. For example, when a user enters the query "passenger flow changes at each station last month", the system can recognize keywords such as "last month", "each station", "passenger flow changes", and understand that the user needs monthly statistics of passenger flow data at each station.

[0093] To ensure the accuracy and completeness of the query, the system will guide the user to provide detailed query information step by step. The user interface is intuitively designed to guide users to enter the required query conditions in a friendly manner, including:

[0094] Time period: Users can select the time range for the query, such as one day, one week, one month or one year.

[0095] Qualifiers: Users can specify specific search criteria, such as specific stations, trains or lines.

[0096] Statistical granularity: Users can select the statistical granularity of the data, such as hourly, daily, monthly or yearly.

[0097] Atomic indicators: Users can select basic atomic indicators, such as passenger flow, energy consumption, train punctuality, etc.

[0098] For example, assuming that the user wants to query "weekly passenger flow changes in the first quarter of this year", the system will guide the user to select the time period as "first quarter", the qualifier as "all sites", the statistical granularity as "weekly", and the atomic indicator as "passenger flow".

[0099] After obtaining the detailed query information provided by the user, the system will build a complete derived indicator query based on this information. Derived indicator query refers to combining multiple atomic indicators to generate more complex comprehensive indicators. For example, if the user provides a query requirement of "weekly passenger flow changes in the first quarter of this year", the system will count the atomic indicator "passenger flow" by week and calculate the changes to generate a complete derived indicator query.

[0100] The system will query the derived indicators constructed, perform query operations, and obtain the required rail transit data. The query results are presented to users in an intuitive and visual way. Common visualization forms include charts, dashboards, maps, etc. For example, for the query result of "weekly passenger flow changes in the first quarter of this year", the system can generate a line chart to show the weekly passenger flow change trend, helping users to intuitively understand the data changes.

[0101] For example, suppose a staff member of a subway company's operations department needs to know the energy consumption of a specific line in the past month in order to evaluate energy efficiency and find energy-saving space. The staff member enters the query requirement through the user interface:

[0102] Time period: Select Past month.

[0103] Qualifier: Select a specific line A.

[0104] Statistical granularity: Select statistics by day.

[0105] Atomic Metrics: Select Energy Consumption.

[0106] The system understands the staff's query intention through the integrated industry big model and guides them to provide query conditions step by step. After completing the input, the system builds a complete derived indicator query based on this information, namely "daily energy consumption of a specific line A in the past month." After executing the query, the system generates a bar chart showing the daily energy consumption of line A in the past month. Different colors in the chart represent the energy consumption of different sections, helping staff to intuitively identify high-energy consumption sections and then formulate energy-saving measures.

[0107] By integrating the industry big model and the friendly user interaction interface, the embodiment of the present invention enables the system to accurately understand the user's query intention, provide step-by-step guidance, and improve the user experience. Guide users to provide detailed query conditions, ensure the accuracy and completeness of the query, and generate reliable rail transit indicators. Automatically construct derived indicator queries based on the information provided by the user to improve the efficiency and accuracy of data analysis. Display query results in a visual way, so that users can intuitively and quickly understand the data and assist in decision-making. The system can flexibly adjust the query conditions and display formats according to user needs to meet the needs of different scenarios and improve the adaptability and flexibility of the system.

[0108] In an exemplary embodiment of the present invention, the following implementation method is also included: classifying the received professional rail transit data into real-time indicator data and offline indicator data; establishing a fast processing channel for the real-time indicator data, and batch processing the offline indicator data.

[0109] In actual application, the system will classify and process the professional rail transit data received. The amount of data in the rail transit system is huge and diverse, including train operation data, passenger flow data, energy consumption data, financial data, etc. In order to process this data more effectively, the system will divide it into two categories: real-time indicator data and offline indicator data.

[0110] Real-time indicator data: These data need to be processed and generated into indicators in a short time to reflect the operation of the rail transit system in a timely manner and support real-time monitoring and decision-making. For example, the operation status of the train, the real-time passenger flow of the station, etc.

[0111] Offline indicator data: These data do not need to be processed immediately, and can be processed in batches during off-peak hours to generate long-term statistics and analysis indicators. For example, monthly financial statements, annual energy consumption statistics, etc.

[0112] To ensure that real-time indicator data can be processed quickly, the system will establish a fast processing channel for it. This means that real-time data will immediately enter the fast processing channel after being received, and will be quickly parsed, processed, and calculated to generate real-time indicators. For example, suppose the system needs to process real-time passenger flow data for each station on a certain subway line. Real-time passenger flow data will be immediately parsed and processed through the fast processing channel to generate real-time passenger flow indicators for each station. These indicators can be used to monitor passenger flow at each station in real time, help operators to dispatch resources in a timely manner, and ensure the smooth operation of the subway system.

[0113] The establishment of a fast processing channel usually includes the following steps:

[0114] 1. Data reception and analysis: Real-time data is received through a high-speed network interface and initially parsed to extract key fields.

[0115] 2. Data processing and calculation: Use efficient algorithms and computing frameworks to process and calculate real-time data and generate real-time indicators.

[0116] 3. Indicator generation and release: The generated real-time indicators are pushed to the real-time monitoring system or display platform for operators to view and use.

[0117] Unlike real-time indicator data, offline indicator data can be processed in batches during off-peak hours. The system will centrally process offline data during appropriate time periods to generate long-term statistics and analysis indicators. For example, the system needs to process the monthly energy consumption data of the subway system. After the monthly energy consumption data is centrally collected at the end of the month, the system will process it in batches during off-peak hours such as at night or on weekends to generate monthly energy consumption reports. These reports can be used for long-term energy consumption analysis and energy-saving decisions, helping subway companies to formulate more scientific energy-saving measures.

[0118] Batch processing of offline indicator data usually includes the following steps:

[0119] 1. Data collection and storage: Collect offline data within a specified period of time and store it in a data lake or data warehouse.

[0120] 2. Data cleaning and conversion: Clean and convert the collected data to ensure data consistency and accuracy.

[0121] 3. Data processing and calculation: Use the big data processing framework to batch process and calculate offline data to generate statistical and analytical indicators.

[0122] 4. Report generation and publication: The generated statistical and analytical indicators are summarized into reports and published to relevant departments or personnel for long-term analysis and decision-making.

[0123] By classifying data into real-time indicator data and offline indicator data and establishing a fast processing channel for real-time data, the embodiment of the present invention can significantly improve the efficiency of data processing and ensure that real-time data can be processed and responded to in a timely manner. Batch processing of offline data is carried out during non-peak hours, which makes full use of the computing resources of the system, avoids resource conflicts and processing bottlenecks during peak hours, and optimizes resource utilization efficiency. By classifying and processing different types of data, the system can more flexibly respond to various data processing needs and improve the flexibility and adaptability of data processing. Through the rapid processing of real-time data and batch processing of offline data, the system can more accurately process and calculate various rail transit indicators and ensure the accuracy and reliability of data processing. The fast processing channel of real-time data supports real-time monitoring and decision-making, and the batch processing of offline data supports long-term statistics and analysis. The system can fully support real-time monitoring and long-term optimization of the rail transit system.

[0124] In an exemplary embodiment of the present invention, an implementation method for continuous task coordination and resource scheduling is to set a timing execution strategy based on the time characteristics of professional rail transit data; build a task execution network based on the dependency relationship between rail transit indicators; dynamically adjust task priority and resource allocation; monitor task execution status and handle abnormal situations.

[0125] The data of the rail transit system has significant time characteristics, and the collection, processing and analysis time of different data types are different. For example, train operation data needs to be processed in real time, while monthly energy consumption data can be processed in batches at the end of each month. Based on these characteristics, the system will set a scheduled execution strategy to ensure that various types of data are processed in the appropriate time period. For example, suppose the system needs to process station passenger flow data and monthly energy consumption data. The system will set a scheduled strategy so that the station passenger flow data is processed once an hour, while the monthly energy consumption data is processed in batches at night at the end of each month. Through this scheduled execution strategy, the system can efficiently utilize resources and avoid resource competition during peak hours.

[0126] There are often dependencies between rail transit data processing tasks. For example, some comprehensive indicators need to be calculated based on multiple basic indicators. The system will build a task execution network based on these dependencies to ensure that tasks are executed in the order of dependencies. For example, suppose the system needs to calculate the comprehensive operating efficiency index of a certain line, which depends on train punctuality, passenger flow and energy consumption data. The system will build a task execution network so that the train punctuality, passenger flow and energy consumption data processing tasks are completed first, and then the comprehensive operating efficiency index is calculated. Through this task execution network, the system can ensure the accuracy and timeliness of various indicator data.

[0127] In order to cope with the ever-changing data processing needs, the system dynamically adjusts the priority of tasks and resource allocation. Priority is adjusted based on the importance and urgency of the task, while resource allocation is optimized based on the current system resource usage. For example, suppose the system needs to process a large amount of passenger flow data and a small amount of energy consumption data during peak hours. The system dynamically adjusts the priority, setting the passenger flow data processing task to a high priority and allocating more computing resources, while setting the energy consumption data processing task to a low priority and allocating fewer resources. Through this dynamic adjustment, the system can ensure that key tasks are completed smoothly during peak hours while avoiding resource waste.

[0128] The system will continuously monitor the execution status of each task to ensure that the task is carried out as planned. If any abnormality is found in the execution of a task, such as task delay or failure, the system will handle it in a timely manner and take measures to recover or retry to ensure the continuity and reliability of data processing. For example, suppose the system finds that a network anomaly causes data transmission failure when processing passenger flow data at a certain site. The system will detect this anomaly and immediately retry or switch to an alternative data source to ensure that passenger flow data can be processed and updated in a timely manner. Through this exception handling mechanism, the system can ensure the reliability of data processing and reduce business interruptions caused by anomalies.

[0129] The embodiments of the present invention use a timing execution strategy and a task execution network, so that the system can reasonably arrange data processing tasks, avoid resource conflicts and task delays, and significantly improve data processing efficiency. Dynamically adjust task priorities and resource allocation so that system resources can be optimally allocated according to actual needs, avoid resource waste and processing bottlenecks, and improve the overall performance of the system. By continuously monitoring the task execution status and handling abnormal situations, the system can respond to various anomalies in a timely manner, ensure the continuity and reliability of data processing, and reduce business interruptions and data loss. Through reasonable task scheduling and resource allocation, the system can simultaneously support real-time data processing and long-term data analysis to meet the diverse needs of the rail transit system. Efficient and reliable data processing capabilities and flexible task scheduling mechanisms can significantly improve user satisfaction, provide strong support for operational decisions, and improve the service quality of the rail transit system.

[0130] Based on the above description of an embodiment of an intelligent rail transit data management and analysis method, the following introduces an intelligent rail transit data comprehensive management and analysis solution based on a data lake and AI. This intelligent rail transit data comprehensive management and analysis solution based on a data lake and AI can be applied to an intelligent rail transit system, referred to as this system. Figure 2 , showing a schematic diagram of the structure of a smart rail transit data comprehensive management and analysis solution based on data lake and AI in an embodiment of the present invention. The smart rail transit data comprehensive management and analysis solution based on data lake and AI is a rail transit indicator comprehensive calculation system that integrates all professional rail transit data and integrates data governance, indicator calculation, and query optimization. The smart rail transit data comprehensive management and analysis solution based on data lake and AI mainly includes the following six modules.

[0131] Module 1: Data Access Module

[0132] The data access module is the interface for the system to interact with external data sources. It is responsible for importing data in professional fields such as driving, passenger flow, energy consumption, and finance into the system in real time or in batches.

[0133] Different data source platforms often provide data using different protocols. This intelligent rail transit data comprehensive management and analysis solution based on data lake and AI is adapted to the access protocols of major manufacturers and can seamlessly access the data of most manufacturers on the market. There are two ways to access data: data access proxy mode and monitoring mode.

[0134] Data access agent mode: deployed near each data source to capture data and send it to the data lake. Figure 3 , showing a schematic diagram of the principle of data access using a data access proxy mode in an embodiment of the present invention. Figure 3 The data sources in the data lake can be the subway automatic fare collection system (Automatic Fare Collection, AFC), the rail transit network command and dispatch system (Train Control Center, TCC), and the subway automatic fare collection system clearing center (Automatic Clearing Center, ACC). Each data source can be configured with its own data access module, which is similar to an agent that transfers data from each data source to the data lake.

[0135] Monitoring mode: By monitoring a specific source server port, access the latest data in real time.

[0136] The data access agent mode extracts data from the data source according to a predetermined scheduling or triggering mechanism. The data adapter is used by the agent to receive data. The data structure and data extraction cycle can be customized according to the characteristics and requirements of different data. The data adapter converts the format of the received data. Most of the accessed data is encrypted data. In this step, the data needs to be decrypted into plain text format to adapt to the storage requirements of the data lake.

[0137] Module 2: Data Storage Module

[0138] The data storage module mainly uses the data lake as the core storage component of the system to store and manage raw data and processed data from different data sources, and provides data retrieval and management functions. In order to ensure that bloodline tracing can be carried out later, data at each stage will be saved. Data can be divided into real-time data and offline data according to its real-time nature. The working principle is that the data storage module receives data from the data access module and stores it persistently. The data indexing service of the data storage module indexes the stored data to support efficient data retrieval. The data storage module is the storage bottom layer, and both modules three and four run on module two.

[0139] Module 3: Quality Management Module

[0140] The data in the data lake can be roughly divided into master data and stop table data. Master data is data that is continuously generated, while stop table data is relatively unchanged. For example, in the subway passenger flow data, every time a passenger swipes his card to exit the station, the system will generate a piece of data containing a detailed record of the passenger's riding process, namely the master data. Relatively unchanged data, such as line tables and station tables, are updated slowly and used frequently. This type of data is called stop table data (for example, the station ID of a passenger getting off the train in the master data is 0124. If you want to know the Chinese name of the station corresponding to 0124, you need to query the station table together). The quality governance module is responsible for quality inspection of the master data in the data lake, implementing data quality monitoring throughout the process, and generating data quality indicators. This is a crucial step in this system. Building data quality indicators in advance will help to divide the boundaries of responsibility. When it is found that the indicator results are inaccurately calculated, it is convenient to judge whether it is a data quality problem or an indicator calculation logic problem based on the data quality indicators, so as to avoid the problem location difficulty caused by the long data chain.

[0141] Data quality indicators are divided into primary data quality indicators and reference data quality indicators according to different data types:

[0142] 1. Reference data quality indicators: code table data timeliness indicators

[0143] Some meter data is time-sensitive. For example, in the ticket table, some ticket products are short-term products. If the data time is beyond the product life cycle, it is necessary to clean up the expired ticket product data in time, otherwise expired tickets may appear in the indicator statistics. This indicator is available for some meter with activation time and end time fields, and the operation cycle is set to run once a day.

[0144] 2. Reference data quality indicators: code table correlation indicators

[0145] The received data is usually only the master data of each profession, while the reference data (meter data) is maintained separately in the data warehouse. Because meter data may be reused by multiple professions, it must be redesigned in combination with the characteristics of multiple professions, and the fields and associations required for all professional data using this meter must be considered. For example, the section information table may be used in calculating passenger flow indicators or driving indicators. Therefore, it is best not to have the section information table come from any system, but to be maintained separately in the data warehouse. Then, when receiving new passenger flow master data and driving master data, it is necessary to check the consistency of the section ID field in the master data with the section ID field in the section information table in the data warehouse. If 100% correlation is detected, it means that there is no problem with the data on both sides. If 90% correlation is detected, it is necessary to check according to the daily report. Perhaps a new station has been opened on a certain line, causing the local meter data to not be updated in time.

[0146] This function is particularly important for energy consumption professional configuration point inspection. Configuration points are all points for collecting data in the subway system. There are configuration points in any device. For example, in an elevator, there are points for collecting the current startup status of the elevator, the current running speed of the elevator, and the current load of the elevator. There may be many configuration points in a device at the same time. They will collect different types of data and send data to the collection end in a scheduled or other way. For example, the configuration points on the train usually send data to the TCC system, and the lights in the station and the configuration point information in the elevator are usually sent to the integrated monitoring system. Configuration points are important devices for collecting data in the rail system. There are huge numbers of them, and they may be added, damaged, or replaced every day. The changes are relatively large. Therefore, the daily point updates need to be synchronized in time to avoid deviations in indicator data. Otherwise, the lack of a lighting configuration point may lead to errors in the overall energy consumption statistics.

[0147] 3. Master data quality indicators: missing and error indicators of important fields in master data

[0148] Missing important fields in master data usually leads to inaccurate aggregation results. Therefore, the system adopts the following approach: if the field is an enumeration value, the system will require a list of all possible enumeration types of the field. When a value other than an enumeration value appears in the data, the data will be pushed out through logs. Users can set the available threshold of the indicator. For example, if the qualified rate of a field of the data (the value meets the requirements and is not empty) is greater than 90%, it is available. If it is less than this value, the data needs to be manually reviewed and the cause of the data error is found for easy maintenance. The indicator will be automatically scheduled for execution when each master data is pulled into the data lake. When the data qualified rate is lower than the threshold, an alarm will be issued immediately to notify the operation and maintenance personnel to handle it.

[0149] 4. Master data quality indicators: Master data real-time indicators

[0150] Different indicators or fusion analysis have different requirements for the real-time performance of master data. When some master data of the system is pulled periodically, it may contain supplementary data, that is, data before the current period, and this part of data is usually not counted in the indicators of the data in this period. This indicator is mainly used to confirm that the data released by the system with high real-time performance may be real-time. Therefore, when such data is interrupted, it is very likely that there is a problem with the source system. The direct impact is that the indicators based on such data as master data cannot be generated. Therefore, the real-time update of master data needs to be monitored as an indicator. In order to find and correct problems in time.

[0151] 5. Master data quality indicators: consistency detection of cross-disciplinary related fields

[0152] Some cross-disciplinary indicators require the association of two sets of master data, and the matching degree of the associated field values ​​also needs to be regarded as a very important indicator.

[0153] For example, to calculate the indicator "Energy consumption distribution of each vehicle in the line network": the data required are: A. Daily operating vehicles; B. Daily vehicle energy consumption. The master data of two different majors have the same field "Vehicle ID";

[0154] To associate two tables, they must have consistent vehicle ID fields. This system has the function of detecting the difference of associated fields in advance, and can automatically analyze the correlation between the associated fields of the two tables before calculating the indicators, that is, how much data cannot be associated between the two tables AB, and whether there are more unassociated data in table A or table B, so as to know in advance the consistency of the associated field data of different systems. Ideally, the vehicle IDs of the two tables AB can all correspond, but there is also a situation where system A also records the data of operation and maintenance vehicles in the system, but system B does not record it. Then, when the two tables are associated, the vehicle ID of A will have more operation and maintenance vehicle IDs than the vehicle ID of B. At this time, the operation and maintenance personnel need to be more specific about the indicator requirements and confirm whether the rescue vehicle needs to be included in the indicator. This is to avoid problems such as the difference in the source system code table that is difficult to detect in the final indicator.

[0155] 6. Master data quality indicators: cross-disciplinary related data update cycle consistency detection indicators

[0156] When the master data of two disciplines are to be associated, the pulling cycles of the two master data are required to be as consistent as possible.

[0157] For example, to calculate the indicator "Energy consumption distribution of each vehicle on the line network": the data needed are: A, daily operating vehicles; B, daily vehicle energy consumption. These two types of data come from the bicycle system and the energy consumption system respectively. The access to the data of these two systems has different update rates. For example, the data in table A is updated every time a vehicle arrives at a station, and the data in table B is updated every 30 minutes. To associate these two types of data, the data with the longest update cycle shall prevail.

[0158] Before calculating the indicator, the time difference between the arrival of the two metadata must be checked (assuming that the source system time of the two data is consistent). For example, if the composite indicator is calculated 29 minutes after the last update of table B, the data time of the operating vehicles in table A will be 29 minutes ahead of table B. The indicator result will be missing the data of the operating vehicles for these 29 minutes.

[0159] Module 4: Indicator calculation module

[0160] The indicator calculation module calculates various indicators of rail transit based on the processed data using specific algorithms and logic. It should be emphasized here that each indicator will list the requirements for master data in advance, distinguishing between single data source indicators (such as passenger volume, which only uses passenger flow master data) and composite data source indicators (such as the distribution of energy consumption of each vehicle on the line network, which uses the master data of both driving and energy consumption systems). There are several points to note:

[0161] 1. If the indicator is from a single data source, it only needs to be run directly after the data pulling and data quality inspection indicators are qualified. If it is multi-source data, it is necessary to consider whether the pulling cycles of the main data from the two sources are consistent or close, otherwise it will lead to many omissions in the calculated data.

[0162] 2. The calculation script of the composite indicator must follow the preceding indicator script, otherwise the calculation result will be empty.

[0163] In addition, the indicator calculation part refers to another set of rail transit standard modeling and indicator calculation system, which will not be elaborated here.

[0164] Module 5: Task Scheduling Module

[0165] In the above modules three and four, many computing tasks are formed, and there are often connections and logical sequences between tasks. This module is used to schedule all tasks to form an automatically operated system, and its characteristic is that a variety of scheduling strategies that are compatible with rail transit business are designed. The time requirements or supply characteristics of rail transit data are different. There are station passenger flow heat map data with extremely high real-time requirements, passenger flow forecast data for the next five minutes that requires the results to be as accurate as possible, T-1 quick report statistics that are calculated only after the end of the day's operation, and T-7 final report data with additional supplementary content. Among them, T represents the number of days.

[0166] After data is extracted from the data source to the data lake, all subsequent tasks are executed by the task scheduling module. In view of the particularity of rail transit data, there are two points to note when configuring the task scheduling module.

[0167] 1. Scheduled execution: In view of the fact that the rail transit system may not be operated until the next natural day, the ETL process of all offline indicators will not start until the end of the operation day. (Different cities have different end times for the operation day. For example, the operation day in City C is the same as the natural day, while the operation day in City B ends at 2 a.m. the next day after the natural day.) Taking City B as an example, all offline indicator ETL will start after 2 a.m.

[0168] 2. The indicator relationship network of many professions is intricate and complex. Figure 4, shows a schematic diagram of the entity relationship (ER) of a small part of the energy consumption indicators of the embodiment of the present invention. When formulating the task scheduling logic, it must be strictly configured according to the ER diagram. Otherwise, it is impossible to calculate all indicators according to the data flow.

[0169] Figure 4 It includes the history of energy consumption, energy consumption classification and sub-items, history of energy consumption classification and sub-items of metering equipment, equipment, metering equipment, as well as electricity energy consumption code (FK), water energy consumption code (FK), gas energy consumption code (FK), heat energy consumption code (FK), cooling energy consumption code (FK), power energy consumption code (FK) and equipment professional code. Among them, the history of energy consumption includes the household party number (FK), energy consumption code (FK), energy consumption relationship sequence number, start time, relationship name, energy consumption publicity, calculation relationship type code (FK), party type code (FK), measurement unit code (FK), whether it is real-time quantity flag, whether it is enabled flag, end time, data source code (FK). Energy consumption classification and sub-items include energy consumption code, superior energy consumption code (FK), energy consumption name, energy consumption unit (FK), conversion standard coal coefficient, conversion carbon dioxide (CO2) dispatch coefficient, whether it is real-time quantity flag, whether it is enabled flag, data source code (FK), data date. The energy consumption classification and item relationship history of metering equipment includes equipment number (FK), energy consumption code (FK), start date, end date, and data source code (FK). Equipment includes equipment number, equipment name, equipment description, commissioning date, end of operation date, equipment management status code (FK), equipment type code (FK), equipment professional code (FK), equipment scrapping reason code (FK), whether it is a high energy consumption equipment mark, data source code (FK), and data date. Metering equipment includes equipment number (FK), source meter number, virtual meter mark, meter purpose description, data source code (FK), and data date. The above FK is the abbreviation of foreign key.

[0170] Module 6: Interaction and Visualization Module

[0171] To query the data of a specific derived indicator, users usually need to include the following four elements in their questions (time period, qualifier, statistical granularity, atomic indicator). By introducing a large model, the interactive experience is improved and the usage threshold is lowered. The interactive and visualization module provides users with an intuitive operation interface for accessing, analyzing and displaying calculation results. For users who are not familiar with the system, when using this module. The questions raised may not be accurately matched. At this time, it is necessary to gradually deepen the questions through the guidance of the large model, so as to finally obtain the four elements of the indicators requested by the user, and finally query the specific derived indicators.

[0172] Modifiers Time period Statistical granularity Atomic Index Derived Indicators Line 1 / High-tech Zone Last week Day / Route Passenger volume Statistics of daily passenger traffic in the Hi-Tech Zone area of ​​Line 1 in the past week Full line network This year Month / Ticket Type / Route Transfer volume Statistics of transfer volume of each ticket type in the whole network this year Nationwide In the past year Province GDP Statistics of GDP of each province in the country in the past year Province S First quarter of 2023 City / Month Average Temperature Monthly average temperature statistics of cities in S Province in the first quarter of 2023

[0173] Table 1. Relationship between the four factors and derived indicators

[0174] Based on the above introduction of a comprehensive management and analysis solution for intelligent rail transit data based on data lake and AI, the following introduces a data processing and analysis platform. Figure 5 , showing a flow chart of a data processing and analysis platform of an embodiment of the present invention. Through the data processing and analysis platform, the smart urban rail simultaneously accesses multi-professional data such as passenger flow data, driving data, energy consumption data, and comprehensive monitoring data. Various professional data have different access methods and different requirements for the real-time performance of the data. The introduction of Nifi can make the process simpler. During the data access process, the data update of the source system is continuously detected, and the latest data is pulled into the data lake for subsequent processing as soon as possible.

[0175] Data collection and integration:

[0176] Nifi: As a powerful data stream processing tool, Nifi can monitor multiple data sources in real time to ensure timely data collection and transmission. Its flexibility and scalability are crucial for building a dynamic rail transit indicator query system.

[0177] Data storage and processing:

[0178] Iceberg: The introduction of data lakes provides a unified data storage solution that can store data in various formats, including structured and unstructured data. It supports long-term storage of data and facilitates historical data analysis and trend prediction.

[0179] Doris: As a high-performance analytical database, Doris can quickly process and analyze large-scale data sets. Its real-time analysis capabilities are crucial for rapid query and response of rail transit indicators.

[0180] Data indexing and caching: To further improve query performance, data indexing and caching mechanisms can be implemented in the data lake. This can reduce query response time and improve user experience.

[0181] Data analysis and optimization:

[0182] Industry big model: By using llama3 big model technology, feeding industry knowledge and defining indicators, the big model can have a very professional rail transit business understanding ability. Users can use non-professional interaction to find the data they want and give interpretations. This ability can significantly improve the quality and efficiency of decision-making and lower the threshold for users.

[0183] User interaction and result output:

[0184] User interaction interface: An intuitive and easy-to-use user interface is designed so that users can easily query rail transit indicators and obtain the required information.

[0185] Professional interpretation and report generation: The system not only provides data query results, but also generates professional interpretations and reports to help users better understand the meaning behind the data.

[0186] Data Security and Disaster Recovery:

[0187] Data backup and disaster recovery: To ensure data security and system reliability, data backup is performed regularly, such as backing up to the Hadoop Distributed File System (HDFS). A disaster recovery mechanism is also established to deal with possible data loss or system failure.

[0188] For the technical solutions in the aforementioned embodiments, there may also be the following alternative solutions, which can achieve the same or similar technical effects.

[0189] 1. Alternative solutions for data access and processing

[0190] Data warehouses as an alternative to data lakes: Although data lakes have the ability to store and process large amounts of structured and unstructured data, data warehouses may be more efficient in storing and managing structured data. Therefore, traditional data warehouses can be used as an alternative specifically for storing and managing structured data in rail transit.

[0191] Distributed computing framework: In addition to the computing power of the data lake itself, distributed computing frameworks such as Apache Spark (a unified analytics engine) and Flink (a stream processing framework for distributed stream and batch processing) can also be introduced to provide more powerful and flexible data processing capabilities.

[0192] 2. Alternatives to Data Quality Governance

[0193] Introducing third-party data quality tools: There are many tools on the market that are specifically used for data quality governance. These tools usually have powerful data verification, anomaly detection, and data repair functions. Introducing these tools can be used as an alternative to improve the effectiveness of data quality governance.

[0194] 3. Alternatives to the Interaction and Visualization Module

[0195] Use other machine learning models: Although industry-leading models have powerful capabilities in specific fields, other machine learning models can be used as substitutes considering development and computing costs. For example, deep learning models or natural language processing models can be used to optimize indicator query and explanation functions.

[0196] IV. Alternatives to the Complete Technical Solution

[0197] Cloud computing-based solution: Deploy the entire rail transit index calculation system on the cloud platform, and use the elastic scaling capabilities of cloud computing to cope with changes in data volume. At the same time, the cloud platform can also provide more abundant and flexible data processing and analysis tools.

[0198] Microservice architecture: The calculation of indicators for each discipline is split into multiple independent microservices, each of which is responsible for processing specific business logic and data. This architecture can improve the scalability and maintainability of the system and reduce the complexity of the system, but it is not friendly to the calculation of cross-disciplinary indicators.

[0199] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0200] Reference Figure 6 , shows a structural block diagram of an intelligent rail transit data management and analysis system according to an embodiment of the present invention. The intelligent rail transit data management and analysis system may specifically include the following modules.

[0201] The data receiving and importing module 61 is used to receive and import professional rail transit data into the data lake;

[0202] A quality monitoring module 62, used to perform full-process quality monitoring on the professional rail transit data and generate data quality indicators;

[0203] A traffic index calculation module 63, used to calculate rail traffic index based on the data quality index;

[0204] A traffic index processing module 64, for providing a user interaction interface to access, analyze and display the rail transit index;

[0205] The coordination and scheduling module 65 is used to continuously coordinate tasks and schedule resources during the steps in which the quality monitoring module 62 performs the full-process quality monitoring and generates data quality indicators and the traffic indicator calculation module 63 calculates rail transit indicators, so as to ensure automated operation according to predetermined strategies and priorities.

[0206] In an exemplary embodiment of the present invention, the data receiving and importing module 61 includes:

[0207] An agent capture module, used to capture the professional rail transit data of a data source through a data access agent mode;

[0208] and / or, a monitoring access module, used to monitor a specific source server port in a monitoring mode to access the professional rail transit data;

[0209] A conversion and decryption module, used for format conversion and decryption processing of the professional rail transit data;

[0210] The data storage module is used to store the professional rail transit data after format conversion and decryption processing in the data lake.

[0211] In an exemplary embodiment of the present invention, the quality monitoring module 62 includes:

[0212] A code table data detection module is used to detect the timeliness and relevance of code table data;

[0213] The master data detection module is used to check the field integrity, real-time performance, cross-disciplinary association consistency and update cycle consistency of the master data;

[0214] The data quality indicator generation module is used to generate the data quality indicators of the code table data and the main data according to the indicator availability threshold set by the user.

[0215] In an exemplary embodiment of the present invention, the traffic index calculation module 63 includes:

[0216] A single data source indicator determination module, for determining, for the single data source indicator in the rail transit indicator, the data quality indicator after the full process quality monitoring is qualified as the single data source indicator;

[0217] A cycle consistency check module, used to check and ensure the consistency of the pull cycle of multi-source data for the composite data source indicators in the rail transit indicators;

[0218] The composite data source indicator calculation module is used to calculate the composite data source indicator according to a predefined calculation logic.

[0219] In an exemplary embodiment of the present invention, the traffic index processing module 64 includes:

[0220] Big model integration module, used to integrate industry big models to understand user query intent;

[0221] A user guidance module is used to guide users to provide the time period, qualifiers, statistical granularity and atomic indicators of the query indicators;

[0222] A derived indicator query construction module, which is used to construct a complete derived indicator query based on the information provided by the user;

[0223] The rail transit indicator presentation module is used to execute queries and present the rail transit indicators in a visual manner.

[0224] In an exemplary embodiment of the present invention, it also includes:

[0225] A rail transit data classification module is used to classify the received professional rail transit data into real-time indicator data and offline indicator data;

[0226] The fast and batch processing module is used to establish a fast processing channel for the real-time indicator data and to perform batch processing on the offline indicator data.

[0227] In an exemplary embodiment of the present invention, the coordination and scheduling module 65 includes:

[0228] An execution strategy setting module, used to set a timing execution strategy according to the time characteristics of the professional rail transit data;

[0229] An execution network construction module, used to construct a task execution network based on the dependency relationship between the rail transit indicators;

[0230] Priority and allocation adjustment module, used to dynamically adjust task priorities and resource allocation;

[0231] The status and exception monitoring module is used to monitor the task execution status and handle exceptions.

[0232] As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0233] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0234] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0235] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0236] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0237] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0238] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0239] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0240] The above is a detailed introduction to an intelligent rail transit data management and analysis method and an intelligent rail transit data management and analysis system provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for intelligent rail transit data management and analysis, characterized in that: The method comprises: Receive and import professional rail transit data into the data lake; Conducting full-process quality monitoring on the professional rail transit data and generating data quality indicators; Calculating rail transit indicators based on the data quality indicators; Providing a user interactive interface to access, analyze and display the rail transit indicators; Among them, during the process of executing the steps of full-process quality monitoring and generating data quality indicators and the step of calculating rail transit indicators, task coordination and resource scheduling are continuously carried out to ensure automated operation according to predetermined strategies and priorities.

2. The method according to claim 1, characterized in that The step of receiving and importing professional rail transit data into the data lake includes: Capturing the professional rail transit data from the data source through a data access agent mode; and / or, monitoring a specific source server port in a monitoring mode to access the professional rail transit data; Performing format conversion and decryption processing on the professional rail transit data; The professional rail transit data after format conversion and decryption processing is stored in the data lake.

3. The method according to claim 1, characterized in that The step of generating the data quality indicator comprises: Check the timeliness and relevance of code table data; Check the field integrity, timeliness, cross-disciplinary association consistency and update cycle consistency of master data; The data quality indicators of the code table data and the main data are generated according to an indicator available threshold value set by a user.

4. The method according to claim 1, characterized in that The step of calculating the rail transit index based on the data quality index comprises: For the single data source indicator in the rail transit indicator, the data quality indicator after the full process quality monitoring is qualified is used as the single data source indicator; For the composite data source indicators in the rail transit indicators, check and ensure the consistency of the pulling cycle of multi-source data; The composite data source indicator is calculated according to a predefined calculation logic.

5. The method according to claim 1, characterized in that The step of providing a user interaction interface to access, analyze and display the rail transit indicators includes: Integrate industry big models to understand user query intent; Guide users to provide query indicators' time period, qualifiers, statistical granularity, and atomic indicators; Build complete derived metric queries based on user-provided information; Execute the query and present the rail transit indicators in a visual manner.

6. The method according to claim 1, characterized in that The following steps are also included: Classifying the received professional rail transit data into real-time indicator data and offline indicator data; A fast processing channel is established for the real-time indicator data, and batch processing is performed on the offline indicator data.

7. The method according to claim 1, characterized in that The steps of continuously coordinating tasks and scheduling resources include: Setting a timing execution strategy according to the time characteristics of the professional rail transit data; Building a task execution network based on the dependency relationship between the rail transit indicators; Dynamically adjust task priorities and resource allocation; Monitor task execution status and handle exceptions.

8. An intelligent rail transit data management and analysis system, characterized in that: The system comprises: Data receiving and importing module, used to receive and import professional rail transit data into the data lake; A quality monitoring module, used to monitor the quality of the professional rail transit data throughout the entire process and generate data quality indicators; A traffic index calculation module, used for calculating rail traffic index based on the data quality index; A traffic index processing module, used for providing a user interaction interface to access, analyze and display the rail transit index; The coordination and scheduling module is used to continuously coordinate tasks and schedule resources during the steps in which the quality monitoring module performs the full-process quality monitoring and generates data quality indicators and the traffic indicator calculation module calculates rail transit indicators, so as to ensure automated operation according to predetermined strategies and priorities.

9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the electronic device to execute the intelligent rail transit data management and analysis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the intelligent rail transit data management and analysis method as described in any one of claims 1 to 7.

Images