Fault diagnosis data pipeline method and system based on heterogeneous cbtc system

Abstract

The application discloses a fault diagnosis data pipeline method and system based on a heterogeneous CBTC system. Independent log boxes are deployed at both ends of a train, non-invasive collection is realized through direct connection with a front network port of a vehicle-mounted controller, and metadata such as a vehicle number, a train end, a system type, an SSID, a frame ID and an NTP time is written in a file name / file header. Through multi-strategy triggering, log files are transmitted to a central server. The central server performs Base template adaptive matching and analysis based on multi-source fingerprints. Unified primary keys and wide table / sub-table strategies are used for high-performance data storage. Rule reasoning, model analysis and knowledge graph linkage are performed on the stored data, a structured diagnosis report is output, and an end-to-end closed loop of 'collection-transmission-analysis-storage-diagnosis' is formed. The application realizes high-reliability collection and safe transmission without modifying existing equipment, improves the accuracy and efficiency of heterogeneous log analysis, and meets the engineering needs of intelligent operation and maintenance of urban rail signal systems.

Description

Technical Field

[0001] This invention relates to the field of urban rail transit communication and signal operation and maintenance technology, and in particular to a fault diagnosis data pipeline method and system based on a heterogeneous CBTC system. Background Technology

[0002] Urban rail transit CBTC (Communication-Based Automatic Train Control) systems typically consist of subsystems such as Zone Controllers (ZC / LC / CI), Onboard Controllers (CC, integrating Automatic Train Protection (ATP) and Automatic Train Operation (ATO) functions), Automatic Train Monitoring Systems (ATS), and Data Transmission Systems (DCS). During operation, this system continuously generates high-frequency, high-dimensional, and heterogeneous format operational logs. A single frame of the signaling system can contain thousands or even tens of thousands of parameters, and the daily log volume of the onboard system can reach hundreds of MB to several GB. Traditional operation and maintenance methods mainly suffer from the following problems: 1. Data collection and modification costs and operational risks: Existing systems mostly rely on manual back-to-the-depot boarding and copying or online downloading through software / hardware modifications to train control equipment. The former is inefficient and has a strong lag, while the latter poses risks of intrusion into the operating system, linkage impact, and security audit.

[0003] 2. Impact of uncontrollable transmission on operations: When the vehicle-to-ground bandwidth is tight or the wireless coverage is uneven, online transmission without bandwidth control can easily occupy critical communication resources and affect mainline train operation.

[0004] 3. Complex parsing of heterogeneous formats: Logs from different lines, manufacturers, and versions differ significantly in file containers, frame structures, and parameter encoding (dimensions, offsets, enumerations). The traditional "manual template selection" method is prone to mismatch, inefficient, and difficult to scale.

[0005] 4. Data ingestion expansion bottleneck: "Ultra-wide tables" with massive fields lead to the upper limit of a single table in traditional databases, frequent DDL operations, and decreased query and write performance; at the same time, it is difficult to guarantee the consistency and retrieval of multi-dimensional primary keys such as vehicle, vehicle direction, frame time, and frame number.

[0006] 5. Insufficient end-to-end closed-loop: The lack of an end-to-end data pipeline results in a long data value conversion cycle, making it difficult to support rapid online / offline diagnosis, trend analysis, and knowledge accumulation.

[0007] The statements herein provide only background information in relation to this invention and do not necessarily constitute prior art. Summary of the Invention

[0008] The purpose of this invention is to provide a fault diagnosis data pipeline method and system based on a heterogeneous CBTC system, which achieves highly reliable data acquisition and secure transmission without modifying existing equipment, improves the accuracy and efficiency of heterogeneous log parsing, and meets the engineering requirements of intelligent operation and maintenance of urban rail signaling systems.

[0009] To achieve the above objectives, this invention provides a fault diagnosis data pipeline method based on a heterogeneous CBTC system, comprising the following steps: S1, the log box initiates acquisition requests to various signal system platforms according to the configuration, receives frame stream logs, and performs rolling segmentation and compression according to preset time slices. The vehicle number, vehicle end, system type, SSID, frame ID, and NTP time are written into the file name and file header. The log box is deployed in the driver's cab at both ends of the train and is directly connected to the front network port of the vehicle controller through the industrial Ethernet port. S2, according to the preset triggering strategy, the log file is transmitted to the central server. The triggering strategy includes at least one of the following: log box timed transmission, central server instruction-triggered transmission, and log box transmission triggered based on vehicle status or location. S3, after receiving the transmitted log file, the central server extracts the fingerprint vector from the log file; matches the fingerprint vector with the templates in the Base template library, selects the target template, and performs parameter-level bit parsing, dimension restoration, and enumeration mapping on the log file; the Base template library is used to store preset templates that include multi-line, multi-version, and multi-vendor log format definitions; S4 uses "Line Number - Train Number - Vehicle Identifier - System Type - Frame Timestamp - Frame Sequence Number - Message Number" as the unified primary key to construct a wide table / partitioned table of the columnar database, infer the field type, and use columnar compression and partitioning to write the parsed structured data into the columnar database in batches. S5 performs fault reasoning and analysis based on the structured data entered into the database, generates and outputs a structured diagnostic report.

[0010] Optionally, the transmission network adopts the FTP / SFTP dual-stack protocol and is encrypted with the TLS protocol; during the transmission process, one or more of the following are implemented: outbound bandwidth rate limiting, concurrent path limit, breakpoint resumption, fragmentation / segmentation verification and failure retransmission.

[0011] Optionally, extracting the fingerprint vector from the log file includes: Extract the compressed header, extension, file name, file header, and frame header features of the log file to generate a multi-source fingerprint vector. The fingerprint vector includes: system platform features, version number range, frame structure summary, message counting pattern, and verification information of unit / offset / enumeration table fragments.

[0012] Optionally, the step of matching the fingerprint vector with templates in the Base template library and selecting a target template includes: In the initial screening stage, templates in the Base template library are roughly screened based on system platform characteristics and version number ranges to obtain a candidate template set; In the fine screening stage, the similarity scores of each template in the candidate template set are calculated based on the frame structure summary, message counting rules, and verification information of dimensional / offset / enumeration table fragments. In the re-verification phase, several sample frames from the log file are used to perform parameter interpretability verification on each template in the candidate template set, and the one that passes the verification and has the highest similarity score is taken as the target template.

[0013] Optionally, when the similarity scores of each template are all below the threshold, a new template is created and semi-automatic annotation is performed, and the new template is included in the Base template library and can be released and rolled back.

[0014] Optionally, the construction of the wide table / partition of the columnar database includes: In columnar databases, wide tables are modeled by "system-version-field group". If the maximum number of fields in a single table exceeds the threshold, the table is automatically split. Each split table retains the primary key and common meta fields.

[0015] Optionally, the method further includes: providing an analytical view that pivots multiple sub-tables into a logically unified table based on the primary key.

[0016] Optionally, the fault reasoning and analysis based on the structured data stored in the database includes: Load a predefined diagnostic rule knowledge base or machine learning model, refer to the knowledge graph to perform online streaming or offline batch fault reasoning and analysis on the input structured data, generate a structured diagnostic report and output it.

[0017] This invention also provides a fault diagnosis data pipeline system based on a heterogeneous CBTC system, comprising: The log box is deployed in the driver's cab at both ends of the train and is directly connected to the front network port of the on-board controller via an industrial Ethernet port. It is used to initiate acquisition requests to various signal system platforms according to configuration, receive frame stream logs, and perform rolling segmentation and compression according to preset time slices. The vehicle number, vehicle end, system type, SSID, frame ID, and NTP time are written into the file name and file header. The log file is then transmitted to the central server according to a preset triggering strategy. The triggering strategy includes at least one of the following: log box timed transmission, central server instruction-triggered transmission, and log box-triggered transmission based on vehicle status or location. The central server includes a parsing engine module, a Base template library, a data storage module, and a diagnostic analysis module; the Base template library is used to store pre-set templates that include multi-line, multi-version, and multi-vendor log format definitions; The parsing engine module is used to extract fingerprint vectors from the log file; match the fingerprint vectors with templates in the Base template library, select a target template, and perform parameter-level bit parsing, dimension restoration, and enumeration mapping on the log file; The data storage module is used to construct a wide table / partitioned table of a columnar database using "line number-train number-vehicle terminal identifier-system type-frame timestamp-frame sequence number-message number" as a unified primary key, and infer the field type, and use columnar compression and partitioning to write the parsed structured data into the columnar database in batches. The diagnostic analysis module is used to perform fault reasoning and analysis based on the structured data stored in the database, generate and output a structured diagnostic report.

[0018] Optionally, the transmission network adopts the FTP / SFTP dual-stack protocol and is encrypted with the TLS protocol; during the transmission process, one or more of the following are implemented: outbound bandwidth rate limiting, concurrent path limit, breakpoint resumption, fragmentation / segmentation verification and failure retransmission.

[0019] Optionally, the parsing engine module is specifically used to extract the compressed header, extension, file name, file header, and frame header features of the log file to generate a multi-source fingerprint vector; the fingerprint vector includes: system platform features, version number range, frame structure summary, message counting pattern, and verification information of unit / offset / enumeration table fragments.

[0020] Optionally, the parsing engine module is specifically used for: a preliminary screening stage, where templates in the Base template library are coarsely screened based on system platform characteristics and version number ranges to obtain a candidate template set; a fine screening stage, where each template in the candidate template set is scored for similarity based on frame structure summary, message counting rules, and verification information of dimensional / offset / enumeration table fragments; and a re-verification stage, where several sample frames from the log file are used to perform parameter interpretability verification on each template in the candidate template set, and the template that passes the verification and has the highest similarity score is taken as the target template.

[0021] Optionally, the parsing engine module is further configured to: create a new template and perform semi-automatic annotation when the similarity scores of each template are all below a threshold, and include the new template in the Base template library and support canary release and rollback.

[0022] Optionally, the data storage module is specifically used to: perform wide table modeling in the columnar database according to "system-version-field group"; if the upper limit of a single table field exceeds the threshold, the table is automatically partitioned, and each partitioned table retains the primary key and common meta fields.

[0023] Optionally, the data storage module is further configured to: provide an analytical view that views multiple sub-tables as a logically unified table by primary key.

[0024] Optionally, the diagnostic analysis module is specifically used to: load a predefined diagnostic rule knowledge base or machine learning model, refer to the knowledge graph to perform online streaming or offline batch fault reasoning and analysis on the structured data in the database, generate a structured diagnostic report and output it.

[0025] Compared with the prior art, the present invention has the following beneficial effects: 1. Non-intrusive and replicable: The log box solution does not require modification of existing train control equipment, has a clear installation location, simple interface, controllable risks, and has cross-line replication capability.

[0026] 2. Secure transmission without disrupting operations: Multiple strategies (timed / centralized / conditional), bandwidth rate limiting, breakpoint resumption, and priority time period retrieval avoid occupying critical bandwidth and meet both operation and maintenance and security constraints.

[0027] 3. Adaptive parsing is accurate and efficient: Based on multi-source fingerprint intelligent matching of the base template, mismatches and manual intervention are significantly reduced, improving the accuracy and efficiency of heterogeneous log parsing.

[0028] 4. High-performance data entry and querying: The combination of wide table / partitioned table strategy and columnar storage supports massive fields and high-concurrency analysis, ensuring the timeliness of diagnosis / visualization and rule / model reasoning.

[0029] 5. End-to-end closed loop: It connects the entire chain of data collection, transmission, parsing, data storage, and diagnosis, and supports rule reasoning and ML / LLM fine-tuning to form a continuous learning and optimization mechanism.

[0030] 6. Engineering-oriented: It matches the actual line conditions (such as coexistence of different types of signal systems, multiple vehicle models, and multiple systems), and can be quickly integrated into existing operation and maintenance platforms to achieve demonstration and promotion. Attached Figure Description

[0031] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the drawings described below are one embodiment of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort: Figure 1This is the overall architecture diagram of the fault diagnosis data pipeline method and system based on heterogeneous CBTC system of the present invention; Figure 2 This is a schematic diagram illustrating the hardware / software and access relationship of the log box according to the present invention; Figure 3 This is a schematic diagram of the multi-strategy transmission and bandwidth rate limiting scheduling of the present invention; Figure 4 This is a schematic diagram of the Base adaptive parsing process and fingerprint matching scoring of the present invention; Figure 5 This is a schematic diagram illustrating the grayscale release and rollback of the template version of this invention; Figure 6 This is a schematic diagram illustrating the wide table / partition table data entry and unified primary key naming convention of the present invention. Detailed Implementation

[0032] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, further illustrates the solution proposed by the present invention. The advantages and features of the present invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, used only to facilitate and clearly illustrate the embodiments of the present invention. Please refer to the drawings to make the objectives, features, and advantages of the present invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only for illustrative purposes to aid those skilled in the art and are not intended to limit the implementation conditions of the present invention. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in proportions, or adjustments to the size, without affecting the effects and objectives achieved by the present invention, should still fall within the scope of the technical content disclosed in the present invention.

[0033] Figure 1 This is a diagram illustrating the overall architecture of the present invention, which includes a closed-loop process encompassing data acquisition, transmission, parsing, storage, and diagnostics. The following provides a detailed description of the solution presented in this invention.

[0034] Combination Figure 1 As shown, this invention provides a fault diagnosis data pipeline method based on a heterogeneous CBTC system, comprising the following steps: S1, the log box initiates acquisition requests to various signal system platforms according to the configuration, receives frame stream logs, and performs rolling segmentation and compression according to preset time slices. The vehicle number, vehicle end, system type, SSID, frame ID, and NTP time are written into the file name and file header. The log box is deployed in the driver's cab at both ends of the train and is directly connected to the front network port of the vehicle controller through the industrial Ethernet port. In this embodiment, independent log collection devices (i.e., log boxes) are arranged near the driver's cab at both ends of each train. The connection relationship of the log boxes is as follows: Figure 2As shown, the log box is directly connected to the front network port of the vehicle controller (CC / ATP / ATO) via the industrial Ethernet port. It can be powered by DC24V and connected to the vehicle's existing power supply panel without requiring any software or hardware modifications to the existing train control equipment.

[0035] The log box runs a data acquisition agent that initiates standard requests (e.g., UR / US protocol frames or existing vendor-exclusive / permitted interfaces) to the signal system platform according to its configuration, and receives OMAP logs as frame streams. During acquisition, the logs are divided into fixed time slices (e.g., 600 seconds) and local real-time compression (Gzip or FDZ), verification, and space quota management (e.g., default retention of the last 30 days, threshold cleanup policy) are performed when the logs are written to disk. For example, the log box acquisition can be configured as follows: acquisition targets: ATP (frameId=65), ATO (frameId=66); acquisition slice: span=600 seconds; acquisition threads: multi-target concurrency, thread-safe writing; compression policy: Gzip level=9; original records are deleted after completion.

[0036] Then, file naming is performed: the vehicle number, vehicle terminal, system type, SSID, frame ID, and NTP time are written into the file name and file header to achieve a strong binding of "file name containing vehicle identity, vehicle terminal, and system information," ensuring a unique correspondence between "log-vehicle-vehicle terminal-system" and avoiding subsequent manual fuzzy matching. For example, the file name naming convention is as follows: {YYYYMMDDhhmmss}{trainNo}{END}{SystemType}{SSID}omap{FrameId}.gz, and an example is: 20250807153000_train099_END1_ATP_UNIVIC_Core1_omap_65.gz. The file header metadata includes: NTP seconds / microseconds, train_no, end_id, system, ssid, frame_id, etc.

[0037] S2, according to the preset triggering strategy, transmits the log file to the central server.

[0038] The transmission network can use FTP / SFTP dual-stack protocol. The default FTP can be upgraded to SFTP and encrypted with TLS protocol. During the transmission process, one or more of the following are implemented: outbound bandwidth rate limit, concurrent path limit, breakpoint resume, fragmentation / segmentation verification and failure retransmission.

[0039] The triggering strategies include at least one of the following: log box timed transmission, central server command-triggered transmission, and log box-based vehicle status or location-based triggering transmission. These three triggering strategies can be implemented in parallel or selectively; that is, different trains or different signaling system platforms can be configured to use the same or different triggering strategies. Multi-strategy transmission and bandwidth-limited speed scheduling are as follows: Figure 3 As shown.

[0040] Log box scheduled transmission refers to the automatic uploading of log boxes within a set time window (usually during non-operational periods). For example, under annual construction and operation constraints, log boxes are set to automatically upload within a fixed window from the end of daily operations to the start of operations the next day, with a rate limiting policy of no more than 2Mbps per connection and no more than 10Mbps for the total outgoing bandwidth.

[0041] Central server command-triggered transmission refers to the central server initiating transmission according to instructions. It allows for non-sequential retrieval of data by specifying a "priority time period / priority event window" and supports pause / resume. For example, maintenance personnel can issue a "priority by time window" command via Web / CLI on the central server (e.g., retrieving all train ATP logs from 15:00 to 15:10 on August 7, 2025). The central server will then issue a task list to the corresponding log boxes, which will transmit the specified window files non-sequentially. If continuity is restricted, the transmission will be paused / resume, and an audit log will be recorded.

[0042] Log box-based transmission triggered by vehicle status or location means that when a vehicle is in a specific state or location (such as entering a depot / a specific station / a specific track section), the log box automatically initiates transmission based on collected log location variables, vehicle status, and auxiliary information such as its own GNSS / odometer / vehicle-to-ground wireless signals. For example, the log box can automatically initiate transmission when a train enters a depot or a designated station based on collected location variables (such as LocalizationState, BeaconID), train entry status (such as yard_mode=TRUE), or station list mapping. If the train's online idle window is extremely short, incremental transmission and breakpoint resume strategies can be used.

[0043] Task orchestration can be performed on the central server, configured as a multi-source, multi-vehicle Pull (data retrieval) / Push (data push) hybrid orchestration, and whitelists, blacklists, priorities, multi-queue QoS and other policies can also be set.

[0044] S3, after receiving the transmitted log file, the central server extracts the fingerprint vector from the log file; matches the fingerprint vector with the templates in the Base template library, selects the target template, and performs parameter-level bit parsing, dimension restoration, and enumeration mapping on the log file; the Base template library is used to store preset templates that include multi-line, multi-version, and multi-vendor log format definitions.

[0045] The templates in the Base template library are stored in XML or JSON format, containing parameter names, starting bits, length, type, units, offsets, enumeration mappings, etc. Fields include: name, type, bit_start, bit_len, quantum, offset, enum_map, valid_range, etc. Multi-instance management based on route, vehicle type, and version dimensions is possible.

[0046] The Base adaptive parsing process is as follows: Figure 4 As shown. In this embodiment, extracting fingerprint vectors from the log file includes: extracting the compressed header, extension, file name, file header, and frame header features of the log file to generate a multi-source fingerprint vector; the fingerprint vector includes: system platform features, version number range, frame structure summary, message counting pattern, and verification information of unit / offset / enumeration table fragments.

[0047] In this embodiment, the step of matching the fingerprint vector with templates in the Base template library and selecting a target template includes: In the initial screening stage, templates in the Base template library are roughly screened based on system platform characteristics and version number ranges to obtain a candidate template set; In the fine screening stage, the similarity score of each template in the candidate template set is performed based on the frame structure summary, message counting rules, and verification information of the unit / offset / enumeration table fragments; that is, the candidate template is comprehensively scored based on "frame header consistency, bit width matching, unit / offset / landing rate, and enumeration hit rate". In the re-verification phase, several sample frames from the log file are used to perform parameter interpretability verification on each template in the candidate template set. The template that passes the verification and has the highest similarity score is selected as the target template. Specifically, N frames are randomly selected for physical interpretability verification (e.g., non-negative speed and not exceeding the upper limit, reasonable mode transitions, and BeaconID from the line dictionary).

[0048] Furthermore, when the matching confidence level is below a threshold (if all templates in the candidate template set fail validation), a new template is created and semi-automatic annotation is performed. The new template is then added to the Base template library, and canary releases and rollbacks are supported. Figure 5 As shown.

[0049] When parsing log files, the target template is subjected to bit-by-bit extraction, unit restoration, and outlier filtering. When an anomaly occurs during parsing, such as "field missing / template deviation", the difference is recorded and entered into the "parsing anomaly table" in the database, and a gray-scale update of the template is prompted.

[0050] S4 uses "Line Number - Train Number - Vehicle Identifier - System Type - Frame Timestamp - Frame Sequence Number - Message Number" as the unified primary key to construct a wide table / partitioned table of the columnar database, infer the field types, and uses columnar compression and partitioning to write the parsed structured data into the columnar database in batches.

[0051] Combination Figure 6 The data entry method shown follows the following naming convention for primary keys: The retrieval key is uniformly set to "line_id, train_no, end_id, system_type, frame_time, frame_index, message_count (for intra-frame information)". The filename corresponds one-to-one with the database record metadata. Within each system, line, train, and timeline at the train end, the frame_time and message_count are unique.

[0052] The construction of wide tables / partitioned tables in a columnar database includes: modeling wide tables in the columnar database according to "system-version-field group"; if the maximum number of fields in a single table exceeds a threshold, the table is automatically partitioned, and each partitioned table retains the primary key and common meta fields. Specifically, in an OLAP-type columnar database (such as Apache Doris), wide tables are modeled according to "system-version-field group"; the maximum number of fields in a single table is set to no more than 4000 columns, and if this limit is exceeded, the table is automatically partitioned (atp_1, _2, ...), and each partitioned table retains the primary key and common meta fields.

[0053] After automatically creating wide tables / partitioned tables, the field types (int / float / enum / string) are determined based on the target template (i.e., the base file) information, so that when writing parameters from the log file to the database, the specific data is converted into the correct values ​​and types; DDL logs are recorded so that subsequent auditing can be performed.

[0054] High-throughput data writing can be achieved through columnar compression and partitioning (e.g., by date or vehicle), with automatic rollback and compensation in case of write failure. Batch writing can handle 10k to 100k rows per batch. When querying inbound data, multi-dimensional searches can be performed by time range, vehicle, system, variable name prefix, and other combinations.

[0055] You can also set retention and cleanup policies for incoming data. For example, you can retain automatically transmitted incoming data for a preset period (e.g., for the last 3 months) and manually uploaded data until manual confirmation, thereby performing rolling cleanup of incoming data.

[0056] In addition, analytical views can be provided to pivot multiple sub-tables into a logically unified table based on the primary key.

[0057] S5 performs fault reasoning and analysis based on the structured data entered into the database, generates and outputs a structured diagnostic report.

[0058] Specifically, predefined diagnostic rule knowledge bases or machine learning models can be loaded, and knowledge graphs can be referenced to perform online streaming or offline batch fault reasoning and analysis on the structured data in the database, generating and outputting structured diagnostic reports. For example, domain rules (such as EB triggering, inaccurate stopping, speed / location anomalies, communication interruptions, etc.) can be loaded from the diagnostic rule knowledge base to perform online / offline reasoning on the structured data stream in the database; or fine-tuning large models based on historical samples or traditional ML (decision trees, SVM, LSTM / Transformer) can be used for case identification, early warning scoring, and suggestion generation.

[0059] In fault analysis, mapping variable events in the structured data stored in the database to a knowledge graph and outputting the shortest root cause path and redundant alarm filtering results can improve the accuracy of fault analysis. The knowledge graph is used to store entities and relationships of devices, faults, variables and rules.

[0060] In the above scheme, the present invention sets the following data governance strategies: 1) Template version control: Each line / each type of equipment / each version of the template is managed with an independent version number. Template changes are fully released after gray-scale verification, and rollback is also supported. 2) Data quality: Quality indicators (such as missing rate, anomaly rate, and out-of-bounds rate) are established for key variables. If the data falls below the threshold, parsing, template, and data collection link checks are triggered. 3) Lifecycle: Automatically and manually uploaded data are retained according to different strategies. Manually uploaded data requires manual confirmation before cleanup and can be exported and archived based on train number or time period.

[0061] In addition, the present invention can also ensure data traceability through the following system auditing methods: 1) Transmission task audit: record task creator, strategy, rate limit, start and end time, file list, number of retries; 2) Parsing audit: record template version, fingerprint scoring, re-verification status, list of abnormal fields; 3) DDL audit: record the history of automatic database / table creation / field addition and modification; 4) Diagnostic audit: record rule hits, model version and output suggested version.

[0062] Based on the same inventive concept, this invention also provides a fault diagnosis data pipeline system based on a heterogeneous CBTC system, comprising: The log box is deployed in the driver's cab at both ends of the train and is directly connected to the front network port of the on-board controller via an industrial Ethernet port. It is used to initiate acquisition requests to various signal system platforms according to the configuration, receive frame stream logs, and perform rolling segmentation and compression according to preset time slices. The vehicle number, vehicle end, system type, SSID, frame ID, and NTP time are written into the file name and file header. The log file is then transmitted to the central server according to a preset triggering strategy. The central server includes a parsing engine module, a Base template library, a data storage module, and a diagnostic analysis module; the Base template library is used to store pre-set templates that include multi-line, multi-version, and multi-vendor log format definitions; The parsing engine module is used to extract fingerprint vectors from the log file; match the fingerprint vectors with templates in the Base template library, select a target template, and perform parameter-level bit parsing, dimension restoration, and enumeration mapping on the log file; The data storage module is used to construct a wide table / partitioned table of a columnar database using "line number-train number-vehicle terminal identifier-system type-frame timestamp-frame sequence number-message number" as a unified primary key, and infer the field type, and use columnar compression and partitioning to write the parsed structured data into the columnar database in batches. The diagnostic analysis module is used to perform fault reasoning and analysis based on the structured data stored in the database, generate and output a structured diagnostic report.

[0063] Furthermore, the triggering strategy includes at least one of the following: log box timed transmission, central server instruction-triggered transmission, and log box-triggered transmission based on vehicle status or location.

[0064] Furthermore, the transmission network adopts the FTP / SFTP dual-stack protocol and uses TLS protocol encryption; during the transmission process, one or more of the following are implemented: outbound bandwidth rate limiting, concurrent path limit, breakpoint resume, fragmentation / segmentation verification and failure retransmission.

[0065] Furthermore, the parsing engine module is specifically used to extract the compressed header, extension, file name, file header, and frame header features of the log file to generate a multi-source fingerprint vector; the fingerprint vector includes: system platform features, version number range, frame structure summary, message counting pattern, and verification information of unit / offset / enumeration table fragments.

[0066] Furthermore, the parsing engine module is specifically used for: a preliminary screening stage, where templates in the Base template library are coarsely screened based on system platform characteristics and version number ranges to obtain a candidate template set; a fine screening stage, where each template in the candidate template set is scored for similarity based on frame structure summary, message counting rules, and verification information of dimensional / offset / enumeration table fragments; and a re-verification stage, where several sample frames from the log file are used to perform parameter interpretability verification on each template in the candidate template set, and the template that passes the verification and has the highest similarity score is taken as the target template.

[0067] Furthermore, the parsing engine module is also used to: create a new template and perform semi-automatic annotation when the similarity scores of each template are all below the threshold, and to include the new template in the Base template library and support canary release and rollback.

[0068] Furthermore, the data storage module is specifically used to: perform wide table modeling in the columnar database according to "system-version-field group"; if the upper limit of a single table field exceeds the threshold, the table is automatically partitioned, and each partitioned table retains the primary key and common meta fields.

[0069] Furthermore, the data storage module is also used to: provide an analytical view that views multiple sub-tables as a logically unified table based on the primary key.

[0070] Furthermore, the diagnostic analysis module is specifically used to: load a predefined diagnostic rule knowledge base or machine learning model, refer to the knowledge graph to perform online streaming or offline batch fault reasoning and analysis on the structured data in the database, generate a structured diagnostic report and output it.

[0071] The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments can be referred to each other.

[0072] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0073] Although the present invention has been described in detail through the preferred embodiments above, it should be understood that the above description should not be considered as a limitation of the present invention. Various modifications and substitutions to the present invention will be apparent to those skilled in the art after reading the above description. Therefore, the scope of protection of the present invention should be defined by the appended claims.

Claims

1. A fault diagnosis data pipeline method based on a heterogeneous CBTC system, characterized in that, Includes the following steps: S1, the log box initiates acquisition requests to various signal system platforms according to the configuration, receives frame stream logs, and performs rolling segmentation and compression according to preset time slices. The vehicle number, vehicle end, system type, SSID, frame ID, and NTP time are written into the file name and file header. The log box is deployed in the driver's cab at both ends of the train and is directly connected to the front network port of the vehicle controller through the industrial Ethernet port. S2, according to the preset triggering strategy, the log file is transmitted to the central server. The triggering strategy includes at least one of the following: log box timed transmission, central server instruction-triggered transmission, and log box transmission triggered based on vehicle status or location. S3, after receiving the transmitted log file, the central server extracts the fingerprint vector from the log file; matches the fingerprint vector with the templates in the Base template library, selects the target template, and performs parameter-level bit parsing, dimension restoration, and enumeration mapping on the log file; the Base template library is used to store preset templates that include multi-line, multi-version, and multi-vendor log format definitions; S4 uses "Line Number - Train Number - Vehicle Identifier - System Type - Frame Timestamp - Frame Sequence Number - Message Number" as the unified primary key to construct a wide table / partitioned table of the columnar database, infer the field type, and use columnar compression and partitioning to write the parsed structured data into the columnar database in batches. S5, based on the structured data entered into the database, performs fault reasoning and analysis, generates and outputs a structured diagnostic report; Extracting the fingerprint vector from the log file includes: Extract the compressed header, extension, file name, file header, and frame header features of the log file to generate a multi-source fingerprint vector; the fingerprint vector includes: system platform features, version number range, frame structure summary, message counting pattern, and verification information of unit / offset / enumeration table fragments; The step of matching the fingerprint vector with templates in the Base template library and selecting a target template includes: In the initial screening stage, templates in the Base template library are roughly screened based on system platform characteristics and version number ranges to obtain a candidate template set; In the fine screening stage, the similarity scores of each template in the candidate template set are calculated based on the frame structure summary, message counting rules, and verification information of dimensional / offset / enumeration table fragments. In the re-verification phase, several sample frames from the log file are used to perform parameter interpretability verification on each template in the candidate template set, and the one that passes the verification and has the highest similarity score is taken as the target template. When the similarity scores of all templates are below the threshold, a new template is created and semi-automatic annotation is performed. The new template is then added to the Base template library and canary release and rollback are supported.

2. The fault diagnosis data pipeline method based on a heterogeneous CBTC system as described in claim 1, characterized in that, The transmission network adopts the FTP / SFTP dual-stack protocol and uses TLS protocol encryption; during the transmission process, one or more of the following are implemented: outbound bandwidth rate limiting, concurrent path limit, breakpoint resume, fragment / segment verification and failure retransmission.

3. The fault diagnosis data pipeline method based on a heterogeneous CBTC system as described in claim 1, characterized in that, The construction of wide tables / partitioned tables in a columnar database includes: In columnar databases, wide tables are modeled by "system-version-field group". If the maximum number of fields in a single table exceeds the threshold, the table is automatically split. Each split table retains the primary key and common meta fields.

4. The fault diagnosis data pipeline method based on a heterogeneous CBTC system as described in claim 3, characterized in that, Also includes: Provides analytical views that can pivot multiple sub-tables into a logically unified table based on the primary key.

5. The fault diagnosis data pipeline method based on a heterogeneous CBTC system according to claim 1, characterized in that, The fault reasoning and analysis based on the structured data stored in the database includes: Load a predefined diagnostic rule knowledge base or machine learning model, refer to the knowledge graph to perform online streaming or offline batch fault reasoning and analysis on the input structured data, generate a structured diagnostic report and output it.

6. A fault diagnosis data pipeline system based on a heterogeneous CBTC system, characterized in that, include: The log box is deployed in the driver's cab at both ends of the train and is directly connected to the front network port of the on-board controller via an industrial Ethernet port. It is used to initiate acquisition requests to various signal system platforms according to configuration, receive frame stream logs, and perform rolling segmentation and compression according to preset time slices. The vehicle number, vehicle end, system type, SSID, frame ID, and NTP time are written into the file name and file header. The log file is then transmitted to the central server according to a preset triggering strategy. The triggering strategy includes at least one of the following: log box timed transmission, central server instruction-triggered transmission, and log box-triggered transmission based on vehicle status or location. The central server includes a parsing engine module, a Base template library, a data storage module, and a diagnostic analysis module; the Base template library is used to store pre-set templates that include multi-line, multi-version, and multi-vendor log format definitions; The parsing engine module is used to extract fingerprint vectors from the log file; match the fingerprint vectors with templates in the Base template library, select a target template, and perform parameter-level bit parsing, dimension restoration, and enumeration mapping on the log file; The data storage module is used to construct a wide table / partitioned table of the columnar database using "line number-train number-vehicle terminal identifier-system type-frame timestamp-frame sequence number-message number" as a unified primary key, and infer the field type, and use columnar compression and partitioning to write the parsed structured data into the columnar database in batches. The diagnostic analysis module is used to perform fault reasoning and analysis based on the structured data stored in the database, generate and output a structured diagnostic report; The parsing engine module is specifically used to extract the compressed header, extension, file name, file header, and frame header features of the log file to generate a multi-source fingerprint vector. The fingerprint vector includes: system platform features, version number range, frame structure summary, message counting pattern, and verification information of unit / offset / enumeration table fragments. The parsing engine module is specifically used for: in the initial screening stage, to perform coarse screening of templates in the Base template library based on system platform characteristics and version number ranges to obtain a candidate template set; in the fine screening stage, to perform similarity scoring on each template in the candidate template set based on frame structure summary, message counting rules, and verification information of dimensional / offset / enumeration table fragments. In the re-verification phase, several sample frames from the log file are used to perform parameter interpretability verification on each template in the candidate template set, and the one that passes the verification and has the highest similarity score is taken as the target template. The parsing engine module is also used to: create a new template and perform semi-automatic annotation when the similarity scores of each template are all below the threshold, and to include the new template in the Base template library and support canary release and rollback.

7. The fault diagnosis data pipeline system based on a heterogeneous CBTC system as described in claim 6, characterized in that, The transmission network adopts the FTP / SFTP dual-stack protocol and uses TLS protocol encryption; during the transmission process, one or more of the following are implemented: outbound bandwidth rate limiting, concurrent path limit, breakpoint resume, fragment / segment verification and failure retransmission.

8. The fault diagnosis data pipeline system based on a heterogeneous CBTC system as described in claim 6, characterized in that, The data storage module is specifically used to: model wide tables in the columnar database according to "system-version-field group"; if the upper limit of a single table field exceeds the threshold, the table will be automatically partitioned, and each partitioned table will retain the primary key and common meta fields.

9. The fault diagnosis data pipeline system based on a heterogeneous CBTC system as described in claim 8, characterized in that, The data storage module is also used to: provide an analytical view that views multiple sub-tables as a logically unified table based on their primary keys.

10. The fault diagnosis data pipeline system based on a heterogeneous CBTC system according to claim 6, characterized in that, The diagnostic analysis module is specifically used to: load a predefined diagnostic rule knowledge base or machine learning model, refer to the knowledge graph to perform online streaming or offline batch fault reasoning and analysis on the structured data in the database, generate a structured diagnostic report and output it.

Images