Collection system and method for railway infrastructure detection and monitoring data

Abstract

The invention discloses a collection system and method for railway infrastructure detection and monitoring data. The system comprises a data acquisition module, a dynamic execution control module, a data quality verification module and a database. The data acquisition module is used for acquiring multi-source data by adopting a corresponding transmission mode according to a data source; the transmission mode comprises wireless communication, a publishing and subscribing mode and client-side batch uploading; the dynamic execution control module is used for configuring verification tasks with different priorities for the multi-source data, calling the data quality verification module to execute the verification tasks and generating a data quality report; the data quality verification is used for verifying whether the normalization, uniqueness, integrity, consistency, availability and timeliness of the multi-source data meet preset specification requirements or not; checking whether the multi-source data meets a preset service requirement or not; checking whether the multi-source data meets a preset process compliance requirement or not; the database is used for storing multi-source data. According to the invention, unified collection management of all-path detection monitoring data can be realized.

Description

Technical Field

[0001] This invention relates to the field of railway infrastructure inspection and data processing technology, and in particular to a system and method for collecting railway infrastructure inspection and monitoring data. Background Art

[0002] This section is intended to provide background or context for the embodiments of the invention set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] With the continuous advancement of railway construction, the railway network is constantly expanding, and facilities and equipment are being upgraded and accelerated. Operations such as mobile monitoring, fixed monitoring, and on-site inspections of infrastructure generate massive amounts of monitoring data. This data can reflect the real-time operational status of railway infrastructure, rolling stock, and equipment, and is a crucial data asset for ensuring the safe operation of railways.

[0004] Existing technologies actively explore effective paths for data collection and management, resulting in the establishment of a series of monitoring and detection data acquisition and management systems, such as the power supply safety monitoring and detection 6C system big data analysis system, the railway engineering safety production management system, and the rail flaw detection and damage management platform. Alternatively, multi-source data acquisition and processing systems have been developed for specific technical issues. However, most of these systems are based on specialized vertical management and operate independently, making data interconnection difficult. Monitoring and detection data not included in the unified system are managed separately by various parties, leading to problems such as diverse data sources and scattered storage, creating numerous "data silos." Furthermore, data quality issues exist, including non-standard data formats, duplicate storage, and missing key information. This severely hinders the effective integration and utilization of data, making it difficult to realize its maximum value in the full life-cycle management and intelligent operation and maintenance of railway infrastructure. Summary of the Invention

[0005] This invention provides a data collection system for railway infrastructure inspection and monitoring, which aims to overcome data barriers, build a cross-professional data collection system, realize unified collection and management of inspection and monitoring data across the entire railway network, and improve the quality of collected data. The system includes: a data acquisition module, a dynamic execution control module, a data quality verification module, and a database.

[0006] The data acquisition module is used to: acquire multi-source data according to the data source and a preset data transmission scheme, using a transmission method corresponding to the data source; the data source includes mobile detection devices, ground fixed monitoring devices, manual observation data, and one or any combination of various unified construction systems; the unified construction system is an existing railway infrastructure data acquisition and management system; the preset data transmission scheme includes a mapping relationship between data sources and transmission methods; the transmission methods include wireless communication, publish-subscribe mode, and batch upload from the client.

[0007] The dynamic execution control module is used to: configure verification tasks with different priorities for multi-source data, call the data quality verification module to execute verification tasks, and when the data quality verification module fails to verify the multi-source data to meet the standards, it will reject data upload or mark the data as erroneous and generate a data quality report.

[0008] The data quality verification module includes a basic layer verification module, a business layer verification module, and a management layer verification module.

[0009] The basic layer verification module is used to verify whether multi-source data meets the preset specification requirements in one or any combination of dimensions such as standardization, uniqueness, integrity, consistency, availability, and timeliness.

[0010] The business layer verification module is used to: verify whether multi-source data meets preset business requirements; the preset business requirements reflect the data logic rationality of multi-source data in the railway infrastructure monitoring scenario;

[0011] The management verification module is used to verify whether multi-source data meets the preset process compliance requirements; the preset process compliance requirements indicate whether the multi-source data complies with upload time limits, planned version consistency, and traceability verification.

[0012] The database is used to store multi-source data after it has been processed by the dynamic execution control module and the data quality verification module.

[0013] This invention also provides a method for collecting railway infrastructure inspection and monitoring data to overcome data barriers, build a cross-professional data collection system, achieve unified collection and management of inspection and monitoring data across the entire railway network, and improve the quality of collected data. The method includes:

[0014] Based on the data sources and a pre-defined data transmission scheme, multi-source data is acquired using a transmission method corresponding to the data sources. The data sources include mobile detection devices, ground-based fixed monitoring devices, manual observation data, and one or any combination of various unified construction systems. The unified construction system is an existing railway infrastructure data acquisition and management system. The pre-defined data transmission scheme includes a mapping relationship between data sources and transmission methods. The transmission methods include wireless communication, publish-subscribe mode, and batch upload from the client.

[0015] Configure verification tasks with different priorities for multi-source data;

[0016] Perform verification tasks; when multi-source data fails verification, reject data uploads or mark erroneous data, and generate a data quality report; the verification tasks include: verifying whether multi-source data meets preset specification requirements in one or any combination of dimensions such as standardization, uniqueness, completeness, consistency, availability, and timeliness; verifying whether multi-source data meets preset business requirements; the preset business requirements reflect the data logical rationality of multi-source data in the railway infrastructure monitoring scenario; and verifying whether multi-source data meets preset process compliance requirements; the preset process compliance requirements indicate whether multi-source data complies with upload time limits, plan version consistency, and traceability verification.

[0017] The verified multi-source data is stored in the database.

[0018] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described method for collecting monitoring data for railway infrastructure.

[0019] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for collecting monitoring and detection data for railway infrastructure.

[0020] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described method for collecting monitoring data for railway infrastructure.

[0021] The data collection system for railway infrastructure inspection and monitoring in this embodiment includes a data acquisition module, a dynamic execution control module, a data quality verification module, and a database. These modules collaborate to address long-standing data barriers in engineering, electrical, and power supply fields, constructing a cross-disciplinary data collection system. This achieves unified collection and management of inspection and monitoring data across the entire railway network. For multiple data sources, including mobile inspection devices, ground-based fixed monitoring devices, manual observation data, and various unified construction systems, it integrates three core data collection methods: wireless communication, publish-subscribe mode, and batch upload access from clients. This establishes an efficient access channel covering multiple data sources. Furthermore, to address data quality issues such as non-standardized, incomplete, and duplicate storage, the data quality verification module performs quality control, improving the quality of the collected data. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0023] Figure 1 This is a schematic diagram of a data collection system for railway infrastructure inspection and monitoring in an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram illustrating the data collection range in an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of the overall architecture of the collection system in an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of vehicle-to-ground wireless data collection in an embodiment of the present invention;

[0027] Figure 5 This is a schematic diagram illustrating the manual data upload and collection in an embodiment of the present invention;

[0028] Figure 6 This is a schematic diagram of data access in the unified construction system according to an embodiment of the present invention;

[0029] Figure 7 This is a schematic diagram of the verification task scheduling process in an embodiment of the present invention;

[0030] Figure 8 This is a flowchart of the data quality verification process in an embodiment of the present invention;

[0031] Figure 9 This is a flowchart illustrating the method for collecting monitoring data for railway infrastructure in an embodiment of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0033] The acquisition, transmission, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0034] In response to the current situation where railway infrastructure inspection and monitoring data are widely sourced, complex in structure, and fragmented in management, this invention proposes to construct a nationwide infrastructure inspection and monitoring data collection system. This system horizontally integrates multiple data collection channels and vertically constructs a "rule mapping-layered matching-dynamic execution" quality control mechanism, thereby achieving efficient collection and unified management of multi-disciplinary and multi-type inspection and monitoring data.

[0035] Figure 1 This is a schematic diagram of a data collection system for railway infrastructure inspection and monitoring in an embodiment of the present invention, such as... Figure 1 As shown, the data collection system 100 includes a data acquisition module 101, a dynamic execution control module 102, a data quality verification module 103, and a database 104.

[0036] The data acquisition module 101 is used to: acquire multi-source data according to the data source and a preset data transmission scheme, using a transmission method corresponding to the data source; the data source includes mobile detection devices, ground fixed monitoring devices, manual observation data, and one or any combination of various unified construction systems; the unified construction system is an existing railway infrastructure data acquisition and management system; the preset data transmission scheme includes a mapping relationship between data sources and transmission methods; the transmission methods include wireless communication, publish-subscribe mode, and batch upload by the client;

[0037] The dynamic execution control module 102 is used to: configure verification tasks with different priorities for multi-source data, call the data quality verification module 103 to execute verification tasks, and when the data quality verification module fails to verify the multi-source data, it will reject data upload or mark the data as erroneous and generate a data quality report.

[0038] The data quality verification module 103 includes a basic layer verification module, a business layer verification module, and a management layer verification module;

[0039] The basic layer verification module is used to verify whether multi-source data meets the preset specification requirements in one or any combination of dimensions such as standardization, uniqueness, integrity, consistency, availability, and timeliness.

[0040] The business layer verification module is used to: verify whether multi-source data meets preset business requirements; the preset business requirements reflect the data logic rationality of multi-source data in the railway infrastructure monitoring scenario;

[0041] The management verification module is used to verify whether multi-source data meets the preset process compliance requirements; the preset process compliance requirements indicate whether the multi-source data complies with upload time limits, planned version consistency, and traceability verification.

[0042] Database 104 is used to store multi-source data after being processed by dynamic execution control module 102 and data quality verification module 103.

[0043] The objectives to be achieved by the collection system in this embodiment of the invention include the following:

[0044] (1) To address the long-standing data barriers in professional fields such as engineering, electrical engineering, and power supply, a cross-professional data collection system is constructed to achieve unified collection and management of detection and monitoring data across the entire railway network.

[0045] (2) For multiple data sources such as mobile detection equipment, fixed monitoring equipment and professional unified construction system for infrastructure, three core data collection methods are integrated: vehicle-to-ground wireless transmission, manual uploading and unified construction system access, to build an efficient access channel covering multiple data sources.

[0046] (3) To address data quality issues such as non-standard, incomplete, and duplicate storage, a quality control mechanism of "rule mapping - hierarchical matching - dynamic execution" is constructed to ensure the quality of collected data.

[0047] In one embodiment, the multi-source data includes one or any combination of the following:

[0048] Data from high-speed integrated inspection trains, 3C inspection data from overhead contact lines, raw data from flaw detection vehicles, inspection data from various specialized inspection vehicles, data from fixed monitoring equipment, data from mobile monitoring equipment, and data from the unified construction system.

[0049] refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the data collection scope in this embodiment of the invention. The data collection scope includes status data of engineering, electrical, and power supply infrastructure obtained through mobile detection equipment, monitoring devices mounted on operating trains, ground-based fixed monitoring devices, small on-site instruments, and manual observation, as well as maintenance information, equipment ledgers, environmental meteorological data, and monitoring management data related to the analysis. Specifically, it includes:

[0050] (1) Dynamic detection data:

[0051] Track maintenance: track geometry, wheel force, vehicle dynamic response, rail inspection, fastener inspection, ballast bed inspection, roadbed inspection, tunnel inspection;

[0052] Electrical engineering: Field strength coverage, track circuits, quality of service, compensation capacitors, real-time spectrum, transponders, frequency sweeping, and onboard ATP (Automatic Train Protection) system;

[0053] Power supply: Contact line 1C, Contact line 3C, Contact line 4C, etc.;

[0054] (2) Maintenance data:

[0055] Engineering maintenance data, electrical maintenance data, power supply maintenance data;

[0056] (3) Monitoring and inspection data:

[0057] GSM-R (Global System for Mobile Communications-Railway, a digital mobile communication system specifically designed for railway communications) interface monitoring and static inspection data;

[0058] (4) Equipment ledger data:

[0059] Railway engineering: tracks, tunnels, gradients, stations, bridges, sleepers, turnouts, roadbeds, broken chains, curves, rails, etc.

[0060] Electrical systems: track circuits, transponders, base stations, repeaters, etc.

[0061] Power supply: support posts, structural markers, suspension types, etc.;

[0062] (5) Meteorological data:

[0063] Temperature, wind speed, precipitation, early warning data, and forecast data;

[0064] (6) Other data:

[0065] Comprehensive testing plan, specialized testing plan, invalid sections, total passing weight, etc.

[0066] By analyzing this data, maintenance personnel can promptly identify potential safety hazards, take proactive measures, and effectively prevent accidents. At the same time, they can also grasp the operational status of infrastructure, explore its changing patterns, and thus make more scientific, reasonable, and economical maintenance decisions.

[0067] Data collection source:

[0068] (1) High-speed integrated inspection train inspection data, catenary 3C inspection data, and raw data from flaw detection vehicles. These data, with the help of onboard equipment, establish a real-time vehicle-to-ground wireless transmission channel, breaking away from the limitations of the traditional manual copying and storage mode. The data can be automatically transferred from the onboard acquisition end to the ground receiving end without manual intervention, improving data transmission efficiency while ensuring data security and timeliness through automated processing.

[0069] (2) Inspection data from various specialized inspection vehicles and comprehensive inspection vehicles of the railway bureau, as well as data from fixed and mobile monitoring equipment. For inspection and monitoring scenarios that do not have vehicle-to-ground wireless transmission capabilities, a convenient data upload interface is created to support batch upload of multi-professional and multi-type inspection and monitoring files, thereby achieving effective data collection and management and ensuring that all types of data can be included in the data collection system.

[0070] (3) Data from unified systems such as PWMIS2.0 (Railway Engineering Management Information System Version 2.0), Engineering Safety Production Management System, Rail Flaw Detection and Damage Management Platform, Power Supply Safety Inspection and Monitoring 6C System Big Data Analysis System, xx Group-level 3C Unified Data Receiving Terminal, and Electrification Dynamic Inspection Data Management. By establishing efficient and stable data interfaces with these unified systems, the data collection system can automatically adapt to different data formats, enabling accurate reception and application interaction of data such as inspection and monitoring, maintenance feedback, and basic ledgers, ensuring "one source for one data," avoiding duplicate entry, and improving the consistency and accuracy of data management.

[0071] Figure 3 This is a schematic diagram of the overall architecture of the aggregation system in an embodiment of the present invention, with reference to... Figure 3 In this embodiment of the invention, the overall architecture of the data collection system integrates three collection channels: vehicle-to-ground wireless transmission, manual upload, and access from the unified construction system. It establishes a verification rule base corresponding to quality dimensions for testing and monitoring data from different sources, specialties, and types. Through dynamic configuration of verification strength and layered execution of quality verification, it achieves standardized data collection from various specialties and uniformly pushes the data to the railway infrastructure testing and monitoring data platform, supporting in-depth data governance and fusion analysis. Data processed by the collection system can be stored and displayed on the railway infrastructure testing data platform via file push or data push, facilitating user access.

[0072] The data sources can include track inspection vehicles, overhead contact line inspection vehicles, electrical inspection vehicles, integrated inspection trains, rail flaw detection vehicles, the group's engineering safety generation management system, and rail flaw detection and damage management platform, etc., and data can be uploaded through various methods such as manual clients, vehicle-to-ground wireless networks, and unified construction system access.

[0073] In one embodiment, the aggregation system may further include a fault-tolerant processing module for:

[0074] During the process of acquiring multi-source data in the data acquisition module 101, automatic retransmission, breakpoint resumption, and abnormal connection management are performed.

[0075] For example, refer to Figure 3 During data upload, the system enables batch uploading of detection data, resume upload of interrupted files, encrypted secure transmission, and abnormal message alerts.

[0076] The preset data transmission scheme in this embodiment of the invention includes a mapping relationship between data source and transmission method. For example, the vehicle-mounted device uses wireless communication to transmit data, and the unified construction system uses a publish-subscribe mode to transmit data.

[0077] In one embodiment, the data acquisition module 101 includes a vehicle-to-ground wireless transmission data collection module; the vehicle-to-ground wireless transmission data collection module is used to acquire multi-source data in real time from mobile detection devices and ground fixed monitoring devices according to priority based on 5G vehicle-to-ground wireless communication technology; a hybrid encryption mechanism of symmetric block cipher algorithm and public key cryptography algorithm based on elliptic curve cryptography is used during data transmission.

[0078] Figure 4 This is a schematic diagram of vehicle-to-ground wireless data collection in an embodiment of the present invention, as shown below. Figure 4 As shown, relying on 4G / 5G vehicle-to-ground wireless communication technology, real-time collection and transmission of detection data is achieved from the onboard equipment acquisition end to the ground receiving end. The onboard equipment acquisition end, such as a track geometry detection system, catenary detection system, communication signal monitoring system, or other detection systems, transmits data to the collection system via the onboard LAN-Internet / private network-railway internal service network. Specifically, the data is transmitted to the infrastructure detection data collection system through the onboard data transmission client, LAN firewall, wireless router, 4G / 5G Internet, internal service network firewall, and ground data receiving service. To ensure data security, a hybrid encryption mechanism using the authoritative certified symmetric block cipher algorithm SM4 and the public-key cryptography algorithm SM2 based on elliptic curve cryptography is employed to encrypt and protect the transmitted data. Data transmission supports both timed triggering and event triggering modes, with the ground receiving end handling data decryption and unpacking. The collection system is compatible with both TCP and UDP dual protocol stacks and features automatic retransmission, breakpoint resumption, and abnormal connection management functions to ensure the integrity and reliability of data transmission. Meanwhile, the aggregation system supports the parallel transmission of files, data streams, and instructions, and can dynamically adjust the transmission order according to business priorities to ensure that critical data is transmitted back first.

[0079] In one embodiment, the data acquisition module 101 includes a manually uploaded data collection module; the manually uploaded data collection module is used to receive file data uploaded by users in batches, divide the file data into data blocks of a specified size, and transmit the data blocks to the server via HTTP (HyperText Transfer Protocol) or Socket protocol, so that the server can reassemble the data blocks into file data; during the data transmission process, the file data is encrypted using a JWT (JSON Web Token) authentication mechanism based on AES (Advanced Encryption Standard).

[0080] Figure 5 This is a schematic diagram of manually uploaded data collection in an embodiment of the present invention, such as... Figure 5As shown, this data collection system includes a manual upload client and a data collection system server. It supports batch file uploads by users. The client automatically segments the files into appropriately sized data blocks and transmits them via HTTP or Socket protocols. Finally, the files are reassembled on the server side. For example, the system demonstrates a client for collecting track geometry data, overhead contact line data, or other data. Through technologies such as data encryption, file slicing, batch upload, and message notifications, the data is transmitted to the data collection system server. The server then performs data decryption, resume interrupted uploads, data verification, file integration, log management, rule configuration management, and data archiving management.

[0081] In terms of technical implementation, the aggregation system converts files into byte streams using FileInputStream (a Java class used to read raw byte streams from files), and dynamically adjusts the block size using an adaptive algorithm. Combined with multi-threaded concurrent transmission technology, it effectively improves the upload efficiency and transmission stability of large-scale files.

[0082] To ensure secure data transmission, the aggregation system integrates a JWT token authentication mechanism based on AES encryption, generating a dynamic token for each transmission session to prevent data loss or tampering during transmission. Simultaneously, the aggregation system possesses a robust fault-tolerance mechanism, including features such as resume capability and automatic retry, effectively handling abnormal situations such as network fluctuations. Furthermore, the aggregation system provides real-time status feedback, displaying users key information such as upload progress and verification results, ensuring users can promptly monitor file transfer status.

[0083] In one embodiment, the data acquisition module 101 includes a unified system access data collection module;

[0084] The unified construction system access data collection module is used to establish data interfaces with each unified construction system based on the publish-subscribe model message queue, and to access data according to the preset data format and push frequency based on the data interface.

[0085] Figure 6 This is a schematic diagram of data access in the unified construction system according to an embodiment of the present invention, such as... Figure 6 As shown, data interfaces are established with various unified construction systems based on Kafka message queues, and data is accessed according to preset data formats and push frequencies. Data providers include, for example, the track maintenance safety production management system, the rail flaw detection and damage management system, and the railway track maintenance management information system. Through the Kafka transmission channel (message queue), the data receiver, i.e., the aggregation system, obtains multi-source data from the unified construction systems. The data providers periodically push data to the message queue, which is then parsed and stored in the aggregation system. The aggregation system transmits reconciliation data, missing data messages, etc., to the data providers through the Kafka transmission channel.

[0086] Communication links are established by configuring core parameters such as the Kafka cluster address and consumer group ID. KafkaConsumer instances are used to subscribe to specified topics and continuously listen for messages. A reliable pull model is employed in the consumption phase. When new messages arrive on the subscribed topic, the consumer retrieves data from the corresponding partition according to the partitioning strategy and converts the byte stream into a Java object through deserialization. After receiving the data, the aggregation system interface module performs format conversion and parsing, then performs strict data validation and persists the data. The entire process is logged in detail, providing support for aggregation system monitoring and optimization.

[0087] Verification is performed after obtaining data from multiple sources. (Reference) Figure 3 Data quality verification can include basic layer verification, business layer verification, and management layer verification. The data quality verification module includes a basic layer verification module, a business layer verification module, and a management layer verification module.

[0088] The basic layer verification module is used to verify whether multi-source data meets the preset specification requirements in one or any combination of dimensions such as standardization, uniqueness, integrity, consistency, availability, and timeliness.

[0089] In one embodiment, the preset specification requirements include specification requirements for the structure and format of the data;

[0090] The basic layer verification module is specifically used to verify whether multi-source data meets the preset specification requirements through file scanning and regular expression matching.

[0091] The business layer verification module is used to verify whether multi-source data meets preset business requirements; the preset business requirements reflect the data logic rationality of multi-source data in the railway infrastructure monitoring scenario.

[0092] In one embodiment, the preset service requirements include one or any combination of the following:

[0093] Track gauge range, signal sequence continuity, planned mileage coverage integrity, record density verification, specified field non-empty, and transmission consistency.

[0094] The management verification module is used to verify whether multi-source data meets the preset process compliance requirements; the preset process compliance requirements indicate whether the multi-source data is compliant in terms of upload time limit, planned version consistency, and traceability verification.

[0095] In this embodiment of the invention, a complete data quality verification rule base is constructed, and a three-level control mechanism of "rule mapping - hierarchical matching - dynamic execution" is used to realize full-process data quality control.

[0096] (1) Rule mapping mechanism: The six quality dimensions of standardization, uniqueness, completeness, consistency, availability, and timeliness are transformed into executable verification rules to ensure that the rules strictly correspond to the quality requirements. Table 1 lists typical verification rules for three types of test data. The rule base supports dynamic expansion. When adding a new rule, the quality dimension to which it belongs, the applicable data type, and the verification logic must be declared.

[0097] Table 1 Typical Verification Rules

[0098]

[0099] (2) Layered Verification Model: The data collection system adopts a gradient verification model of basic layer → business layer → management layer to carry out data quality verification. The basic layer relies on a file scanner and regular expression engine to achieve second-level interception; the business layer loads a professional threshold library and association rules, and completes logical verification through a rule engine; the management layer relies on a workflow engine to verify consistency and data timeliness and other process compliance requirements. Table 2 shows the layered verification model.

[0100] Table 2

[0101]

[0102] The dynamic execution control module is used to: configure verification tasks with different priorities for multi-source data, call the data quality verification module to execute verification tasks, and when the data quality verification module fails to verify the multi-source data to meet the standards, it will reject the data upload or mark the data as erroneous and generate a data quality report.

[0103] The aggregation system supports dynamic configuration of rule strength, applying differentiated processing to blocking and non-blocking rules based on the rule strength matrix. Table 3 shows the rule execution control matrix. For example, blocking issues such as file format errors are intercepted in real time, while non-blocking anomalies such as discontinuous signal devices are marked in near real time to ensure uninterrupted critical business operations.

[0104] Table 3 Rule Enforcement Control Matrix

[0105]

[0106]

[0107] In a preferred embodiment, the dynamic execution control module is specifically used to: optimize preset specification requirements, preset business requirements, and preset process compliance requirements through a feedback mechanism; the optimization includes the canary release and rollback of the preset specification requirements, preset business requirements, and preset process compliance requirements.

[0108] During implementation, all anomalies are optimized through a closed-loop feedback mechanism. For example, recurring non-blocking anomalies can be upgraded to blocking rules. The aggregation system's version control function supports rule canary releases and rollbacks, ensuring the continuous evolution of verification strategies.

[0109] In one embodiment, the dynamic execution control module is specifically used to: dynamically adjust the priority of verification tasks based on data importance, business urgency, and data type; dynamically schedule the data quality verification module to execute verification tasks based on priority; and construct a multi-threaded resource pool during the scheduling process to allocate thread resources to verification tasks of different priorities according to a specified ratio gradient.

[0110] The data collection system employs a priority-based dynamic task scheduling mechanism, dividing verification tasks into high, medium, and low priorities, and dynamically adjusting priorities based on data importance, business urgency, and data type. The scheduling process utilizes multi-threaded resource pool management, allocating thread resources in a 4:2:1 gradient to ensure high-priority tasks are executed first. When the system load is high, low-priority tasks are automatically paused from being enqueued, and ongoing low-priority tasks are slowed down to maintain the stable operation of critical tasks. Task management is implemented using a priority queue, with tasks enqueued according to priority. Tasks of the same priority follow a first-in, first-out (FIFO) strategy. The scheduler continuously retrieves tasks from the head of the queue and assigns them to threads for execution. After a task is completed, it is automatically removed, and the next task is processed, until the queue is empty.

[0111] Figure 7 This is a schematic diagram of the verification task scheduling process in an embodiment of the present invention, such as... Figure 7 As shown, the collection system automatically triggers new verification task instances by monitoring data retransmission and correction in real time, and dynamically inserts them into the execution queue to ensure timely quality verification after data changes. Specifically, this includes:

[0112] 1) Data quality verification configuration;

[0113] 2) Task Scheduler:

[0114] Task priorities are divided into high, medium, and low;

[0115] Use the pre-generated list of today's task execution plans to generate a list of today's task instance execution plans;

[0116] Create and execute task instances by triggering them on schedule;

[0117] Add them to the queue to form a queue of task instances to be executed;

[0118] Submit for execution, and route the call according to the service name;

[0119] 3) Task executor:

[0120] Batch-based standardization verification service;

[0121] Batch integrity verification service;

[0122] Batch timeliness verification service;

[0123] Batch uniqueness verification service;

[0124] Batch availability verification service; ...

[0126] 4) Automatic trigger mode:

[0127] Update data object entities through data producers;

[0128] The data monitor monitors data object entities;

[0129] Data update messages are sent back to the task instance.

[0130] Figure 8 This is a flowchart of the data quality verification process in an embodiment of the present invention, for reference only. Figure 8 The data collection system employs a three-level progressive verification mechanism, conducting data quality verification sequentially in the order of the basic layer, business layer, and management layer.

[0131] The basic layer validation uses a file scanner and regular expression engine to rigorously verify the structure and format specifications of the data, such as naming convention validation, file format validation, file duplication detection, and file read validation. For detected blocking anomalies, the aggregation system automatically performs differentiated processing based on the data source.

[0132] If the vehicle-to-ground wireless data transmission is abnormal, the collection system will automatically send a retransmission command to the vehicle terminal.

[0133] If a manually uploaded data error occurs, the error details will be provided in real time and the upload will be blocked, requiring the user to correct and resubmit.

[0134] If there is a problem with the data accessed by the unified system, a data repair notification will be triggered, allowing the source system to repair or resynchronize the data.

[0135] Data that passes the basic layer verification will then proceed to the business layer and management layer verification, where business rule logic verification and management indicator checks will be performed. These checks include field non-empty verification, key information matching verification, consistency with the detection plan verification, field value range verification, upload timeliness verification, detection frequency integrity verification, system data synchronization verification, and traceability verification. Blocking anomalies will be handled using the basic layer's handling strategy, while non-blocking anomalies will be marked and recorded in a quality report. Data that passes verification will be stored in the database. After verification, the system automatically generates a report containing multi-dimensional quality assessment results and specific rectification suggestions. Simultaneously, through rule optimization, the rule base will be dynamically updated, and high-frequency non-blocking anomalies will be upgraded to blocking rules. Through a tiered interception mechanism and differentiated error handling strategies, data quality is ensured while verification efficiency is considered, constructing a complete data quality management closed loop of "verification-interception-repair-feedback".

[0136] In a preferred embodiment, the data collection system for railway infrastructure inspection and monitoring adopts a layered architecture design, which is divided into a system layer, a data layer, a service layer, a transmission layer and a presentation layer, with clear division of labor and collaborative work among the layers.

[0137] The system layer includes the basic runtime environment settings used by the underlying layers.

[0138] The system layer mainly includes the underlying Java development toolkit, operating system and other basic operating environment, Windows Server (Windows Server operating system, for example, Windows Server 12 in this embodiment of the invention) and CentOS (CentOS is an abbreviation for Community Enterprise Operating System, for example, CentOS 7 in this embodiment of the invention). The software runs on the open source JDK 1.8 as support, providing a stable operating foundation for the system.

[0139] The data layer includes a database and file-related data management components; the file-related data management components include a Redis component for caching data.

[0140] The data layer mainly includes databases, file storage, and related data management software. It uses MySQL as a relational database for structured data storage, Redis as a caching component, Mybatis as the ORM (Object Relational Mapping Framework), and Minio (an open-source, high-performance object storage server) for file storage, ensuring efficient data storage and retrieval.

[0141] The service layer adopts a microservice architecture and is used to implement the functions of the dynamic execution control module and the data quality verification module.

[0142] The service layer is a backend function that supports the overall data collection, verification and other business operations of the system. It mainly adopts the Spring Cloud microservice architecture and implements access control through Spring Security. It has good scalability and maintainability, and functional modules can be added and modified according to business needs.

[0143] The transport layer is used to implement file upload and download functions by employing 5G vehicle-to-ground wireless communication technology, web servers, and reverse proxy servers.

[0144] The transport layer supports file upload and download functions. For example, it uses 4G / 5G vehicle-to-ground wireless communication technology and Caddy high-performance web services to ensure file transfer speed and security.

[0145] The presentation layer includes the client and the server. The client uses Vue and JavaScript technologies to implement graphical display.

[0146] The presentation layer mainly consists of user-facing service pages such as client-side and server-side components. The client-side uses JavaScript, HTML, and CSS to build cross-platform desktop applications; for example, it uses Electron for one-click installation and leverages technologies like Vue and JavaScript to display data graphically, enhancing the user experience.

[0147] This invention, for the first time, constructs a cross-disciplinary data collection system, achieving unified collection and management of monitoring data across the entire railway network. It establishes an efficient access channel covering three core data collection methods: vehicle-to-ground wireless transmission, manual uploading, and unified system access. A "rule mapping-layered matching-dynamic execution" quality control mechanism ensures the quality of the collected data. Through this invention, a dynamic data resource pool covering multiple fields such as track, overhead contact line, communication, and signaling will be formed, completely changing the traditional "data silo" pattern and providing data support for cross-disciplinary collaborative analysis and infrastructure lifecycle management and operation and maintenance decisions.

[0148] The beneficial effects of the embodiments of the present invention are:

[0149] (1) The embodiments of the present invention successfully broke through the long-standing data barriers in professional fields such as engineering, electrical engineering, and power supply, and for the first time achieved the unified collection of 57 types of detection and monitoring data in 23 categories across the entire railway.

[0150] (2) The embodiments of the present invention integrate three core data collection methods: vehicle-to-ground wireless transmission, manual uploading, and unified system access, to build an efficient access channel covering multi-source data.

[0151] (3) The embodiments of the present invention construct a quality control mechanism of "rule mapping-layered matching-dynamic execution", which effectively ensures the quality of collected data through dynamic rule engine and progressive verification process.

[0152] This invention also provides a method for collecting monitoring data for railway infrastructure, as described in the following embodiments. Since the principle behind this method is similar to that of the system for collecting monitoring data for railway infrastructure, its implementation can refer to the implementation of the system for collecting monitoring data for railway infrastructure; details will not be repeated.

[0153] Figure 9 This is a flowchart illustrating the method for collecting monitoring and inspection data for railway infrastructure in an embodiment of the present invention, as shown below. Figure 9 As shown, the method includes:

[0154] Step 901: Based on the data source and the preset data transmission scheme, acquire multi-source data using a transmission method corresponding to the data source; the data source includes mobile detection devices, ground fixed monitoring devices, manual observation data, and one or any combination of various unified construction systems; the unified construction system is the existing railway infrastructure data acquisition and management system; the preset data transmission scheme includes a mapping relationship between data sources and transmission methods; the transmission methods include wireless communication, publish-subscribe mode, and batch upload from the client.

[0155] Step 902: Configure verification tasks with different priorities for multi-source data;

[0156] Step 903: Perform verification tasks; if the multi-source data fails to meet the standards, reject the data upload or mark the data as erroneous, and generate a data quality report; the verification tasks include: verifying whether the multi-source data meets preset standard requirements in one or any combination of dimensions such as standardization, uniqueness, completeness, consistency, availability, and timeliness; verifying whether the multi-source data meets preset business requirements; the preset business requirements reflect the data logic rationality of the multi-source data in the railway infrastructure monitoring scenario; verifying whether the multi-source data meets preset process compliance requirements; the preset process compliance requirements indicate whether the multi-source data complies with upload time limits, plan version consistency, and traceability verification.

[0157] Step 904: Store the verified multi-source data in the database.

[0158] In one embodiment, the database includes a relational database for structured data storage and a Redis database for caching data.

[0159] In one embodiment, multi-source data is acquired by using a transmission method corresponding to the data source, based on the data source and a preset data transmission scheme, including:

[0160] Based on 5G vehicle-to-ground wireless communication technology, multi-source data is acquired in real time from mobile detection devices and ground-based fixed monitoring devices according to priority; a hybrid encryption mechanism using symmetric block cipher algorithm and public-key cryptography algorithm based on elliptic curve cryptography is used during data transmission.

[0161] In one embodiment, multi-source data is acquired by using a transmission method corresponding to the data source, based on the data source and a preset data transmission scheme, including:

[0162] It receives batch file data uploaded by users, divides the file data into data blocks of a specified size, and transmits the data blocks to the server via HTTP or Socket protocol so that the server can reassemble the data blocks into file data; during the data transmission process, the file data is encrypted using a JWT authentication mechanism based on AES encryption.

[0163] In one embodiment, multi-source data is acquired by using a transmission method corresponding to the data source, based on the data source and a preset data transmission scheme, including:

[0164] The message queue based on the publish-subscribe model establishes a data interface with various unified construction systems, and data is accessed based on the data interface according to the preset data format and push frequency.

[0165] In one embodiment, multi-source data is acquired by adopting a transmission method corresponding to the data source according to the data source and a preset data transmission scheme, including: automatic retransmission, breakpoint resumption, and abnormal connection management during the acquisition of multi-source data.

[0166] In one embodiment, configuring verification tasks with different priorities for multi-source data includes: optimizing preset specification requirements, preset business requirements, and preset process compliance requirements through a feedback mechanism; the optimization includes the canary release and rollback of preset specification requirements, preset business requirements, and preset process compliance requirements.

[0167] In one embodiment, configuring verification tasks with different priorities for multi-source data includes: dynamically adjusting the priority of verification tasks based on data importance, business urgency, and data type; dynamically scheduling the data quality verification module to execute verification tasks based on priority; and constructing a multi-threaded resource pool during the scheduling process to allocate thread resources to verification tasks with different priorities according to a specified ratio gradient.

[0168] In one embodiment, the preset specification requirements include specification requirements for the structure and format of the data;

[0169] Perform verification tasks, including: verifying whether multi-source data meets preset specification requirements through file scanning and regular expression matching.

[0170] In one embodiment, the preset service requirements include one or any combination of the following:

[0171] Track gauge range, signal sequence continuity, planned mileage coverage integrity, record density verification, specified field non-empty, and transmission consistency.

[0172] In one embodiment, the multi-source data includes one or any combination of the following:

[0173] Data from high-speed integrated inspection trains, 3C inspection data from overhead contact lines, raw data from flaw detection vehicles, inspection data from various specialized inspection vehicles, data from fixed monitoring equipment, data from mobile monitoring equipment, and data from the unified construction system.

[0174] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described method for collecting monitoring data for railway infrastructure.

[0175] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for collecting monitoring and detection data for railway infrastructure.

[0176] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described method for collecting monitoring data for railway infrastructure.

[0177] The data collection system for railway infrastructure inspection and monitoring in this embodiment includes a data acquisition module, a dynamic execution control module, a data quality verification module, and a database. These modules collaborate to address long-standing data barriers in engineering, electrical, and power supply fields, constructing a cross-disciplinary data collection system. This achieves unified collection and management of inspection and monitoring data across the entire railway network. For multiple data sources, including mobile inspection devices, ground-based fixed monitoring devices, manual observation data, and various unified construction systems, it integrates three core data collection methods: wireless communication, publish-subscribe mode, and batch upload access from clients. This establishes an efficient access channel covering multiple data sources. Furthermore, to address data quality issues such as non-standardized, incomplete, and duplicate storage, the data quality verification module performs quality control, improving the quality of the collected data.

[0178] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0179] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0180] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0181] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0182] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A data collection system for railway infrastructure inspection and monitoring, characterized in that, include: Data acquisition module, dynamic execution control module, data quality verification module, database; The data acquisition module is used to: acquire multi-source data according to the data source and a preset data transmission scheme, using a transmission method corresponding to the data source; the data source includes mobile detection devices, ground fixed monitoring devices, manual observation data, and one or any combination of various unified construction systems; the unified construction system is an existing railway infrastructure data acquisition and management system; the preset data transmission scheme includes a mapping relationship between data sources and transmission methods; the transmission methods include wireless communication, publish-subscribe mode, and batch upload from the client. The dynamic execution control module is used to: configure verification tasks with different priorities for multi-source data, call the data quality verification module to execute verification tasks, and when the data quality verification module fails to verify the multi-source data to meet the standards, it will reject data upload or mark the data as erroneous and generate a data quality report. The data quality verification module includes a basic layer verification module, a business layer verification module, and a management layer verification module. The basic layer verification module is used to verify whether multi-source data meets the preset specification requirements in one or any combination of dimensions such as standardization, uniqueness, integrity, consistency, availability, and timeliness. The business layer verification module is used to: verify whether multi-source data meets preset business requirements; the preset business requirements reflect the data logic rationality of multi-source data in the railway infrastructure monitoring scenario; The management verification module is used to verify whether multi-source data meets the preset process compliance requirements; The preset process compliance requirements indicate whether multi-source data complies with upload time limits, planned version consistency, and traceability verification. The database is used to store multi-source data after it has been processed by the dynamic execution control module and the data quality verification module.

2. The data collection system for railway infrastructure inspection and monitoring as described in claim 1, characterized in that, The data collection system for railway infrastructure monitoring adopts a layered architecture design, consisting of a system layer, a data layer, a service layer, a transmission layer, and a presentation layer. The system layer includes the basic runtime environment settings used by the underlying layers; The data layer includes a database and file-related data management components; the file-related data management components include a Redis component for caching data. The service layer adopts a microservice architecture and is used to implement the functions of the dynamic execution control module and the data quality verification module. The transport layer is used to implement file upload and download functions by employing 5G vehicle-to-ground wireless communication technology, a web server, and a reverse proxy server. The presentation layer includes the client and the server. The client uses Vue and JavaScript technologies to implement graphical display.

3. The data collection system for railway infrastructure inspection and monitoring as described in claim 1, characterized in that, The data acquisition module includes a vehicle-to-ground wireless transmission data collection module; The vehicle-to-ground wireless transmission data collection module is used to acquire multi-source data in real time from mobile detection devices and ground fixed monitoring devices according to priority, based on 5G vehicle-to-ground wireless communication technology. The data transmission process employs a hybrid encryption mechanism that combines symmetric block cipher algorithms with public-key cryptography algorithms based on elliptic curve cryptography.

4. The data collection system for railway infrastructure inspection and monitoring as described in claim 1, characterized in that, The data acquisition module includes a manual data upload and collection module; The manual upload data collection module is used to receive batch file data uploaded by users, divide the file data into data blocks of a specified size, and transmit the data blocks to the server via HTTP or Socket protocol so that the server can reassemble the data blocks into file data. During data transmission, file data is encrypted using a network token JWT authentication mechanism based on the Advanced Encryption Standard (AES).

5. The data collection system for railway infrastructure inspection and monitoring as described in claim 1, characterized in that, The data acquisition module includes a unified system access data collection module; The unified construction system access data collection module is used to establish data interfaces with each unified construction system based on the publish-subscribe model message queue, and to access data according to the preset data format and push frequency based on the data interface.

6. The data collection system for railway infrastructure inspection and monitoring as described in any one of claims 3 to 5, characterized in that, It also includes a fault tolerance module, used for: During the process of acquiring data from multiple sources in the data acquisition module, automatic retransmission, breakpoint resumption, and abnormal connection management are implemented.

7. The data collection system for railway infrastructure inspection and monitoring as described in claim 1, characterized in that, The dynamic execution control module is specifically used for: The preset specification requirements, preset business requirements, and preset process compliance requirements are optimized through a feedback mechanism; the optimization includes the canary release and rollback of the preset specification requirements, preset business requirements, and preset process compliance requirements.

8. The data collection system for railway infrastructure inspection and monitoring as described in claim 1, characterized in that, The dynamic execution control module is specifically used for: The priority of verification tasks is dynamically adjusted based on data importance, business urgency, and data type. The data quality verification module is dynamically scheduled to execute verification tasks based on priority. During the scheduling process, a multi-threaded resource pool is built, and thread resources are allocated to verification tasks of different priorities according to a specified ratio gradient.

9. The data collection system for railway infrastructure inspection and monitoring as described in claim 1, characterized in that, The preset specifications include requirements for the structure and format of the data; The basic layer verification module is specifically used to verify whether multi-source data meets the preset specification requirements through file scanning and regular expression matching.

10. The data collection system for railway infrastructure inspection and monitoring as described in claim 1, characterized in that, The preset service requirements include one or any combination of the following: Track gauge range, signal sequence continuity, planned mileage coverage integrity, record density verification, specified field non-empty, and transmission consistency.

11. The data collection system for railway infrastructure inspection and monitoring as described in claim 1, characterized in that, The multi-source data includes one or any combination of the following: Data from high-speed integrated inspection trains, 3C inspection data from overhead contact lines, raw data from flaw detection vehicles, inspection data from various specialized inspection vehicles, data from fixed monitoring equipment, data from mobile monitoring equipment, and data from the unified construction system.

12. A method for collecting monitoring and inspection data for railway infrastructure, characterized in that, include: Based on the data sources and a pre-defined data transmission scheme, multi-source data is acquired using a transmission method corresponding to the data sources. The data sources include mobile detection devices, ground-based fixed monitoring devices, manual observation data, and one or any combination of various unified construction systems. The unified construction system is an existing railway infrastructure data acquisition and management system. The pre-defined data transmission scheme includes a mapping relationship between data sources and transmission methods. The transmission methods include wireless communication, publish-subscribe mode, and batch upload from the client. Configure verification tasks with different priorities for multi-source data; Perform the verification task; When multi-source data fails to meet the standards, the data upload is rejected or erroneous data is marked, and a data quality report is generated. The verification tasks include: verifying whether the multi-source data meets preset standard requirements in one or any combination of dimensions such as standardization, uniqueness, completeness, consistency, availability, and timeliness; verifying whether the multi-source data meets preset business requirements; these preset business requirements reflect the logical rationality of the multi-source data in the railway infrastructure monitoring scenario; and verifying whether the multi-source data meets preset process compliance requirements; these preset process compliance requirements indicate whether the multi-source data complies with upload time limits, plan version consistency, and traceability verification. The verified multi-source data is stored in the database.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of claim 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of claim 12.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of claim 12.

Images