Railway bridge monitoring data aggregation method

Abstract

The invention relates to the technical field of railway traffic, and particularly discloses a railway bridge monitoring data aggregation method, which comprises the following steps of: pushing monitoring data to a data aggregation system through a monitoring device according to a preset data format; the data aggregation system performs data cleaning treatment on the received monitoring data, wherein the data cleaning treatment comprises data legality verification and duplicate removal; packaging and transmitting the cleaned and treated monitoring data to a first-level convergence center; the first-level convergence center carries out analysis and object-model mapping on the received data, and finally writes the data into a database cluster for storage; wherein the data aggregation system is adapted to a wired transmission mode or a wireless transmission mode to complete data aggregation according to a network environment where the monitoring device is located.

Description

Technical Field

[0001] This invention relates to the field of railway transportation technology, and specifically to a method for collecting monitoring data from railway bridges. Background Technology

[0002] In accordance with the needs of railway bridge maintenance and operation, various large-span railway bridges and key bridges have successively constructed and installed bridge health monitoring, navigation bridge collision monitoring, and collision monitoring devices for railway-crossing public overpasses and height-limited protective frames. As required by the railway authorities, the monitoring data generated by the above-mentioned bridge monitoring devices need to be aggregated into the railway bridge monitoring information management system in real time.

[0003] Currently, railway bridge monitoring devices generally use RESTful API interfaces, message queues, and file transfers to transmit monitoring data from front-end devices to back-end information systems, which presents the following problems: (1) The front-end monitoring device directly calls the server interface to push data. As the number of front-end devices increases, the server's concurrent processing capability becomes a bottleneck.

[0004] (2) Data access verification, information deduplication and other data cleaning and governance work are usually carried out on the server side. The operation and calculation are concentrated on the server side, which can easily lead to excessive server load and untimely data processing.

[0005] (3) Some front-end monitoring devices are located in the Internet environment, and it is necessary to realize cross-network transmission of monitoring data from the Internet to the internal railway network and ensure information security during the transmission process.

[0006] Therefore, how to solve the problem of real-time aggregation and cross-network transmission of large amounts of bridge monitoring data, which is difficult to handle in existing technologies, is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0007] To achieve the objective of this invention, this application provides a method for collecting monitoring data from railway bridges, comprising: The monitoring device pushes monitoring data to the data aggregation system according to a preset data format; The data aggregation system performs data cleaning and processing on the received monitoring data, including data validity verification and deduplication. The monitoring data, after cleaning and treatment, is packaged and transmitted to the primary aggregation center; The primary aggregation center parses the received data, performs object-model mapping, and finally writes it into the database cluster for storage. The data aggregation system adapts to either wired or wireless transmission methods to aggregate data, depending on the network environment in which the monitoring device is located.

[0008] In some specific embodiments, when a wired transmission method is used, the data aggregation system includes a load cluster, an interface service cluster, and a cache cluster deployed at the secondary aggregation point, as well as a message queue cluster and a listening service cluster deployed at the primary aggregation center. The data cleaning and governance are performed by the cache cluster. The encapsulation is performed by the interface service cluster, which reads the managed data from the cache cluster through a scheduled task, encapsulates it, and writes it into the message queue cluster of the primary aggregation center.

[0009] In some specific embodiments, the monitoring device calls the interface exposed by the load cluster to push data; the load cluster uses a round-robin method to reverse proxy the request to a node in the interface service cluster.

[0010] In some specific embodiments, the listening service cluster of the primary aggregation center listens to the message queue cluster in real time, obtains and parses the data, performs object-model mapping, and writes it to the database cluster.

[0011] In some specific embodiments, the following are included: When using wireless transmission, the monitoring device pushes the encrypted monitoring data to the external interface service cluster deployed in the Internet environment via the wireless network.

[0012] In some specific embodiments, when a wireless transmission method is used, the method further includes: The external network interface service cluster will forward the request to the internal network interface service cluster. The external network to internal interface service cluster requests authentication service to obtain authentication information, and after encapsulating the authentication information together with encrypted data packets, pushes it to the railway internal network through the security platform; The intranet interface service cluster deployed in the railway's internal network receives data packets from the security platform, parses and decrypts them, performs object-model mapping, and writes them to the database cluster.

[0013] In some specific embodiments, the monitoring data includes at least one of railway bridge health monitoring data, navigation bridge collision avoidance monitoring data, and collision monitoring data of railway-crossing public overpasses and height-restricted protective frames.

[0014] In some specific embodiments, the nodes performing critical services in the data aggregation system are deployed in a clustered manner.

[0015] In some specific embodiments, in the wired transmission method, the data cleaning and processing is performed at the secondary aggregation point beforehand, thereby reducing the computational load on the primary aggregation center.

[0016] In some specific embodiments, cross-network transmission is achieved through a data encryption and security platform in the wireless transmission method to ensure data transmission security.

[0017] The beneficial effects of the above technical solution are as follows: (1) The wired and wireless transmission methods provided by the present invention can be adapted to different bridge monitoring network environments to realize the real-time aggregation of bridge monitoring data.

[0018] (2) The wired transmission method in this invention sets up a secondary aggregation point, and performs data verification, deduplication, encapsulation and other governance actions in advance, thereby reducing the computational pressure on the aggregation center and improving the concurrent response capability.

[0019] (3) The wireless transmission method in this invention achieves secure cross-network transmission of monitoring data through technologies such as data encryption, authentication services, and security platforms.

[0020] (4) The key nodes involved in this invention all adopt a cluster approach, which can effectively avoid the interruption of the transmission process due to single-point equipment failure and improve the reliability of bridge monitoring data aggregation. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating a method for aggregating monitoring data of a railway bridge according to Embodiment 1 of the present invention. Figure 2 This is a schematic diagram illustrating the wired transmission method of a railway bridge monitoring data aggregation method provided in Embodiment 1 of the present invention. Figure 3 This is a flowchart illustrating a method for aggregating monitoring data of a railway bridge according to Embodiment 2 of the present invention. Figure 4 This is a schematic diagram illustrating the specific process of wireless transmission in a railway bridge monitoring data aggregation method provided in Embodiment 2 of the present invention. Detailed Implementation

[0023] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0024] Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the invention, and should not be construed as limiting the invention.

[0025] Example 1 One embodiment of the present invention provides a method for data aggregation of railway bridge monitoring, referring to... Figure 1 , Figure 2 As shown, it includes: The monitoring device pushes monitoring data to the data aggregation system according to a preset data format; The data aggregation system performs data cleaning and processing on the received monitoring data, including data validity verification and deduplication. The monitoring data, after cleaning and treatment, is packaged and transmitted to the primary aggregation center; The primary aggregation center parses the received data, performs object-model mapping, and finally writes it into the database cluster for storage. The data aggregation system adapts to the wired transmission method to complete the data aggregation based on the network environment in which the monitoring device is located.

[0026] In one specific embodiment of the present invention, it includes: When the wired transmission method is used, the data aggregation system includes a load cluster, an interface service cluster, and a cache cluster deployed at the secondary aggregation point, as well as a message queue cluster and a listening service cluster deployed at the primary aggregation center. The data cleaning and governance are performed by the cache cluster; The encapsulation and transmission are performed by the interface service cluster. The data is read from the cache cluster by a scheduled task, encapsulated, and then written into the message queue cluster of the primary aggregation center.

[0027] In one specific embodiment of the present invention, the monitoring device calls the interface exposed by the load cluster to push data; the load cluster uses a round-robin method to reverse proxy the request to a node in the interface service cluster.

[0028] In one specific embodiment of the present invention, the listening service cluster of the primary aggregation center listens to the message queue cluster in real time, obtains and parses the data, performs object-model mapping, and writes it to the database cluster.

[0029] In one specific embodiment of the present invention, the monitoring data includes at least one of railway bridge health monitoring data, navigation bridge collision avoidance monitoring data, and collision monitoring data of railway-crossing public overpasses and height-restricted protective frames.

[0030] In one specific embodiment of the present invention, the nodes performing key services in the data aggregation system are all deployed in a clustered manner.

[0031] In one specific embodiment of the present invention, in the wired transmission method, the data cleaning and processing is performed at the secondary aggregation point beforehand, thereby reducing the computational load on the primary aggregation center.

[0032] This embodiment details the specific process of data aggregation using wired transmission within a railway internal network environment. The core of this method lies in setting up secondary aggregation points to bring data cleaning and processing forward, thereby alleviating the pressure on the primary aggregation center and improving the overall concurrent processing capacity and reliability of the system.

[0033] The specific implementation steps are as follows: Monitoring data push: Monitoring devices (such as sensors and data collectors) deployed at railway bridge sites actively push real-time collected monitoring data according to preset and agreed data formats (such as JSON or XML) by calling the unified API interface exposed by the load cluster. This embodiment provides a unified and single access point for all front-end monitoring devices, simplifying front-end configuration and avoiding front-end devices directly facing the complex service topology of the back-end.

[0034] Load balancing and reverse proxy: After receiving a request from the monitoring device, the load balancing cluster (such as Nginx or HAProxy) uses a round-robin strategy to reverse proxy the request to an available node in the interface service cluster (a microservice cluster composed of multiple processing nodes). This embodiment distributes massive concurrent requests across multiple interface service nodes through load balancing, effectively preventing a single service node from crashing due to request overload, greatly improving the system's concurrent response capability and throughput, and solving the server-side concurrency bottleneck problem mentioned in the background technology.

[0035] Asynchronous write to the cache: Upon receiving a request, the interface service node quickly writes the monitoring data to a cache cluster (such as Redis or Memcached) and then immediately returns a response to the monitoring device. This decouples data persistence from business processing. This embodiment implements asynchronous data processing, allowing the interface service to respond quickly to the front end, reducing request waiting time and avoiding request blocking caused by slow backend processing. Simultaneously, the high read / write performance of the cache cluster provides a high-speed buffer for subsequent data cleaning.

[0036] Pre-processing data cleaning and governance: The cache cluster, or its associated dedicated data processing service, performs data cleaning and governance tasks on the temporarily stored data, including data validity verification (such as checking data format, range, and validity) and information deduplication (such as removing duplicate reports based on data identifiers and timestamps). This is the key improvement of this method. Moving the computationally intensive cleaning work, originally performed at the primary aggregation center, to the secondary aggregation point significantly reduces the computational load on the primary aggregation center, fundamentally solving the problems of excessive server load and untimely data processing. The secondary aggregation point can be horizontally scaled and dedicated to data governance.

[0037] Scheduled Encapsulation and Transmission: The interface service cluster, as a whole, executes a scheduled task (e.g., once per minute) to proactively read cleaned and processed monitoring data from the cache cluster, encapsulate it according to the specific format required by the primary aggregation center (e.g., package it into batch messages), and then write it to the primary aggregation center's message queue cluster (e.g., Kafka, RocketMQ). This consolidates scattered requests into batches of standardized data packets for transmission, reducing network communication and improving transmission efficiency. Asynchronous communication via message queues decouples the secondary aggregation point from the primary aggregation center. Even if the aggregation center is temporarily unavailable, data is not lost but temporarily stored in the message queue, ensuring data reliability.

[0038] Real-time monitoring and parsing: A monitoring service cluster deployed in the primary aggregation center monitors the message queue cluster in real time. Once new data arrives, it is immediately acquired, parsed, and the structured monitoring information is reconstructed. The monitoring service cluster also employs a clustered deployment, enabling parallel consumption of data from the message queue to ensure timely processing and storage, preventing data backlog.

[0039] Data Mapping and Persistence: The monitoring service cluster uses technologies such as Object-Relational Mapping (ORM) to populate the parsed data into the corresponding data model entities, and finally writes the complete entity objects to the database cluster (such as a MySQL cluster or PostgreSQL cluster) for persistent storage. This embodiment realizes the conversion of data from transmission format to business model, and finally securely stores it in the database for querying, analysis, and display by the upper-level railway bridge monitoring information management system.

[0040] This embodiment, through the aforementioned hierarchical architecture and process, successfully distributes the computational pressure, solves the concurrency bottleneck and high server load problem when large-scale monitoring data is aggregated in real time, and significantly improves the system's processing power and reliability.

[0041] Example 2 One embodiment of the present invention provides a method for data aggregation of railway bridge monitoring, referring to... Figure 3 , Figure 4 As shown, it includes: The monitoring device pushes monitoring data to the data aggregation system according to a preset data format; The data aggregation system performs data cleaning and processing on the received monitoring data, including data validity verification and deduplication. The monitoring data, after cleaning and treatment, is packaged and transmitted to the primary aggregation center; The primary aggregation center parses the received data, performs object-model mapping, and finally writes it into the database cluster for storage. The data aggregation system adapts to the wireless transmission method based on the network environment in which the monitoring device is located to complete the data aggregation.

[0042] In one specific embodiment of the present invention, it includes: When the wireless transmission method is used, the monitoring device pushes the encrypted monitoring data to the external interface service cluster deployed in the Internet environment via the wireless network.

[0043] In one specific embodiment of the present invention, the wireless transmission method further includes: The external network interface service cluster will forward the request to the internal network interface service cluster. The external network to internal interface service cluster requests authentication service to obtain authentication information, and after encapsulating the authentication information together with encrypted data packets, pushes it to the railway internal network through the security platform; The intranet interface service cluster deployed in the railway's internal network receives data packets from the security platform, parses and decrypts them, performs object-model mapping, and writes them to the database cluster.

[0044] In one specific embodiment of the present invention, the monitoring device is equipped with a directional IoT card, which accesses the Internet via a 4G or 5G network.

[0045] In one specific embodiment of the present invention, the monitoring data includes at least one of railway bridge health monitoring data, navigation bridge collision avoidance monitoring data, and collision monitoring data of railway-crossing public overpasses and height-restricted protective frames.

[0046] In one specific embodiment of the present invention, the nodes performing key services in the data aggregation system are all deployed in a clustered manner.

[0047] The specific implementation steps are as follows: Encrypted Data Push: Monitoring devices deployed in internet environments (such as remote sensors connected via 4G / 5G networks) first encrypt the monitoring data using a hybrid encryption scheme combining the national standard SM2 elliptic curve public-key cryptography algorithm and the SM4 block cipher algorithm. Then, monitoring devices equipped with directional IoT cards, via the 4G / 5G network, call the external network interface service cluster deployed in the cloud to access the internet's exposed API interface, pushing the encrypted monitoring data in a pre-defined format. Data encryption ensures that even if intercepted during public network transmission, the information will not be leaked, guaranteeing data confidentiality. Using directional IoT cards enhances the controllability of network access.

[0048] Internal Interface Forwarding: The external network interface service cluster acts as an external barrier. It does not handle core business processes itself, but only forwards received requests to the internal network interface service cluster located in the same network area but at a higher level. This embodiment achieves network-level isolation, allowing the external service cluster to focus on responding to internet attacks (such as DDoS) and protecting the security of the internal business logic cluster.

[0049] Security Authentication Request: Upon receiving a forwarded request, the external network interface service cluster does not immediately process the data. Instead, it first sends a request to an independent authentication service, providing its own identity credentials to obtain authentication information (such as an access token). This strict identity authentication mechanism ensures that only legitimate and authorized services can conduct subsequent cross-network data transmission, preventing unauthorized devices from accessing the network and malicious data from being pushed.

[0050] Packet encapsulation and secure invocation: The external network-to-internal interface service cluster re-encapsulates the obtained authentication information and received encrypted data packets to form a new data packet that conforms to security specifications. Subsequently, it invokes the interface specified by the security platform (such as a network gateway or secure data exchange system) to push the encapsulated data packet to the security platform. The security platform is a key device for enabling cross-network transmission. It typically employs technologies such as physical isolation or protocol stripping to establish a controlled, unidirectional data exchange channel only at the network boundary, thereby completely blocking the possibility of direct access to the railway's internal network from the Internet and meeting extremely high network security requirements.

[0051] Secure Data Forwarding: Following a preset security policy, the security platform performs deep content inspection and virus scanning on data packets before securely forwarding them from the internet area to the internal network interface service cluster within the railway's internal network area. This embodiment achieves secure and compliant cross-network transmission of data from the public network to the private network, resolving the cross-network transmission information security issues mentioned in the background section.

[0052] Packet parsing and decryption: After receiving a data packet from the security platform, the intranet interface service cluster first parses the packet to extract the original encrypted monitoring data, and then uses the corresponding key to decrypt it, restoring the plaintext monitoring data. This embodiment completes the decryption operation in a secure internal network environment, ensuring the security of the key.

[0053] Data Mapping and Persistence: Similar to wired transmission, the intranet interface service cluster performs object-model mapping and population on the decrypted plaintext monitoring data, and writes the final generated data entity into the database cluster for storage. This application enables data aggregated wirelessly and wiredly to achieve a unified format and standard at the final storage level, facilitating unified processing and analysis by the management system.

[0054] This embodiment constructs an end-to-end secure link that includes encryption, authentication, and a security platform, thereby efficiently achieving real-time and reliable aggregation of monitoring data in an Internet environment while ensuring compliance with the strict security standards of the railway's internal network.

[0055] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0056] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes. Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the invention. Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0057] The methods and apparatus provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

[0058] In the description of this specification, references to terms such as "an embodiment," "some embodiments," "example," "specific example," or "a specific embodiment" or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A railway bridge monitoring data aggregation method, characterized by, include: The monitoring device pushes monitoring data to the data aggregation system according to a preset data format; The data aggregation system performs data cleaning and processing on the received monitoring data, including data validity verification and deduplication. The monitoring data, after cleaning and treatment, is packaged and transmitted to the primary aggregation center; The primary aggregation center parses the received data, performs object-model mapping, and finally writes it into the database cluster for storage. The data aggregation system adapts to either wired or wireless transmission methods to aggregate data, depending on the network environment in which the monitoring device is located.

2. The method for collecting railway bridge monitoring data according to claim 1, characterized in that, When using wired transmission, the data aggregation system includes a load cluster, an interface service cluster, and a cache cluster deployed at the secondary aggregation point, as well as a message queue cluster and a listening service cluster deployed at the primary aggregation center. The data cleaning and governance are performed by the cache cluster. The encapsulation is performed by the interface service cluster, which reads the managed data from the cache cluster through a scheduled task, encapsulates it, and writes it into the message queue cluster of the primary aggregation center.

3. The railway bridge monitoring data aggregation method of claim 2, wherein, The monitoring device calls the interface exposed by the load cluster to push data; the load cluster uses a round-robin method to reverse proxy the request to a node in the interface service cluster.

4. The method for collecting railway bridge monitoring data according to claim 2, characterized in that, The listening service cluster of the primary aggregation center listens to the message queue cluster in real time, obtains and parses the data, performs object-model mapping, and writes it to the database cluster.

5. The railway bridge monitoring data aggregation method of claim 1, wherein, include: When using wireless transmission, the monitoring device pushes the encrypted monitoring data to the external interface service cluster deployed in the Internet environment via the wireless network.

6. The railway bridge monitoring data aggregation method of claim 5, wherein, When using wireless transmission, it also includes: The external network interface service cluster will forward the request to the internal network interface service cluster. The external network to internal interface service cluster requests authentication service to obtain authentication information, and after encapsulating the authentication information together with encrypted data packets, pushes it to the railway internal network through the security platform; The intranet interface service cluster deployed in the railway's internal network receives data packets from the security platform, parses and decrypts them, performs object-model mapping, and writes them to the database cluster.

7. The railway bridge monitoring data aggregation method of claim 1, wherein, The monitoring data includes at least one of the following: railway bridge health monitoring data, navigation bridge collision avoidance monitoring data, and collision monitoring data of railway-crossing public overpasses and height-restricted protective frames.

8. The railway bridge monitoring data aggregation method of claim 1, wherein, The nodes that perform critical services in the data aggregation system are all deployed in a clustered manner.

9. The railway bridge monitoring data aggregation method of claim 2, wherein, In the wired transmission method, the data cleaning and processing is performed at the secondary aggregation point beforehand, thereby reducing the computational load on the primary aggregation center.

10. The railway bridge monitoring data aggregation method of claim 6, wherein, In the aforementioned wireless transmission method, cross-network transmission is achieved through a data encryption and security platform, ensuring data transmission security.

Images