Railway freight train data acquisition method and system based on multi-sensor fusion

Abstract

The invention discloses a railway freight train data acquisition method and system based on multi-sensor fusion, and relates to the technical field of railway transportation, and the method comprises the steps: detecting the operation state and position of a train in real time through wireless communication, and carrying out the self-inspection of each sensor when the train is detected to be about to enter a monitoring region; constructing a three-dimensional health state model of the train through three-dimensional reconstruction and synchronous positioning according to the comprehensive state data of the train; in combination with a three-dimensional health state model and a side cloud collaborative architecture, layered processing and dynamic optimization of train comprehensive state data are accelerated by using a quantum algorithm, and abnormal features are identified in real time; the abnormal features are analyzed through deep learning and a time sequence analysis algorithm, and alarm information is generated and fed back to the edge computing terminal; according to the method, the running state of the train and the cargo state data are deeply fused through the convolution operation of the three-dimensional health state model and the space-time diagram, so that the global perception capability of the data is improved.

Description

Technical Field

[0001] The present invention relates to the field of railway transportation technology, and in particular to a railway freight train data acquisition method and system based on multi-sensor fusion. Background Art

[0002] In recent years, with the increasing requirements for intelligence and automation in railway freight logistics, railway train operation status monitoring and cargo status detection technologies have been widely studied and applied. Traditional monitoring methods mainly rely on a single sensor to collect data. Although this method can provide certain monitoring capabilities, it has problems such as insufficient data accuracy, limited coverage, and low information fusion. With the development of sensor technology and computing power, multi-sensor fusion technology has gradually become an effective means to improve the accuracy and reliability of train monitoring. Based on this, researchers began to explore how to achieve accurate monitoring of train operation status and cargo status by fusing multiple sensor data and combining advanced computing models and algorithms.

[0003] However, existing technologies still have shortcomings in multi-sensor data fusion, real-time monitoring and anomaly detection. For example, traditional cargo status detection methods have large delays when processing large-scale, high-dimensional data, and the real-time synchronization and accuracy of multi-sensor data remain a challenge. In addition, existing anomaly detection methods mostly rely on manually set thresholds or simple rules, lack dynamic adaptability and intelligence, and are difficult to cope with the complex and changing railway transportation environment. Therefore, how to effectively improve data fusion efficiency and enhance the intelligence and real-time nature of anomaly detection has become a core issue that needs to be solved urgently. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a railway freight train data collection method based on multi-sensor fusion to solve the problems of poor real-time performance and insufficient intelligence level of multi-sensor data fusion and anomaly detection in the prior art.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a railway freight train data collection method based on multi-sensor fusion, which includes detecting the running status and position of the train in real time through wireless communication, and performing self-check on each sensor when it is detected that the train is about to enter the monitoring area;

[0008] Based on the multi-sensor after self-inspection, the cargo status is monitored through edge detection and semantic segmentation methods, and the comprehensive status data of the train is collected in real time;

[0009] According to the comprehensive status data of the train, a three-dimensional health status model of the train is constructed through three-dimensional reconstruction and synchronous positioning;

[0010] Combining the three-dimensional health status model with the edge-cloud collaborative architecture, quantum algorithms are used to accelerate the hierarchical processing and dynamic optimization of the train's comprehensive status data, and identify abnormal features in real time;

[0011] Abnormal features are analyzed through deep learning and time series analysis algorithms, alarm information is generated and fed back to the edge computing terminal.

[0012] As a preferred solution of the railway freight train data collection method based on multi-sensor fusion described in the present invention, wherein: the sensors include a door frame top visual sensor, door frame side visual sensors, a rail side state sensor and a rail bottom state sensor;

[0013] The running status and position of the train are detected in real time through wireless communication, and each sensor is self-checked when it is detected that the train is about to enter the monitoring area. The specific steps are as follows:

[0014] Through GNSS positioning and radio frequency identification methods, the running status and position of the train are monitored in real time to generate train running status data;

[0015] The time when the train arrives at the monitoring area is calculated through the Kalman filter prediction algorithm;

[0016] Set a safety distance threshold. When it is predicted that the distance between the train and the monitoring area is less than the preset safety distance threshold, the edge computing terminal sends a self-check start instruction to each sensor.

[0017] After receiving the self-test start command, each sensor performs status detection in parallel according to the self-test sequence and outputs sensor data;

[0018] Upload sensor data to the edge computing terminal for real-time aggregation.

[0019] As a preferred solution of the railway freight train data collection method based on multi-sensor fusion of the present invention, wherein: the multi-sensor based on self-inspection monitors the cargo status through edge detection and semantic segmentation methods, and collects the comprehensive status data of the train in real time, the specific steps are as follows:

[0020] After the status detection is completed, each sensor synchronously collects 2D images and 3D point cloud data;

[0021] De-noising, enhancement and geometric calibration of collected 2D images and 3D point cloud data;

[0022] Use the Canny edge detection algorithm to extract the contour features of the goods in the 2D image;

[0023] Apply deep learning semantic segmentation models to classify cargo types and structures in 3D point cloud data and identify the loading status of different cargoes;

[0024] Combine the identified loading status of different goods with the contour features of the goods to generate a panoramic cargo identification view;

[0025] Detect the cargo shape and internal residue in the panoramic cargo identification view to form cargo status data;

[0026] The Kalman filtering method is used to combine the train operation status data with the cargo status data to generate the comprehensive train status data;

[0027] The comprehensive train status data is fed back to the edge computing terminal in real time.

[0028] As a preferred solution of the railway freight train data collection method based on multi-sensor fusion of the present invention, wherein: according to the comprehensive status data of the train, a three-dimensional health status model of the train is constructed through three-dimensional reconstruction and synchronous positioning, and the specific steps are as follows:

[0029] Each time step in the train running status data is regarded as a node of the graph, and the time dimension is regarded as the time series of the graph;

[0030] The cargo status data is used as the attribute of the corresponding node;

[0031] Based on the nodes of the graph, the time series of the graph, and the attributes of the corresponding nodes, the information between the nodes is streamed and fused through the spatiotemporal graph convolution operation;

[0032] Constructing spatiotemporal adaptive functions during spatiotemporal graph convolution operations The expression is:

[0033]

[0034] in, represents the context information vector of node i at time t, α represents The influence weight of represents the L2 norm of the context information vector of node i at time t, represents the state difference between node i and its neighboring nodes at time t, and β represents The weight of influence;

[0035] Based on spatiotemporal adaptive function The relationship between nodes in the graph convolution process is dynamically adjusted to construct a heterogeneous spatiotemporal graph convolution model, which is expressed as:

[0036]

[0037] in, represents the feature vector of node i after the graph convolution operation at time t+1, σ represents the nonlinear activation function, N represents the number of all nodes in the graph, j represents the adjacent node, and A ij represents the adjacency matrix, W i (t) represents the influence weight of node i on the adjacent node information at time t, represents the feature vector of the adjacent node j at time t, B i represents the bias term;

[0038] The feature vectors of all nodes i after the graph convolution operation The time steps are integrated to form the health status feature set of the train at different time nodes;

[0039] Based on the health status feature set, a three-dimensional health status model is constructed through feature aggregation.

[0040] As a preferred solution of the railway freight train data collection method based on multi-sensor fusion described in the present invention, the three-dimensional health status model is combined with the edge-cloud collaborative architecture, and the quantum algorithm is used to accelerate the hierarchical processing and dynamic optimization of the comprehensive status data of the train, and to identify abnormal features in real time. The specific steps are as follows:

[0041] Adopting edge-cloud collaborative architecture, the edge computing terminal processes and monitors the health status feature set in real time, and the cloud performs global optimization and anomaly identification tasks;

[0042] The train running status data is used as the first-level optimization variable, the cargo status data is used as the second-level optimization variable, and the sensor data is used as the third-level optimization variable;

[0043] Using the quantum approximate optimization algorithm, each layer is optimized through the loss function in the quantum circuit, and the expression is:

[0044]

[0045] in, represents the loss function in the quantum circuit, represents the state difference between node i and its neighboring nodes at time t+1, represents the L2 norm of the feature vector of node i after the graph convolution operation at time t+1, γ is the influence factor of quantum control, f(θ) represents the quantum state control function, M represents the number of quantum bits, and θ k represents the control parameter of quantum bit k, cos(θ k ) represents the projection of the control qubit state in the ground state direction, sin(θ k) represents the projection of the control qubit state in the excited state direction, ω k Indicates the control cos(θ k ) weight coefficient, δ k represents the control sin(θ k )’s weight coefficient;

[0046] Time series analysis and deep neural network models are used to model the time series patterns of the optimized train operation status data, cargo status data and sensor data, and autoencoders are used to detect abnormal features.

[0047] As a preferred solution of the railway freight train data collection method based on multi-sensor fusion described in the present invention, wherein: the time series analysis and deep neural network model are used to model the time series pattern of the optimized train operation status data, cargo status data and sensor data, and the autoencoder is used to detect abnormal features. The specific steps are as follows:

[0048] Clean and standardize the optimized train operation status data, cargo status data and sensor data;

[0049] Long short-term memory networks are used to model the cleaned and standardized train operation status data, cargo status data, and sensor data, extract time series features, and capture the time series dependencies between various state variables.

[0050] Use convolutional neural networks to extract features from the cleaned and standardized train operation status data, cargo status data, and sensor data to identify time series features;

[0051] Based on the time series dependency and time series features, the autoencoder model is used to analyze the reconstruction error, identify abnormal data points that deviate from the normal state, and mark the abnormal features.

[0052] As a preferred solution of the railway freight train data collection method based on multi-sensor fusion described in the present invention, wherein: the abnormal characteristics are analyzed by deep learning and time series analysis algorithms, alarm information is generated and fed back to the edge computing terminal, and the specific steps are as follows:

[0053] Use deep learning models to analyze abnormal features and identify abnormal behavior information that deviates from normal patterns;

[0054] Combined with the time series analysis algorithm, further identification and confirmation of abnormal patterns are performed based on the identified abnormal behavior information;

[0055] Integrate confirmed abnormal behavior information to form an abnormal detection report;

[0056] Generate alarm information based on the anomaly detection report and indicate the anomaly type and severity;

[0057] The generated alarm information is transmitted to the edge computing terminal for real-time notification and response.

[0058] In the second aspect, the present invention provides a railway freight train data acquisition system based on multi-sensor fusion, including a self-check module, a monitoring module, a modeling module, an abnormality recognition module and an alarm generation module; the self-check module is used to detect the running status and position of the train in real time through wireless communication, and perform self-check on each sensor when it is detected that the train is about to enter the monitoring area; the monitoring module is used to monitor the cargo status based on the multi-sensors after self-checking through edge detection and semantic segmentation methods, and collect the comprehensive status data of the train in real time; the modeling module is used to construct a three-dimensional health status model of the train through three-dimensional reconstruction and synchronous positioning according to the comprehensive status data of the train; the abnormality recognition module is used to combine the three-dimensional health status model with the edge-cloud collaborative architecture, use quantum algorithms to accelerate the hierarchical processing and dynamic optimization of the comprehensive status data of the train, and identify abnormal features in real time; the alarm generation module is used to analyze the abnormal features through deep learning and time series analysis algorithms, generate alarm information and feedback to the edge computing terminal.

[0059] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the railway freight train data collection method based on multi-sensor fusion as described in the first aspect of the present invention is implemented.

[0060] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the railway freight train data collection method based on multi-sensor fusion as described in the first aspect of the present invention.

[0061] The beneficial effects of the present invention are as follows: the present invention effectively improves the accuracy and real-time performance of railway freight train data collection through the combination of multi-sensor fusion and intelligent algorithms; the reliability and stability of each sensor in operation are ensured by self-checking and status monitoring of sensors; at the same time, the cargo status is efficiently analyzed through edge detection and deep learning semantic segmentation models, thereby realizing accurate identification of different cargo loading states. In addition, through the three-dimensional health status model and spatiotemporal graph convolution operation, the train's operating status and cargo status data are deeply integrated, improving the global perception of data; the introduction of quantum algorithms further accelerates the data processing process and enhances the ability to respond to complex environmental changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0063] Figure 1 This is a flow chart of the railway freight train data collection method based on multi-sensor fusion in Example 1.

[0064] Figure 2 This is a module diagram of the railway freight train data acquisition system based on multi-sensor fusion in Example 1. DETAILED DESCRIPTION

[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0066] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0067] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0068] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a railway freight train data collection method based on multi-sensor fusion, comprising the following steps:

[0069] S1. Detect the running status and position of the train in real time through wireless communication, and perform self-check on each sensor when it is detected that the train is about to enter the monitoring area.

[0070] Furthermore, each sensor includes a door frame top vision sensor, door frame side vision sensors, rail side status sensors, and rail bottom status sensors;

[0071] Specifically, the visual sensor on the top of the door frame includes a 2D / 3D image sensor; the visual sensors on both sides of the door frame include a 2D / 3D image sensor; the track side status sensor includes a magnetic steel measurement and control sensor including a 2D / 3D visual sensor; the rail bottom status sensor includes a 2D / 3D visual sensor and an infrared thermal imaging sensor.

[0072] Real-time monitoring of train operation status and location through GNSS positioning and RFID methods;

[0073] The time when the train arrives at the monitoring area is calculated through the Kalman filter prediction algorithm;

[0074] Set a safety distance threshold. When it is predicted that the distance between the train and the monitoring area is less than the preset safety distance threshold, the edge computing terminal sends a self-check start instruction to each sensor.

[0075] It should be noted that when setting the safety distance threshold, it is first necessary to determine the appropriate safety distance based on the train's running speed, the size of the monitoring area, and the response time of the sensor. First, by combining the train's real-time speed data with GNSS positioning, the estimated time for the train to reach the monitoring area from its current position is calculated. Next, taking into account the sensor's self-test time and data transmission delay, in order to ensure that the sensor completes self-test before the train enters the monitoring area, a reserved time is set, usually 3 to 5 seconds, which can be flexibly adjusted according to different scenarios. Based on the train's speed and reserved time, a suitable safety distance threshold can be calculated;

[0076] For example, when the train speed is 60km / h, the set safety distance threshold may be 150 meters. When the train distance monitoring area is less than the safety distance threshold, the edge computing terminal will send a self-check start instruction to each sensor in advance according to the predicted arrival time to ensure that each sensor can complete the status detection on time, ensuring the normal operation of the monitoring and data accuracy.

[0077] After receiving the self-test start command, the sensor performs status detection in parallel according to the self-test sequence and outputs sensor data;

[0078] Specifically, the operating steps for the self-inspection of the visual sensor on the top of the door frame are as follows: start the 2D / 3D image sensor to collect the baseline image and point cloud data of the top area of ​​the car; automatically calibrate the optical imaging accuracy of the image acquisition equipment to ensure that the resolution and focal length are within the standard range; check the data transmission delay of the image acquisition to ensure that it is lower than the preset threshold.

[0079] The operating steps for the self-check of the visual sensors on both sides of the door frame are as follows: start the 2D / 3D image sensor to collect reference images and point cloud data on both sides of the car; automatically perform image recognition correction to ensure that key components such as the car door locks and bolt positions are clearly captured; perform a lighting adaptability test to confirm the performance under different lighting conditions during the image acquisition process.

[0080] The operating steps for the self-test of the trackside status sensor are as follows: start the magnetic steel measurement and control sensor, check the wheel approach detection status, and verify the approach distance and speed measurement accuracy; start the 2D / 3D visual sensor, collect trackside status data, and check the track appearance, bogie, and wheel flange status; verify the data synchronization between the magnetic steel sensor and the visual sensor to ensure that all data is transmitted on time and correctly matched.

[0081] The operating steps of the rail bottom status sensor self-inspection are as follows: start the 2D / 3D visual sensor to perform rail bottom detection, check the bottom structure of the car and the status of the suspension parts; start the infrared thermal imaging sensor to detect the temperature of the bottom of the car and identify whether there is a temperature abnormality; verify the data synchronization of the infrared thermal imaging and visual sensors, and confirm the correlation between the temperature abnormality area and the structural problem.

[0082] Upload sensor data to the edge computing terminal for real-time aggregation.

[0083] Among them, the edge computing terminal refers to a computing device deployed at the data collection source end close to the sensor. It is responsible for the preliminary processing, analysis and aggregation of the data collected from each sensor (including the visual sensor on the top of the door frame, the visual sensors on both sides of the door frame, the trackside status sensor and the track bottom status sensor). Unlike the traditional method of transmitting all data to the cloud for processing, the edge computing terminal can perform efficient data processing and analysis locally, reducing dependence on the central server and significantly improving the response speed and real-time performance. It can not only complete the storage, processing and transmission of data, but also monitor and execute automated decisions in real time to quickly respond to abnormal situations during train operation.

[0084] It should be noted that uploading the status detection results of the sensors to the edge computing terminal for real-time aggregation is intended to quickly process the status data fed back by the sensors through centralized local computing power, thereby ensuring that various status information of the train can be detected in real time. This process can reduce data transmission delays and avoid the time required for data processing after uploading to the cloud, thereby improving the efficiency and accuracy of data processing.

[0085] For example, when a sensor detects an abnormality (such as unstable cargo status or equipment failure), the edge computing terminal will immediately conduct a preliminary analysis of the abnormal status and trigger an alarm mechanism or execute relevant emergency measures. In addition, the edge computing terminal can also reduce the burden on cloud servers, optimize the use of network bandwidth, and improve overall stability and reliability.

[0086] S2. Based on the multi-sensors after self-inspection, the cargo status is monitored through edge detection and semantic segmentation methods, and the comprehensive status data of the train is collected in real time.

[0087] Furthermore, after the state detection is completed, each sensor synchronously collects 2D images and 3D point cloud data;

[0088] Specifically, the current position of the train is determined through GNSS positioning, the distance to the monitoring area is calculated in real time, and the time when the train is about to enter the monitoring area is predicted through built-in algorithms (such as Kalman filter), ensuring that the time point of data collection matches the running status of the train. Next, the edge computing terminal sends synchronous collection instructions to the sensors on the top, sides, trackside and bottom of the door frame based on the position and time information. After receiving the instructions, the sensor starts to collect 2D images and 3D point cloud data according to the preset configuration. The collected data includes information such as the appearance of the train carriage, cargo loading status, structural characteristics, and track conditions.

[0089] Furthermore, the sensor on the top of the door frame uses a high-resolution camera and LiDAR to collect 2D images and 3D point cloud data to monitor the loading and stability of the top of the car and the cargo; the visual sensors on both sides of the door frame use high-frame rate cameras to capture images of both sides of the car, and analyze the cargo edge and car status in combination with LiDAR data; the sensors on the trackside and track bottom use laser scanning and infrared sensing technology to monitor the contact between the bottom of the train and the track in real time to ensure that there are no obstacles or damage. The acquisition process of all sensors maintains the temporal consistency of the data through a synchronization mechanism, ensuring that the acquired 2D images match the 3D point cloud data and that the timestamp of the acquired data is accurate.

[0090] De-noising, enhancement and geometric calibration of collected 2D images and 3D point cloud data;

[0091] Use the Canny edge detection algorithm to extract the contour features of the goods in the 2D image;

[0092] Apply deep learning semantic segmentation models to classify cargo types and structures in 3D point cloud data and identify the loading status of different cargoes;

[0093] Preferably, the deep learning semantic segmentation model is applied to classify the cargo types and structures in 3D point cloud data, which can efficiently identify the loading status of different cargoes.

[0094] Furthermore, through deep learning models (such as PointNet++, DeepLab V3+, etc.), 3D point cloud data can be accurately processed in high-dimensional space, and the shape, structure and loading status of goods can be automatically identified, overcoming the limitations of traditional methods in processing complex structures and large-scale data. Compared with traditional classification methods based on manual features, deep learning models can automatically extract richer and higher-dimensional features and accurately distinguish different types of goods.

[0095] For example, bulk cargo, containers, etc., and can still maintain high accuracy and real-time performance in a high-speed moving train environment. In addition, the deep learning model can handle occlusion, noise and complex geometric shapes, improving the robustness in complex environments, making it more accurate and intelligent in monitoring the status of cargo, effectively improving the detection accuracy of cargo loading status, and providing more reliable data support for train operation safety.

[0096] Combine the identified loading status of different goods with the contour features of the goods to generate a panoramic cargo identification view;

[0097] Specifically, the loading status of each cargo is identified through a deep learning model to obtain its stability and whether there are any residues. At the same time, the Canny edge detection algorithm is used to extract the contour features of the cargo to ensure that the shape and position of the cargo are accurately depicted. Then, a weighted fusion algorithm is used to fuse the loading status information with the contour feature data, and image registration technology is used to align 2D images from different perspectives with 3D point cloud data to form a complete view in a unified coordinate system. Finally, the data fusion algorithm is used to synthesize the cargo status data from all perspectives to generate a high-precision panoramic cargo identification view, presenting the true shape, loading status and surrounding environment of each cargo, providing comprehensive perspective support for subsequent abnormality detection and monitoring.

[0098] Detect the cargo shape and internal residue in the panoramic cargo identification view to form cargo status data;

[0099] Among them, cargo status data includes cargo integrity, loading stability, presence of internal residues, morphological deformation, tilt angle, edge damage and other abnormal features.

[0100] Specifically, the generated panoramic view is used to accurately locate the shape of the cargo, and a deep learning model (such as a convolutional neural network (CNN)) is used to extract the geometric features of the cargo, including tilt angle, deformation, and edge damage. Then, semantic segmentation technology is applied to carefully divide each cargo area to detect whether there are internal residues, such as debris, dust, or unreinforced items. Combined with object detection algorithms (such as YOLO), abnormal areas are located and marked, and image enhancement technology is used to improve the detection accuracy of low-contrast areas.

[0101] Furthermore, during the inspection process, the data of each inspection result is first fused based on the geometric features extracted by deep learning models (such as convolutional neural networks (CNN)) (including tilt angles, deformations, edge damage, etc.), the cargo area division information obtained by semantic segmentation technology (including whether there are internal residues such as debris, dust or unreinforced items), and the abnormal areas located by object detection algorithms (such as YOLO) (such as cracks, deformation points, damaged areas, etc.). The specific steps include: combining the geometric features of each cargo area with the segmentation information for consistency verification to ensure the accuracy of the area division; prioritizing the abnormal areas and classifying them according to the severity of the damage, the scope of impact and the urgency; using image enhancement technology to improve the data clarity of low-contrast areas to ensure the accurate identification of various features; combining the above analysis results to generate cargo status data, including the appearance damage level, residue detection results, reinforcement status, etc., to form a detailed cargo health report.

[0102] The Kalman filtering method is used to combine the train operation status data with the cargo status data to generate the comprehensive train status data;

[0103] The comprehensive train status data is fed back to the edge computing terminal in real time.

[0104] It should be noted that real-time feedback to the edge computing terminal can realize instant monitoring and analysis of the status of goods and trains. The edge computing terminal can quickly process this data locally, detect potential anomalies in time, and take rapid response measures, thereby greatly reducing delays and improving overall response speed and reliability.

[0105] S3. Based on the comprehensive status data of the train, a three-dimensional health status model of the train is constructed through three-dimensional reconstruction and synchronous positioning.

[0106] Furthermore, each time step in the train operation status data is regarded as a node of the graph, and the time dimension is regarded as the time series of the graph, so as to capture the dynamics of the train status changing over time;

[0107] The cargo status data is used as the attribute of the corresponding node;

[0108] Among them, the attributes of the node include the loading status, shape changes, damage conditions, etc. of the cargo, which are used to ensure that each node contains the train operation status data and can also reflect the relevant status of the cargo.

[0109] Based on the nodes of the graph, the time series of the graph, and the attributes of the corresponding nodes, the information between the nodes is streamed and fused through the spatiotemporal graph convolution operation;

[0110] It should be noted that based on nodes, time series and attributes, the information between nodes is streamed and fused through spatiotemporal graph convolution operations, which can effectively capture the spatiotemporal dependencies during train operation. Each node represents a time step, and the feature vector of the node contains the running status and cargo status data of the train. Through graph convolution operations, the information between nodes can be mutually transmitted and fused in time and space dimensions, ensuring that the state of each node is not only affected by its own information, but also reflects the dynamic changes of surrounding nodes. This process dynamically adjusts the association strength between nodes through spatiotemporal graph convolution, so that each node can take into account the previous and next time steps and the status of adjacent nodes when updating, thereby generating a more accurate train health status model.

[0111] Constructing spatiotemporal adaptive functions during spatiotemporal graph convolution operations The expression is:

[0112]

[0113] in, represents the context information vector of node i at time t, α represents The influence weight of represents the L2 norm of the context information vector of node i at time t, represents the state difference between node i and its neighboring nodes at time t, and β represents The weight of influence;

[0114] It should be noted that by introducing the spatiotemporal adaptive function, the context information of node i at time t can be and the state difference between the neighboring nodes Dynamically adjust the information flow between nodes during graph convolution. Specifically, the spatiotemporal adaptive function is based on the L2 norm of the node context information. The influence weights α and β of the state differences between nodes are adjusted in real time to ensure that the graph convolution can flexibly transmit information between different time steps and different nodes. This process enhances the adaptability of the spatiotemporal graph convolution model, making the relationship between nodes more accurate and able to effectively capture the spatiotemporal changes in the train running status and cargo status. Compared with the existing technology, by introducing the spatiotemporal adaptive function, the limitations of fixed weights and static adjacency relationships in traditional methods are avoided, thereby improving the flexibility and accuracy of graph convolution operations in complex and dynamic environments.

[0115] Preferably, constructing a spatiotemporal adaptive function during the spatiotemporal graph convolution operation can significantly enhance the ability to accurately model the status of trains and cargo, improve the real-time response capability in dynamic monitoring and anomaly detection, thereby optimizing the train's operating status monitoring and fault prediction, and comprehensively improving the accuracy and intelligence level of the overall solution.

[0116] Based on spatiotemporal adaptive function The relationship between nodes in the graph convolution process is dynamically adjusted to construct a heterogeneous spatiotemporal graph convolution model, which is expressed as:

[0117]

[0118] in, represents the feature vector of node i after the graph convolution operation at time t+1, σ represents the nonlinear activation function, N represents the number of all nodes in the graph, j represents the adjacent node, and A ij represents the adjacency matrix, W i (t) represents the influence weight of node i on the adjacent node information at time t, represents the feature vector of the adjacent node j at time t, B i represents the bias term;

[0119] It should be noted that when constructing a heterogeneous spatiotemporal graph convolution model, the relationship between nodes in the graph convolution process is dynamically adjusted based on the spatiotemporal adaptive function, which can more accurately capture the spatiotemporal dependency between sensors in the ever-changing train operation process.

[0120] Furthermore, the running status of the train and the status of the cargo change over time, so the correlation between nodes will also change at different time steps and under different environmental conditions. The spatiotemporal adaptive function dynamically adjusts the information transmission mode between nodes according to the context information of the nodes and the state difference between the nodes, so that the graph convolution operation can adapt to this change, avoiding the limitation that the traditional static graph convolution cannot flexibly cope with the dynamic changes of the spatiotemporal environment. Through this adjustment, the heterogeneous spatiotemporal graph convolution model can accurately fuse data from different sensors at each moment to ensure that the health status of the train and the status of the cargo are reflected in a timely and accurate manner. In addition, the use of this dynamic adjustment mechanism can optimize the interaction and fusion of sensor data according to the actual situation during the train operation, enhance the overall flexibility and prediction ability, and thus improve the accuracy and robustness of the train health status modeling.

[0121] The feature vectors of all nodes i after the graph convolution operation The time steps are integrated to form the health status feature set of the train at different time nodes;

[0122] Specifically, the integration can be performed through weighted averaging, pooling or splicing.

[0123] Based on the health status feature set, a three-dimensional health status model is constructed through feature aggregation (such as weighted averaging, pooling or deep learning methods).

[0124] S4. Combine the three-dimensional health status model with the edge-cloud collaborative architecture, use quantum algorithms to accelerate the hierarchical processing and dynamic optimization of the train's comprehensive status data, and identify abnormal features in real time.

[0125] Furthermore, a feature aggregation method is used to merge the health status feature vectors of multiple time steps into a multi-dimensional state matrix;

[0126] Adopting edge-cloud collaborative architecture, the edge computing terminal processes and monitors the health status feature set in real time, and the cloud performs global optimization and anomaly identification tasks;

[0127] Preferably, an edge-cloud collaborative architecture is adopted, and the monitored health status feature set is processed in real time by the edge computing terminal, which can significantly reduce the data transmission delay and ensure the instant processing and feedback of the train operation status and cargo monitoring data, thereby improving the response speed and real-time performance.

[0128] Specifically, edge computing terminals can process data nearby, conduct preliminary screening and anomaly detection, reduce the pressure of transmitting large amounts of data to the cloud, optimize bandwidth usage, and reduce the computing burden on the cloud. The cloud is responsible for global optimization and complex anomaly identification tasks, and can combine comprehensive data from different regions and levels for in-depth analysis to form a global perspective. Compared with existing technologies, the edge-cloud collaborative architecture can effectively share computing tasks and achieve more efficient data processing and resource allocation, especially in the face of large-scale data streams and application scenarios with extremely high real-time requirements. It can ensure efficient operation and flexible response, and improve the overall level of intelligence and real-time performance. The adoption of this architecture means that train operation and cargo status monitoring no longer rely on a single centralized computing, avoiding the delays and computing bottlenecks of traditional centralized methods, promoting the efficiency of information flow and the timeliness of response, and greatly improving real-time data processing capabilities and stability.

[0129] The train running status data is used as the first-level optimization variable, the cargo status data is used as the second-level optimization variable, and the sensor data is used as the third-level optimization variable;

[0130] Preferably, the train operation status data is used as the first-level optimization variable, which can focus on the real-time operation efficiency, speed control and safety of the train; the cargo status data is used as the second-level optimization variable, which can focus on optimizing the loading status, vibration monitoring and safety during cargo transportation; the sensor data is used as the third-level optimization variable, which can mainly optimize the accuracy, response time and stability of various sensors.

[0131] Using the quantum approximate optimization algorithm, each layer is optimized through the loss function in the quantum circuit, and the expression is:

[0132]

[0133] in, represents the loss function, represents the state difference between node i and its neighboring nodes at time t+1, represents the L2 norm of the feature vector of node i after the graph convolution operation at time t+1, γ is the influence factor of quantum control, f(θ) represents the quantum state control function, M represents the number of quantum bits, and θ k represents the control parameter of quantum bit k, cos(θ k ) represents the projection of the state of the control qubit in the ground state direction, that is, the "stable" part of the qubit, describing the state of the qubit in the "horizontal" direction, sin(θ k ) represents the projection of the control qubit state in the excited state direction, that is, the "dynamic" part of the qubit, which describes the change of the qubit in the "vertical" direction, ω k Indicates the control cos(θk ) weight coefficient, δ k represents the control sin(θ k )’s weight coefficient;

[0134] It should be noted that in quantum optimization, the rotation of the quantum bit (i.e., θ k The change of cos(θ k ) and sin(θ k ) These two parts enable quantum algorithms to achieve more efficient search and optimization in quantum state space.

[0135] Preferably, the quantum approximate optimization algorithm (QAOA) is used to optimize each layer through the loss function in the quantum circuit, which can significantly improve the processing efficiency and accuracy of the comprehensive state data of the train. Specifically, the quantum approximate optimization algorithm (QAOA) optimizes the state variables of each layer. First, the train's operating status, cargo status, and sensor data are divided into different levels, and optimization variables are defined for each layer. Optimize the state of each layer.

[0136] Furthermore, unlike traditional classical optimization algorithms (such as genetic algorithms), the quantum approximate optimization algorithm (QAOA) can use the characteristics of quantum superposition and quantum entanglement to achieve parallel search in the search space of the optimization problem, and avoid local optimal solutions through the superposition and entanglement states of quantum bits, and quickly explore the global optimal solution. Specifically, quantum algorithms can process and integrate large-scale, multi-dimensional state data in a short period of time, greatly improve processing efficiency, ensure the efficiency and accuracy of anomaly detection and dynamic adjustment, and ultimately promote the intelligence and precision of train operation status and cargo monitoring. This optimization not only accelerates multi-level data processing, but also can dynamically adjust anomaly identification, data processing efficiency, and resource allocation, providing higher computing performance and global optimization capabilities.

[0137] Time series analysis and deep neural network models are used to model the time series patterns of the optimized train operation status data, cargo status data and sensor data, and autoencoders are used to detect abnormal features.

[0138] Specifically, the optimized train operation status data, cargo status data and sensor data are cleaned and standardized; the long short-term memory network is used to model the cleaned and standardized train operation status data, cargo status data and sensors, extract time series features, and capture the time series dependencies between each state variable; the convolutional neural network is used to extract features of the cleaned and standardized train operation status data, cargo status data and sensors, identify time series features, and enhance the model's learning ability for complex patterns; based on time series dependencies and time series features, the reconstruction error is analyzed through the autoencoder model, abnormal data points that deviate from the normal state are identified, and abnormal features are marked.

[0139] S5. Analyze abnormal features through deep learning and time series analysis algorithms, generate alarm information and feed it back to the edge computing terminal.

[0140] Furthermore, deep learning models are used to analyze abnormal features and identify abnormal behavior information that deviates from normal patterns;

[0141] Specifically, after data cleaning and standardization, deep learning models (such as convolutional neural networks (CNN), long short-term memory networks (LSTM), autoencoders, etc.) are used to analyze the train operation status, cargo status, and sensor data. The deep learning model first extracts features from the train operation status data, cargo status data, and sensor data to identify abnormal behavior information that deviates from the normal pattern. By calculating the anomaly score for each data point, a higher score indicates a significant deviation. Next, the high-scoring data points are further analyzed to identify specific anomaly types (such as speed fluctuations, changes in cargo shape, or sensor failures). These abnormal information are marked as potential risks and are ready for further confirmation.

[0142] Combined with the time series analysis algorithm, further identification and confirmation of abnormal patterns are performed based on the identified abnormal behavior information;

[0143] Specifically, these abnormal information marked as potential risks are sorted in chronological order and a time series data set is constructed to capture the evolution trend of abnormal behavior in the time dimension through the time series analysis algorithm. Then, the time series analysis algorithm further identifies the specific pattern of abnormal behavior by learning the time series pattern in the data, and confirms the persistence and severity of the abnormality based on the trend and mutation point within the time window.

[0144] For example, if the speed of a train fluctuates or the amount of cargo changes significantly within a specific time period, the algorithm will determine whether it is a long-term anomaly or a short-term sudden anomaly based on the fluctuation characteristics of the time series pattern. The analysis results will generate a further confirmation report on the abnormal pattern, indicating whether the anomaly is persistent, its type, and possible root causes. Ultimately, these time series analysis results can more accurately classify abnormal behaviors, confirm their overall impact, and provide more accurate information for subsequent alarm generation and response.

[0145] Integrate confirmed abnormal behavior information to form an abnormal detection report;

[0146] Among them, the anomaly detection report includes the anomaly type, occurrence time, duration, severity, possible root cause, scope of impact and related risk assessment.

[0147] It should be noted that the anomaly detection report helps evaluate the potential impact of anomalies on overall operations through detailed analysis of abnormal patterns, and provides data support for subsequent alarm generation, decision optimization and response strategies; at the same time, the anomaly detection report not only provides anomaly tracking records, but also promotes rapid identification and accurate response to abnormal situations, ensuring effective adjustments and optimizations in the shortest time, thereby improving the reliability and efficiency of the monitoring process.

[0148] Generate alarm information based on the anomaly detection report and indicate the anomaly type and severity;

[0149] Specifically, the types of anomalies in the report are classified. Possible anomalies include abnormal train operation status (such as speed exceeding the limit, emergency braking, etc.), abnormal cargo status (such as tilting, damage, etc.), and sensor failure. According to the duration of the anomaly, the scope of impact, and the potential risk to the whole, the severity of each anomaly is evaluated. The anomalies are usually divided into mild, moderate and severe. Severe anomalies require immediate response and corresponding measures. When the alarm information is generated, it contains the type of anomaly, severity, time of occurrence, affected area, recommended response measures, etc., and different priorities are set for different abnormal situations according to the set rules to ensure that important anomalies can be handled first.

[0150] For example, after receiving an anomaly detection report, the anomaly type in the report is first classified. If it is detected that the speed of the train suddenly exceeds the preset safety limit, it is marked as "abnormal train operation status"; if the cargo is tilted or damaged, it is classified as "abnormal cargo status". Then, the severity of each anomaly is evaluated based on the duration of the anomaly, the area where it occurs, and the potential impact on the overall operation.

[0151] For example, if a train exceeds the speed limit by more than 5 seconds and is on a busy traffic line, this type of anomaly will be assessed as a "severe anomaly" and immediate deceleration, alarm, and operator notification will be required. If the cargo is slightly tilted and does not affect safety, it will be assessed as a "mild anomaly" and a later inspection will be recommended. When an alarm message is generated, the type of anomaly, severity, time of occurrence, affected area, and recommended response measures will be included. For severe anomalies, it may be recommended to decelerate immediately and initiate emergency braking, while mild anomalies will prompt routine inspections. According to preset rules, different types of anomalies will be prioritized to ensure that severe anomalies such as speed limit violations are handled first.

[0152] The generated alarm information is transmitted to the edge computing terminal for real-time notification and response.

[0153] Specifically, the alarm information will be transmitted to the edge computing terminal through wireless communication or wired network. After receiving the alarm information, the edge computing terminal will first prioritize the alarm information, and identify which anomalies are of high priority (for example, train speed exceeds the limit, cargo deformation is serious, etc.) according to the set priority rules, and determine whether immediate action is needed through real-time algorithms.

[0154] Furthermore, for high-priority alarms, the edge computing terminal will immediately trigger the preset emergency response mechanism, such as adjusting the train speed through automatic scheduling methods, changing the transportation route of goods, or immediately starting the vehicle detection method. At the same time, the edge terminal will send a real-time notification to the control center or relevant operators, including the type of abnormality, location, time, severity, recommended response measures, and expected impact range, to ensure that operators can obtain accurate information at the first time. In this way, the alarm information can not only be quickly transmitted to the processing module within a few seconds, but also ensure that the abnormal situation is handled quickly and accurately, ensuring the safety of train operation and the integrity of goods.

[0155] The present embodiment also provides a railway freight train data acquisition system based on multi-sensor fusion, including: a self-check module, a monitoring module, a modeling module, an abnormality recognition module and an alarm generation module; the self-check module is used to detect the running status and position of the train in real time through wireless communication, and perform self-check on each sensor when it is detected that the train is about to enter the monitoring area; the monitoring module is used to monitor the cargo status based on the multi-sensors after self-checking through edge detection and semantic segmentation methods, and collect the comprehensive status data of the train in real time; the modeling module is used to construct a three-dimensional health status model of the train through three-dimensional reconstruction and synchronous positioning according to the comprehensive status data of the train; the abnormality recognition module is used to combine the three-dimensional health status model with the edge-cloud collaborative architecture, use quantum algorithms to accelerate the hierarchical processing and dynamic optimization of the comprehensive status data of the train, and identify abnormal features in real time; the alarm generation module is used to analyze the abnormal features through deep learning and time series analysis algorithms, generate alarm information and feedback to the edge computing terminal.

[0156] This embodiment also provides a computer device, which is suitable for the railway freight train data collection method based on multi-sensor fusion, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the railway freight train data collection method based on multi-sensor fusion as proposed in the above embodiment.

[0157] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0158] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the railway freight train data collection method based on multi-sensor fusion proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0159] In summary, the present invention effectively improves the accuracy and real-time performance of railway freight train data collection through the combination of multi-sensor fusion and intelligent algorithms; through self-inspection and status monitoring of sensors, the reliability and stability of each sensor in operation are ensured; at the same time, through edge detection and deep learning semantic segmentation model, the cargo status is efficiently analyzed, thereby realizing accurate identification of different cargo loading states. In addition, through the three-dimensional health status model and spatiotemporal graph convolution operation, the train's operating status and cargo status data are deeply integrated, improving the global perception of data; the introduction of quantum algorithms further accelerates the data processing process and enhances the ability to respond to complex environmental changes.

[0160] Example 2, referring to Table 1, is the second example of the present invention. To further verify the technical solution of the present invention, experimental simulation data of a railway freight train data acquisition method based on multi-sensor fusion is provided.

[0161] A real-time railway freight train is used to monitor the train's operating status and cargo status in real time through the combination of a wireless communication system and on-board sensors. The experimental process will include the following steps:

[0162] Firstly, GNSS positioning and radio frequency identification (RFID) are used to detect the running status and position of the train in real time through wireless communication. A variety of sensors are configured, including visual sensors on the top of the door frame, visual sensors on both sides of the door frame, trackside status sensors and track bottom status sensors, to obtain various running data of the train. The Kalman filter algorithm is used to predict the time when the train arrives at the monitoring area, and a safety distance threshold is set. When it is predicted that the train is about to approach the monitoring area, a self-test start command is sent to each sensor to ensure that there is no failure in the sensor during operation.

[0163] Then, after the self-check is completed, each sensor synchronously collects 2D images and 3D point cloud data. The 2D image is used for cargo contour detection, and the 3D point cloud data is used for cargo type and structure classification. The Canny edge detection algorithm is used to extract the cargo contour in the 2D image, and the deep learning semantic segmentation model is used to analyze the 3D point cloud data to classify the cargo type and identify the loading status of the cargo. The above data is denoised, enhanced, and geometrically calibrated to improve the accuracy of the image and point cloud data.

[0164] Secondly, based on the train's operating status data and cargo status data, the spatiotemporal graph convolution method is used to build a three-dimensional health status model of the train. Based on the quantum algorithm, the hierarchical processing and dynamic optimization of the train's comprehensive status data are accelerated, abnormal features are identified in real time, alarm information is generated, and it is fed back to the edge computing terminal.

[0165] Furthermore, time series analysis and deep neural network models are used to model abnormal features, analyze the status data of trains and cargo, and identify abnormal data points that deviate from the normal state. Alarm information is generated based on abnormal features, indicating the type and severity of the abnormality, and real-time feedback is provided to the operator through wireless communication.

[0166] Finally, the following is the data collected during the experiment, which shows the advantages of implementing a railway freight train data collection method based on multi-sensor fusion compared to traditional methods. The data covers multiple aspects such as train operation status, cargo status monitoring, 3D health status model, anomaly detection, etc.

[0167] The details are shown in Table 1 below:

[0168] Table 1 Experimental data comparison table

[0169]

[0170]

[0171] Through the above data comparison, it can be seen that the present invention has obvious advantages in terms of train running status collection, cargo status monitoring, three-dimensional health status model and abnormal detection reaction time:

[0172] Accuracy of train running status collection: In traditional methods, due to reliance on a single sensor (such as a single GNSS positioning and basic track sensors), the accuracy of train running status collection is only 85%. However, through the method based on multi-sensor fusion, multiple data sources such as the combination of GNSS and RFID and sensor self-checking mechanism are utilized, and the accuracy is improved to 98%. This shows that the method based on multi-sensor fusion can effectively eliminate the errors that may be caused by a single sensor and provide more accurate train running status data.

[0173] Cargo status monitoring accuracy: Traditional methods for cargo status monitoring mainly rely on simple visual sensors and position recognition, with an accuracy of 80%. By using Canny edge detection and deep learning semantic segmentation methods, combined with 2D images and 3D point cloud data, the invented method based on multi-sensor fusion has increased the accuracy of cargo status monitoring to 95%. This shows that the monitoring method that combines image processing and deep learning technology can more accurately identify the loading status and type of cargo, especially in complex environments.

[0174] Accuracy of three-dimensional health status model: Traditional methods can only provide basic status monitoring, but cannot comprehensively model the health status of trains, with an accuracy of 75%. In the method of the present invention, the running status of the train and the cargo status are dynamically integrated through spatiotemporal graph convolution and three-dimensional reconstruction technology, and the accuracy is improved to 92%. This shows that the present invention can reflect the health status of the train in real time through spatiotemporal information and three-dimensional modeling technology, providing a more accurate basis for subsequent maintenance and early warning.

[0175] Abnormal detection reaction time: The abnormal detection reaction time of the traditional method is 25 seconds, mainly because the data processing process is lengthy and the efficient computing model is not fully utilized. The invention method based on multi-sensor fusion accelerates data processing through quantum algorithms, and combines deep learning and time series analysis models to shorten the abnormal detection reaction time to 12 seconds. This shows that the present invention has obvious advantages in improving response speed and real-time performance, can identify and feedback abnormal information more quickly, and improve the safety and reliability of train operation.

[0176] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A railway freight train data collection method based on multi-sensor fusion, characterized in that: include, The running status and location of the train are detected in real time through wireless communication, and each sensor is self-checked when it is detected that the train is about to enter the monitoring area; Based on the multi-sensor after self-inspection, the cargo status is monitored through edge detection and semantic segmentation methods, and the comprehensive status data of the train is collected in real time; According to the comprehensive status data of the train, a three-dimensional health status model of the train is constructed through three-dimensional reconstruction and synchronous positioning; Combining the three-dimensional health status model with the edge-cloud collaborative architecture, quantum algorithms are used to accelerate the hierarchical processing and dynamic optimization of the train's comprehensive status data, and identify abnormal features in real time; Abnormal features are analyzed through deep learning and time series analysis algorithms, alarm information is generated and fed back to the edge computing terminal.

2. The railway freight train data collection method based on multi-sensor fusion according to claim 1, characterized in that: The sensors include a door frame top vision sensor, door frame side vision sensors, rail side status sensors and rail bottom status sensors; The running status and position of the train are detected in real time through wireless communication, and each sensor is self-checked when it is detected that the train is about to enter the monitoring area. The specific steps are as follows: Through GNSS positioning and radio frequency identification methods, the running status and position of the train are monitored in real time to generate train running status data; The time when the train arrives at the monitoring area is calculated through the Kalman filter prediction algorithm; Set a safety distance threshold. When it is predicted that the distance between the train and the monitoring area is less than the preset safety distance threshold, the edge computing terminal sends a self-check start instruction to each sensor. After receiving the self-test start command, each sensor performs status detection in parallel according to the self-test sequence and outputs sensor data; Upload sensor data to the edge computing terminal for real-time aggregation.

3. The railway freight train data collection method based on multi-sensor fusion as claimed in claim 2, characterized in that: The multi-sensor based on self-inspection performs cargo status monitoring through edge detection and semantic segmentation methods, and collects comprehensive train status data in real time. The specific steps are as follows: After the status detection is completed, each sensor synchronously collects 2D images and 3D point cloud data; De-noising, enhancement and geometric calibration of collected 2D images and 3D point cloud data; Use the Canny edge detection algorithm to extract the contour features of the goods in the 2D image; Apply deep learning semantic segmentation models to classify cargo types and structures in 3D point cloud data and identify the loading status of different cargoes; Combine the identified loading status of different goods with the contour features of the goods to generate a panoramic cargo identification view; Detect the cargo shape and internal residue in the panoramic cargo identification view to form cargo status data; The Kalman filtering method is used to combine the train operation status data with the cargo status data to generate the comprehensive train status data; The comprehensive train status data is fed back to the edge computing terminal in real time.

4. The railway freight train data collection method based on multi-sensor fusion as claimed in claim 3, characterized in that: The three-dimensional health status model of the train is constructed based on the comprehensive status data of the train through three-dimensional reconstruction and synchronous positioning. The specific steps are as follows: Each time step in the train running status data is regarded as a node of the graph, and the time dimension is regarded as the time series of the graph; The cargo status data is used as the attribute of the corresponding node; Based on the nodes of the graph, the time series of the graph, and the attributes of the corresponding nodes, the information between the nodes is streamed and fused through the spatiotemporal graph convolution operation; Constructing spatiotemporal adaptive functions during spatiotemporal graph convolution operations The expression is: in, represents the context information vector of node i at time t, α represents The influence weight of represents the L2 norm of the context information vector of node i at time t, represents the state difference between node i and its neighboring nodes at time t, and β represents The weight of influence; Based on spatiotemporal adaptive function The relationship between nodes in the graph convolution process is dynamically adjusted to construct a heterogeneous spatiotemporal graph convolution model, which is expressed as: in, represents the feature vector of node i after the graph convolution operation at time t+1, σ represents the nonlinear activation function, N represents the number of all nodes in the graph, j represents the adjacent node, and A ij represents the adjacency matrix, W i (t) represents the influence weight of node i on the adjacent node information at time t, represents the feature vector of the adjacent node j at time t, B i represents the bias term; The feature vectors of all nodes i after the graph convolution operation The time steps are integrated to form the health status feature set of the train at different time nodes; Based on the health status feature set, a three-dimensional health status model is constructed through feature aggregation.

5. The railway freight train data collection method based on multi-sensor fusion as claimed in claim 4, characterized in that: The three-dimensional health status model and edge-cloud collaborative architecture are combined to use quantum algorithms to accelerate the hierarchical processing and dynamic optimization of the comprehensive status data of the train, and identify abnormal features in real time. The specific steps are as follows: Adopting edge-cloud collaborative architecture, the edge computing terminal processes and monitors the health status feature set in real time, and the cloud performs global optimization and anomaly identification tasks; The train running status data is used as the first-level optimization variable, the cargo status data is used as the second-level optimization variable, and the sensor data is used as the third-level optimization variable; Using the quantum approximate optimization algorithm, each layer is optimized through the loss function in the quantum circuit, and the expression is: in, represents the loss function in the quantum circuit, represents the state difference between node i and its neighboring nodes at time t+1, represents the L2 norm of the feature vector of node i after the graph convolution operation at time t+1, γ is the influence factor of quantum control, f(θ) represents the quantum state control function, M represents the number of quantum bits, and θ k represents the control parameter of quantum bit k, cos(θ k ) represents the projection of the control qubit state in the ground state direction, sin(θ k ) represents the projection of the control qubit state in the excited state direction, ω k Indicates the control cos(θ k ) weight coefficient, δ k represents the control sin(θ k )’s weight coefficient; Time series analysis and deep neural network models are used to model the time series patterns of the optimized train operation status data, cargo status data and sensor data, and autoencoders are used to detect abnormal features.

6. The railway freight train data collection method based on multi-sensor fusion as claimed in claim 5, characterized in that: The time series analysis and deep neural network model are used to model the time series patterns of the optimized train operation status data, cargo status data and sensor data, and the abnormal features are detected by the autoencoder. The specific steps are as follows: Clean and standardize the optimized train operation status data, cargo status data and sensor data; Long short-term memory networks are used to model the cleaned and standardized train operation status data, cargo status data, and sensor data, extract time series features, and capture the time series dependencies between various state variables. Use convolutional neural networks to extract features from the cleaned and standardized train operation status data, cargo status data, and sensor data to identify time series features; Based on the time series dependency and time series features, the autoencoder model is used to analyze the reconstruction error, identify abnormal data points that deviate from the normal state, and mark the abnormal features.

7. The railway freight train data collection method based on multi-sensor fusion according to claim 6, characterized in that: The abnormal features are analyzed by deep learning and time series analysis algorithms, alarm information is generated and fed back to the edge computing terminal. The specific steps are as follows: Use deep learning models to analyze abnormal features and identify abnormal behavior information that deviates from normal patterns; Combined with the time series analysis algorithm, further identification and confirmation of abnormal patterns are performed based on the identified abnormal behavior information; Integrate confirmed abnormal behavior information to form an abnormal detection report; Generate alarm information based on the anomaly detection report and indicate the anomaly type and severity; The generated alarm information is transmitted to the edge computing terminal for real-time notification and response.

8. A railway freight train data acquisition system based on multi-sensor fusion, based on the railway freight train data acquisition method based on multi-sensor fusion according to any one of claims 1 to 7, characterized in that: Including, self-check module, monitoring module, modeling module, abnormality identification module and alarm generation module; The self-check module is used to detect the running status and position of the train in real time through wireless communication, and to perform self-check on each sensor when it is detected that the train is about to enter the monitoring area; The monitoring module is used to monitor the cargo status based on the multi-sensor after self-inspection through edge detection and semantic segmentation methods, and collect the comprehensive status data of the train in real time; The modeling module is used to construct a three-dimensional health status model of the train through three-dimensional reconstruction and synchronous positioning according to the comprehensive status data of the train; The anomaly identification module is used to combine the three-dimensional health status model with the edge-cloud collaborative architecture, use quantum algorithms to accelerate the hierarchical processing and dynamic optimization of the comprehensive status data of the train, and identify abnormal characteristics in real time; The alarm generation module is used to analyze abnormal features through deep learning and time series analysis algorithms, generate alarm information and feed it back to the edge computing terminal.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the railway freight train data collection method based on multi-sensor fusion described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the railway freight train data collection method based on multi-sensor fusion described in any one of claims 1 to 7 are implemented.

Images