Multi-terminal data fusion transmission platform based on watchtower

Abstract

The invention relates to the technical field of data fusion transmission, in particular to a multiterminal data fusion transmission platform based on a watchtower. The technical problems that a traditional data transmission platform often depends on single or limited sensor equipment, the data acquisition range is limited, data fusion processing is relatively simple, and monitoring requirements in a complex and changeable environment are difficult to meet exist. According to the technical scheme, the multiterminal data fusion transmission platform based on the watchtower comprises a watchtower module, a data acquisition module, a data transmission module, a data fusion module and a data storage module, according to the invention, a multi-modal data fusion method is adopted through the data fusion module, cross-modal information extraction and association can be carried out on heterogeneous data from different sensors, the comprehensive utilization rate of the data is improved, data fusion is carried out by using a graph theory algorithm, the potential relationship and mode between the sensor data are revealed, and the data fusion efficiency is improved. And large-range, high-precision and real-time data fusion transmission is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data fusion and transmission, and particularly to a multi-terminal data fusion and transmission platform based on a watchtower. Background Art

[0002] In the current monitoring and data transmission fields, traditional data transmission platforms often rely on single or limited sensor devices, with limited data acquisition scope and relatively simple data fusion processing, making it difficult to meet the monitoring requirements in complex and changeable environments; especially when it is necessary to simultaneously monitor multiple data types such as video images, moving targets, meteorological parameters, and sound information, the data processing capabilities and real-time performance of traditional platforms are inadequate; in addition, in the face of sensor failures or data anomalies, traditional platforms often lack effective fault tolerance mechanisms, resulting in a decline in data accuracy and reliability;

[0003] Therefore, in view of the above problems, a multi-terminal data fusion and transmission platform based on a watchtower is proposed, which integrates multiple sensor devices and utilizes the high-point observation advantages of the watchtower to achieve large-range, high-precision, and real-time data monitoring and fusion transmission. Summary of the Invention

[0004] In order to overcome the problems existing in the process of using traditional data transmission platforms in daily work, where the data transmission platform often relies on single or limited sensor devices, with limited data acquisition scope and relatively simple data fusion processing, making it difficult to meet the monitoring requirements in complex and changeable environments.

[0005] The technical solution of the present invention is as follows: A multi-terminal data fusion and transmission platform based on a watchtower, comprising:

[0006] A watchtower module, which is used to provide a high-point observation platform and expand the monitoring range to improve the accuracy and real-time performance of data acquisition;

[0007] A data acquisition module, which is used to collect data on video images, moving targets, meteorological parameters, and sound information by using multi-type sensing devices;

[0008] A data transmission module, which is used to transmit the collected data to the data fusion module through a wireless network;

[0009] A data fusion module, which is used to perform cross-modal information extraction and association on heterogeneous data from different sensors by using a multi-modal data fusion method, and perform data fusion by using a graph theory algorithm. At the same time, it is used for an algorithm that adaptively adjusts the fusion strategy according to environmental changes and data characteristics, and adopts a fault tolerance mechanism to handle sensor failures and data anomalies;

[0010] A data storage module, which is used to receive, process, and store data from the watchtower.

[0011] Preferably, the watchtower module provides a high - point observation platform and is used to expand the monitoring range to improve the accuracy and real - time performance of data acquisition; through the data acquisition module, various types of sensing devices are used to collect data on video images, moving targets, meteorological parameters, and sound information; through the data transmission module, the collected data is transmitted to the data fusion module via a wireless network; through the data fusion module, a multi - modal data fusion method is used to perform cross - modal information extraction and association on heterogeneous data from different sensors, and a graph - theory algorithm is used for data fusion. At the same time, an algorithm for adaptively adjusting the fusion strategy according to environmental changes and data characteristics is adopted, and a fault - tolerance mechanism is used to handle sensor failures and data anomalies; through the data storage module, data from the watchtower is received, processed, and stored.

[0012] Preferably, the data acquisition module includes a high - definition camera, a radar, a meteorological sensor, and a sound sensor. The high - definition camera is used to obtain real - time video images of the monitoring area, the radar is used to detect moving targets in the air, on the ground, and on the water surface, the meteorological sensor is used to detect temperature, humidity, and wind speed information, and the sound sensor is used to collect sound information in the monitoring area. Various sensors of the acquisition module collect data in the monitoring area according to preset sampling frequencies and accuracies.

[0013] Preferably, when the data acquisition module is working, it performs the following specific steps:

[0014] S301: The high - definition camera is started, and real - time video image acquisition is performed at a preset video frame rate to obtain dynamic visual information of the monitoring area;

[0015] S302: The radar is started, a specific detection range and accuracy are set, and continuous monitoring of moving targets in the air, on the ground, and on the water surface is carried out, and the position, speed, and direction data of the targets are recorded in real - time;

[0016] S303: The meteorological sensor is started, and according to the preset sampling frequency, the temperature, humidity, and wind speed meteorological parameters in the monitoring area are monitored and recorded in real - time;

[0017] S304: The sound sensor is started, and sound information in the monitoring area, including environmental noise and sound source information, is collected at a set audio sampling rate and bit depth;

[0018] S305: The data acquisition module synchronizes and integrates various data collected by the high - definition camera, radar, meteorological sensor, and sound sensor according to a unified timestamp;

[0019] Among them, the specific steps for the data acquisition module to synchronize and integrate various data collected by the high - definition camera, radar, meteorological sensor, and sound sensor according to a unified timestamp are as follows:

[0020] S3051: Set a unified time synchronization mechanism for each type of sensor so that the data collected by all sensors has the same time reference;

[0021] S3052: During the data collection process, for each type of sensor, when it collects data, record the timestamp corresponding to the data;

[0022] S3053: After the data collection module receives the data collected by various sensors, sort and organize the data according to the timestamps to ensure the temporal consistency of the data;

[0023] S3054: Package the organized data according to the preset data format to form a unified data packet.

[0024] Preferably, when the data transmission module performs data transmission, it uses 5G network transmission and Wi-Fi network transmission, which can be applicable to scenarios where wired networks cannot be laid.

[0025] Preferably, the data fusion module uses a multi-modal data fusion method to perform cross-modal information extraction and association on heterogeneous data from different sensors, and uses graph theory algorithms for data fusion. Consider the sensor network as a graph structure, where nodes represent sensors and edges represent the relationships or data correlations between sensors. Use the shortest path algorithm for data fusion to reveal the potential relationships and patterns between sensor data. At the same time, an algorithm that adaptively adjusts the fusion strategy according to environmental changes and data characteristics is used to meet the data fusion requirements in different scenarios. A fault tolerance mechanism is used to handle sensor failures and data anomalies. When a certain sensor fails or has data anomalies, the data fusion module can automatically exclude the faulty data and use the data of other normal sensors for fusion processing.

[0026] Preferably, when the data fusion module is working, it performs the following specific steps:

[0027] S601: Set the parameters of the graph theory algorithm, with each sensor as a node in the graph structure and the relationships and data correlations between sensors as the edges in the graph;

[0028] S602: Receive the data packets from the data transmission module. Each data packet contains the data collected by multiple sensors and has been synchronized and integrated according to a unified timestamp;

[0029] S603: Unpack the received data packets to extract the data collected by each sensor;

[0030] S604: Construct a sensor network graph. According to the shortest path algorithm, initialize a graph object to store sensor nodes and edges;

[0031] S605: Add each sensor as a node to the graph and set the attributes of the node;

[0032] S606: Define the weights and attributes of the edges according to the types of sensors and the types of data collected;

[0033] S607: Add the defined edges to the graph to connect the corresponding nodes;

[0034] S608: Perform cross-modal information extraction and association;

[0035] S609: Monitor environmental changes and data characteristics and adjust the fusion strategy;

[0036] S610: Detect sensor failures and data anomalies. When a failure or data anomaly of a certain sensor is detected, exclude the data of that sensor and use the data of other normal sensors for fusion processing;

[0037] S611: Input the preprocessed sensor data into the fusion algorithm to perform fusion processing, where weighted average method and Kalman filtering method are used for data fusion;

[0038] S612: Organize the fused data into a unified vector and matrix form and output the organized data to the data storage module.

[0039] Preferably, in step S608, performing cross-modal information extraction and association includes the following specific steps:

[0040] S701: For the video images collected by the high-definition camera, extract the contour, texture, and color features of the target object;

[0041] S702: For the data collected by the radar, extract the position, speed, and direction motion features of the target object;

[0042] S703: For the meteorological sensors, extract the environmental features of temperature, humidity, and wind speed;

[0043] S704: Represent the visual features as vectors and matrices, where the contour is represented as a set of coordinates of edge points, and the texture is represented as a statistic of texture features;

[0044] S705: Represent the motion features as vectors, where position, speed, and direction form a motion feature vector;

[0045] S706: Represent the environmental features and audio features as vectors and matrices respectively;

[0046] S707: Take the feature vector of each sensor as one dimension of a multi-dimensional feature vector space, and construct a high-dimensional feature vector space according to the number of sensors and the dimension of the feature vector;

[0047] S708: Take the points in the feature vector space, i.e., the feature vectors, as the nodes of the graph;

[0048] S709: Define the edges and the weights of the edges between the nodes according to the correlation between the feature vectors;

[0049] S710: Use Dijkstra's shortest path algorithm to find the optimal association path on the graph;

[0050] Among them, using Dijkstra's shortest path algorithm to find the optimal association path on the graph, the specific steps are as follows:

[0051] S7101: Initialize all the nodes in the graph, set a starting node, set the distance of the starting node to 0, and set the distances of all the other nodes to infinity;

[0052] S7102: Create a priority queue, add all the nodes to the queue, and sort them according to the distances of the nodes;

[0053] S7103: Take the node with the smallest distance from the priority queue as the current node, and mark it as visited;

[0054] S7104: Traverse all the adjacent nodes of the current node, and calculate the distances from the starting node to each adjacent node; if the distance to an adjacent node through the current node is less than the current distance value of the adjacent node, then update the distance value of the adjacent node, and record the current node as the predecessor node of the adjacent node;

[0055] S7105: Repeat steps S7103 and S7104 until the priority queue is empty, that is, all the nodes have been visited, or the target node has been found;

[0056] S7106: According to the recorded predecessor node information, backtrack from the target node to construct the optimal association path from the starting node to the target node;

[0057] S7107: Output the optimal association path, which represents the optimal association method between cross-modal information.

[0058] Preferably, in step S609, monitor the environmental changes and data characteristics, and adjust the fusion strategy, including the following specific steps:

[0059] S801: Input the real-time data from various sensors;

[0060] S802: The data fusion module continuously receives the data from these sensors and performs denoising and calibration processing on the data;

[0061] S803: Using time series analysis methods, analyze the changing trend of the pre-processed real-time data. Through statistical methods, detect the abnormal points and abnormal patterns in the data, and at the same time extract the key features in the data, including the number, speed, and direction of moving targets;

[0062] S804: According to the extracted environmental change information and data characteristics, adjust the weights of different sensor data;

[0063] S805: Select a fusion algorithm according to actual needs; among them, in cases where high real-time performance is required, the weighted average method is selected; in cases where high-precision fusion results are required, the Kalman filtering method is selected;

[0064] S806: According to the selected fusion algorithm, set the corresponding algorithm parameters, including the weights in the weighted average method, the state transition matrix and the observation matrix in the Kalman filtering method;

[0065] S807: Output the selected fusion algorithm and its parameters.

[0066] Preferably, in step S610, detect sensor faults and data anomalies. When a fault or data anomaly is detected in a certain sensor, exclude the data of this sensor and perform fusion processing using the data of other normal sensors, including the following specific steps:

[0067] S901: When the data fusion module starts up, initialize the fault detection mechanism. The fault detection mechanism includes setting the normal range, abnormal threshold, and fault determination rules of sensor data;

[0068] S902: The data fusion module continuously receives the data from each sensor and performs real-time monitoring on these data;

[0069] S903: Evaluate the data quality of each sensor, and check whether there are data missing, abnormal jumps, and obvious inconsistencies with the data of other sensors;

[0070] S904: According to the preset fault determination rules, perform fault determination on the sensor data; among them, when the data of a certain sensor exceeds the normal range continuously for multiple times or the data change rate is abnormal, it is determined that this sensor may have a fault;

[0071] S905: When a fault or data anomaly is detected in a certain sensor, the data fusion module marks the data of this sensor as abnormal data and temporarily excludes it from the fusion processing, and at the same time records the information of the faulty sensor and the time of the fault occurrence;

[0072] S906: After excluding the data of the faulty sensor, the data fusion module readjusts the fusion strategy and performs fusion processing using the data of other normal sensors;

[0073] S907: Send a fault alarm message to the maintenance personnel, including the location and type of the faulty sensor;

[0074] S908: After the faulty sensor is repaired or replaced, the data fusion module reconnects the data of this sensor and repeats steps S902 - S904 for a period of verification and observation; if the verification result shows that the sensor data returns to normal, then incorporate it back into the fusion processing; otherwise, continue to exclude it from the fusion processing and notify the relevant personnel for further inspection and handling.

[0075] Preferably, the data storage module includes a data receiving interface, a data processing unit, a data storage unit, a data backup and recovery unit, and a data security unit; the data receiving interface is used to provide an interface with the data fusion module for receiving the processed data in real time; the data processing unit is responsible for the pre - processing of data, including data cleaning, format conversion, and compression; the data storage unit includes a relational database, a non - relational database, and a file storage system for storing the processed data; the data backup and recovery unit is responsible for implementing the data backup strategy and recovering data when needed; the data security unit is responsible for the security management of data, including encryption, access control, and auditing.

[0076] Advantages of the present invention:

[0077] 1. Through the data fusion module, this platform adopts a multi - modal data fusion method, which can perform cross - modal information extraction and association on heterogeneous data from different sensors, improving the comprehensive utilization rate of data, and using graph theory algorithms for data fusion to reveal the potential relationships and patterns between sensor data, providing more comprehensive information support for decision - making;

[0078] 2. Through the application of the adaptive adjustment of the fusion strategy and the fault - tolerance mechanism, this platform enables the system to meet the data fusion requirements in different scenarios and effectively handle the situations of sensor faults and data anomalies. Brief Description of the Drawings

[0079] Figure 1 Shown is a schematic structural diagram of the multi - terminal data fusion and transmission platform based on a watchtower of the present invention;

[0080] Figure 2 Shown is a schematic flowchart of the working steps of the data fusion module of the multi - terminal data fusion and transmission platform based on a watchtower of the present invention. Detailed Embodiments

[0081] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0082] Please refer to Figure 1 - Figure 2 , the present invention provides an embodiment: a multi-terminal data fusion transmission platform based on a watchtower, including:

[0083] A watchtower module, which is used to provide a high-point observation platform and expand the monitoring range to improve the accuracy and real-time performance of data acquisition;

[0084] A data acquisition module, which is used to collect data of video images, moving targets, meteorological parameters, and sound information by using multi-type sensing devices;

[0085] A data transmission module, which is used to transmit the collected data to the data fusion module through a wireless network;

[0086] A data fusion module, which is used to extract and correlate cross-modal information of heterogeneous data from different sensors by using a multi-modal data fusion method, and perform data fusion by using a graph theory algorithm. At the same time, it is used for an algorithm that adaptively adjusts the fusion strategy according to environmental changes and data characteristics, and adopts a fault tolerance mechanism to handle sensor failures and data anomalies;

[0087] A data storage module, which is used to receive, process, and store data from the watchtower.

[0088] Preferably, a high-point observation platform is provided through the watchtower module and used to expand the monitoring range to improve the accuracy and real-time performance of data acquisition; through the data acquisition module, multi-type sensing devices are used to collect data of video images, moving targets, meteorological parameters, and sound information; the collected data is transmitted to the data fusion module through the data transmission module via a wireless network; through the data fusion module, a multi-modal data fusion method is used to extract and correlate cross-modal information of heterogeneous data from different sensors, and a graph theory algorithm is used for data fusion. At the same time, it is used for an algorithm that adaptively adjusts the fusion strategy according to environmental changes and data characteristics, and adopts a fault tolerance mechanism to handle sensor failures and data anomalies; through the data storage module, data from the watchtower is received, processed, and stored.

[0089] Preferably, the data acquisition module includes a high-definition camera, a radar, a meteorological sensor, and a sound sensor. The high-definition camera is used to obtain real-time video images of the monitoring area, the radar is used to detect moving targets in the air, on the ground, and on the water surface, the meteorological sensor is used to detect temperature, humidity, and wind speed information, and the sound sensor is used to collect sound information in the monitoring area. All types of sensors in the acquisition module collect data in the monitoring area according to preset sampling frequencies and precisions.

[0090] Preferably, when the data acquisition module is working, it performs the following specific steps:

[0091] S301: The high-definition camera is started, and real-time video image acquisition is performed at a preset video frame rate to obtain dynamic visual information of the monitoring area;

[0092] S302: The radar is started, a specific detection range and accuracy are set, moving targets in the air, on the ground, and on the water surface are continuously monitored, and the position, speed, and direction data of the targets are recorded in real time;

[0093] S303: The meteorological sensor is started, and the temperature, humidity, and wind speed meteorological parameters in the monitoring area are monitored and recorded in real time according to a preset sampling frequency;

[0094] S304: The sound sensor is started, and the sound information in the monitoring area, including environmental noise and sound source information, is collected at a set audio sampling rate and bit depth;

[0095] S305: The data acquisition module synchronizes and integrates various data collected by the high-definition camera, radar, meteorological sensor, and sound sensor according to a unified timestamp;

[0096] Among them, the data acquisition module synchronizes and integrates various data collected by the high-definition camera, radar, meteorological sensor, and sound sensor according to a unified timestamp. The specific steps are as follows:

[0097] S3051: Set a unified time synchronization mechanism for each sensor so that the data collected by all sensors has the same time reference;

[0098] S3052: During the data acquisition process, each sensor records the timestamp corresponding to the data while collecting the data;

[0099] S3053: After receiving the data collected by various sensors, the data acquisition module sorts and organizes the data according to the timestamp to ensure the chronological consistency of the data;

[0100] S3054: Package the organized data according to a preset data format to form a unified data packet.

[0101] Preferably, when the data transmission module performs data transmission, it uses 5G network transmission and Wi-Fi network transmission, which can be applicable to scenarios where wired networks cannot be laid.

[0102] Preferably, the data fusion module uses a multi-modal data fusion method to perform cross-modal information extraction and association on heterogeneous data from different sensors, and uses graph theory algorithms for data fusion. The sensor network is regarded as a graph structure, where nodes represent sensors and edges represent the relationships or data correlations between sensors. The shortest path algorithm is used for data fusion to reveal the potential relationships and patterns between sensor data. At the same time, an algorithm that adaptively adjusts the fusion strategy according to environmental changes and data characteristics is used to meet the data fusion requirements in different scenarios. A fault tolerance mechanism is adopted to handle sensor failures and data anomalies. When a certain sensor fails or has data anomalies, the data fusion module can automatically exclude the faulty data and use the data of other normal sensors for fusion processing.

[0103] Preferably, when the data fusion module is working, the following specific steps are executed:

[0104] S601: Set the parameters of the graph theory algorithm, take each sensor as a node in the graph structure, and take the relationships and data correlations between sensors as the edges in the graph;

[0105] S602: Receive the data packets from the data transmission module. Each data packet contains the data collected by multiple sensors and has been synchronized and integrated according to a unified timestamp;

[0106] S603: Unpack the received data packets and extract the data collected by each sensor;

[0107] S604: Construct a sensor network graph. According to the shortest path algorithm, initialize a graph object to store sensor nodes and edges;

[0108] S605: Add each sensor as a node to the graph and set the attributes of the nodes;

[0109] S606: Define the weights and attributes of the edges according to the types of sensors and the types of data collected;

[0110] S607: Add the defined edges to the graph to connect the corresponding nodes;

[0111] S608: Perform cross-modal information extraction and association;

[0112] S609: Monitor environmental changes and data characteristics and adjust the fusion strategy;

[0113] S610: Detect sensor failures and data anomalies. When it is detected that a certain sensor fails or has data anomalies, exclude the data of this sensor and use the data of other normal sensors for fusion processing;

[0114] S611: Input the preprocessed sensor data into the fusion algorithm, perform fusion processing, where the weighted average method and the Kalman filtering method are used for data fusion;

[0115] S612: Organize the fused data to form a unified vector and matrix form, and output the organized data to the data storage module.

[0116] Preferably, in step S608, cross-modal information extraction and association are performed, including the following specific steps:

[0117] S701: For the video images collected by the high-definition camera, extract the contour, texture, and color features of the target object;

[0118] S702: For the data collected by the radar, extract the position, speed, and direction motion features of the target object;

[0119] S703: For the meteorological sensor, extract the temperature, humidity, and wind speed environmental features;

[0120] S704: Represent the visual features as vectors and matrices, where the contour is represented as a set of coordinates of edge points, and the texture is represented as a statistic of texture features;

[0121] S705: Represent the motion features as vectors, where the position, speed, and direction form a motion feature vector;

[0122] S706: Represent the environmental features and audio features as vectors and matrices respectively;

[0123] S707: Take the feature vector of each sensor as a dimension of the multi-dimensional feature vector space, and construct a high-dimensional feature vector space according to the number of sensors and the dimension of the feature vector;

[0124] S708: Take the points in the feature vector space, i.e., the feature vectors, as the nodes of the graph;

[0125] S709: Define the edges and the weights of the edges between the nodes according to the correlation between the feature vectors;

[0126] S710: Use the Dijkstra shortest path algorithm to find the optimal association path on the graph;

[0127] Among them, using the Dijkstra shortest path algorithm to find the optimal association path on the graph, the specific steps are as follows:

[0128] S7101: Initialize all the nodes in the graph, set a starting node, set the distance of the starting node to 0, and set the distances of all the other nodes to infinity;

[0129] S7102: Create a priority queue, add all nodes to the queue, and sort them according to the distance of the nodes;

[0130] S7103: Take out the node with the minimum distance from the priority queue as the current node, and mark it as visited;

[0131] S7104: Traverse all adjacent nodes of the current node, calculate the distance from the starting node to each adjacent node; if the distance to the adjacent node through the current node is less than the current distance value of the adjacent node, update the distance value of the adjacent node, and record the current node as the predecessor node of the adjacent node;

[0132] S7105: Repeat steps S7103 and S7104 until the priority queue is empty, that is, all nodes have been visited, or the target node has been found;

[0133] S7106: According to the recorded predecessor node information, trace back from the target node to construct the optimal association path from the starting node to the target node;

[0134] S7107: Output the optimal association path, which represents the optimal association method between cross-modal information.

[0135] Preferably, in step S609, monitor environmental changes and data characteristics, and adjust the fusion strategy, including the following specific steps:

[0136] S801: Input real-time data from various sensors;

[0137] S802: The data fusion module continuously receives the data of these sensors, and performs denoising and calibration processing on the data;

[0138] S803: Use time series analysis methods to analyze the change trend of the pre-processed real-time data, detect abnormal points and abnormal patterns in the data through statistical methods, and extract key features in the data, including the number, speed, and direction of moving targets;

[0139] S804: According to the extracted environmental change information and data characteristics, adjust the weights of different sensor data;

[0140] S805: Select a fusion algorithm according to actual needs; among them, in occasions where high real-time performance is required, the weighted average method is selected; in occasions where high-precision fusion results are required, the Kalman filter method is selected;

[0141] S806: According to the selected fusion algorithm, set the corresponding algorithm parameters, including the weights in the weighted average method, the state transition matrix and the observation matrix in the Kalman filter method;

[0142] S807: Output the selected fusion algorithm and its parameters.

[0143] Preferably, in step S610, sensor faults and data anomalies are detected. When a fault or data anomaly is detected in a certain sensor, the data of that sensor is excluded, and the data of other normal sensors is used for fusion processing, including the following specific steps:

[0144] S901: When the data fusion module starts up, it initializes the fault detection mechanism, which includes setting the normal range of sensor data, the anomaly threshold, and the fault determination rule;

[0145] S902: The data fusion module continuously receives data from each sensor and monitors these data in real time;

[0146] S903: Evaluate the data quality of each sensor to check whether there are data missing, abnormal jumps, and obvious inconsistencies with the data of other sensors;

[0147] S904: According to the preset fault determination rule, determine whether the sensor data has a fault; among them, when the data of a certain sensor exceeds the normal range continuously for multiple times or the data change rate is abnormal, it is determined that the sensor may have a fault;

[0148] S905: When a fault or data anomaly is detected in a certain sensor, the data fusion module marks the data of that sensor as abnormal data and temporarily excludes it from the fusion processing, and at the same time records the information of the faulty sensor and the time of the fault occurrence;

[0149] S906: After excluding the data of the faulty sensor, the data fusion module readjusts the fusion strategy and uses the data of other normal sensors for fusion processing;

[0150] S907: Send a fault alarm message to the maintenance personnel, including the location and type of the faulty sensor;

[0151] S908: After the faulty sensor is repaired or replaced, the data fusion module reconnects the data of that sensor and repeats steps S902 - S904 for a period of verification and observation; if the verification result shows that the sensor data returns to normal, it is re - incorporated into the fusion processing; otherwise, it continues to be excluded from the fusion processing and relevant personnel are notified for further inspection and processing.

[0152] Preferably, the data storage module includes a data receiving interface, a data processing unit, a data storage unit, a data backup and recovery unit, and a data security unit; the data receiving interface is used to provide an interface with the data fusion module for real-time receiving of the processed data; the data processing unit is responsible for the preprocessing of the data, including data cleaning, format conversion, and compression; the data storage unit includes a relational database, a non-relational database, and a file storage system for storing the processed data; the data backup and recovery unit is responsible for implementing the data backup strategy and restoring the data when needed; the data security unit is responsible for the security management of the data, including encryption, access control, and auditing.

[0153] Embodiment

[0154] Optionally, when it is set to be applied in the fields of transportation, border defense, and urban security monitoring using a multi-terminal data fusion and transmission platform based on a watchtower of the present invention, by integrating various sensing devices such as high-definition cameras, radars, meteorological sensors, and sound sensors, and utilizing the high-point observation advantage of the watchtower, large-scale, high-precision, and real-time data monitoring and fusion transmission are realized, providing strong support for monitoring vehicle and personnel flow and preventing illegal activities and criminal acts; the specific implementation steps are as follows:

[0155] S1: Deploy a watchtower at a transportation artery, the border line, or a key urban area, with a tower height of 30 meters to ensure that the monitoring range covers an area of several kilometers around;

[0156] S2: Adopt a high-definition camera with a 4K resolution, preset the video frame rate to 30 frames per second, and the monitoring range covers an area of 2 kilometers around the watchtower;

[0157] S3: Adopt a Doppler radar with a detection range of 5 kilometers around the watchtower, which can accurately record the position, speed, and direction data of the target;

[0158] S4: Adopt a high-precision meteorological sensor with a sampling frequency of once per minute to provide meteorological data support for environmental analysis and data fusion;

[0159] S5: Adopt a high-sensitivity sound sensor with an audio sampling rate of 44.1 kHz and a bit depth of 16 bits, which can clearly capture the sound details in the monitoring area;

[0160] S6: Start each sensor and collect data according to the preset sampling frequency and accuracy. The data acquisition module synchronizes and integrates the data collected by various sensors according to a unified timestamp; the specific steps include:

[0161] Set a unified time synchronization mechanism (NTP protocol) for each sensor to ensure consistent data time bases;

[0162] The sensor records timestamps when collecting data;

[0163] The data acquisition module sorts and organizes the data according to the timestamps;

[0164] The organized data is packed in a preset JSON format to form a unified data packet;

[0165] S7: Use 5G network and Wi-Fi network for data transmission to ensure the real-time and stability of data transmission; in remote areas where wired networks cannot be laid, such as the border line, mainly rely on 5G network for data transmission;

[0166] S8: Set the parameters of the graph theory algorithm through the data fusion module to construct a sensor network graph;

[0167] S9: Cross-modal information extraction and association, the specific steps include:

[0168] Extract the feature vectors of various sensors and construct a high-dimensional feature vector space;

[0169] Use the Dijkstra shortest path algorithm to find the optimal association path on the feature vector space graph to reveal the optimal association method between cross-modal information;

[0170] S10: Monitor environmental changes and data characteristics, and adjust the fusion strategy, the specific steps include:

[0171] Analyze the real-time data, extract key features, adjust the sensor weights according to environmental changes, select a suitable fusion algorithm (weighted average method or Kalman filtering method) and set parameters;

[0172] S11: Detect sensor failures and data anomalies, and exclude faulty data, the specific steps include:

[0173] Initialize the fault detection mechanism, monitor the sensor data in real time, evaluate the data quality, determine and handle faults, send fault alarm information, and conduct verification and observation after the faults are repaired;

[0174] S12: Input the preprocessed sensor data into the fusion algorithm to perform fusion processing;

[0175] S13: Organize the fused data into a unified vector and matrix form and output it to the data storage module;

[0176] S14: The data storage module receives the data output by the data fusion module and stores it in a relational database (MySQL) for subsequent query and analysis. The data storage module provides a real-time monitoring interface to display the fused video images, moving target trajectories, meteorological parameters, sound information, etc. for the monitoring personnel to view in real time;

[0177] S15: Set warning rules. For example, when an abnormal moving target or environmental parameters exceed the normal range is detected, an early warning signal is automatically triggered to notify relevant personnel for handling;

[0178] Among them, the specific applications of this platform in urban security monitoring include:

[0179] Deployment location: On the observation towers in key urban areas (such as transportation hubs, commercial centers, government buildings, etc.);

[0180] Monitoring targets: Monitor the flow of vehicles and personnel to prevent illegal activities and criminal acts;

[0181] Data collection:

[0182] High-definition cameras: Continuously collect video images of the monitored area and capture the dynamics of vehicles and personnel in real time;

[0183] Radar: Detect the movement trajectories of vehicles and personnel and record position, speed, and direction data;

[0184] Meteorological sensors: Monitor environmental temperature, humidity, and wind speed to provide data support for environmental analysis;

[0185] Sound sensors: Collect sound information in the monitored area and capture abnormal sounds or suspicious conversations;

[0186] Data transmission: Use 5G network to transmit the collected data to the data fusion module in real time;

[0187] Data fusion:

[0188] Cross-modal information extraction: Extract target contours, textures, and color features from video images, position, speed, and direction features from radar data, temperature, humidity, and wind speed features from meteorological data, and audio features from sound data;

[0189] Graph theory algorithm fusion: Construct a sensor network graph, use the Dijkstra shortest path algorithm to find the optimal association path, and reveal the associations between cross-modal information;

[0190] Adaptive adjustment of fusion strategy: Adjust the sensor weights and select appropriate fusion algorithms according to real-time data and environmental changes;

[0191] Fault tolerance processing: Monitor sensor data in real time. When a fault or abnormal data is detected, it is automatically excluded and the fusion strategy is readjusted;

[0192] Early warning function: Set warning rules. When an abnormal moving target or suspicious behavior is detected, an early warning signal is automatically triggered to notify security personnel for handling;

[0193] The comparison between the present invention and the existing data transmission platform is shown in the following table:

[0194]

[0195]

[0196] Therefore, through the above comparison, it can be seen that the present invention is superior to the existing technology in terms of monitoring scope, data acquisition accuracy, data transmission real-time performance, data fusion ability, adaptive adjustment of fusion strategy, fault tolerance mechanism, early warning function, data storage and query, and application scenario adaptability, showing significant beneficial effects.

[0197] The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments, and various changes can be made without departing from the spirit of the present invention within the scope of knowledge possessed by those skilled in the art.

Claims

1. A multi-terminal data fusion and transmission platform based on a watchtower, characterized in that: It includes: A watchtower module, which is used to provide a high-point observation platform and expand the monitoring range to improve the accuracy and real-time performance of data acquisition; A data acquisition module, which is used to collect data on video images, moving targets, meteorological parameters, and sound information by using multi-type sensing devices; A data transmission module, which is used to transmit the collected data to the data fusion module through a wireless network; A data fusion module, which is used to extract and correlate cross-modal information from heterogeneous data from different sensors by using a multi-modal data fusion method, perform data fusion by using a graph theory algorithm, and at the same time, is used for an algorithm that adaptively adjusts the fusion strategy according to environmental changes and data characteristics, and adopts a fault tolerance mechanism to handle sensor failures and data anomalies; A data storage module, which is used to receive, process, and store data from the watchtower.

2. The multi-terminal data fusion and transmission platform based on a watchtower according to claim 1, characterized in that: The data acquisition module includes a high-definition camera, a radar, a meteorological sensor, and a sound sensor. The high-definition camera is used to obtain real-time video images of the monitoring area. The radar is used to detect moving targets in the air, on the ground, and on the water surface. The meteorological sensor is used to detect temperature, humidity, and wind speed information. The sound sensor is used to collect sound information in the monitoring area. All kinds of sensors of the acquisition module collect data in the monitoring area according to the preset sampling frequency and accuracy.

3. The multi-terminal data fusion and transmission platform based on the watchtower according to claim 2, characterized in that: When the data acquisition module is working, it executes the following specific steps: S301: The high-definition camera is started, and real-time video image acquisition is performed according to the preset video frame rate to obtain dynamic visual information of the monitoring area; S302: The radar is started, a specific detection range and accuracy are set, and moving targets in the air, on the ground, and on the water surface are continuously monitored, and the position, speed, and direction data of the targets are recorded in real time; S303: The meteorological sensor is started, and according to the preset sampling frequency, the temperature, humidity, and wind speed meteorological parameters in the monitoring area are monitored and recorded in real time; S304: The sound sensor is started, and sound information in the monitoring area, including environmental noise and sound source information, is collected at the set audio sampling rate and bit depth; S305: The data acquisition module synchronizes and integrates various data collected by the high-definition camera, radar, meteorological sensor, and sound sensor according to a unified timestamp; Among them, the data acquisition module synchronizes and integrates various data collected by the high-definition camera, radar, meteorological sensor, and sound sensor according to a unified timestamp. The specific steps are as follows: S3051: Set a unified time synchronization mechanism for each sensor so that the data collected by all sensors has the same time reference; S3052: During the data acquisition process, each sensor records the timestamp corresponding to the data while collecting the data; S3053: After the data acquisition module receives the data collected by various sensors, it sorts and arranges the data according to the timestamp to ensure the temporal consistency of the data; S3054: Pack the sorted data according to the preset data format to form a unified data packet.

4. The multi-terminal data fusion and transmission platform based on a watchtower according to claim 1, characterized in that: When the data transmission module performs data transmission, it adopts 5G network transmission and Wi-Fi network transmission, which can be applied to scenarios where wired networks cannot be laid.

5. The multi-terminal data fusion and transmission platform based on a watchtower according to claim 1, characterized in that: The data fusion module uses a multi-modal data fusion method to perform cross-modal information extraction and association on heterogeneous data from different sensors, and uses graph theory algorithms for data fusion. The sensor network is regarded as a graph structure, where nodes represent sensors and edges represent the relationships or data correlations between sensors. The shortest path algorithm is used for data fusion to reveal the potential relationships and patterns between sensor data. At the same time, an algorithm that adaptively adjusts the fusion strategy according to environmental changes and data characteristics is used to meet the data fusion requirements in different scenarios. A fault tolerance mechanism is adopted to handle sensor failures and data anomalies. When a certain sensor fails or data is abnormal, the data fusion module can automatically exclude the faulty data and use the data of other normal sensors for fusion processing.

6. The multi-terminal data fusion and transmission platform based on the watchtower according to claim 5, characterized in that: When the data fusion module is working, it performs the following specific steps: S601: Set the parameters of the graph theory algorithm, and regard each sensor as a node in the graph structure, and regard the relationships and data correlations between sensors as edges in the graph; S602: Receive data packets from the data transmission module. Each data packet contains data collected by multiple sensors and has been synchronized and integrated according to a unified timestamp; S603: Unpack the received data packets and extract the data collected by each sensor; S604: Construct a sensor network graph. According to the shortest path algorithm, initialize a graph object to store sensor nodes and edges; S605: Add each sensor as a node to the graph and set the attributes of the node; S606: Define the weights and attributes of the edges according to the types of sensors and the types of collected data; S607: Add the defined edges to the graph to connect the corresponding nodes; S608: Perform cross-modal information extraction and association; S609: Monitor environmental changes and data characteristics and adjust the fusion strategy; S610: Detect sensor failures and data anomalies. When a certain sensor is detected to have failed or data is abnormal, exclude the data of this sensor and use the data of other normal sensors for fusion processing; S611: Input the preprocessed sensor data into the fusion algorithm to perform fusion processing, where the weighted average method and the Kalman filtering method are used for data fusion; S612: Organize the fused data into a unified vector and matrix form and output the organized data to the data storage module.

7. The multi-terminal data fusion transmission platform based on a watchtower according to claim 6, characterized in that: In step S608, for cross-modal information extraction and association, it includes the following specific steps: S701: For the video images collected by the high-definition camera, extract the contour, texture, and color features of the target object; S702: For the data collected by the radar, extract the position, speed, and direction motion features of the target object; S703: For the meteorological sensor, extract the environmental features of temperature, humidity, and wind speed; S704: Represent the visual features as vectors and matrix forms, where the contour is represented as a set of coordinates of edge points, and the texture is represented as the statistics of texture features; S705: Represent the motion features as vectors, where the position, speed, and direction form a motion feature vector; S706: Represent the environmental features and audio features in the form of vectors and matrices respectively; S707: Take the feature vector of each sensor as a dimension of a multi-dimensional feature vector space, and construct a high-dimensional feature vector space according to the number of sensors and the dimension of the feature vector; S708: Take the points in the feature vector space, i.e., the feature vectors, as the nodes of the graph; S709: Define the edges and the weights of the edges between the nodes according to the correlation between the feature vectors; S710: Use the Dijkstra shortest path algorithm to find the optimal association path on the graph; Among them, using the Dijkstra shortest path algorithm to find the optimal association path on the graph, the specific steps are as follows: S7101: Initialize all the nodes in the graph, set a starting node, set the distance of the starting node to 0, and set the distances of all the other nodes to infinity; S7102: Create a priority queue, add all the nodes to the queue, and sort them according to the distances of the nodes; S7103: Take out the node with the smallest distance from the priority queue as the current node, and mark it as visited; S7104: Traverse all the adjacent nodes of the current node, and calculate the distances from the starting node to each adjacent node; If the distance to an adjacent node through the current node is less than the current distance value of the adjacent node, then update the distance value of the adjacent node, and record the current node as the predecessor node of the adjacent node; S7105: Repeat steps S7103 and S7104 until the priority queue is empty, that is, all the nodes have been visited, or the target node has been found; S7106: According to the recorded predecessor node information, backtrack from the target node to construct the optimal association path from the starting node to the target node; S7107: Output the optimal association path, which represents the optimal association method between cross-modal information.

8. The multi-terminal data fusion and transmission platform based on a watchtower according to claim 6, characterized in that: In step S609, monitor the environmental changes and data characteristics, and adjust the fusion strategy, including the following specific steps: S801: Input the real-time data from various sensors; S802: The data fusion module continuously receives the data of these sensors, and performs denoising and calibration processing on the data; S803: Use time series analysis methods to analyze the change trend of the pre-processed real-time data, detect the abnormal points and abnormal patterns in the data through statistical methods, and extract the key features in the data, including the number, speed, and direction of moving targets; S804: Adjust the weights of the data of different sensors according to the extracted environmental change information and data characteristics; S805: Select a fusion algorithm according to the actual needs; among them, in the case of high real-time requirements, select the weighted average method; in the case of high-precision fusion results, select the Kalman filtering method; S806: Set the corresponding algorithm parameters according to the selected fusion algorithm, including the weights in the weighted average method, the state transition matrix and the observation matrix in the Kalman filtering method; S807: Output the selected fusion algorithm and its parameters.

9. The multi-terminal data fusion and transmission platform based on a watchtower according to claim 6, characterized in that: In step S610, sensor faults and data anomalies are detected. When a fault or data anomaly is detected in a certain sensor, the data of that sensor is excluded, and the data of other normal sensors is used for fusion processing, including the following specific steps: S901: When the data fusion module starts up, it initializes the fault detection mechanism, which includes setting the normal range of sensor data, anomaly thresholds, and fault determination rules; S902: The data fusion module continuously receives data from each sensor and monitors this data in real time; S903: Evaluate the data quality of each sensor, and check whether the data is missing, has abnormal jumps, and is significantly inconsistent with the data of other sensors; S904: According to the preset fault determination rules, determine whether there is a fault in the sensor data; among them, when the data of a certain sensor exceeds the normal range continuously for multiple times or the data change rate is abnormal, it is determined that the sensor may have a fault; S905: When a fault or data anomaly is detected in a certain sensor, the data fusion module marks the data of that sensor as abnormal data and temporarily excludes it from the fusion processing. At the same time, the information of the faulty sensor and the time of the fault occurrence are recorded; S906: After excluding the data of the faulty sensor, the data fusion module readjusts the fusion strategy and uses the data of other normal sensors for fusion processing; S907: Send a fault alarm message to the maintenance personnel, including the location and type of the faulty sensor; S908: After the faulty sensor is repaired or replaced, the data fusion module reconnects the data of that sensor and repeats steps S902 - S904 for a period of verification and observation; if the verification result shows that the sensor data returns to normal, it is re - incorporated into the fusion processing; otherwise, it continues to be excluded from the fusion processing, and relevant personnel are notified for further inspection and processing.

10. The multi-terminal data fusion and transmission platform based on a watchtower according to claim 1, wherein: The data storage module includes a data receiving interface, a data processing unit, a data storage unit, a data backup and recovery unit, and a data security unit; the data receiving interface is used to provide an interface with the data fusion module for receiving the processed data in real time; The data processing unit is responsible for the pre - processing of data, including data cleaning, format conversion, and compression; the data storage unit includes a relational database, a non - relational database, and a file storage system for storing the processed data; The data backup and recovery unit is responsible for implementing the data backup strategy and recovering data when needed; the data security unit is responsible for the security management of data, including encryption, access control, and auditing.

Images