Dual-mode terahertz detection data synchronization method and system

Abstract

The application belongs to the field of industrial nondestructive testing, and relates to a terahertz detection technology, and is used for solving the problems of low data synchronization accuracy, rigid filtering model and weak network adaptability in the prior art, and specifically relates to a dual-mode terahertz detection data synchronization method and system, which comprises a protocol layer, an algorithm layer, a perception layer, a visualization layer and a system layer; in the protocol layer, a length prefix method is adopted to realize message frame delimiting, and the receiving end completely drives analysis, without the need of the pushing end to maintain the application layer sending buffer state, so that the problem of accurately identifying and separating variable-length data packets in a continuous TCP byte stream is effectively solved, and data analysis errors or data loss caused by message boundary ambiguity are avoided; according to real-time changes of network quality, the time step calculation strategy of the Kalman filter is intelligently adjusted, so that the reliability of data synchronization is ensured.

Description

Technical Field

[0001] This invention belongs to the field of industrial nondestructive testing and relates to terahertz detection technology, specifically a method and system for synchronizing dual-mode terahertz detection data. Background Technology

[0002] Wind turbine blades are the core components of wind turbine generators. During long-term operation, they are prone to internal defects such as holes, cracks, and delamination. Terahertz waves have advantages such as strong penetration, sensitivity to non-metallic materials, and no ionizing radiation, and have become a cutting-edge technology for detecting internal defects in blades. In practical applications, drones equipped with terahertz detectors are used for mobile scanning to simultaneously collect reflection imaging data and spatial coordinate data. The three-dimensional location of defects is achieved through post-processing.

[0003] Multi-sensor data synchronization is a common problem in the Industrial Internet of Things (IIoT), and typical solutions include:

[0004] Hardware synchronization: Clock alignment is achieved through GPS PPS pulses and dedicated synchronization cables, but this increases hardware cost and weight, making it unsuitable for the lightweight requirements of drones;

[0005] Software timestamps: Each data source generates a timestamp independently, and the receiving end aligns based on the timestamp. However, the accuracy is limited due to system clock drift and network latency jitter.

[0006] Fixed-frequency interpolation: linear interpolation between coordinate data frames, but it cannot handle non-linear motions such as acceleration, deceleration, and turning of UAVs, and the error is relatively large.

[0007] The invention patent with publication number CN109900713B discloses a camera-guided UAV-based dynamic detection system and method for wind turbine blade defects. The main technical solutions of the system include: UAV equipped with a high-definition camera; global motion measurement and contour extraction of the blade; overall fault detection and analysis of the wind turbine blade; and UAV-based local area fault detection and analysis of the blade.

[0008] The invention patent with publication number CN120142224A discloses a method and related device for detecting the web of a wind turbine blade. The main technical solution of the method includes: performing Fourier transform on first sample data and first reference data to obtain second sample data and second reference data; deconvolving the second sample data based on the second sample data and second reference data to obtain third sample data; performing noise reduction processing on the third sample data to obtain target sample data; and generating a cross-sectional image of the web to be detected based on the target sample data.

[0009] However, the system still has the following technical shortcomings:

[0010] Low data synchronization accuracy: Images and coordinates are transmitted independently without a unified data packet structure, and timestamp alignment errors exceed 100ms, resulting in defect location deviations exceeding 10cm.

[0011] Rigid filtering model: The traditional filtering model has a fixed time step of 0.1s, which cannot adapt to the variable speed movement of the drone and the frame rate fluctuation of 10-30fps. The prediction error increases by 3-5 times in non-uniform speed scenarios.

[0012] Poor network adaptability: It cannot perceive network conditions such as TCP latency and packet loss rate. In weak network environments with latency >200ms, the coordinate prediction error deteriorates from 2cm to more than 8cm, causing detection failure. Summary of the Invention

[0013] The purpose of this invention is to provide a dual-mode terahertz detection data synchronization method and system to solve the problems of low data synchronization accuracy, rigid filtering model and weak network adaptability in the prior art;

[0014] The technical problem that this invention needs to solve is: how to provide a dual-mode terahertz detection data synchronization system.

[0015] The objective of this invention can be achieved through the following technical solutions:

[0016] A dual-modal terahertz detection data synchronization system, comprising a protocol layer, an algorithm layer, a sensing layer, a visualization layer, and a system layer;

[0017] Protocol layer: Dual-channel data packets are uniformly encapsulated, supporting dynamic session creation and automatic metadata completion; a single TCP port multiplexing architecture is adopted to achieve mixed transmission and automatic splitting of dual-type data streams on a single TCP long connection;

[0018] Algorithm layer: Design a dual-mode adaptive Kalman filter that automatically switches the time base according to network quality;

[0019] Dual-mode adaptive Kalman filter design: The state vector contains complete UAV kinematic parameters: 3D position, 3D attitude, 3D linear velocity, and 3D angular velocity; the state transition matrix A is dynamically updated according to the time step dt, with the velocity term integrated to the position and the angular velocity term integrated to the attitude.

[0020] Perception layer: Real-time TCP latency and FPS difference measurement, driving dynamic adjustment of filtering strategies; specifically including TCP latency measurement, FPS difference calculation and automatic switching decision engine;

[0021] Visualization layer: 12-dimensional parametric panoramic 3D rendering, providing transparent monitoring of network quality;

[0022] System layer: Lightweight memory management, supporting 24 / 7 industrial-grade continuous operation; the lightweight memory management architecture includes zero-copy frame data transmission, forced memory reclamation mechanism, and dual-buffer separation strategy.

[0023] Furthermore, at the protocol layer, all data packets follow a unified frame header format, and heterogeneous data is transmitted from the same source through type identifiers. Data packet types include:

[0024] 0x00000000: Terahertz frame data packet;

[0025] 0x00000001: UAV coordinate data packet;

[0026] 0x00000002-0xFFFFFFFF: Reserved extended types;

[0027] Payload Size: The length of the payload data in bytes, excluding the 8-byte frame header;

[0028] Payload Data: Different encoding schemes are used depending on the Packet Type.

[0029] Furthermore, at the protocol layer, the length prefix method is used to delimit message frames, which is completely driven by the receiver. The pusher does not need to maintain the state of the application layer's sending buffer. The receiver maintains a loop reading logic. First, it checks whether the buffer has at least an 8-byte frame header. After unpacking, it obtains the payload_size and then determines whether the buffer contains a complete data packet. If it is complete, it extracts the payload and removes the processed data. Otherwise, it waits for the next network event to trigger.

[0030] Furthermore, the push terminal is deployed on the UAV's onboard micro industrial control computer, responsible for sending terahertz frame data and coordinate data, including:

[0031] Terahertz detector interface module: The airborne micro industrial computer obtains the detector's original 64×64 float64 matrix via a USB interface;

[0032] Coordinate interface module: The airborne micro industrial computer acquires real-time flight control data of the UAV through the USB interface and generates a 12-dimensional coordinate vector;

[0033] Data packetization module: encapsulates frame data and coordinate data into byte streams with header and payload respectively;

[0034] TCP client module: Pushes data packets via TCP protocol, supports reconnection after disconnection and heartbeat detection.

[0035] Furthermore, the monitoring terminal is deployed on the ground station's industrial control computer, responsible for receiving, synchronizing, filtering, visualizing, and storing data, including:

[0036] TCP server module: Listens on port 50000 and parses both types of data packets;

[0037] Adaptive filtering module: 12-dimensional Kalman filter, supporting dual-mode switching;

[0038] Network quality monitoring module: measures latency and FPS differences, and automatically switches drivers;

[0039] 3D visualization module: OpenGL rendering of drone models and 12-dimensional parametric HUD;

[0040] Data processing module: Supports 14 image processing algorithms including differential and cumulative noise reduction, interpolation, pseudo-color, and edge detection;

[0041] Storage Management Module: Creates session folders to save PNG images and JSON format metadata.

[0042] Furthermore, the dynamic session creation process includes:

[0043] After receiving the start command from the industrial control computer software, set the timeout threshold to 10 seconds;

[0044] Upon receiving the first frame of the image, it is cached in first_frame_data and not processed immediately.

[0045] If valid coordinates are received within 10 seconds, call the session creation function to process the cached image;

[0046] If the coordinates are not received within 10 seconds, a session will be created using zero coordinates, and a warning log will be logged to prevent the system from being permanently blocked.

[0047] Furthermore, at the algorithmic level, the dual-mode time step selection process includes:

[0048] Theoretical timing mode: Δt=1 / push_fps, trusting the hardware clock of the push end, suitable for weak network environments, avoiding the impact of latency jitter on filtering stability;

[0049] Measurement timing mode: Δt = measured packet reception interval, trusting the receiver's local clock, suitable for ideal networks, can accurately reflect the actual motion dynamics changes of the UAV.

[0050] Furthermore, TCP latency measurement: The receiving end records the arrival time of each data packet, calculates the time difference with the previous packet, and converts it into millisecond-level latency;

[0051] FPS difference calculation: The receiver counts the actual number of data packets received per second to obtain measured_fps, and calculates the absolute difference between measured_fps and push_fps declared by the pusher.

[0052] Automatic switching of decision engines:

[0053] A forced switch to theoretical timing mode will be made when any of the following conditions are met:

[0054] a. TCP latency > 100ms;

[0055] b.|measured_fps-push_fps|>5fps;

[0056] When network quality recovers to latency <80ms and FPS difference <3fps, switching back to measurement timing mode is allowed.

[0057] Furthermore, zero-copy frame data transmission: the receiving end uses memory mapping technology to directly convert the TCP byte stream into a NumPy array, avoiding Python object serialization, and the single frame processing time is <1ms;

[0058] Forced memory reclamation mechanism: Performs three cleanup steps when data collection stops:

[0059] Clear all buffers;

[0060] Disconnect all object references;

[0061] Explicitly calling gc.collect() forces garbage collection;

[0062] Dual cache separation strategy:

[0063] FrameBuffer: Used for real-time acquisition, including image processing such as differential and cumulative processing;

[0064] Playback cache: Stores the processed 8-bit image, with a memory usage of only 1 / 8 of the original data.

[0065] A dual-modal terahertz detection data synchronization method includes the following steps:

[0066] Step S1: Data Acquisition and Packaging: The terahertz detector on the UAV acquires 64×64 floating-point matrix imaging data of the target area, while the UAV flight control system outputs a 12-dimensional motion state vector in real time.

[0067] Step S2: Data reception and protocol parsing: The ground station TCP server listens on the specified port, receives the data stream, parses the frame header, and identifies the data packet type;

[0068] Step S3: Network quality awareness and filtering mode decision: Real-time measurement of data packet arrival time interval, calculation of TCP latency and frame rate difference;

[0069] Step S4: Adaptive Kalman filtering: Construct a 12-dimensional state vector and dynamically update the time step dt according to the selected mode;

[0070] Step S5: Image Processing and Memory Management: Perform at least one of the following processing methods on the received terahertz image data: differential processing, noise reduction, and pseudo-color mapping;

[0071] Step S6: Panoramic visualization and data storage: Render the drone model in the 3D scene and simultaneously display 12-dimensional motion parameters and network quality HUD information;

[0072] Step S7: System resource maintenance: Explicitly release the memory buffer at the end of the data collection and force the garbage collection mechanism to ensure that the system's memory usage remains stable during long-term operation.

[0073] The present invention has the following beneficial effects:

[0074] 1. Message frame delimitation is achieved using a length prefix method at the protocol layer, and the parsing is fully driven by the receiving end. This eliminates the need for the pushing end to maintain the application layer's sending buffer state. This design effectively solves the problem of accurately identifying and separating variable-length data packets in a continuous TCP byte stream, avoiding data parsing errors or data loss due to ambiguous message boundaries. The receiving end maintains a loop reading logic. This not only significantly reduces the resource consumption and system complexity of the pushing end (such as an airborne micro-industrial control computer on a UAV), allowing it to focus more on data acquisition and transmission, but also improves the efficiency and reliability of data processing at the receiving end (such as a ground station industrial control computer). It ensures accurate synchronization of dual-modal terahertz detection data in complex network environments, thus providing stable and high-quality data input for subsequent adaptive Kalman filtering and real-time visualization.

[0075] 2. This application can intelligently adjust the time step calculation strategy of the Kalman filter according to the real-time changes in network quality. This enables the system to effectively avoid the adverse effects of network latency jitter on filter stability in weak network environments, ensuring the reliability of data synchronization. In ideal network environments, it can make full use of actual measurement data to accurately reflect the dynamic changes of UAV motion, thereby providing higher precision synchronization results. This adaptive time step selection mechanism significantly improves the robustness and accuracy of the dual-modal terahertz detection data synchronization system in complex and variable network environments, providing a solid technical guarantee for the real-time processing and visualization of UAV-borne terahertz detection data.

[0076] 3. By measuring TCP latency and FPS differences in real time, the system can accurately assess the health of the current network environment. Based on these quantitative indicators, the automatic switching decision engine can intelligently determine when to adjust the timing mode of the Kalman filter, thereby switching to a more stable theoretical timing mode in a timely manner when the network conditions are poor. This effectively avoids the decrease in filtering accuracy and unstable state estimation caused by network jitter or packet loss. At the same time, when the network recovers, the system can smoothly switch back to the measurement timing mode, making full use of the dynamic characteristics of the actual data stream, improving the accuracy of filtering and the ability to respond to the actual movement of the UAV. This dynamic adaptability significantly enhances the robustness of the dual-mode terahertz detection data synchronization system and the reliability of data synchronization in complex and ever-changing network environments.

[0077] 4. Zero-copy transmission and memory mapping technologies significantly improve data processing efficiency, reduce single-frame processing time, and ensure that the system can respond to data streams in real time. The forced memory reclamation mechanism fundamentally eliminates the risk of memory leaks, ensuring the long-term stability and reliability of the system under industrial-grade continuous operation conditions. The dual-buffer separation strategy optimizes the allocation and utilization of memory resources, enabling data from different processing stages to be stored in a way that best suits their purpose, further improving the overall system resource efficiency. These improvements together ensure that the dual-modal terahertz detection data synchronization system can operate in a highly efficient, stable, and reliable manner, meeting the stringent requirements of industrial applications for continuous operation and data processing performance. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0079] Figure 1 This is a system block diagram of Embodiment 1 of the present invention;

[0080] Figure 2 This is a schematic diagram of heterogeneous data transmission from the same source in Embodiment 1 of the present invention;

[0081] Figure 3 This is a flowchart of the method in Embodiment 2 of the present invention. Detailed Implementation

[0082] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0083] In terahertz inspection systems for wind turbine blades, multi-sensor data synchronization faces technical challenges. Hardware synchronization solutions rely on GPS PPS pulses or dedicated synchronization cables for clock alignment, but this increases system complexity and weight, failing to meet the lightweight requirements of drones. Software timestamping methods, where each data source timestamps independently and the receiver aligns based on these timestamps, suffer from insufficient synchronization accuracy due to system clock drift and network latency jitter. Fixed-frequency interpolation performs linear interpolation between coordinate data frames, which cannot adapt to non-linear motions such as acceleration, deceleration, and turning of the drone, leading to increased data synchronization errors. These factors directly affect the accuracy of 3D defect localization, reducing the reliability of inspection results.

[0084] For example, when inspecting blades in wind farms with complex terrain, drones need to perform non-uniform scanning trajectories along the blade surface, including frequent acceleration, deceleration, and turning maneuvers. Network signals are unstable due to terrain obstruction, and TCP transmission latency fluctuates significantly. Software timestamp alignment mechanisms suffer from time deviations due to clock drift, resulting in inaccurate matching between coordinate data and terahertz images. Fixed linear interpolation cannot accurately predict position changes during turns, causing predicted coordinate values ​​to deviate from the actual trajectory. Consequently, terahertz reflection data and spatial coordinates are misaligned, and defect locations are offset in 3D reconstruction, affecting the reliability of the inspection results.

[0085] If the aforementioned data synchronization issues are not resolved, defect location deviations will be uncontrollable and further amplified under conditions of UAV dynamic movement and network fluctuations. The rigid design of the filtering model makes the system difficult to adapt to changes in actual flight conditions, increasing the uncertainty of data processing. Insufficient network adaptability leads to system performance degradation in weak network environments, causing detection interruptions or data loss. These technical deficiencies will limit the application effectiveness of terahertz detection technology in the field inspection of wind turbine blades, reducing the robustness and practicality of the detection system.

[0086] Example 1: As Figure 1 As shown, the dual-modal terahertz detection data synchronization system is mainly used in scenarios where UAVs equipped with terahertz detectors perform mobile scanning. It aims to simultaneously acquire reflection imaging data and UAV spatial coordinate data, and achieve three-dimensional location of defects through post-processing. Dual-modal refers to two different types of data: terahertz frame data and UAV coordinate data. The system includes a protocol layer, an algorithm layer, a perception layer, a visualization layer, and a system layer.

[0087] For ease of understanding, the following explains some key terms in this embodiment:

[0088] The protocol layer is responsible for data encapsulation, transmission, and distribution. Its core function is to ensure that different types of data streams can be transmitted efficiently and reliably under a unified communication mechanism.

[0089] The algorithm layer focuses on data filtering and synchronization. By designing specific filtering algorithms, this layer can handle noise and uncertainty in the data and adjust the filtering strategy according to the actual operating environment.

[0090] The perception layer is responsible for monitoring the network status and data flow characteristics in real time. This layer provides a basis for the dynamic adjustment of the algorithm layer by measuring key indicators.

[0091] The visualization layer is used to present the processed data and system status to operators in an intuitive way, providing a comprehensive monitoring view through technologies such as 3D rendering.

[0092] The system layer is responsible for underlying resource management and system stability. This layer ensures that the system can run stably for a long time by optimizing memory usage and other methods.

[0093] Adaptive Kalman filtering is a Kalman filtering algorithm that can dynamically adjust the filter gain based on the statistical characteristics of system noise and measurement noise. Its advantage lies in its ability to better adapt to changes in the uncertainty of the system model and measurement model, thereby improving the filtering accuracy.

[0094] Single TCP port multiplexing architecture refers to the simultaneous transmission of multiple types of data streams over a single long TCP connection, with automatic stream splitting at the receiving end through protocol mechanisms. This architecture can reduce connection establishment overhead and improve communication efficiency.

[0095] Dynamic session creation refers to the system's ability to automatically identify and establish a corresponding data processing session when it receives a specific data stream, in order to ensure that the data can be processed and stored correctly.

[0096] Metadata autocomplete refers to the ability of a system to automatically generate or supplement missing descriptive data based on existing information during data transmission, thereby improving data integrity and usability.

[0097] Dual-mode adaptive Kalman filtering refers to a filter that can switch between two preset time base modes based on different operating conditions or network quality in order to optimize the filtering effect.

[0098] The state vector is a set of variables used in a Kalman filter to describe the current state of the system. In this system, it includes kinematic parameters such as the UAV's position, attitude, linear velocity, and angular velocity.

[0099] The state transition matrix A is a matrix in the Kalman filter used to describe how the system state evolves from one time step to the next. This matrix is ​​dynamically updated according to the time step to reflect the motion characteristics of the UAV.

[0100] TCP latency refers to the time required for a data packet to travel from the sender to the receiver in a TCP network. This metric reflects the real-time transmission performance of the network.

[0101] FPS difference refers to the difference between the actual received data frame rate and the frame rate declared by the sender. This metric can reflect the smoothness and integrity of data transmission.

[0102] Dynamic adjustment of filtering strategy refers to the system's ability to automatically adjust the parameters of the Kalman filter or switch the filtering mode based on real-time monitoring of network quality and data flow characteristics, in order to adapt to the ever-changing operating environment.

[0103] 12-dimensional parametric panoramic 3D rendering refers to the visualization of 12 kinematic parameters of a drone (position 3D, attitude 3D, linear velocity 3D, angular velocity 3D) using 3D graphics technology, providing a comprehensive perspective.

[0104] Transparent network quality monitoring refers to a system that can display network performance indicators such as TCP latency and FPS differences in an intuitive way, allowing users to clearly understand the current network status.

[0105] Lightweight memory management refers to reducing the memory footprint of the system during runtime, improving resource utilization, and supporting long-term stable operation by optimizing memory allocation, usage, and reclamation mechanisms.

[0106] Protocol layer: Dual-channel data packets are uniformly encapsulated, supporting dynamic session creation and automatic metadata completion; a single TCP port multiplexing architecture is adopted to achieve mixed transmission and automatic splitting of dual-type data streams on a single TCP long connection, which reduces firewall configuration complexity and system resource consumption compared to traditional multi-port solutions;

[0107] like Figure 2 As shown, all data packets follow a unified frame header format, and heterogeneous data are transmitted from the same source through type identification;

[0108] A unified frame header format defines a fixed-length structure containing specific fields at the beginning of a data packet. This format aims to provide a standardized starting point for all transmitted data packets, allowing the receiving end to consistently begin parsing any incoming packet. For example, the header might include a magic number for quick packet validation, a version number for protocol compatibility management, and a type identifier field. Alternatively, the header could contain a fixed-length field, with one part indicating the overall length of the packet and another part indicating the packet type.

[0109] The type identifier is a specific field in the uniform frame header format that explicitly indicates the data type carried by the current data packet. By assigning unique type identifiers to different types of heterogeneous data (such as terahertz frame data and UAV coordinate data), the receiving end can quickly identify the content category of the data packet without parsing the entire packet. For example, the type identifier can be an enumeration value or a bit field, where different bit combinations represent different data types.

[0110] Data packet types include:

[0111] 0x00000000: Terahertz frame data packet;

[0112] 0x00000001: UAV coordinate data packet;

[0113] 0x00000002-0xFFFFFFFF: Reserved extended types;

[0114] Payload Size: The length of the payload data in bytes, excluding the 8-byte frame header;

[0115] Payload Size is a field that indicates the length of the payload data in a data packet, typically in bytes. Located in the frame header, this field informs the receiver the precise length of the actual data content (payload data) of the current data packet. Using Payload Size, the receiver can accurately extract the payload data and determine packet integrity without relying on specific terminators or preset lengths. It's important to note that Payload Size only calculates the length of the payload data and does not include the length of the frame header itself; for example, this application does not include an 8-byte frame header.

[0116] Payload Data: Different encoding schemes are used depending on the Packet Type;

[0117] Payload data is the core component of a data packet, carrying the actual information. Its content and encoding scheme are determined by the packet type identifier. For example, if the packet type indicates a terahertz frame data packet, the payload data may use a specific image encoding format (such as raw binary data or a compressed format); if the packet type indicates a UAV coordinate data packet, the payload data may use a structured binary format to represent 12-dimensional coordinate information such as position and attitude. This mechanism of using different encoding schemes based on the type ensures the flexibility and efficiency of data transmission.

[0118] Among them, terahertz frame data packets (Packet Type=0).

[0119] Its load structure includes:

[0120] Data dimensions: 64 rows × 64 columns × 8 bytes = 32,768 bytes;

[0121] Data type: float64;

[0122] Numerical range: [0,1], representing the normalized terahertz reflection intensity.

[0123] Memory layout:

[0124] payload=[pixel(0,0),pixel(0,1),...,pixel(0,63),

[0125] pixel(1,0),pixel(1,1),...,pixel(1,63), ...

[0127] pixel(63,0),pixel(63,1),...,pixel(63,63)].

[0128] The parsing method includes: the receiving end uses zero-copy memory mapping technology to directly convert the byte stream into a NumPy array to avoid data copying overhead, and then converts it into an 8-bit grayscale image for display using OpenCV;

[0129] The generation rules include:

[0130] Guide frame: The first LEAD_IN_FRAMES frames (5 frames by default) after the start of acquisition are sent as pure noise background for system calibration;

[0131] Defect frame: Subsequent frames are defect feature data collected by the detector;

[0132] Signal-to-noise ratio (SNR): The ratio of defect signal strength to noise standard deviation is SNR = 20 dB;

[0133] Transmission frequency: controlled by FRAME_FPS (default 30Hz), using precise timestamps to control the Frame Timer.

[0134] Among them, the drone coordinate data packet (Packet Type=1).

[0135] Its load structure includes:

[0136] Data dimension: 13 elements × 8 bytes = 104 bytes;

[0137] Data type: float64;

[0138] Array semantics definition:

[0139] index field name describe 0 x meters (m) drone's northward position 1 y meters (m) drone's eastward position 2 z meters (m) Drone altitude 3 roll degree (°) Roll angle, rotation about the X-axis 4 pitch (degrees) Pitch angle, rotation around the Y-axis 5 yaw degree(°) Yaw angle, rotation about the Z-axis 6 vx meters per second (m / s) Northbound linear velocity 7 vy meters per second (m / s) Eastward linear velocity 8 vz meters per second (m / s) Vertical linear velocity 9 vroll (degrees per second) Roll rate 10 vpitch degrees / second (° / s) Pitch angular velocity 11 vyaw (degrees per second) Yaw angular velocity 12 push_fps Hertz (Hz) The frame rate baseline value declared by the push client ;

[0140] The generation rules include:

[0141] Independent update frequency: controlled by COORD_FPS (default 10Hz), decoupled from frame data;

[0142] Caching mechanism: When the coordinate update cycle has not been reached, the coordinates of the previous cycle are sent again, but the push_fps field remains constant;

[0143] Motion model: Real-time flight control data injection;

[0144] Timestamp synchronization: Instead of transmitting timestamps separately, the receiving end records the arrival time using time.time(), reducing protocol overhead.

[0145] The message frame delimitation is achieved using the length prefix method, and the parsing is entirely driven by the receiver. There is no need for the pusher to maintain the application layer sending buffer state: The receiver maintains a loop reading logic. First, it checks whether the buffer has at least an 8-byte frame header. After unpacking, it obtains the payload_size and then determines whether the buffer contains a complete data packet. If it is complete, the payload is extracted and the processed data is removed. Otherwise, it waits for the next network event to trigger.

[0146] This application's protocol layer employs a length prefix method for message frame delimitation, combined with a receiver-driven parsing mechanism, ensuring efficient and reliable separation and processing of dual-type data streams transmitted over a single TCP long connection. Specifically, when sending terahertz frame data and coordinate data, the push end (e.g., a UAV-borne micro industrial control computer) only needs to write the encapsulated data packet (including frame header and payload) as a continuous byte stream into the TCP send buffer, without needing to concern itself with the integrity of application layer messages or the receiver's state. This design significantly reduces the processing burden and memory consumption of the push end. The receiver end (e.g., a ground station industrial control computer) maintains a loop reading logic, continuously listening to and processing data in the TCP receive buffer. When data arrives, the receiver first attempts to read a fixed-length frame header (e.g., 8 bytes) from the buffer. Once the frame header is successfully read and unpacked, the receiver can obtain the "payload_size" of the current data packet. Based on this "payload_size," the receiver can accurately determine whether the buffer contains all the bytes required to constitute a complete data packet (frame header + payload). If the data packet is complete, the receiver immediately extracts the payload data for subsequent processing (e.g., automatic packet splitting based on packet type) and removes processed data from the buffer, thus maintaining buffer conciseness and efficiency. If the data packet is incomplete, the receiver pauses parsing the current packet, waiting for more network data to arrive until the next network event triggers. This mechanism allows the receiver to independently and robustly handle byte boundary issues in TCP streams. Even if network transmission causes packets to be segmented or merged, the receiver can correctly reconstruct the original data packet using length prefix information. By centralizing the complexity of message delimitation and parsing at the receiver and employing a lightweight push-end design, this solution effectively solves the problem of reliable packet splitting and efficient processing of mixed data streams in real-time data synchronization systems. It ensures accurate and timely synchronization of terahertz frame data and UAV coordinate data, providing a high-quality data source for subsequent adaptive Kalman filtering and visualization.

[0147] The exception handling strategies include:

[0148] Frame header corruption: If packet_type is not in the range [0,1], it is considered a protocol error. Clear the buffer and record the error log.

[0149] Excessively long packets: If payload_size > 32768*2, it is considered an attack or an anomaly, and the connection is closed;

[0150] Incomplete packets: Retained in the buffer, waiting for subsequent data to arrive. If the incomplete packet is not completed within 5 seconds, it will be discarded.

[0151] The push notification is deployed on a drone-borne micro industrial computer and includes:

[0152] Terahertz detector interface module: The airborne micro industrial control computer obtains the detector's original 64×64 float64 matrix through the USB interface; its function is to ensure that the airborne micro industrial control computer can correctly identify and initialize the terahertz detector, and obtain the original terahertz frame data in an efficient and lossless manner.

[0153] Coordinate Interface Module: The airborne micro industrial computer acquires real-time flight control data of the UAV through the USB interface and generates a 12-dimensional coordinate vector; its function is to ensure that the airborne micro industrial computer can accurately and timely acquire key parameters such as the position, attitude, and speed of the UAV.

[0154] Data packet module: Encapsulates frame data and coordinate data into byte streams with header and payload respectively; its function is to encapsulate raw data (terahertz frame data and coordinate data) from different sources and in different formats into byte streams that conform to the transmission protocol.

[0155] TCP client module: Pushes data packets via TCP protocol, supports reconnection after disconnection and heartbeat detection; its function is to utilize the reliability of TCP protocol to ensure that the encapsulated data packets can be transmitted accurately and orderly to the receiving end.

[0156] As the data source, the deployment location of the push terminal determines the real-time performance and environmental adaptability of data acquisition. Deploying it on a UAV-borne micro industrial computer aims to leverage the computer's miniaturization, high integration, low power consumption, and tolerance to harsh environments (such as vibration and temperature changes) to ensure the shortest physical distance between the data acquisition device and the UAV flight control system, thereby reducing data transmission latency and guaranteeing the stability and reliability of data acquisition during UAV flight. Possible implementations include using an industrial-grade single-board computer designed specifically for embedded applications, or a customized embedded system built on a high-performance ARM processor. The core function of the push terminal is to act as the convergence point and external interface for dual-modal data, responsible for integrating terahertz frame data and UAV coordinate data acquired from different sensors and preparing them for transmission. These two data types have different characteristics and update frequencies, but both require real-time transmission to support subsequent synchronization and filtering. Possible implementations include running a main control program on the industrial computer, which processes the acquisition and transmission tasks of both types of data in parallel through multi-threading or asynchronous task management mechanisms, or using an event-driven model to trigger the transmission process when data is ready.

[0157] The monitoring terminal is deployed on the industrial control computer at the ground station and includes:

[0158] TCP server module: Listens on port 50000, parses dual-type data packets; responsible for establishing and maintaining TCP connections with the push client, and receiving data streams from the push client;

[0159] Adaptive filtering module: 12-dimensional Kalman filter, supporting dual-mode switching; responsible for dynamically adjusting filtering parameters based on received data and system status to optimize filtering effect;

[0160] Network quality monitoring module: measures latency and FPS differences, drives automatic switching; responsible for real-time evaluation of network connection performance. This module can calculate latency by periodically sending probe packets and measuring round-trip time (RTT), or by analyzing the timestamps of received data packets.

[0161] 3D Visualization Module: This module renders the drone model using OpenGL and displays a 12-dimensional parametric HUD. It is responsible for displaying the drone's motion state and terahertz detection data in 3D graphics. This module can utilize graphics APIs such as OpenGL, Vulkan, or DirectX for low-level rendering, or be developed using a high-level 3D rendering engine. Employing OpenGL rendering technology, it uses the OpenGL graphics library to draw 3D graphics, creating realistic 3D scenes. It can render drone models, displaying a virtual model representing the drone in a 3D scene, whose position and attitude are updated in real-time based on received coordinate data. Simultaneously, this module can also display a 12-dimensional parametric HUD (Head-Up Display), which overlays the 3D scene onto the screen, displaying the drone's 12-dimensional kinematic parameters in real-time, such as position, attitude, linear velocity, and angular velocity. The HUD can be presented in various formats, including text, dashboards, and charts.

[0162] Data Processing Module: This module supports image processing algorithms for preprocessing, image transformation and correction, image enhancement, and feature extraction. Specific image processing algorithms include the following 14 types: difference algorithm, cumulative noise reduction, median filtering, Gaussian filtering, bilateral filtering, geometric transformation, frequency domain transformation, histogram equalization, pseudo-color processing, edge detection, image interpolation, threshold segmentation, morphological processing, and image sharpening. It is responsible for performing various image processing operations on the received terahertz frame data to improve image quality and extract useful information.

[0163] Storage Management Module: Creates session folders to save PNG images and JSON format metadata (including complete 12-dimensional coordinates and push_fps, raw 64×64 terahertz detector data); responsible for organizing and saving received raw data, processed images, and related metadata.

[0164] The listening end is the core component of the entire data synchronization system on the receiving side. Deployed on the ground station's industrial control computer, it is a dedicated computing platform with high stability, reliability, and processing power, capable of continuous operation for extended periods and processing large amounts of real-time data. The core responsibilities of the listening end are receiving, synchronizing, filtering, visualizing, and storing the dual-modal data transmitted from the UAV pusher. Receiving data refers to acquiring the terahertz frame data and coordinate data pushed by the UAV through a network communication interface, which can be achieved using the Socket API provided by the operating system or existing network communication libraries. Synchronizing data aims to precisely align the received terahertz frame data and coordinate data in time, ensuring accurate correlation. This is typically achieved through mechanisms such as timestamp comparison or packet sequence numbers. Filtering data involves processing the received data to eliminate noise and improve data accuracy, for example, using the Kalman filter algorithm. Visualizing data presents the processed data to the user in an intuitive graphical or image format, which can be achieved through graphics rendering libraries or user interface frameworks. Storing data involves saving the received and processed data to persistent storage media for subsequent querying, analysis, or playback, which can be achieved through file systems or databases.

[0165] The dynamic session creation process includes:

[0166] After receiving the START command from the industrial control computer software, set the timeout threshold to 10 seconds;

[0167] Upon receiving the first frame of the image, it is cached in first_frame_data and not processed immediately.

[0168] Scenario A: Valid coordinates are received within 10 seconds, the session creation function is called, and the cached image is processed;

[0169] Scenario B: If the coordinates are not received after 10 seconds, force the creation of a session using zero coordinates and log a warning to avoid permanent system blockage.

[0170] The "dynamic session creation process" refers to the mechanism by which the system automatically establishes and initializes a data processing session based on specific conditions and logic after receiving a data stream. This process aims to ensure the correctness and continuity of data processing, especially when the data source is started or the connection is unstable. Its implementation can include event-triggered creation, data integrity check-based creation, or creation based on preset rules. The "industrial control computer software start command" in "after receiving the start command from the industrial control computer software, set a timeout threshold of 10 seconds" refers to the instruction issued by the software on the ground station's industrial control computer to initiate the data reception and processing flow. Upon receiving this command, the system starts a timer and sets a "timeout threshold of 10 seconds." This threshold limits the maximum time for waiting for critical data (such as UAV coordinate data) to prevent the system from waiting indefinitely and becoming blocked. This timeout mechanism can be implemented through a timer service provided by the operating system, a separate timing thread, or timestamp comparison in the event loop. The "first frame image" in "when the first frame image is received, cache it in first_frame_data and do not process it immediately" typically refers to the first valid image data in the terahertz frame data packet. When the system receives this data, it temporarily stores it in a memory area or buffer named "first_frame_data". The image is not processed immediately to wait for other necessary data (such as drone coordinate data) to arrive, ensuring the session can be initialized with a complete and meaningful dataset. Caching can be implemented through memory allocation, data structure storage, or temporary file writing. The "valid coordinates" in "If valid coordinates are received within 10 seconds, call the session creation function to process the cached image" refers to drone coordinate data that conforms to a preset format and validation rules. If the system successfully receives this valid coordinate data within the set 10-second timeout threshold, it will trigger the "session creation function". This function is responsible for initializing all necessary components and states of the data processing session and then processing the previously cached "first_frame_data", incorporating it into the newly created session. The session creation function can be a standalone subroutine, a class method, or a service interface. The phrase "If no coordinates are received within 10 seconds, force the creation of a session using zero coordinates and log a warning to avoid permanent system blocking" means that if the system still fails to receive valid drone coordinate data after the 10-second timeout threshold, it will take mandatory measures to ensure continuous system operation and avoid blocking. Specifically, it will use a preset "zero coordinate" (e.g., a coordinate vector with all dimensions equal to zero) to replace the missing actual coordinate data and call the session creation function using this "zero coordinate." Simultaneously, the system will "log a warning" to record this anomaly for subsequent investigation. This mechanism ensures that the system can start and run even with incomplete data, thus "avoiding permanent system blocking."The creation of the zero coordinate can be a hard-coded default value, and the recording of warning logs can be implemented through a standard logging library or a custom logging module.

[0171] Algorithm layer: Dual-mode adaptive Kalman filter, automatically switching the time base according to network quality;

[0172] Dual-mode adaptive Kalman filter design: The state vector contains complete UAV kinematic parameters: 3D position, 3D attitude, 3D linear velocity, and 3D angular velocity; the state transition matrix A is dynamically updated according to the time step dt, with the velocity term integrated to the position and the angular velocity term integrated to the attitude.

[0173] Dual-mode time step selection:

[0174] Theoretical timing mode: Δt=1 / push_fps, trusting the hardware clock of the push end, suitable for weak network environments, avoiding the impact of latency jitter on filtering stability;

[0175] Measurement timing mode: Δt = measured packet reception interval, trusting the receiver's local clock, suitable for ideal networks, can accurately reflect the actual motion dynamics changes of the UAV;

[0176] When a coordinate duplication is detected (the difference between the coordinates and the previous frame is <1e-6), the observation noise covariance R is automatically amplified by 100 times to reduce the pollution of the filter by the cached coordinates, making the prediction results more dependent on model calculations rather than unreliable measurements.

[0177] Specifically, the dual-mode time step selection process refers to a mechanism for dynamically determining the state transition time interval (Δt) of a Kalman filter. This process aims to intelligently select the most suitable Δt calculation method based on the current network conditions to ensure the stability and accuracy of the filter. Its implementation may include a decision module that triggers mode switching based on network quality indicators (such as latency, packet loss rate, frame rate differences, etc.). Theoretical timing mode is a time step calculation method where Δt is set to the reciprocal of the frame rate (push_fps) declared by the pusher, i.e., 1 / push_fps. The core of this mode is trusting the data generation rate determined by the internal hardware clock of the pusher (e.g., a micro-industrial control computer on a drone), rather than the actual network transmission time. The advantage of this approach is that even with significant network latency and jitter, the filter's prediction step size remains consistent, effectively avoiding interference from network instability on the filtering results, making it particularly suitable for environments with poor network conditions. Another implementation method is to pre-configure a fixed Δt value corresponding to the nominal frame rate of the pusher. The measurement timing model is another method for calculating the time step, where Δt is determined based on the actual time interval between data packets received by the receiver. This model trusts the receiver's local clock and assumes that, under ideal network conditions, the actual arrival interval of data packets accurately reflects the generation interval of data at the push end and the actual motion dynamics of the UAV. This model provides higher accuracy because it directly utilizes actual observation data. Another implementation is to use a moving average of the reception intervals of the most recent N data packets to smooth out minor measurement errors. Trusting the push end's hardware clock means that, in the theoretical timing model, the system relies on frame rate information (push_fps) provided or preset by the data sender (such as the UAV) to determine the time step. This dependence ensures that the filter's predictions are consistent with the generation rhythm of the data source, thus maintaining the inherent stability of the filtering process under unstable network conditions. It is suitable for weak network environments and avoids the impact of latency jitter on filtering stability, highlighting the crucial role of the theoretical timing model in poor network conditions. In situations with high network latency and large jitter, if the Kalman filter's state predictions rely entirely on the actual packet arrival time, the irregular changes in the time step will cause the predictions to become unstable or even diverge. By employing a fixed theoretical time step, uncertainties in network transmission can be effectively isolated, ensuring the smooth progress of the filter's prediction process. Trusting the receiver's local clock means that in measurement timing mode, the system uses the arrival timestamps of data packets recorded by the receiver to calculate time intervals. This approach leverages the accuracy of the receiver's local clock, ensuring that the calculated time step is synchronized with the actual received data stream. Suitable for ideal networks, it accurately reflects the actual dynamic changes of the UAV, demonstrating the significant advantages of measurement timing mode under favorable network conditions.In environments with stable networks and low latency, the actual arrival interval of data packets can highly accurately reflect the true rhythm of UAV data generation and movement. In this case, employing a measurement timing mode allows the Kalman filter to more precisely track the UAV's real-time position, attitude, velocity, and other kinematic parameters, thereby providing more refined synchronization and positioning effects.

[0178] Perception layer: Real-time measurement of TCP latency and FPS difference, driving dynamic adjustment of filtering strategies; the specific process includes:

[0179] TCP latency measurement: The receiving end records the arrival time of each data packet, calculates the time difference with the previous packet, and converts it into millisecond-level latency;

[0180] FPS difference calculation: The receiver counts the actual number of data packets received per second to obtain measured_fps, and calculates the absolute difference between measured_fps and push_fps declared by the pusher.

[0181] Automatic switching of decision engines:

[0182] A forced switch to theoretical timing mode will be made when any of the following conditions are met:

[0183] a. TCP latency > 100ms;

[0184] b.|measured_fps-push_fps|>5fps;

[0185] When network quality recovers to latency <80ms and FPS difference <3fps, switching back to measurement timing mode is allowed.

[0186] The automatic switching decision engine forces a switch to theoretical timing mode when either of the following conditions is met: a) TCP latency greater than 100ms; b) |measured_fps - push_fps| greater than 5fps. Switching back to measurement timing mode is allowed when network quality recovers to latency less than 80ms and FPS difference less than 3fps. Specifically, this engine is a logical decision unit that continuously receives and analyzes TCP latency measurements and FPS difference calculations. When any preset threshold condition is triggered, such as TCP latency exceeding 100ms, or the absolute difference between the actual received frame rate and the expected frame rate exceeding 5 frames per second, the decision engine immediately issues a command, forcing the dual-mode adaptive Kalman filter in the algorithm layer to switch to theoretical timing mode. This forced switching ensures that the filter can rely on a more stable theoretical time step for state estimation when network conditions are poor. Simultaneously, to avoid frequent switching near network quality critical points, the engine also sets a recovery threshold. The decision engine only allows the filter to switch back to measurement timing mode when TCP latency drops below 80ms and FPS difference narrows to below 3 frames per second. This decision-making mechanism with hysteresis effectively prevents unstable switching of filters when network quality fluctuates.

[0187] By introducing precise TCP latency measurement and FPS difference calculation mechanisms at the perception layer, combined with an intelligent automatic switching decision engine, real-time, quantitative assessment of network quality is achieved. TCP latency measurement provides a direct indicator of network response speed by recording packet arrival times and calculating time differences. FPS difference calculation reveals the integrity and continuity of the data stream by comparing the actual received frame rate with the expected frame rate. These key network quality indicators are input to the automatic switching decision engine in real time. Based on preset dynamic thresholds, the engine intelligently determines whether the current network conditions are sufficient to support Kalman filtering based on actual measurement timing. When network quality deteriorates, such as excessively high TCP latency or excessively large FPS differences, the decision engine decisively forces the dual-mode adaptive Kalman filter in the algorithm layer to switch to theoretical timing mode, thereby avoiding state estimation errors caused by network jitter or packet loss. Conversely, when network quality recovers and reaches a stable standard, the decision engine allows the filter to switch back to measurement timing mode to fully utilize the dynamic characteristics of the actual data stream. This closely coordinated mechanism enables the system to dynamically adjust its filtering strategy according to the constantly changing network environment, ensuring that the dual-mode terahertz detection data synchronization system can maintain high-precision data synchronization and state estimation under various network conditions.

[0188] Visualization Layer: 12-dimensional parametric panoramic 3D rendering, providing transparent monitoring of network quality: Employing a third-person perspective, the camera is fixed 8 meters behind the drone and 3 meters above it, always pointing at the drone model. The background is a 20×20 meter moving grid ground, dynamically shifting according to the drone's horizontal position to create a visual effect of relative flight motion.

[0189] HUD Information Overlay: Top Panel: Black semi-transparent background, displaying 12-dimensional parameters in four groups: position (m), attitude (°), linear velocity (m / s), and angular velocity (° / s).

[0190] System layer: Lightweight memory management, supporting 24 / 7 industrial-grade continuous operation; the lightweight memory management architecture includes zero-copy frame data transmission, forced memory reclamation mechanism, and dual-buffer separation strategy;

[0191] Zero-copy frame data transmission: The receiving end uses memory mapping technology to directly convert the TCP byte stream into a NumPy array, avoiding Python object serialization, and the single frame processing time is <1ms;

[0192] Forced memory reclamation mechanism: Performs three cleanup steps when data collection stops:

[0193] Clear all buffers;

[0194] Disconnect all object references;

[0195] Explicitly calling gc.collect() forces garbage collection;

[0196] Dual cache separation strategy:

[0197] FrameBuffer: Used for real-time acquisition, including image processing such as differential and cumulative processing;

[0198] Playback cache: Stores the processed 8-bit image, with a memory usage of only 1 / 8 of the original data.

[0199] Zero-copy frame data transmission refers to minimizing data copying between different memory regions during data processing, thereby reducing CPU overhead and memory bandwidth usage. Specifically, this can be achieved in several ways. For example, the receiving end can utilize memory mapping technology provided by the operating system to directly map the TCP byte stream to the process's virtual address space, or use Direct Memory Access (DMA) technology to allow the Network Interface Controller (NIC) to directly write data to a pre-allocated buffer without CPU intervention.

[0200] The receiving end uses memory mapping technology, which essentially maps the physical memory region of a file or device (such as a network socket) directly into the application's virtual address space. This allows the application to directly access this memory, just like accessing a regular array, without needing to copy data through system calls. Besides functions like "mmap" provided by the operating system, memory mapping can also be implemented through specific hardware drivers or network protocol stack optimizations. For example, some high-performance network cards support directly mapping received data into user space.

[0201] The following example will provide a more detailed explanation of the above technical solution:

[0202] Imagine a wind turbine blade inspection scenario where a drone equipped with a terahertz detector scans the blades from the air. The drone needs to transmit terahertz frame data and its own 12-dimensional kinematic coordinates to a ground station in real time for processing and analysis. In traditional solutions, terahertz frame data and coordinate data may be transmitted independently through different channels or aligned using simple software timestamps. This can easily lead to insufficient data synchronization accuracy; for example, there may be a time deviation of tens or even hundreds of milliseconds between the terahertz image and the corresponding drone position, resulting in significant errors in the 3D localization of defects. Furthermore, the drone may experience nonlinear motions such as acceleration, deceleration, and turning during flight, and the network environment may be unstable, causing data transmission delays and frame rate fluctuations. These factors further reduce the accuracy of data processing.

[0203] The system in this embodiment solves the above problems in the following way:

[0204] First, at the UAV end, terahertz frame data and UAV coordinate data are uniformly encapsulated by the protocol layer. For example, the two data streams can be packaged into data packets with a unified format and sent to the ground station via a single long TCP connection. This single TCP port multiplexing architecture ensures that the terahertz frame data and coordinate data remain correlated throughout transmission, avoiding timestamp alignment errors caused by independent transmission in traditional solutions. When the ground station's listening end receives these data packets, the protocol layer can automatically separate the terahertz frame data and coordinate data based on the identifiers within the data packets and initiate a dynamic session creation process to prepare for subsequent data processing. For example, when the system receives terahertz frame data for the first time, it will attempt to establish a processing session with subsequently received coordinate data to ensure that each frame of image can be associated with the corresponding spatial coordinates.

[0205] Secondly, a dual-mode adaptive Kalman filter was designed at the ground station's algorithm layer. This filter can construct a state vector based on the UAV's 12-dimensional kinematic parameters (position, attitude, linear velocity, angular velocity) and dynamically update the state transition matrix according to the time step. For example, when the UAV performs non-uniform motion such as acceleration or turning, the Kalman filter can adjust the filtering parameters through its adaptive characteristics to more accurately predict the UAV's real-time state. The perception layer measures TCP latency and FPS differences in real time. For example, when the network latency exceeds a certain threshold or there is a large deviation between the actual frame rate and the expected frame rate, the perception layer drives the algorithm layer to switch the Kalman filter's time reference mode. For example, when the network conditions are good, a measurement timing mode based on the actual packet reception interval can be used to accurately reflect the UAV's motion; while when the network conditions deteriorate, it switches to a theoretical timing mode based on the push terminal's hardware clock to enhance the stability of the filter and avoid the impact of network jitter on prediction accuracy. This dynamic adjustment mechanism solves the problems of rigid filtering models and inability to adapt to network fluctuations and nonlinear motion of UAVs in traditional schemes.

[0206] Finally, the system layer ensures continuous 24 / 7 operation through lightweight memory management. For example, by optimizing data storage and access in memory, unnecessary memory copying is reduced, and memory is periodically reclaimed, thereby lowering memory usage and resource consumption during system operation. The visualization layer renders the processed 12-dimensional parameters and network quality indicators in panoramic 3D. For instance, operators can intuitively see the drone's real-time position, attitude, and overlaid terahertz images on a 3D interface, while also clearly monitoring current TCP latency and FPS differences, thus achieving transparent monitoring of the entire detection process. This comprehensive visualization and stable system operation capability provide reliable assurance for industrial applications.

[0207] Through the above collaborative work, the system in this embodiment can effectively solve problems such as low synchronization accuracy of multi-sensor data, rigid filtering model and weak network adaptability, and ensure that terahertz detection data can be synchronized, filtered and visualized with high precision in complex UAV detection environment, thereby achieving accurate three-dimensional positioning of internal defects of wind turbine blades.

[0208] Based on the above examples, the technical concept of this embodiment demonstrates a significant technical contribution. In traditional UAV terahertz inspection systems, data synchronization accuracy is limited by the independent transmission of images and coordinates and simple software timestamp alignment, resulting in large defect location errors. For example, in the aforementioned wind turbine blade inspection scenario, traditional solutions may cause defect location deviations of more than 10cm due to data synchronization errors. In contrast, this embodiment implements unified encapsulation of dual-channel data packets and a single TCP port multiplexing architecture at the protocol layer, ensuring close correlation and synchronization between terahertz frame data and UAV coordinate data at the transmission layer, thereby improving the accuracy of data synchronization from the source.

[0209] Furthermore, existing filtering models typically employ fixed linear interpolation or simple Kalman filtering with fixed time steps, making it difficult to adapt to the variable speed motion and frame rate fluctuations inherent in actual UAV flight. This leads to a significant increase in prediction errors in non-uniform speed scenarios. For example, when a UAV accelerates rapidly or turns, the prediction error of traditional filters may increase by 3-5 times. This embodiment designs a dual-mode adaptive Kalman filter at the algorithm layer. Its state vector contains complete 12-dimensional kinematic parameters, and the state transition matrix A is dynamically updated according to the time step. Further, the perception layer measures TCP latency and FPS differences in real time and drives dynamic adjustments to the filtering strategy, for example, automatically switching between theoretical timing mode and measured timing mode based on network quality. This adaptive and dynamically adjusted filtering mechanism enables the system to better adapt to the complex motion of the UAV and changes in the network environment, significantly improving the prediction accuracy of kinematic parameters and the stability of the filtering.

[0210] Example 2: Figure 3 As shown, the dual-mode terahertz detection data synchronization method includes the following steps:

[0211] Step S1: Data Acquisition and Packaging: The terahertz detector on the UAV acquires 64×64 floating-point matrix imaging data of the target area, while the UAV flight control system outputs a 12-dimensional motion state vector in real time.

[0212] Step S2: Data reception and protocol parsing: The ground station TCP server listens on the specified port, receives the data stream, parses the frame header, and identifies the data packet type;

[0213] Step S3: Network quality awareness and filtering mode decision: Real-time measurement of data packet arrival time interval, calculation of TCP latency and frame rate difference;

[0214] Step S4: Adaptive Kalman filtering: Construct a 12-dimensional state vector and dynamically update the time step dt according to the selected mode;

[0215] Step S5: Image Processing and Memory Management: Perform at least one of the following processing methods on the received terahertz image data: differential processing, noise reduction, and pseudo-color mapping;

[0216] Step S6: Panoramic visualization and data storage: Render the drone model in the 3D scene and simultaneously display 12-dimensional motion parameters and network quality HUD information;

[0217] Step S7: System resource maintenance: Explicitly release the memory buffer at the end of the data collection and force the garbage collection mechanism to ensure that the system's memory usage remains stable during long-term operation.

[0218] A dual-modal terahertz detection data synchronization method and system are disclosed. During operation, the terahertz detector on a UAV acquires 64×64 floating-point matrix imaging data of the target area, while the UAV flight control system outputs a 12-dimensional motion state vector in real time. The ground station TCP server listens to a designated port, receives the data stream, parses the frame header, and identifies the data packet type. The arrival time interval of data packets is measured in real time, and the TCP latency and frame rate difference are calculated. A 12-dimensional state vector is constructed, and the time step dt is dynamically updated according to the selected mode. At least one of the following processing methods is performed on the received terahertz image data: differential processing, noise reduction, and pseudo-color mapping. The UAV model is rendered in a 3D scene, and the 12-dimensional motion parameters and network quality HUD information are displayed synchronously. At the end of the acquisition, the memory buffer is explicitly released, and the garbage collection mechanism is forcibly triggered to ensure stable memory usage during long-term operation.

[0219] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

[0220] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0221] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A dual-mode terahertz detection data synchronization system, characterized in that, It includes the protocol layer, algorithm layer, perception layer, visualization layer, and system layer; Protocol layer: Dual-channel data packets are uniformly encapsulated, supporting dynamic session creation and automatic metadata completion; a single TCP port multiplexing architecture is adopted to achieve mixed transmission and automatic splitting of dual-type data streams on a single TCP long connection; the dynamic session creation process refers to the mechanism by which the system automatically establishes and initializes a data processing session after receiving a data stream, based on specific conditions and logic; Algorithm layer: Design a dual-mode adaptive Kalman filter that automatically switches the time base according to network quality; Dual-mode adaptive Kalman filter design: The state vector contains complete UAV kinematic parameters: 3D position, 3D attitude, 3D linear velocity, and 3D angular velocity; the state transition matrix A is dynamically updated according to the time step Δt, with the velocity term integrated to the position and the angular velocity term integrated to the attitude. Perception layer: Real-time measurement of TCP latency and FPS difference, driving dynamic adjustment of filtering strategy; Specifically, this includes TCP latency measurement, FPS difference calculation, and an automatic switching decision engine; Visualization layer: 12-dimensional parametric panoramic 3D rendering, providing transparent monitoring of network quality; System layer: Lightweight memory management, supporting 24 / 7 industrial-grade continuous operation; The lightweight memory management architecture includes zero-copy frame data transmission, a forced memory reclamation mechanism, and a dual-buffer separation strategy; The dual-mode time step selection process includes: Theoretical timing mode: Δt=1 / push_fps, trusting the hardware clock of the push end, suitable for weak network environments, avoiding the impact of latency jitter on filtering stability; Measurement timing mode: Δt = measured packet reception interval, trusting the receiver's local clock, suitable for ideal networks, can accurately reflect the actual motion dynamics changes of the UAV; TCP latency measurement: The receiving end records the arrival time of each data packet, calculates the time difference with the previous packet, and converts it into millisecond-level latency; FPS difference calculation: The receiver counts the actual number of data packets received per second to obtain measured_fps, and calculates the absolute difference between measured_fps and push_fps declared by the pusher. Automatic switching of decision engines: A forced switch to theoretical timing mode will be made when any of the following conditions are met: a. TCP latency > 100ms; b.|measured_fps-push_fps|>5fps; When network quality recovers to latency <80ms and FPS difference <3fps, switching back to measurement timing mode is allowed.

2. The dual-mode terahertz detection data synchronization system according to claim 1, characterized in that, At the protocol layer, all data packets follow a uniform frame header format, and heterogeneous data is transmitted from the same source through type identifiers. Data packet types include: 0x00000000: Terahertz frame data packet; 0x00000001: UAV coordinate data packet; 0x00000002-0xFFFFFFFF: Reserved extended types; Payload Size: The length of the payload data in bytes, excluding the 8-byte frame header; Payload Data: Different encoding schemes are used depending on the Packet Type.

3. The dual-mode terahertz detection data synchronization system according to claim 2, characterized in that, At the protocol layer, the length prefix method is used to delimit message frames. The parsing is entirely driven by the receiver, and the pusher does not need to maintain the state of the application layer's sending buffer. The receiver maintains a loop reading logic. First, it checks whether the buffer has at least an 8-byte frame header. After unpacking, it obtains the payload_size and then determines whether the buffer contains a complete data packet. If it is complete, it extracts the payload and removes the processed data. Otherwise, it waits for the next network event to trigger.

4. The dual-mode terahertz detection data synchronization system according to claim 3, characterized in that, The push terminal is deployed on the drone's onboard micro industrial control computer and is responsible for sending terahertz frame data and coordinate data, including: Terahertz detector interface module: The airborne micro industrial computer obtains the detector's original 64×64 float64 matrix via a USB interface; Coordinate interface module: The airborne micro industrial computer acquires real-time flight control data of the UAV through the USB interface and generates a 12-dimensional coordinate vector; Data packetization module: encapsulates frame data and coordinate data into byte streams with header and payload respectively; TCP client module: Pushes data packets via TCP protocol, supports reconnection after disconnection and heartbeat detection.

5. The dual-mode terahertz detection data synchronization system according to claim 4, characterized in that, The monitoring terminal is deployed on the industrial control computer at the ground station and is responsible for receiving, synchronizing, filtering, visualizing, and storing data, including: TCP server module: Listens on port 50000 and parses both types of data packets; Adaptive filtering module: 12-dimensional Kalman filter, supporting dual-mode switching; Network quality monitoring module: measures latency and FPS differences, and automatically switches drivers; 3D visualization module: OpenGL rendering of drone models and 12-dimensional parametric HUD; Data processing module: Supports image processing algorithms for image preprocessing, image transformation and correction, image enhancement, and feature extraction; Storage Management Module: Creates session folders to save PNG images and JSON format metadata.

6. The dual-mode terahertz detection data synchronization system according to claim 5, characterized in that, The dynamic session creation process includes: After receiving the start command from the ground station industrial control computer software, set the timeout threshold to 10 seconds; Upon receiving the first frame of the image, it is cached in first_frame_data and not processed immediately. If valid coordinates are received within 10 seconds, call the session creation function to process the cached image; If the coordinates are not received within 10 seconds, a session will be created using zero coordinates, and a warning log will be logged to prevent the system from being permanently blocked.

7. The dual-mode terahertz detection data synchronization system according to claim 6, characterized in that, Zero-copy frame data transmission: The receiving end uses memory mapping technology to directly convert the TCP byte stream into a NumPy array, avoiding Python object serialization, and the single frame processing time is <1ms; Forced memory reclamation mechanism: Performs three cleanup steps when data collection stops: Clear all buffers; Disconnect all object references; Explicitly calling gc.collect() forces garbage collection; Dual cache separation strategy: FrameBuffer: Used for real-time acquisition, for differential and cumulative image processing; Playback cache: Stores the processed 8-bit image, with a memory usage of only 1 / 8 of the original data.

8. A dual-mode terahertz detection data synchronization method, applied in the dual-mode terahertz detection data synchronization system as described in any one of claims 1-7, characterized in that, Includes the following steps: Step S1: Data Acquisition and Packaging: The terahertz detector on the UAV acquires 64×64 floating-point matrix imaging data of the target area, while the UAV flight control system outputs a 12-dimensional motion state vector in real time. Step S2: Data reception and protocol parsing: The ground station TCP server listens on the specified port, receives the data stream, parses the frame header, and identifies the data packet type; Step S3: Network quality awareness and filtering mode decision: Real-time measurement of data packet arrival time interval, calculation of TCP latency and frame rate difference; Step S4: Adaptive Kalman filtering: Construct a 12-dimensional state vector and dynamically update the time step Δt according to the selected mode; Step S5: Image Processing and Memory Management: Perform at least one of the following processing methods on the received terahertz image data: differential processing, noise reduction, and pseudo-color mapping; Step S6: Panoramic visualization and data storage: Render the drone model in the 3D scene and simultaneously display 12-dimensional motion parameters and network quality HUD information; Step S7: System resource maintenance: Explicitly release the memory buffer at the end of the data collection and force the garbage collection mechanism to ensure that the system's memory usage remains stable during long-term operation.

Images