Embedded electric trigger high-precision timing video recording system and method

Abstract

The application discloses an embedded electric trigger high-precision timing video recording system and method, and relates to the technical field of spaceflight test monitoring.The system comprises a network camera module, an embedded main control module, a storage module and an upper computer module.The embedded main control module receives RTP data packets and TTL trigger electric signals, acquires a trigger time, encapsulates the RTP data packets, sampling time carried by the RTP data packets and the acquired system receiving time into a self-defined format file and stores the file into the storage module;the upper computer module acquires the self-defined format file, obtains actual sampling time of each frame of image through time calibration and calculates a time difference, and generates a video recording file containing timing information by adding the time difference to each frame of image.The application can effectively eliminate delay interference caused by coding, transmission and network fluctuation, and significantly improve timing precision.

Description

Technical Field

[0001] This invention relates to the field of aerospace test monitoring technology, and in particular to an embedded electrically triggered high-precision timing and recording system and method. Background Technology

[0002] Key aspects of aerospace testing require real-time video monitoring and precise recording of the time difference between the actual acquisition time of each frame and the start time of the test. This time difference is the core foundation for multi-source data linkage analysis and directly determines the efficiency of fault tracing and the accuracy of timing optimization.

[0003] Currently, commercial video recorders are commonly used for image acquisition and recording in aerospace testing and monitoring. These devices employ a frame-level timing mechanism, defaulting to the moment when the image frame decoding is completed as the actual acquisition moment of the image frame.

[0004] However, the existing timing mechanisms mentioned above do not take into account the delays caused by image encoding, transmission, decoding, and network fluctuations, resulting in timing errors as high as 50~300ms, which cannot meet the timing accuracy requirements of multi-source data linkage analysis. Summary of the Invention

[0005] The technical problem to be solved by this invention is to address the shortcomings of existing technologies, specifically the large timing delay and wide fluctuation range of existing aerospace test recording devices, which makes it difficult to meet the requirements of accurate timing for multi-source data linkage analysis and time-series fault tracing. Specifically, this invention provides an embedded electrically triggered high-precision timing and recording system and method, as detailed below: 1) In a first aspect, the present invention provides an embedded electrically triggered high-precision timing and recording system, the specific technical solution of which is as follows: a network camera module, an embedded main control module, a storage module, and a host computer module; The network camera module is used to collect video data and generate and output RTP data packets; The embedded main control module includes a GPIO interface, a network interface, and a storage interface; it establishes an RTSP connection with the network camera module through the network interface to receive the RTP data packets, receives the TTL trigger electrical signal from an external start signal generator through the GPIO interface and obtains the trigger time, and connects to the storage module through the storage interface to write data. The embedded main control module is also used to parse and extract the sampling time carried in the RTP data packet, encapsulate the sampling time, the real-time acquired system reception time and the RTP data packet into a custom format file, and write the custom format file into the storage module; The storage module is connected to the embedded main control module through the storage interface and is used to store the custom format file; The host computer module is connected to the embedded main control module through a communication network. It is used to obtain the custom format file stored in the storage module, perform time calibration on the custom format file to obtain the actual acquisition time of each frame image in the video data, determine the time difference of each frame image based on the actual acquisition time and the trigger time, and superimpose the time difference onto each frame image to generate a video file containing timing information.

[0006] The beneficial effects of the embedded electrically triggered high-precision timing and recording system provided by this invention are as follows: The embedded main control module captures TTL trigger signals and obtains the trigger time via the GPIO interface. It receives RTP data packets from a network camera via the network interface, parses and extracts the sampling time and system reception time, and then encapsulates them into a custom format file, storing it in the storage module. The host computer module obtains this custom format file, extracts the sampling time of key parameter packets, and forms an effective time pair with the system reception time. A linear relationship model is established, and the transformation parameters are solved using transmission delay compensation to determine the mapping relationship between the sampling time and the actual acquisition time, thereby determining the actual acquisition time of each frame. Based on the actual acquisition time and trigger time, the time difference of each frame is calculated and superimposed onto each frame to generate a video file containing timing information. This system effectively eliminates delay interference caused by image encoding, transmission, and network fluctuations, significantly improving timing accuracy, surpassing existing commercial equipment. It provides precise timing support for multi-source data linkage analysis and time-series fault tracing. Furthermore, the embedded architecture design results in a small device size, controllable cost, and flexible deployment, adapting to complex field environments such as aerospace experiments, balancing accuracy requirements with economic efficiency.

[0007] Based on the above solution, the present invention can be further improved as follows.

[0008] Furthermore, the step of performing time calibration on the custom format file to obtain the actual acquisition time of each frame in the video data specifically includes: Extract the sampling time of the key parameter packet sent before I-frame encoding and the system reception time from the custom format file to form multiple valid time pairs; A linear relationship model is established for the multiple effective time pairs, and the conversion parameters are obtained by combining transmission delay compensation; based on the conversion parameters, the mapping relationship between the sampling time and the actual acquisition time is determined. Based on the mapping relationship, the actual acquisition time corresponding to the sampling time is determined.

[0009] Furthermore, the establishment of a linear relationship model for the multiple effective time pairs, combined with transmission delay compensation, to obtain the conversion parameters specifically includes: Linear fitting is performed on the multiple valid time pairs, and abnormal data with residuals exceeding a preset threshold are removed to obtain the clock conversion coefficients and conversion bias. The conversion parameters are obtained based on the clock conversion coefficients and the conversion bias, combined with transmission delay compensation.

[0010] Furthermore, the key parameter package is an SPS package, a PPS package, or an SEI package.

[0011] Furthermore, determining the time difference of each frame image based on the actual acquisition time and the trigger time specifically includes: Select the most recent I-frame before the triggering time as the anchor point, and convert the sampling time of the anchor point into the corresponding actual acquisition time according to the mapping relationship; Based on the incremental step size of the sampling time and the actual acquisition time corresponding to the anchor point, the actual acquisition time of each frame image is determined, and then the time difference of each frame image relative to the trigger time is calculated.

[0012] Furthermore, the step of superimposing the time difference onto each frame image to generate a video file containing timing information specifically includes: Encoded data is extracted from the custom format file, the bitstream is reassembled according to a preset encoding standard, the time difference is superimposed on a specified position of each frame image in a recognizable form, compressed into a standard format, and a video file containing timing information is generated.

[0013] Furthermore, the embedded main control module also includes a circular buffer for continuously storing the custom format file; When the embedded main control module receives the TTL trigger signal, it writes the custom format file already stored in the ring buffer into the storage module.

[0014] 2) In a second aspect, the present invention also provides an embedded electrically triggered high-precision timing and recording method, the specific technical solution of which is as follows: S1 configures and starts the monitoring thread of the GPIO interface through the embedded main control module; establishes an RTSP connection with the network camera module; initializes the ring buffer and starts the RTP receiving thread to continuously receive RTP data packets; S2, the receiving thread receives RTP data packets in real time, parses and extracts the sampling time carried in the RTP data packets, encapsulates the sampling time, the real-time acquired system receiving time and the RTP data packets into a custom format file, and writes the custom format file into the circular buffer; S3, when the monitoring thread receives the TTL trigger signal, it triggers a hardware interrupt and obtains the current time as the trigger time; it writes the custom format file stored in the circular buffer into the storage module; S4, obtain the custom format file stored in the storage module through the host computer module, perform time calibration on the custom format file to obtain the actual acquisition time of each frame image in the video data, and determine the time difference of each frame image based on the actual acquisition time and the trigger time; S5, the time difference is superimposed onto each frame image through the host computer module to generate a video file containing timing information.

[0015] Based on the above solution, the present invention can be further improved as follows.

[0016] Furthermore, the step of performing time calibration on the custom format file to obtain the actual acquisition time of each frame in the video data specifically includes: Extract the sampling time of the key parameter packet sent before I-frame encoding and the system reception time from the custom format file to form multiple valid time pairs; A linear relationship model is established for the multiple effective time pairs, and the conversion parameters are obtained by combining transmission delay compensation; based on the conversion parameters, the mapping relationship between the sampling time and the actual acquisition time is determined. Based on the mapping relationship, the actual acquisition time corresponding to the sampling time is determined.

[0017] Furthermore, the establishment of a linear relationship model for the multiple effective time pairs, combined with transmission delay compensation, to obtain the conversion parameters specifically includes: Linear fitting is performed on the multiple valid time pairs, and abnormal data with residuals exceeding a preset threshold are removed to obtain the clock conversion coefficients and conversion bias. The conversion parameters are obtained based on the clock conversion coefficients and the conversion bias, combined with transmission delay compensation.

[0018] Furthermore, the key parameter package is an SPS package, a PPS package, or an SEI package.

[0019] Furthermore, determining the time difference of each frame image based on the actual acquisition time and the trigger time specifically includes: Select the most recent I-frame before the triggering time as the anchor point, and convert the sampling time of the anchor point into the corresponding actual acquisition time according to the mapping relationship; Based on the incremental step size of the sampling time and the actual acquisition time corresponding to the anchor point, the actual acquisition time of each frame image is determined, and then the time difference of each frame image relative to the trigger time is calculated.

[0020] Furthermore, the step of superimposing the time difference onto each frame image to generate a video file containing timing information specifically includes: Encoded data is extracted from the custom format file, the bitstream is reassembled according to a preset encoding standard, the time difference is superimposed on a specified position of each frame image in a recognizable form, compressed into a standard format, and a video file containing timing information is generated.

[0021] 3) In a third aspect, the present invention also provides a computer device, the computer device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the computer device to implement any of the above methods.

[0022] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to enable a computer to implement any of the above methods.

[0023] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0024] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the structure of an embedded electrically triggered high-precision timing and recording system according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the steps of an embedded electrically triggered high-precision timing and recording method according to an embodiment of the present invention. Figure 3 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0026] The following is a unified explanation of the English abbreviations used throughout the entire implementation method: The Peripheral Component Interconnect Express (PCIe) standard is a high-speed serial interface bus standard used to connect external devices (such as solid-state drives, graphics cards, etc.) to the computer motherboard, enabling high-bandwidth data transmission.

[0027] Real-time Transport Protocol (RTP) is a real-time audio and video transmission protocol used to encapsulate data and carry timing information such as timestamps and sequence numbers.

[0028] General-purpose input / output (GPIO) is a general-purpose programmable pin interface that can be configured by software to input or output modes for digital signal interaction with external circuits.

[0029] Real-Time Streaming Protocol (RTSP) is an application-layer protocol used to establish and control media sessions, allowing for negotiation of encoding parameters, transmission methods, and communication ports during session establishment.

[0030] Transistor-Transistor Logic (TTL) is a standard for transmitting trigger electrical signals, with a high level of 3.3V or 5V and a low level of 0V.

[0031] First In First Out (FIFO) is a data caching management mechanism in which data written to the cache first is read or removed first.

[0032] User Datagram Protocol (UDP) is a connectionless, low-latency transport layer protocol used to carry RTP packets.

[0033] An intra-coded frame (I-frame) is a video frame in the H.264 encoding standard that uses intra-frame coding. It contains complete image information and can be used as a starting point for decoding. Its encoding does not depend on other video frames.

[0034] The Network Abstraction Layer Unit (NALU) is the basic transmission unit of an H.264 bitstream, containing coded slices or control information.

[0035] Non-Volatile Memory Express Solid State Drive (NVMe SSD) is a high-speed storage medium based on the Non-Volatile Memory Host Controller Interface Specification protocol, used to ensure data read and write speeds.

[0036] The Sequence Parameter Set (SPS) is the core control parameter set for H.264 encoding. It is sent before I-frame encoding and is the core reference node for timing calibration.

[0037] The Picture Parameter Set (PPS) is a key control parameter unit in the H.264 coding standard. It is sent before I-frame encoding and contains image coding parameter information, such as entropy coding mode and initial values ​​of quantization parameters.

[0038] Supplemental Enhancement Information (SEI) is an auxiliary information unit in the H.264 coding standard. It is sent before or after I-frame encoding and contains supplementary enhancement information generated during the encoding process, such as user-defined data and image timing information.

[0039] File Transfer Protocol (FTP) is a standard application layer protocol used to transfer files between clients and servers in a computer network. It supports operations such as uploading, downloading, deleting, and renaming files.

[0040] Example 1, such as Figure 1 As shown, an embedded electrically triggered high-precision timing and recording method according to an embodiment of the present invention is applied to an embedded electrically triggered high-precision timing and recording system, the system including a network camera module (corresponding to...). Figure 1 "1-Network Camera" in the text), embedded main control module (corresponding to Figure 1 "2-Embedded motherboard" and storage module (corresponding to Figure 1 The “5-high-speed solid-state drive” and the host computer module (corresponding to) Figure 1 (3-Host computer in the text).

[0041] The network camera module is used to capture video data and generate and output RTP data packets. An RTP data packet is a data unit encapsulated according to RTP, containing header information and payload data. The header information carries timing information such as sampling time and sequence number, while the payload data carries H.264 encoded video data. H.264 encoding is a video compression standard used to significantly compress video data while maintaining video quality, facilitating video transmission and storage.

[0042] The embedded main control module includes GPIO interfaces, a network interface, and a storage interface. It establishes an RTSP connection with the network camera module via the network interface to receive RTP data packets. The embedded main control module receives the TTL trigger signal from an external start signal generator and obtains the trigger time via the GPIO interface. The external start signal generator, located outside the embedded main control module and part of the measurement and control system, outputs a TTL trigger signal as the start command for the recording system. The embedded main control module also parses and extracts the sampling time carried in the RTP data packets, encapsulates the sampling time, the real-time acquired system reception time, and the RTP data packets into a custom format file, and writes this custom format file to the storage module via the storage interface.

[0043] The storage module is connected to the embedded main control module through a storage interface and is used to store a custom format file composed of sampling time, real-time acquired system reception time, and RTP data packet encapsulation.

[0044] The host computer module connects to the embedded main control module through a communication network to obtain the custom format file stored in the storage module. It performs time calibration on the custom format file to obtain the actual acquisition time of each frame in the video data. Based on the actual acquisition time and the trigger time, it determines the time difference of each frame and superimposes the time difference onto each frame to generate a video file containing timing information.

[0045] In a preferred embodiment, the embedded main control module further includes a circular buffer. The circular buffer is a reusable memory area used to store custom format files. The receiving thread continuously and in real-time receives RTP data packets, parses the RTP packets to obtain the sampling time carried within them, encapsulates the sampling time, system reception time, and RTP data packets into a custom format file, and continuously writes it to the circular buffer. The circular buffer employs a FIFO mechanism; if the buffer is full when new data is written, the oldest read data is automatically overwritten. When the embedded main control module receives a TTL trigger signal, it records the read / write pointer position of the circular buffer and writes the custom format file already stored in the circular buffer to a temporary data file in the storage module. Throughout the process, RTP data packets are continuously parsed and the encapsulated custom format file is written to the circular buffer. The read / write pointer position is used to mark the boundaries of valid data in the circular buffer at the trigger time, ensuring that the storage thread can accurately read the existing data in the buffer. By pre-storing video data several seconds before the trigger in the circular buffer, it is ensured that key frames before the trigger are not lost, achieving a complete transition of recording data before and after the trigger. Compared to writing custom format files to the storage module via the storage interface, this implementation only writes the custom format files to the storage module after receiving the TTL trigger signal, which reduces the resource consumption of the storage module. By storing the parsed custom format files in a circular buffer, the read / write throughput can be reduced.

[0046] The aforementioned recording system works in collaboration with the host computer module through an embedded main control module to achieve a complete process from trigger signal capture, video data reception and storage, time calibration to time difference calculation and recording generation.

[0047] The specific hardware selection and connection methods for each module are as follows: The embedded main control module uses a Raspberry Pi 5 development board with 8GB of memory and runs on Ubuntu 20.04 64-bit operating system. The network camera module uses a Hikvision DS-2CD3T25-I3 camera, supporting H.264 encoding and RTSP protocol. The storage module uses a 512GB NVMe SSD with read / write speeds of at least 500MB / s. Auxiliary accessories include a 2-meter shielded GPIO connection cable (for interference-resistant transmission), a gigabit network cable, and a 5V DC power adapter.

[0048] The connections between the modules are as follows: the external start signal generator is connected to the GPIO17 pin of the Raspberry Pi 5 development board via a shielded GPIO connection cable through the GPIO interface to transmit TTL trigger electrical signals; the network camera module, Raspberry Pi 5 development board, and host computer module are all connected to a gigabit switch via gigabit Ethernet cables to form a dedicated local area network to ensure transmission bandwidth; the NVMe SSD is inserted into the PCIe interface (i.e., the storage interface of the embedded main control module) of the Raspberry Pi 5 development board via a PCIe adapter card to achieve high-speed data read and write.

[0049] The hardware parameters for each module are configured as follows: GPIO interface configuration: Enable GPIO17 pin via the gpiod library, configure it as rising edge triggered mode, and set a 10ms debouncing delay; verify the interface status using the "gpiodetect" command-line tool to ensure normal interrupt response. The gpiod library is a development library and toolset for accessing and controlling GPIO at the application layer on Linux systems. Network camera configuration: Log in to the camera's web management interface and configure the parameters: set the encoding format to H.264, the I-frame interval to 25 frames, the resolution to 1920×1080 (i.e., 1080P), the frame rate to 25 FPS, the bitrate to 4 Mbps (constant bitrate), the transmission protocol to RTSP / RTP / UDP, and the RTSP connection address to rtsp: / / 192.168.1.100:554 / stream1. Storage configuration: In the Raspberry Pi 5 system, use the "fdisk" disk management tool to partition and format the solid-state drive as an ext4 file system, mount it to the specified directory ( / mnt / ssd), and set read and write permissions using the "chmod 777 / mnt / ssd" permission setting command to ensure that the software can write data normally.

[0050] The development environment is set up as follows: Embedded side: A Raspberry Pi 5 development board is used as the embedded main control module, with an Ubuntu 20.04 64-bit operating system installed and a C++17 compilation environment configured; core dependency libraries are installed, including gpiod (for GPIO control), libcurl (for network communication), and openssl (for data encryption). Host computer side: A regular PC is used as the host computer module, with a Python 3.8 runtime environment installed; core dependency libraries are installed, including NumPy (for residual fitting calculation), FFmpeg (for audio and video processing), and paramiko (for FTP file transfer).

[0051] The core logic of the embedded main control module is implemented using the C++17 programming language, while the host computer module uses the Python3.8 programming language combined with the FFmpeg library to implement video processing.

[0052] In a preferred embodiment, time calibration of the custom format file is performed to obtain the actual acquisition time of each frame in the video data. Specifically, this includes: extracting the sampling time and system reception time of the key parameter packet sent before I-frame encoding from the custom format file to form multiple valid time pairs; establishing a linear relationship model for the multiple valid time pairs, and solving for the conversion parameters by combining transmission delay compensation; determining the mapping relationship between the sampling time and the actual acquisition time based on the conversion parameters; and determining the actual acquisition time corresponding to the sampling time based on the mapping relationship.

[0053] Specifically, the critical parameter packet sent before I-frame encoding refers to the parameter data packet carrying decoding control information sent by the network camera before encoding and outputting the I-frame. Its sampling time t is consistent with the current I-frame, and the system reception time T is closest to the actual acquisition time Tt of each frame image. The critical parameter packet belongs to NALU in the RTP packet payload, and different parameter packet types can be distinguished by identifying the NALU header type value.

[0054] A linear relationship model is established for multiple valid time pairs (t, T), and the transformation parameters are obtained by combining the transmission delay compensation dT. The linear relationship model describes the linear correspondence between the sampling time t and the actual acquisition time Tt, and the transformation parameters are the coefficients that quantize this linear relationship. Substituting the transformation parameters back into the linear relationship model yields the mapping relationship between the sampling time and the actual acquisition time.

[0055] Based on the above mapping relationship, the actual acquisition time Tt corresponding to the sampling time t is determined. That is, for any sampling time t, substituting it into the mapping relationship for calculation will yield the corresponding actual acquisition time Tt. The actual acquisition time Tt is the time when the network camera actually acquires each frame of image, and it is the basic data for subsequent time difference calculation.

[0056] In a preferred embodiment, a linear relationship model is established for multiple valid time pairs, and the conversion parameters are obtained by combining transmission delay compensation. Specifically, this includes: performing linear fitting on multiple valid time pairs, removing abnormal data with residuals exceeding a preset threshold, and obtaining clock conversion coefficients and conversion bias; and obtaining conversion parameters based on clock conversion coefficients and conversion bias, combined with transmission delay compensation.

[0057] Specifically, a linear relationship model T = k×t + B is established with sampling time t as the independent variable and system reception time T as the dependent variable. The least squares method is used to perform linear residual fitting on multiple valid time pairs (t, T). After removing outlier data with residuals exceeding a preset threshold (10ms), the clock conversion coefficient k and conversion bias B are obtained. It is important to note that the conversion bias B includes the actual conversion bias b and transmission delay compensation dT, i.e., B = b + dT.

[0058] The average estimated value of the transmission delay compensation dT is obtained through LAN UDP self-transmission and self-reception tests. The transmission delay compensation dT is then subtracted from the delay offset B, i.e., b = B - dT, to obtain the true conversion offset b. Based on the clock conversion coefficient k and the true conversion offset b, the mapping relationship between the sampling time t and the actual acquisition time Tt is determined: Tt = k×t + b. The conversion parameters include the clock conversion coefficient k and the true conversion offset b.

[0059] In a preferred embodiment, the key parameter packet is an SPS packet, a PPS packet, or an SEI packet. SPS is the H.264 encoding core control parameter set, sent before I-frame encoding, containing all parameter information of the encoded sequence, with a NALU type value of 0x07. PPS contains image encoding parameter information, with a NALU type value of 0x08. SEI contains supplementary enhancement information, with a NALU type value of 0x06. All three key parameter packets can be sent before I-frame encoding, and any one can be selected as a reference node for timing calibration. Only the packet type identification logic needs to be adjusted to achieve timing calibration with the same precision. In this embodiment, the SPS packet is preferred as the key parameter packet. The packet type identification logic is as follows: Parse the NALU header in the RTP data packet payload and read the lower 5 bits of the first byte of the header as the type value. When the type value equals 0x07, the current parameter packet is determined to be an SPS packet; when the type value equals 0x08, the current parameter packet is determined to be a PPS packet; when the type value equals 0x06, the current parameter packet is determined to be an SEI packet.

[0060] All three parameter packets can be sent before intra-frame encoding. Any one of them can be used as a reference node for timing calibration to achieve timing calibration with the same accuracy. Simply adjust the type value in the packet type identification logic from 0x07 to 0x08 or 0x06 to switch to the corresponding parameter packet type. The rest of the calibration process remains unchanged.

[0061] In a preferred embodiment, the time difference of each frame image is determined based on the actual acquisition time and the trigger time. Specifically, this includes: selecting the most recent I-frame before the trigger time as the anchor point, converting the sampling time of the anchor point into the corresponding actual acquisition time according to the mapping relationship; determining the actual acquisition time of each frame image based on the sampling time increment step and the actual acquisition time corresponding to the anchor point, and then obtaining the time difference of each frame image relative to the trigger time.

[0062] Specifically, the nearest I-frame before the trigger time T0 is selected as the anchor point, and the sampling time t of the anchor point is determined according to the mapping relationship Tt = k×t + b. I Convert to the corresponding actual acquisition time Tt IAn anchor point refers to the I-frame that serves as the reference point for timing calculation. The most recent I-frame before the trigger time T0 is selected as the anchor point, and the actual acquisition time Tt of that frame is used as the reference point. I As a baseline reference value for timing calculation, the actual acquisition time Tt of each of the remaining frames is used. i All calculations are based on this anchor point.

[0063] The sampling time increment step d refers to the time interval t between two adjacent image frames. It is determined by both the encoding type and the frame rate. Taking a frame rate of 25 FPS as an example, the sampling time increment step d is 90000 ÷ 25 = 3600. The actual acquisition time Tt of each image frame... i According to sampling time t i Relative to anchor sampling time t I The difference is divided by the sampling time increment step d, and then divided by the actual acquisition time Tt of the anchor point. I Adding them together, we get Tt. i = (t i - t I ) / d + Tt I This allows us to determine the actual acquisition time Tt of each frame of the image. i Subtracting the trigger time T0, we obtain the time difference ΔT = Tt between each frame and the trigger time. i - T0.

[0064] In a preferred embodiment, the time difference is superimposed on each frame image to generate a video file containing timing information. Specifically, this includes: extracting encoded data from a custom format file, recombining the bitstream according to a preset encoding standard, superimposing the time difference in a recognizable form onto a specified position of each frame image, compressing it into a standard format, and generating a video file containing timing information.

[0065] Specifically, the encoded data extracted from RTP packets, namely H.264 NALU data, is the original H.264 encoded bitstream segment carried in the payload of the RTP packets. The preset encoding standard is a pre-defined video encoding standard; in this scheme, it is H.264. The NALU bitstream is reassembled according to the H.264 protocol. "Recognizable format" refers to the way the time difference is superimposed onto the video frame in a way that is directly perceptible to the human eye, including text, graphics, or time watermarks. "Specified position" refers to the upper left corner of the video frame. In this scheme, text superimposed onto the upper left corner of the video frame is preferred. "Standard format" refers to common video file container formats, including MP4, AVI, and MKV formats. After compression encoding, a video file that can be directly played by mainstream players is generated. This scheme preferably uses MP4 format for container output.

[0066] By overlaying the time difference as text onto the video frames, users can directly view the precise time difference information of each frame relative to the trigger time through mainstream media players, which facilitates subsequent multi-source data linkage analysis and timing fault tracing.

[0067] Example 2, as Figure 2 As shown, the present invention also provides an embedded electrically triggered high-precision timing and recording method, which is based on the above-mentioned embedded electrically triggered high-precision timing and recording system and specifically includes the following steps.

[0068] S1 configures and starts the monitoring thread of the GPIO interface through the embedded main control module; establishes an RTSP connection with the network camera module; initializes the ring buffer and starts the RTP receiving thread to continuously receive RTP data packets. The specific implementation method is as follows.

[0069] Configure the GPIO interface: Initialize the GPIO interface pins using the gpiod library, register the interrupt callback function trigger_callback(), which is used to capture the trigger time T0 and start the storage thread.

[0070] Start the GPIO interface: Start the monitoring thread of the GPIO interface, set the monitoring thread priority to 10, and ensure the priority of triggering electrical signals.

[0071] Establish an RTSP connection: Implement an RTSP client using the C++ programming language, connect to the RTSP address of the network camera module rtsp: / / 192.168.1.100:554 / stream1, set the connection timeout to 5 seconds, and negotiate and determine the encoding parameters, transmission protocol, and communication port after successful connection. In this embodiment, the preferred transmission protocol is UDP, and the default UDP receive port is 5540.

[0072] Initialize the circular buffer: Create 16 circular buffers, each 4MB in size, for a total capacity of 64MB, capable of pre-storing 8 seconds of 4Mbps video data. Start the RTP receive thread to continuously receive RTP packets and write them to the circular buffers using a FIFO mechanism.

[0073] A pre-caching mechanism ensures that video data is not lost before triggering, while a high-priority monitoring thread guarantees the real-time response to the triggering electrical signal.

[0074] S2 receives RTP data packets in real time via a receiving thread, parses and extracts the sampling time carried in the RTP data packets, encapsulates the sampling time, the real-time acquired system reception time, and the RTP data packets into a custom format file, and writes the custom format file into a circular buffer. The specific implementation method is as follows.

[0075] The RTP receive thread runs continuously, receiving RTP packets in real time, parsing them according to the RTP protocol, and extracting the 12-byte header information. The first two bytes contain the version number (version number V equals 2) and padding bits (padding bit P is 0 or 1); bytes 3 and 4 contain the payload type (payload type PT value 96 corresponds to H.264 encoding) and a flag bit (flag bit M is 1 to indicate the end of the frame); bytes 5 to 8 contain the sampling time (in the form of a timestamp t); bytes 9 to 12 contain the sequence number; the following is the original H.264 data.

[0076] Extract the sampling time, call the microsecond-level system time function time.time() to obtain the system reception time T (microsecond-level precision), encapsulate it according to the sampling time t (8 bytes), system reception time T (8 bytes), data length (4 bytes) and the custom format of the original RTP data to obtain a custom format file, and write the custom format file into the circular buffer.

[0077] The sampling time t and the system receiving time T are paired and saved in a custom format file to provide basic data for the host computer time calibration. The custom encapsulation format retains the original RTP data to ensure data integrity.

[0078] S3: When the monitoring thread receives a TTL trigger signal, it triggers a hardware interrupt and obtains the current time as the trigger moment; it then writes the custom format file stored in the circular buffer to the storage module. The specific implementation method is as follows.

[0079] The monitoring thread for the GPIO interface is blocked and waiting.

[0080] When the GPIO interface pin detects the rising edge of the TTL trigger signal (3.3V high level, lasting ≥10ms), the interrupt callback function trigger_callback() is triggered; the nanosecond-level system time function time.time_ns() is called to obtain the UTC timestamp as the trigger time T0 (nanosecond-level precision).

[0081] Synchronously record the read and write pointer positions of the circular buffer, start the storage thread, and write the existing custom format file in the circular buffer into the temporary data file of the storage module. The file path is / mnt / ssd / temp_data.bin.

[0082] By recording the read / write pointer position, the boundary of valid data in the circular buffer at the trigger moment is accurately marked, ensuring that the key images a few seconds before the trigger are seamlessly connected with the data after the trigger and written to the storage module.

[0083] S4: Obtain the custom format file stored in the storage module through the host computer module, perform time calibration on the custom format file to obtain the actual acquisition time of each frame in the video data, and determine the time difference of each frame based on the actual acquisition time and the trigger time. The specific implementation method is as follows.

[0084] The host computer module uses FTP to retrieve the custom format file temp_data.bin from the storage module. This custom format file contains the sampling time of the RTP data packets, the system reception time, and the original RTP data packets. Time calibration is used to determine the mapping relationship between the sampling time and the actual acquisition time, and based on this mapping relationship, the actual acquisition time of each frame is determined. The time calibration process is as follows: After parsing the custom format file, extract the key parameter packets transmitted before I-frame encoding. In this embodiment, the SPS packet is preferred as the key parameter packet, thus identifying the parameter packet with a NALU header type value of 0x07. Record valid time pairs (sampling time t, system reception time T), for example, record 10 sets of valid time pairs: (t1= 520000, T1= 1781278325.312873); (t2= 610000, T2= 1781278326.313022); (t3= 700000, T3= 1781278327.312987); (t4= 790000, T4= 1781278328.313081); (t5= 880000, T5= 1781278329.312801); (t6= 970000, T6= 1781278330.312858); (t7= 1060000, T7= 1781278331.312831); (t8= 1150000, T8= 1781278332.312903); (t9= 1240000, T9= 1781278333.312912); (t) 10 = 1330000, T 10 = 1781278334.312866).

[0085] The least squares linear fitting method (with sampling time t as the independent variable and system receiving time T as the dependent variable) was implemented using the polyfit function of the NumPy library, resulting in the fitting equation T = 1.1111087 × 10⁻⁶. -5×t + 1781277746.666667 (Theoretical slope 0.04 / 3600≈1.11×10) -5 The residuals for each group were all less than 10ms, and there was no abnormal data.

[0086] A UDP self-transmission and self-reception test was performed using the Python socket library. The average estimated value of the transmission delay compensation dT was measured to be 1.3ms. After incorporating the transmission delay compensation dT, the mapping relationship between the sampling time t and the actual acquisition time Tt was determined: Tt = 1.1111087 × 10 -5 ×t + 1781277746.665367.

[0087] The most recent I-frame before the trigger time is selected as the anchor point to calculate the actual acquisition time of each frame image, and then the time difference of each frame image is obtained.

[0088] By effectively eliminating delay interference caused by encoding, transmission and network fluctuations through linear fitting and delay compensation mechanisms, timing accuracy is significantly improved, providing accurate timing information for multi-source data linkage analysis.

[0089] S5, through the host computer module, superimposes the time difference onto each frame image to generate a video file containing timing information. The specific implementation method is as follows.

[0090] The host computer module extracts H.264 encoded data from the RTP packet payload of a custom format file, reassembles the NALU bitstream according to the H.264 protocol, calls the FFmpeg 5.1 video processing library to read the raw H.264 data, and uses the filter function to overlay the time difference ΔT in red Arial 24-point font onto the upper left corner of the video frame. Compression is performed using the libx264 encoder, with a bitrate of 4Mbps, a frame rate of 25FPS, and a resolution of 1920×1080, packaged in MP4 format. The generated MP4 file can be played by mainstream players such as PotPlayer. By finding the time difference corresponding to the first frame after the start signal is emitted (an LED is connected in parallel to the start circuit; the light indicates the signal has been emitted), after 100 repeated tests, the timing error is less than ±40ms.

[0091] The calibrated time difference information is overlaid onto the video screen in a visual form, generating a video file containing timing information that can be directly played and analyzed, facilitating subsequent fault tracing and timing optimization analysis.

[0092] The embedded terminal is responsible for real-time data reception and storage, ensuring the real-time nature of the trigger response and data integrity; the host computer is responsible for data parsing, time calibration, and video recording generation, making full use of the host computer's computing power to complete complex calibration algorithms. This architecture ensures both the real-time nature of the trigger response and the computational accuracy of the calibration algorithm, achieving high-precision timing and recording.

[0093] Compared with existing technologies, this invention has the following advantages: By fitting and calibrating the residual between the sampling time of the SPS packet of an I-frame and the system reception time, combined with a network latency compensation mechanism, interference from image accumulation, encoding, and network fluctuations is effectively eliminated. The timing error can be controlled within ±40ms, which is significantly better than the fluctuation range of 50~300ms of existing commercial equipment, providing accurate timing support for multi-source data linkage analysis and time series fault tracing. At the same time, it adopts an embedded architecture design, which makes the device small in size, cost-controllable, and flexible in deployment, and can be adapted to complex field environments such as different types of rocket tests, balancing accuracy requirements and economy.

[0094] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation, and these situations are also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0095] Furthermore, the acquisition process of the data involved in this application follows the principles of legality, legitimacy, and necessity. Based on obtaining the explicit authorization and consent of the user, only the minimum necessary information required to achieve the purpose is collected, and data security protection obligations are fulfilled in accordance with the law.

[0096] It should be noted that the beneficial effects of the embedded electrically triggered high-precision timing and recording system provided in the above embodiments are the same as those of the embedded electrically triggered high-precision timing and recording method described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0097] like Figure 3As shown, an embodiment of the present invention provides a computer device 300, which includes a processor 320 coupled to a memory 310. The memory 310 stores at least one computer program 330, which is loaded and executed by the processor 320 to enable the computer device 300 to implement any of the above-described methods. Specifically: The computer device 300 can vary considerably due to differences in configuration or performance. It may include one or more processors 320 (Central Processing Units, CPUs) and one or more memories 310. The one or more memories 310 store at least one computer program 330, which is loaded and executed by the one or more processors 320 to enable the computer device 300 to implement the embedded electrically triggered high-precision timing and recording method provided in the above embodiments. Of course, the computer device 300 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The computer device 300 may also include other components for implementing device functions, which will not be elaborated here.

[0098] An embodiment of the present invention provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to enable a computer to implement any of the above-described methods.

[0099] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0100] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the above-described embedded electrically triggered high-precision timing and recording methods.

[0101] It should be noted that the terms "first," "second," etc., used in the specification of this application are used to distinguish similar objects and do not imply a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown in the figures or description.

[0102] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0103] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0104] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An embedded electrically triggered high-precision timing and recording system, characterized in that, include: Network camera module, embedded main control module, storage module, and host computer module; The network camera module is used to collect video data and generate and output RTP data packets; The embedded main control module includes a GPIO interface, a network interface, and a storage interface; it establishes an RTSP connection with the network camera module through the network interface to receive the RTP data packets, receives the TTL trigger electrical signal from an external start signal generator through the GPIO interface and obtains the trigger time, and connects to the storage module through the storage interface to write data. The embedded main control module is also used to parse and extract the sampling time carried in the RTP data packet, encapsulate the sampling time, the real-time acquired system reception time and the RTP data packet into a custom format file, and write the custom format file into the storage module; The storage module is connected to the embedded main control module through the storage interface and is used to store the custom format file; The host computer module is connected to the embedded main control module through a communication network. It is used to obtain the custom format file stored in the storage module, perform time calibration on the custom format file to obtain the actual acquisition time of each frame image in the video data, determine the time difference of each frame image based on the actual acquisition time and the trigger time, and superimpose the time difference onto each frame image to generate a video file containing timing information.

2. The embedded electrically triggered high-precision timing and recording system according to claim 1, characterized in that, The step of performing time calibration on the custom format file to obtain the actual acquisition time of each frame in the video data specifically includes: Extract the sampling time of the key parameter packet sent before I-frame encoding and the system reception time from the custom format file to form multiple valid time pairs; A linear relationship model is established for the multiple effective time pairs, and the conversion parameters are obtained by combining transmission delay compensation; based on the conversion parameters, the mapping relationship between the sampling time and the actual acquisition time is determined. Based on the mapping relationship, the actual acquisition time corresponding to the sampling time is determined.

3. The embedded electrically triggered high-precision timing and recording system according to claim 2, characterized in that, The process of establishing a linear relationship model for the multiple effective time pairs, combined with transmission delay compensation, and solving for the conversion parameters specifically includes: Linear fitting is performed on the multiple valid time pairs, and abnormal data with residuals exceeding a preset threshold are removed to obtain the clock conversion coefficients and conversion bias. The conversion parameters are obtained based on the clock conversion coefficients and the conversion bias, combined with transmission delay compensation.

4. The embedded electrically triggered high-precision timing and recording system according to claim 2, characterized in that, The key parameter package is an SPS package, a PPS package, or a SEI package.

5. The embedded electrically triggered high-precision timing and recording system according to claim 2, characterized in that, The step of determining the time difference of each frame image based on the actual acquisition time and the trigger time specifically includes: Select the most recent I-frame before the triggering time as the anchor point, and convert the sampling time of the anchor point into the corresponding actual acquisition time according to the mapping relationship; Based on the incremental step size of the sampling time and the actual acquisition time corresponding to the anchor point, the actual acquisition time of each frame image is determined, and then the time difference of each frame image relative to the trigger time is calculated.

6. The embedded electrically triggered high-precision timing and recording system according to claim 1, characterized in that, The step of superimposing the time difference onto each frame image to generate a video file containing timing information specifically includes: Encoded data is extracted from the custom format file, the bitstream is reassembled according to a preset encoding standard, the time difference is superimposed on a specified position of each frame image in a recognizable form, compressed into a standard format, and a video file containing timing information is generated.

7. The embedded electrically triggered high-precision timing and recording system according to claim 1, characterized in that, The embedded main control module also includes a circular buffer for storing the custom format file; When the embedded main control module receives the TTL trigger signal, it writes the custom format file already stored in the ring buffer into the storage module.

8. An embedded electrically triggered high-precision timing and recording method, characterized in that, include: S1, configures and starts the monitoring thread of the GPIO interface through the embedded main control module; Establish an RTSP connection with the webcam module; Initialize the circular buffer and start the RTP receive thread to continuously receive RTP packets; S2, the receiving thread receives RTP data packets in real time, parses and extracts the sampling time carried in the RTP data packets, encapsulates the sampling time, the real-time acquired system receiving time and the RTP data packets into a custom format file, and writes the custom format file into the circular buffer; S3, when the monitoring thread receives the TTL trigger signal, it triggers a hardware interrupt and obtains the current time as the trigger moment; Write the custom format file already stored in the circular buffer into the storage module; S4, obtain the custom format file stored in the storage module through the host computer module, perform time calibration on the custom format file to obtain the actual acquisition time of each frame image in the video data, and determine the time difference of each frame image based on the actual acquisition time and the trigger time; S5, the time difference is superimposed onto each frame image through the host computer module to generate a video file containing timing information.

9. A computer device, characterized in that, The computer device includes a processor coupled to a memory, the memory storing at least one computer program, which is loaded and executed by the processor to enable the computer device to implement the embedded electrically triggered high-precision timing and recording method as described in claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer to implement the embedded electrically triggered high-precision timing and recording method as described in claim 8.

Images