Heterogeneous multi-core data acquisition and backpressure transmission method and system

Abstract

The invention discloses a heterogeneous multi-core data acquisition and backpressure transmission method and system, and relates to the technical field of embedded systems.The method comprises the steps that S1, a time-sharing relay assembly line is constructed on an acquisition core; S2, in an acquisition task, sampling is periodically performed from a data source, and samples are written into an intermediate message queue; s3, in the ingestion task, reading the sample from the intermediate message queue, writing the sample into the annular buffer area, and monitoring the occupation amount of the annular buffer area; s4, performing pressure sensing packing in the packing task; s5, in the communication task, deducting the credit value when a new data batch is sent; s6, the processing core receives the data batch, generates a credit control message including the credit adjustment amount and an accumulation confirmation message according to the current buffer water level and the processing time delay index, and sends the credit control message and the accumulation confirmation message to the collection core; and S7, in the communication task, receiving and responding to the credit control message and the accumulated confirmation message, updating the credit value according to the information, and releasing the confirmed data batch.

Description

Technical Field

[0001] This application relates to the field of embedded systems technology, and more specifically, to a heterogeneous multi-core data acquisition and backpressure transmission method and system. Background Technology

[0002] In fields such as industrial control, intelligent sensing, and edge computing, heterogeneous multi-core processors are widely used in high-throughput data acquisition and real-time processing systems due to their advantages in balancing real-time processing and complex computational capabilities. A common architecture consists of an acquisition core running a real-time operating system (such as the Cortex-M series) responsible for high-precision, equally spaced signal sampling, and a processing core running a general-purpose operating system (such as the Cortex-A series) responsible for data aggregation and reporting. Data exchange between the two cores occurs through inter-process communication mechanisms such as shared memory and message queues. The key to achieving optimal system performance lies in ensuring continuous and stable data acquisition, efficient and reliable cross-core transmission, and maintaining deterministic system response under fluctuating loads within this architecture.

[0003] However, some common problems affecting stability still exist in typical implementations of such systems. For example, due to the potential coupling or blocking between tasks such as acquisition, caching, packet assembly, and transmission, when the cross-core transmission link latency increases or the receiving end processing is temporarily delayed, the acquisition task can easily be blocked, thereby disrupting the strict periodicity and continuity of data sampling.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a heterogeneous multi-core data acquisition and backpressure transmission method and system to solve the above-mentioned technical problems.

[0006] This application provides a heterogeneous multi-core data acquisition and backpressure transmission method, comprising: S1, constructing a time-division relay pipeline on the acquisition core, wherein the time-division relay pipeline cyclically executes acquisition tasks, ingestion tasks, packet assembly tasks, and communication tasks; S2, in the acquisition task, periodically sampling from the data source and writing the samples into an intermediate message queue; S3, in the ingestion task, reading samples from the intermediate message queue and writing them into a circular buffer, and monitoring the occupancy of the circular buffer; S4, in the packet assembly task, performing pressure-sensing packet assembly; S5, in the communication task, maintaining a credit value, allowing the transmission of new data batches when the credit value is greater than zero, and deducting the credit value for each new data batch transmitted; S6, the processing core receiving the data batches and generating a credit control message including a credit adjustment amount and a cumulative confirmation message based on its current buffer level and processing latency index, and sending it to the acquisition core; S7, in the communication task, receiving and responding to the credit control message and the cumulative confirmation message, thereby updating the credit value and releasing the confirmed data batches.

[0007] This application provides a heterogeneous multi-core data acquisition and backpressure transmission system, comprising: a construction module for constructing a time-sharing pipeline on the acquisition core, wherein the time-sharing pipeline cyclically executes acquisition tasks, ingestion tasks, packet assembly tasks, and communication tasks; a sampling module for periodically sampling from the data source and writing the samples into an intermediate message queue during the acquisition tasks; a monitoring module for reading samples from the intermediate message queue and writing them into a circular buffer during the ingestion tasks, and monitoring the occupancy of the circular buffer; and a sensing module for performing pressure sensing during the packet assembly tasks. The system comprises: a maintenance module for maintaining a credit value during the communication task, allowing the transmission of new data batches when the credit value is greater than zero, and deducting the credit value for each new data batch transmitted; a control module for receiving the data batches and generating a credit control message including a credit adjustment amount and a cumulative confirmation message based on the current buffer level and processing latency index, and sending them to the acquisition core; and a response module for receiving and responding to the credit control message and the cumulative confirmation message during the communication task, thereby updating the credit value and releasing the confirmed data batches.

[0008] Based on the embodiments provided in this application, a time-sharing pipeline is constructed by cyclically executing acquisition, ingestion, packet assembly, and communication tasks, thereby achieving temporal decoupling between the acquisition and communication stages from a task scheduling architecture perspective. This allows the acquisition task to continue operating independently according to its fixed cycle even when there is a delay in the cross-core link or a temporary lag in the processing core response. Its sampling action is not affected by any subsequent possible transmission blockages, thus ensuring the continuity and strictly equal-interval characteristics of data acquisition at the system level, meeting the core requirements of high real-time acquisition tasks.

[0009] A closed-loop mechanism is implemented to achieve dynamic, feedback-based adjustment of cross-core data transmission flow. This mechanism involves the processing core dynamically generating credit control messages based on its buffer level, and the acquisition core's communication task updating the credit value and controlling transmission based on these messages. This allows the data transmission rate to adaptively match the real-time processing capability of the processing core and the link status, avoiding transmission overload or insufficient utilization caused by a fixed transmission strategy. Consequently, it helps maintain a more stable and efficient cross-core data transmission throughput under varying loads.

[0010] In other words, by combining time-sharing pipeline and dynamic credit feedback, the reliability and efficiency of cross-core data transmission are effectively improved while ensuring real-time data acquisition, thus enhancing the overall stability and performance determinism of heterogeneous multi-core systems under high-load data acquisition scenarios. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0012] Figure 1 This is a flowchart of an optional heterogeneous multi-core data acquisition and backpressure transmission method according to an embodiment of this application;

[0013] Figure 2 This is an optional system architecture diagram according to an embodiment of this application;

[0014] Figure 3 This is an optional four-segment relay timing diagram according to an embodiment of this application;

[0015] Figure 4 This is a diagram illustrating an optional data batch type-length-value structure and batch header structure according to an embodiment of this application.

[0016] Figure 5 This is an optional back pressure control and credit change histogram according to an embodiment of this application;

[0017] Figure 6 This is an optional power-off coordination timing diagram according to an embodiment of this application;

[0018] Figure 7 This is an optional set of experimental result graphs according to an embodiment of this application;

[0019] Figure 8 This is another optional set of experimental result graphs according to an embodiment of this application;

[0020] Figure 9This is yet another optional set of experimental result graphs according to an embodiment of this application;

[0021] Figure 10 This is yet another optional set of experimental result graphs according to an embodiment of this application;

[0022] Figure 11 This is yet another optional set of experimental result graphs according to an embodiment of this application;

[0023] Figure 12 This is yet another optional set of experimental result graphs according to an embodiment of this application;

[0024] Figure 13 This is an optional six-group test statistical comparison table according to an embodiment of this application.

[0025] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0027] This invention belongs to the field of high-throughput real-time data acquisition and cross-core transmission technology in heterogeneous multi-core embedded systems. Specifically, it relates to a method and system for zero-blocking acquisition, congestion adaptive packetization, and reliable backpressure transmission between an acquisition core running a real-time operating system (such as FreeRTOS) and a processing core running a general-purpose operating system (such as Linux), based on a virtual serial port / message channel (such as OpenAMP / VIRT_UART / RPMsg).

[0028] With the increasing prevalence of heterogeneous multi-core platforms in industrial control, edge AI, and sensor fusion scenarios, a common division of labor is as follows: the acquisition core continuously samples from ADCs / various sensors at a fixed interval, while the processing core is responsible for aggregation, reporting, or display. The two cores typically exchange data and control information via links such as rpmsg / virtio / shared memory message queues. In such systems, existing implementations generally suffer from the following problems:

[0029] Link congestion causes data acquisition interruptions. When cross-core links experience short-term congestion or receiver processing lags, if the sender uses blocking transmission or lock contention exists, the data acquisition task can easily be "stuck," disrupting the real-time and equal-interval characteristics of sampling. Buffer level fluctuations lead to jitter / packet loss. Single-threshold, hysteresis-free buffer level control oscillates repeatedly between "nearly full / nearly empty," triggering frequent packet dropping, excessive packet reassembly, or invalid wake-ups, exacerbating system jitter. Fixed acknowledgment / timeout parameters. With fixed ACK timeout, retransmission threshold, and other parameters, under different load and jitter conditions, the response is either too aggressive (misjudging timeouts and causing unnecessary retransmissions) or too conservative (reducing throughput and increasing queuing latency), lacking adaptive convergence capability to link states. High coupling between data and control. Mixing control plane messages such as credit, ACK, and heartbeat with business data easily amplifies packet reassembly and parsing complexity, causing interference between control and data, affecting the overall transmission rhythm and observability.

[0030] In other words, in related technologies, the acquisition side is easily affected by blockages. Once transmission is blocked or lock contention occurs, the acquisition thread / service will be slowed down, resulting in missed samples, distortion, or sampling interval jitter. There is a lack of adaptive batch and retransmission control. Batch size, ACK timeout, and retransmission strategies lack dynamic tuning. When the link switches from "smooth → congested → recovering," system convergence is slow, leading to either erroneous retransmissions or reduced throughput. The cost and difficulty of timeline reconstruction are unbalanced. There is no lightweight timeline reconstruction design: either timestamping each sample results in high storage and bandwidth costs, or there is a lack of batch-level time anchors, making it difficult to align the timeline after an anomaly. There is no negotiation between shutdown and anomaly cleanup. The shutdown / reset / anomaly recovery phase lacks an orderly drain / flush protocol and final confirmation, easily leading to the loss of tail packets or legacy resources (unreleased credits, residual data in the queue). This introduces uncertainty and additional maintenance costs for subsequent restarts.

[0031] The aforementioned deficiencies make it difficult to guarantee the core objective of uninterrupted data acquisition, and it is also difficult to simultaneously meet the requirements of stable throughput and time consistency under high load. This invention addresses these pain points by proposing mechanisms such as non-blocking acquisition protection, adaptive control plane, lightweight timeline reconstruction, and orderly shutdown negotiation to overcome these deficiencies.

[0032] This invention aims to address heterogeneous multi-core (e.g., A35 / M33) scenarios by maintaining high throughput and timeline consistency under conditions of link congestion and system fluctuations without interrupting data acquisition, while reducing implementation and operational complexity. Specific objectives include: ensuring uninterrupted data acquisition; suppressing data level fluctuations and reducing jitter and packet loss; implementing adaptive acknowledgment and retransmission strategies; decoupling the data plane and control plane; lightweight timeline reconstruction; achieving anomaly self-healing and eventual consistency; ensuring verifiability, portability, and integrability; and facilitating easy operation and maintenance with online adjustability.

[0033] This invention can be widely applied to scenarios requiring highly reliable data acquisition and stable cross-core uploading, such as industrial control, medical monitoring, edge AI sensing, unmanned equipment, and vehicle domain control. It has the advantages of low implementation cost, strong compatibility, and engineering feasibility.

[0034] With the above objectives, this invention aims to overcome the problems in the prior art such as "transmission blocking entangles acquisition, fixed parameters are difficult to adapt, timeline reconstruction is costly, and shutdown termination lacks negotiation", and to form a cross-core data transmission and control method and system that is more reliable in both steady state and transient state for industrial / edge scenarios.

[0035] According to one aspect of the embodiments of this application, such as Figure 1 As shown, the data acquisition and backpressure transmission method, which is jointly executed by a data acquisition core running a real-time operating system and a processing core running a general-purpose operating system, provides a heterogeneous multi-core data acquisition and backpressure transmission method, including:

[0036] S1, construct a time-sharing pipeline on the acquisition core, and the time-sharing pipeline cyclically executes acquisition tasks, capture tasks, packet assembly tasks and communication tasks;

[0037] like Figure 2 (System Overall Architecture: Relationship Diagram of Acquisition Core / Processing Core / Shared Memory / Message Channel and Four Task Segments) and Figure 3 (The four-stage relay timing diagram: the handover timeline and time slice budget for Acq→Ingest→Pack→Comm) shows that the method of this invention constructs a time-division relay pipeline consisting of four stages: acquisition, ingestion, packet assembly, and communication on the acquisition core, such as Cortex-M33. During system initialization, through... The function creates four tasks with equal priority (osPriorityAboveNormal, higher than normal operating system priority): Acq, Ingest, Packet, and Comm, and configures four semaphores. (Acquisition of start signal) (Acquire ready signal) (Packet ready semaphore) (Communication ready semaphore).

[0038] Release after initialization Initiate the first round of data collection. The four tasks form a closed-loop scheduling mechanism using semaphores: the Acq task releases its semaphore upon completion. Release after the Ingest task is completed Release after Pack task is completed Release after the Comm task is completed This process repeats continuously. Each task is limited by a preset maximum execution time slice and maximum processing capacity within its time slice. Once either constraint is met, the next task semaphore is released and the task is suspended, forming a "time-limited / capacity-limited advancement" mechanism. This design avoids long critical sections, suppresses priority inversion, and ensures the real-time response capability of the acquisition core, forming the basic architecture for achieving high-precision acquisition and stable transmission.

[0039] S2, in the data collection task, periodically samples from the data source and writes the samples to the intermediate message queue;

[0040] The acquisition task (Acq_Thread_Entry) is responsible for sampling from the data source (such as ADC channel ADC_CHANNEL_15, PB15→ADC_INP15) at a fixed period of SAMPLE_PERIOD_US (sampling period) (derived from ACQ_RATE_SPS (sampling rate per second), for example, 2000 SPS corresponds to 500 μs). In each round of execution, the task continues to sample until it reaches the maximum number of samples (ACQ_ROUND_MAX_SAMPLES, maximum number of samples per acquisition round) or the maximum time (ACQ_ROUND_MAX_MS, maximum time per acquisition round).

[0041] Each collected sample (sample_t structure, containing only 16 bits) is immediately delivered to the AcqMQ message queue (acquisition message queue) (depth ACQ_MQ_DEPTH=64). If AcqMQ is full, the "drop oldest" strategy (overwrite the oldest sample at the head of the queue) is executed before writing the new sample, ensuring that the acquisition process is never blocked and maintaining the absolute stability of the sampling period.

[0042] S3, in the ingestion task, reads samples from the intermediate message queue and writes them to the circular buffer, monitoring the occupancy of the circular buffer; the ingestion task (Ingest_Thread_Entry, the ingestion thread entry point) reads samples in batches from AcqMQ (up to INGEST_BATCH_MAX (maximum batch size) messages or INGEST_FLUSH_MAX_MS (maximum refresh time)) and continuously writes them to the single-writer lock-free circular buffer sample_ring_mem[]. The capacity of the circular buffer is calculated according to...

[0043] Estimate (example: 2000 × 4 × 0.2 × 3 ≈ 4800 bytes), then round up to the smallest power of 2 that is not less than need, and clamp to no more than (Number of bytes in the static circular buffer) (e.g., 8192 bytes).

[0044] A single-writer lock-free strategy is used during writes: if the write length exceeds the current free space, the read pointer is first synchronized. The circular read pointer advances forward by (write length - remaining space) bytes to discard the oldest data before writing; data copying is performed twice using memcpy (memory copy) to copy the tail and head segments. The task continuously calculates the used space of the circular buffer. . It is the Sample Ring Write Pointer, which is usually a monotonically increasing counter that indicates the position (index) where the next piece of data should be written to the circular buffer.

[0045] S4, in the package assembly task, perform pressure-aware package assembly;

[0046] S5, in the communication task, maintains a credit value. When the credit value is greater than zero, a new batch of data is allowed to be sent, and the credit value is deducted for each new batch of data sent.

[0047] The communication task (Comm_Thread_Entry, the entry point for the communication thread) uses shared variables. (Number of Credit Packets Transmitted) Maintains a credit value, initialized to INITIAL_CREDIT_PKTS (initial number of credit packets) (default 8). During each transmission opportunity, the task first checks the credit value: if tx_credit_pkts > 0, it can call try_send_one_packet() (attempt to send a single data packet) to send a new data batch, decrementing the credit value by 1 for each new batch sent. If the credit value is 0, sending new data batches is paused until a CREDIT control message is received from the processing core to replenish the credit. This mechanism implements receiver-driven transmission rate control (backpressure), ensuring the processing core is not overwhelmed by sudden data bursts.

[0048] S6, the processing core receives the data batch and, based on its current buffer level and processing latency index, generates a credit control message including the credit adjustment amount and a cumulative confirmation message, and sends them to the acquisition core;

[0049] The processing core (Cortex-A35 / Linux) receives data batches on the data plane via rpmsg (Remote Processor Message) or shared memory. The application parses the TLV header, batch header, and sample array, performs CRC checks, and reconstructs the sampling timeline based on the first sample timestamp ts_us_first (in microseconds) and the sampling period period_us (in microseconds) in the batch header. The application treats the receive buffer queue as a finite resource, maintaining its backlog (the backlog of pending data) and processing latency metrics.

[0050] S7, in the communication task, receives and responds to credit control messages and cumulative confirmation messages, updates the credit value accordingly, and releases the confirmed data batch.

[0051] The control plane callback VIRT_UART_CTRL_RxCpltCallback() (virtual serial port control receive completion callback) of the communication task is responsible for receiving and parsing control messages. It enters different branches for processing based on the tlv_t.type (TLV type) field, thus achieving a closed control loop.

[0052] Furthermore, S3 also includes: when the occupancy of the annular buffer reaches or exceeds the first high water level threshold, the annular buffer enters the pressure mode; when the occupancy of the annular buffer drops to or below the first low water level threshold, the annular buffer exits the pressure mode.

[0053] In this embodiment, the first high-water threshold is set to 3 / 4 of the buffer capacity (e.g., calculated using the function WM_HIGH()), and the first low-water threshold is set to 1 / 4 of the buffer capacity (e.g., calculated using the function WM_LOW()). The buffer enters "pressure mode" when the used space (used) is greater than or equal to the first high-water threshold; it exits pressure mode only when used is less than or equal to the first low-water threshold. This hysteresis mechanism, switching between different thresholds, effectively avoids frequent mode switching caused by minor fluctuations near the thresholds, enhancing system stability. This water level determination is executed periodically in the Ingest and Pack tasks, and the pressure_mode flag is recorded for subsequent packing logic.

[0054] S4, In the packet assembly task, pressure-aware packet assembly is performed, including: when the circular buffer is in pressure mode, the number of packets is determined based on the portion of the circular buffer occupancy that exceeds the first low water level threshold; when the circular buffer is not in pressure mode, the number of packets is determined based on the number of available samples and the link upper limit; the packets are encapsulated into data batches including batch number, first sample timestamp, sample quantity and sampling period.

[0055] The packet assembly task (Pack_Thread_Entry, the packet assembly thread entry point) executes the core pressure-aware packet assembly logic. Under pressure mode, it calculates the excess capacity. (Where LOW is the first low watermark threshold), the number of samples packed in this batch must at least cover this excess (at least 1 sample) to achieve rapid "flood discharge"—that is, a "flood discharge" strategy used to quickly reduce buffer pressure; in normal mode, it tries to fill the available samples as much as possible under the constraints of PKT_PAYLOAD_LIMIT (packet load limit). Each round of the task attempts to build no more than PACK_ROUND_MAX_PKTS (maximum number of packets per round) or PACK_ROUND_MAX_MS (maximum time for rounds of packet assembly).

[0056] Regardless of the stress mode, a package is created during packaging as follows: Figure 4 The data batch (Data Batch TLV and Header Structure) shows a TLV (Type-Length-Value) structure. This structure consists of the following parts: a 2-byte TLV header (... The process begins with a batch header (batch_hdr_t, a header structure) containing an 8-byte batch sequence number (seq), an 8-byte first sample timestamp (Ts_us_first, the first microsecond-level timestamp, i.e., the batch-level time anchor), a 2-byte sample count (batch_cnt, i.e., the number of samples in this batch), and a 2-byte reserved field (storing period_us, the microsecond-level period), followed by a continuous array of numerical samples (sample[]. Finally, a 32-bit CRC-32 checksum is calculated and appended (generating polynomial 0xEDB88320). After the packets are assembled, ring_pop(copy_bytes) is called to pop the packaged samples from the circular buffer, and pq_push() is used to push them into the in-transit queue.

[0057] S5 also includes: monitoring for timeouts of data batches that have been sent but not yet acknowledged, and prioritizing the retransmission of the earliest sent data batch that has timed out; retransmission operations do not consume credit points;

[0058] The communication task maintains the in-transit queue pktq[PKT_QUEUE_SLOTS] (data packet queue slots), recording the status of each sent data batch. During each transmission opportunity, the task first checks if the unacknowledged data batch at the head of the queue (corresponding to pq_inflight_head (in-transit queue head index)) has timed out: if... If so, the batch should be retransmitted immediately.

[0059] After retransmission, its ts_first is updated to the current time, but the credit value is not consumed. This design decouples reliability assurance (retransmission) from flow control (credit), avoiding network fluctuations that could lead to credit depletion and a complete halt in transmission, ensuring that data in transit is eventually reached. S7 also includes: dynamically adjusting the timeout threshold based on the round-trip delay of accumulated acknowledgments.

[0060] Each time a cumulative acknowledgment message is received, the communication task calculates the sampled round-trip time (RTT) from the time the message was sent to the acknowledgment, and dynamically adjusts the ack_timeout_ms based on the EWMA (Exponential Weighted Moving Average) algorithm, so that the timeout threshold adapts to changes in network and processing latency.

[0061] Further, S6 processes the received data batch and, based on its current buffer level and processing latency metrics, generates a credit control message including a credit adjustment amount, and a cumulative acknowledgment message, including:

[0062] The processing core triggers the calculation of the credit adjustment amount based on a first preset cycle or based on an event that a preset number of data batches have been processed.

[0063] The processing core can be configured to trigger a credit calculation once at a fixed period (e.g., 10ms) or after processing a preset number of data batches (e.g., after receiving 4 batches) to balance control overhead and response speed.

[0064] The processing core sets a second high-water threshold and a second low-water threshold for its receive buffer queue. The second high-water threshold is set to 3 / 4 of the queue capacity, and the second low-water threshold is set to 1 / 4 of the queue capacity to form hysteresis control.

[0065] like Figure 5 As shown in the bar chart (Backpressure Control and Credit Change), the processing core sets a second high-water threshold (HIGH, e.g., 3 / 4 of the queue capacity) and a second low-water threshold (LOW, e.g., 1 / 4 of the queue capacity) for its receive buffer queue, forming a hysteresis control symmetrical to the acquisition core side, avoiding frequent credit adjustments. When the occupancy of the receive buffer queue falls below the second low-water threshold, the processing core generates a credit control message, in which the credit adjustment amount is a positive value.

[0066] When the backlog of the receive buffer queue (current backlog of pending tasks) falls below the second low watermark threshold (LOW), the processing core calculates a positive delta_credit (credit adjustment) (e.g., +4) and generates a TLV_TYPE_CONTROL_CREDIT message (type 0x9002) which is then sent to the acquisition core. This is equivalent to granting the acquisition core more sending quota to expand its sending window.

[0067] When the occupancy of the receiving buffer queue reaches or exceeds the second high water level threshold, the processing core generates a credit control message, in which the credit adjustment amount is negative, and the absolute value of the negative credit adjustment amount does not exceed the current credit value of the acquisition core. Moreover, the adjustment only applies to the future sending quota of the acquisition core and does not require the withdrawal or affect the batch of data that has been sent but not yet confirmed.

[0068] When the receive buffer queue reaches or exceeds the second high watermark threshold (HIGH), the processing core generates a negative delta_credit (e.g., -4) to quickly shrink the acquisition core's sending window. The absolute value of this negative adjustment does not exceed the acquisition core's current credit value tx_credit_pkts, and this adjustment only applies to the acquisition core's future sending quota, without requiring the withdrawal or affecting sent but unacknowledged batches of data in transit.

[0069] When the backlog is between high and low levels, the processing core can issue delta_credit=0, creating natural back pressure. This causes the acquisition core's credit value tx_credit_pkts to gradually deplete and eventually stop sending new batches. The processing core calculates a timeout recommendation value based on the actual latency of its processed data batches (i.e., processing latency metric) and fills this value into the credit control message.

[0070] The processing core calculates a suggested timeout value (tmo_ms, typically twice the actual delay) based on its processing latency (such as scheduling latency or application processing time) and fills this value into the ack_timeout_ms field of the CREDIT message. The sending end updates this value upon receipt. This ensures that the timeout threshold is synchronized with the actual processing capacity of the receiving end.

[0071] The processing core sends a keep-alive probe message with type identifier 0x9F01 to the acquisition core through the control channel according to the second preset cycle, and immediately replies upon receiving a response message with type identifier 0x9F02 to maintain session activity;

[0072] like Figure 2 As shown, the processing core (also known as the processing kernel) periodically sends a TLV_TYPE_CONTROL_PING (keep-alive probe type identifier) ​​(0x9F01) keep-alive probe. Upon receiving this, the acquisition core (also known as the acquisition kernel) immediately replies with a TLV_TYPE_CONTROL_PONG (keep-alive response type identifier) ​​(0x9F02). If the communication task does not receive any control frames within 2 seconds, the peer's status is marked as PEER_CONGESTED (peer congestion) (credit is 0 but there are packets in transit) or PEER_SUSPECT_DOWN (peer suspected disconnection) (no packets in transit), and a conservative sending strategy is entered.

[0073] When the number of data batches arriving reaches a preset threshold, a preset time period is reached, or the amount of data in the receiving buffer queue exceeds a preset alarm threshold, the processing core generates a cumulative confirmation message, which carries a cumulative confirmation sequence number.

[0074] For example, a preset quantity threshold can be set, such as 4 batches. That is, when the number of data batches continuously received and buffered by the processing core reaches 4, a cumulative acknowledgment message will be immediately generated and sent, regardless of whether the timer has expired.

[0075] A preset alarm threshold can be set, for example, to 85% of the receive buffer queue capacity. This is an "alarm" level higher than the second highest watermark threshold (e.g., 75%). When the backlog occupancy of the receive buffer queue exceeds this threshold, it indicates that the queue is about to be full, and the system will immediately trigger cumulative acknowledgments. This is designed to accelerate the release of data at the receiving end by rapidly advancing the acknowledgment sequence number, thereby relieving pressure.

[0076] For example, cumulative acknowledgments for the processing core can be triggered by any of the following conditions: Quantity trigger: Every four data batches received. Periodic trigger: An independent acknowledgment timer times out every 20 milliseconds (i.e., a preset timing period). Pressure trigger: The receive queue occupancy exceeds 85% of the total capacity (i.e., a preset alarm threshold).

[0077] These three conditions are related by "OR", and a cumulative confirmation message is generated when any one of the conditions is met. This ensures the timeliness of confirmation, maintaining basic progress through periodic confirmation, and responding quickly when data arrives in large quantities or when processing pressure is high.

[0078] The processing core triggers cumulative acknowledgments when one of the following conditions is met: four batches of received data are received, a timer period (e.g., 20ms) is reached, or the receive queue occupancy exceeds the alarm threshold. The generated TLV_TYPE_CONTROL_ACK (cumulative acknowledgment type identifier) ​​(0x9001) message carries the cumulative acknowledgment sequence number ack_seq (cumulative acknowledgment sequence number), indicating all... The batch has been successfully received.

[0079] It should be noted that the first high / low water level threshold is used for the ring buffer of the acquisition core to control whether it enters pressure mode, thereby affecting the packet assembly strategy. The second high / low water level threshold is used for the receive buffer queue of the processing core and is the basis for determining the generation of credit control messages (positive / negative adjustment amounts).

[0080] Furthermore, in S7, receiving and responding to credit control messages and cumulative confirmation messages includes:

[0081] Upon receiving the credit control message, use the following formula: Update local credit score, where This is a clamping function, and the preset maximum credit window value is a constant statically configured by the system during initialization.

[0082] The communication task parses the CREDIT message, extracts the delta_credit (credit adjustment amount) field, and updates it. . It is typically set to half of PKT_QUEUE_SLOTS (the number of packet queue slots) (e.g., 8). This clamping function ensures that the credit value is always within a valid range, preventing overflow or negative values.

[0083] Upon receiving the timeout suggestion value in the credit control message, the timeout threshold maintained by the data collection core is updated to max(timeout threshold, timeout suggestion value);

[0084] For example, upon receiving a credit control message, the suggested timeout value tmo_ms (in milliseconds) is extracted. Then, the adaptive timeout threshold ack_timeout_ms maintained by the acquisition core is updated to max(ack_timeout_ms, tmo_ms), meaning the larger of the current threshold and the suggested value is used as the new threshold. This strategy ensures that the timeout threshold incorporates processing delay information from the receiver, making the threshold setting more conservative and avoiding misjudged timeouts and unnecessary retransmissions due to slower receiver scheduling or processing. Upon receiving a cumulative acknowledgment message, data batches with batch numbers not exceeding the cumulative acknowledgment number in the en route queue are released.

[0085] Calculate the round-trip delay sample value based on the sending time of the data batch corresponding to the cumulative confirmation message.

[0086] For the confirmed batch, the round-trip delay sample value is calculated based on the first transmission time ts_first recorded in the in-transit queue for that batch.

[0087] This sampled value will be used in subsequent EWMA (Exponentially Weighted Moving Average) updates to dynamically adjust the timeout threshold.

[0088] Furthermore, timeout monitoring and priority retransmission in S5, and dynamic adjustment of timeout thresholds in S7, are achieved through the following steps:

[0089] The data acquisition core records the transmission time for each batch of data sent.

[0090] Each slot in the in-transit queue stores the ts_first (first transmission time) field, which records the microsecond time of the first transmission or the most recent retransmission of this batch.

[0091] The acquisition kernel maintains a round-trip delay estimate. Its initial value is the preset baseline timeout value. Half of;

[0092] System startup initialization (Baseline confirmation timeout in milliseconds) / 2 (e.g., 25ms) is used as the initial value for RTT estimation.

[0093] Each time a cumulative confirmation message is received, a new round-trip delay sample value is calculated based on the sending time of the corresponding data batch. and according to the formula

[0094] Update the round-trip time delay estimate; where, Indicates the state before the update ; Indicates the updated ;

[0095] Perform EWMA smoothing upon receiving an ACK: The "+1" ensures that the estimated value converges within 1ms, preventing numerical underflow. This formula more clearly illustrates the difference between the old and new values.

[0096] After the update, the timeout threshold will be set to... ;

[0097] 20ms is the minimum safety margin, and a 2x factor provides sufficient margin to handle jitter. This value is dynamically adjusted up or down based on the RTT estimate.

[0098] In each sending opportunity, check whether the earliest sent and unacknowledged data batch in the in-transit queue has timed out (i.e., whether the current time minus its sending time is greater than the timeout threshold). If it has timed out, the data batch is retransmitted first. After retransmission, its sending time is updated to the current time, and this retransmission does not consume credit value.

[0099] The `try_send_one_packet()` function first checks for a timeout at the `pq_inflight_head` (the index at the head of the in-transit queue). If a timeout occurs, the batch is immediately retransmitted and updated. The credit score remains unchanged. This ensures that the first packet lost by the team leader is repaired first.

[0100] If the same data batch is retransmitted more than 3 times due to timeout, the backoff mechanism is activated, and the timeout threshold is raised to a preset upper limit of no more than 500 milliseconds. When the cumulative confirmation message is received again, the backoff mechanism is deactivated, and the timeout threshold is dynamically adjusted based on the round-trip delay estimate.

[0101] To avoid retransmission storms during congestion, if the same batch times out more than 3 times consecutively, backoff will be triggered: Once an ACK is received confirming the batch, the process backs off and resumes EWMA dynamic adjustment.

[0102] Furthermore, the design rules for the circular buffer in S3 and the data batches encapsulated in S4 include:

[0103] The capacity of the circular buffer is determined by the following steps:

[0104] Based on formula Estimate the required number of bytes, then take a value not less than The smallest non-negative integer power of 2 is used to calculate the result, which is then clamped to a value not exceeding the maximum capacity of a preset circular buffer. ;

[0105] In one embodiment, ACQ_RATE_SPS=2000U (sampling rate per second) and SAMPLE_BYTES=4 (length of a single sample byte). , Calculated The value is raised to the power of 2, resulting in 8192 bytes. If this exceeds STATIC_RING_BYTES (the number of bytes in the static ring buffer) (default 8192 bytes), it is clamped. This capacity ensures sufficient buffering even under worst-case link latency.

[0106] in, The preset sampling rate per second, The preset length of a single sample byte. The preset worst-case link latency, The estimation result is based on a preset safety factor greater than 1. The unit is bytes;

[0107] These parameters are all compile-time macros and can be adjusted in config.h according to the actual application scenario.

[0108] Write operations to the circular buffer employ a single-writer lock-free strategy. When the length of data to be written exceeds the current remaining free space in the circular buffer, the read pointer is synchronously advanced first. The oldest data is discarded before writing. The data copying process is divided into two memory copies: the tail segment and the head segment. The two memory copies are only performed when the data to be written logically crosses the physical tail boundary of the buffer and are protected by a short critical section.

[0109] The specific implementation steps of ring_write() (circular write function) are as follows: First, calculate the free space of the current circular buffer. If the length of the data to be written in this request, len (the number of bytes to be written), is greater than free, then execution will proceed. The read pointer is moved forward, discarding the oldest data to free up space. Then, the write operation is performed: the data copy operation is completed in two memcpy (memory copy) steps—first copying to the remaining space at the end of the buffer (tail segment), and if the data is not completely written, wrapping around to the beginning of the buffer to continue copying (head segment). The entire calculation, pointer adjustment, and copying process is completed within a short critical section, ensuring atomicity by disabling interrupts or using mutexes, thus avoiding data races in a multi-tasking environment.

[0110] The encapsulated data batch adopts a type-length-value structure, which consists of: a 2-byte data batch type identifier, a 2-byte length field, a batch header including an 8-byte batch sequence number and an 8-byte first sample timestamp, a continuous sample array containing only numerical values, and a cyclic redundancy check (CRC) code covering the batch header and sample array. The CRC code is generated using the CRC-32 algorithm, and the generator polynomial of the CRC-32 algorithm is 0xEDB88320.

[0111] like Figure 4 As shown in the (Data Batch TLV and Header Structure), in this embodiment, a complete data batch is encapsulated using a Type-Length-Value (TLV) structure, which includes the following from front to back:

[0112] TLV header (tlv_t): 4 bytes in total, including 2 bytes of type (type identifier, such as 0x0001 representing batch data) and 2 bytes of length (length field, indicating the length of all subsequent data).

[0113] Batch header (batch_hdr_t): 20 bytes in total, it is the core metadata of the data batch, containing: seq (8 bytes, batch sequence number): used for cumulative confirmation and order preservation. ts_us_first (8 bytes, first sample timestamp, in microseconds): serves as the time reference anchor for all samples in this batch. batch_cnt (2 bytes, number of samples in this batch): indicates the number of samples immediately following. period_us (2 bytes, sampling period, in microseconds): stored in the reserved field, used together with ts_us_first at the receiving end to reconstruct the equally spaced sampling timeline.

[0114] The sample array (sample_t[batch_cnt]): contains batch_cnt samples stored contiguously. Each sample_t structure is 4 bytes in size (16-bit sample value plus 2 bytes of padding). Here, "4-byte alignment" means that each sample occupies memory and starts at a 4-byte boundary, which facilitates efficient access by the processor.

[0115] CRC-32 checksum (4 bytes): Its calculation covers the batch_hdr_t header and the sample_t[] sample array, and is used to verify the integrity of data during transmission. Any batch that fails the check will be discarded.

[0116] This architecture is compact and efficient. Compared to schemes that carry independent timestamps for each sample, the batch-level timestamp (ts_us_first) combined with a fixed period_us saves approximately batch_cnt × 6 bytes of bandwidth per batch (assuming an independent timestamp occupies 8 bytes and a sample value occupies 2 bytes, and considering alignment). CRC-32 checksum provides reliable end-to-end integrity assurance.

[0117] Furthermore, the implementation of the time-sharing pipeline constructed in S1 includes:

[0118] The four tasks—acquisition, ingestion (also known as data intake), packaging (also known as data packaging), and communication (also known as communication module)—are assigned the same operating system priority.

[0119] In FreeRTOS, all four tasks are created with an osPriorityAboveNormal priority to avoid priority inversion. Because the semaphore mechanism guarantees sequential execution, a strict pipeline is still formed under the same priority, and prolonged blocking of any one task will not cause starvation in other tasks.

[0120] During system initialization, four start semaphores are configured for the four tasks, corresponding to the data acquisition tasks. Corresponding to the acquisition task Corresponding package task and corresponding communication tasks ;

[0121] Semaphores are created in MX_FREERTOS_Init() using osSemaphoreNew() (semaphore creation), with initial counts of 0. This design ensures tasks start in sequence, avoiding race conditions.

[0122] Release the semaphore after initialization. To initiate the first round of data collection;

[0123] After completing the initialization of all resources (ADC, OpenAMP, ring buffer), the main function explicitly calls osSemaphoreRelease(S_acq_start) (releasing the semaphore) to trigger the Acq task to run for the first time.

[0124] Closed-loop scheduling is achieved between tasks by releasing the semaphore for the next task, including: releasing the semaphore after the data acquisition task is completed. Released after the task is completed. Release after the package assembly task is completed Release after the communication task is completed To initiate the next round of data collection;

[0125] At the end of each task, after releasing the next semaphore using osSemaphoreRelease(), osDelay(0) is called to suspend the task, forming a "passing the baton" loop. For example... Figure 3 As shown, the four tasks on the timeline are activated sequentially without overlap, ensuring full utilization of single-core resources.

[0126] Each task, during its execution, is limited by preset maximum execution time slices and maximum processing capacity, including: data collection task limitations. Sample size or Time; intake task limitations Message count or Time; Package assembly task limitations Number of packages or Time; Communication task limitations Number of packages or Time; among which , , , , , , , All are preset system constants. Once any constraint is met, the next task semaphore is released and the task is suspended. When a data acquisition task writes a sample to the intermediate message queue, if the intermediate message queue is full, the oldest sample in the intermediate message queue is discarded before writing the sample.

[0127] Typical values ​​in the examples are: ACQ_ROUND_MAX_SAMPLES = 8 (maximum number of samples per collection round), ACQ_ROUND_MAX_MS = 1 (maximum collection round time, milliseconds); INGEST_BATCH_MAX = 16 (maximum batch size), INGEST_FLUSH_MAX_MS = 2 (maximum refresh time, milliseconds); PACK_ROUND_MAX_PKTS = 4 (maximum number of packets per packet assembly round), PACK_ROUND_MAX_MS = 2 (maximum packet assembly round time, milliseconds); COMM_ROUND_MAX_SEND = 4 (maximum number of messages sent per communication round), COMM_ROUND_MAX_MS = 3 (maximum communication round time, milliseconds). These quotas can be tuned in config.h to balance throughput and response latency.

[0128] The implementation of queues for managing data batches that have been sent but not yet acknowledged in a communication task includes:

[0129] The communication task maintains an in-transit queue for managing batches of data that have been sent but not yet acknowledged. This in-transit queue is a circular array queue, which includes... Each slot, and maintains a header index. Tail index and in-transit head index ;

[0130] The in-transit queue (PKTQ, the data packet queue) is a fixed-size array (by default). The array is managed using a circular array. All three indices are monotonically increasing, and access is performed by taking the modulo operation (& (PKT_QUEUE_SLOTS-1)) to avoid rollback issues. pq_inflight_head (head index of the in-transit queue) points to the earliest unacknowledged batch, and pq_tail (tail index of the queue) points to the next position to be written.

[0131] Each slot is used to store metadata for a data batch, including: a pointer to the data batch's storage buffer, the data length, the batch number, the time of the first transmission, and a status flag indicating the current status of the slot;

[0132] The slot structure contains: a buf pointer (pointing to the actual data storage), len (data length), seq (batch sequence number), ts_first (first transmission time), and an in_flight flag (a boolean value indicating whether it has been sent but not acknowledged). This design enables constant complexity queue management, supporting O(1) enqueue, dequeue, and timeout checks.

[0133] Upon receiving the cumulative confirmation message, release all slots whose batch sequence number is not greater than the cumulative confirmation sequence number and whose status flag is in transit.

[0134] pq_ack_until() (queue acknowledgment until) iterates from pq_inflight_head to pq_tail, releasing all queue acknowledgments. and The slot is set to `in_flight` and the index is advanced. Unconfirmed batches are held. Waiting for the next timeout or ACK.

[0135] Furthermore, the method of the present invention also includes a mechanism for determining and recovering lost contact:

[0136] If a communication task does not receive a credit control message or cumulative confirmation message from the processing core within 2 consecutive seconds, it is determined that the other end has entered a congestion or suspected disconnection state.

[0137] Once in this state, the communication task stops sending new data batches and only performs timeout retransmissions of the in-transit queue based on the timeout threshold;

[0138] When a cumulative confirmation message is received again, the system exits this state and resumes the normal process of sending new data batches when the credit value is greater than zero.

[0139] Furthermore, the method also includes an ordered shutdown protocol implemented between the processing core and the acquisition core via an inter-core communication controller, including:

[0140] The processing core sends a shutdown request to the acquisition core via the inter-core communication controller;

[0141] The processing core application calls CoproSync_Shutdown() (coprocessor synchronous shutdown), which sends a shutdown request signal to the acquisition core M33 via IPCC (Inter-Processor Communication Controller).

[0142] The acquisition core responds to the interrupt triggered by the shutdown request, stops the cyclic execution of the four tasks: acquisition, ingestion, packet assembly, and communication, and emptys the data in the in-transit queue and the circular buffer in sequence.

[0143] The CoproSync_ShutdownCb interrupt callback of the acquisition core is triggered, calling shutdown_thread_entry() (the shutdown thread entry point). This function sets a global flag, causing the four tasks to proactively exit and stop sampling from the ADC upon detecting the flag in the next loop.

[0144] During the emptying process, the communication task continuously attempts to send the remaining data batches in the in-transit queue and waits for the corresponding cumulative confirmation messages so that all tail batch data can be finally confirmed.

[0145] like Figure 6 (As shown in the power-off coordination sequence: interaction flow of peer initiation → cleanup → notification → power-off), the communication task enters the emptying state during the power-off process: it repeatedly calls try_send_one_packet() to retransmit the remaining batches in the en route queue and waits for the processing core to return the cumulative ACK to ensure "final batch reachability". If the credit value is greater than 0, a new batch is sent first; otherwise, only retransmission is performed.

[0146] Specifically, such as Figure 6The shutdown coordination sequence shown describes the complete interactive process from the peer (A55 Linux) initiating a shutdown request, to the acquisition core (M33FreeRTOS) clearing its own tasks and resources, then notifying the processing core (main CPU), and finally the processing core executing the power-off. The specific steps are as follows:

[0147] Shutdown Request: The peer processor sends a shutdown synchronization request (CoproSync) to the acquisition core.

[0148] Confirm and begin cleanup: Collect the core confirmation request and begin the local cleanup process.

[0149] Stop tasks: Acquisition core (stop each of its running tasks in sequence, including acquisition, ingestion, packaging, and communication tasks (Acq / Ingest / Pack / Comm).

[0150] Hardware deinitialization: The acquisition core deinitializes the hardware resources it manages, such as virtual serial ports, inter-core communication frameworks, and analog-to-digital converters (VIRT_UART / OPENAMP / ADC).

[0151] Emptying and Notification: The acquisition core enters the emptying state, retransmitting data in a loop until the in-transit queue is empty and the credit value is zero. After completion, it sends a "cleanup done / ready to power off" notification to the processing core through the IPCC (Inter-core Communication Controller) receive channel.

[0152] Power-off confirmation: After receiving the notification, the processing core safely closes the rpmsg (remote processor message) channel and releases resources, and finally performs a power-off or reset operation.

[0153] This embodiment ensures that all sent data is acknowledged during shutdown by using a request, cleanup, notification, and power-down sequence, resulting in a clean system state and providing deterministic initial conditions for the next startup.

[0154] The collection core resets the credit value to zero and sends a cleanup completion notification to the processing core via the inter-core communication controller;

[0155] After emptying, the acquisition core resets the credit value tx_credit_pkts to 0 and sends a "cleanup done / ready to power off" confirmation signal to the processing core via the IPCC notification channel. At this point, both the circular buffer and the in-transit queue are empty, with no residual credits.

[0156] After receiving the cleanup completion notification, the processing core executes resource release and channel closure.

[0157] Upon receiving the cleanup completion notification, the processing core safely shuts down the rpmsg channel, releases application resources, and finally performs a power-down or reset operation. This protocol ensures that all sent data batches are acknowledged and the credit value is reset to zero during shutdown, resulting in a clean system state and providing deterministic initial conditions for the next startup.

[0158] Furthermore, the method supports platform portability and performance tuning through compile-time macro configuration, with configuration parameters including: static upper limit of the circular buffer. Number of slots in the in-transit queue Data and control channel separation switch Sampling rate and safety factor ,in A compile-time configuration switch with a value of 0 or 1, used to control whether the data and control channels are physically separated;

[0159] All key parameters are defined as macros in config.h for easy platform portability. For example When data batches and control messages share the same rpmsg virtual serial port, they are distinguished by tlv_t.type; when DUAL_CHANNEL=1, data and control each occupy an independent virtual UART, reducing interference. STATIC_RING_BYTES, PKT_QUEUE_SLOTS, ACQ_RATE_SPS, SAFETY_FACTOR_K, etc., can all be adjusted according to hardware resources (such as the SRAM size of the STM32MP257) and link characteristics.

[0160] When the method runs on a processor platform with data caching, a cache cleanup operation is performed before data batches are sent and a cache invalidation operation is performed after data batches are received to ensure memory consistency.

[0161] On platforms with DCache (data cache) (such as Cortex-A35), before sending, SCB_CleanDCache_by_Addr() (system control block cleans the data cache by address) (macro CACHE_CLEAN()) is called to flush data from the cache into shared memory; after receiving, SCB_InvalidateDCache_by_Addr() (system control block invalidates the data cache by address) (macro CACHE_INVALIDATE()) is called to invalidate the cache, forcing a reload from memory. This avoids data inconsistency issues in multi-core shared memory.

[0162] During runtime, the method periodically records and outputs key metrics for system observation, including: used space and total capacity of the circular buffer, length of the in-transit queue, credit value, estimated round-trip delay, timeout threshold, and stress mode status.

[0163] In some embodiments, the communication task outputs logs every round or every second: STAT: ring=used / cap pq=count inflight=count credit=... rtt=...ms to=...ms. When switching pressure modes, it prints [PACK] pressure ON / OFF (used=... hi=... lo=...). When the ring buffer overflows, it records ring overflow: drop XB. When retransmitting, it prints retransmit: seq=... len=... (timeout=...ms). These logs provide observability support for system debugging, performance analysis, and online tuning.

[0164] Through the coordinated operation of steps S1 to S7 and various refined features, the present invention achieves the following in embodiments (such as the STM32MP257 platform): Figure 7 , Figure 8 , Figure 9 , Figure 10 , Figure 11 , Figure 12 and Figure 13 The excellent performance shown, among which, Figure 7 , Figure 8 , Figure 9 , Figure 10 , Figure 11 , Figure 12 The following are six optional sets of experimental result curves according to embodiments of this application, showing the changes of the received sample rate, duplicate packet rate, and packet loss rate over time under the following scenarios: baseline-1, baseline-2, baseline-3, the complete scheme of this invention, congestion scenario, and acknowledgment loss scenario. Specifically:

[0165] In this experiment, Figure 7 (Baseline-1, no pressure / no credit / fixed timeout) and Figure 8 (Baseline-2, credit backpressure only) Figure 9 (Baseline-3, pressure hysteresis packaging only) serves as a baseline control group, demonstrating the performance limitations when key features of the invention are missing. The complete embodiment of the invention... Figure 10 (Pressure hysteresis + credit backpressure + RTT-EWMA + queue-first retransmission) In a 30-minute test, the average recv_sps (received sample rate) remained stable at 1998 samples / second, with jitter less than 0.1%. , The P95 latency is less than 15ms. Figure 11 (Congestion scenario: ACK delay + small window) and Figure 12 (ACK loss scenario: 5% ACK loss) verified the robustness of the system under extreme conditions. Figure 13 (Statistical comparison table of six groups of 30-minute tests) The summary of the six comparisons shows that the present invention is superior to the baseline scheme in terms of steady-state throughput, jitter suppression, retransmission rate and robustness, which fully verifies the effectiveness and engineering value of the present invention.

[0166] According to another aspect of the embodiments of this application, a heterogeneous multi-core data acquisition and backpressure transmission system is also provided. The system includes:

[0167] The construction module is used to construct a time-sharing pipeline on the acquisition core. The time-sharing pipeline cyclically executes acquisition tasks, capture tasks, packet assembly tasks, and communication tasks.

[0168] The sampling module is used to periodically sample data from the data source during the data collection task and write the samples to an intermediate message queue.

[0169] The monitoring module is used to read samples from the intermediate message queue and write them to the circular buffer during the ingestion task, and to monitor the occupancy of the circular buffer.

[0170] The perception module is used to perform stress-sensing packet assembly during packet assembly tasks.

[0171] The maintenance module is used to maintain the credit value during communication tasks. When the credit value is greater than zero, new data batches are allowed to be sent, and the credit value is deducted for each new data batch sent.

[0172] The control module is used to process the data batch received by the core, and generate a credit control message including the credit adjustment amount and a cumulative confirmation message based on its current buffer level and processing latency index, and send them to the acquisition core.

[0173] The response module is used in communication tasks to receive and respond to credit control messages and cumulative confirmation messages, thereby updating the credit value and releasing the confirmed data batch.

[0174] It should be noted that the embodiments implemented on the heterogeneous multi-core data acquisition and backpressure transmission system side in this application can be referenced to each other with the embodiments implemented on the heterogeneous multi-core data acquisition and backpressure transmission method side, and will not be described in detail in this application.

[0175] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A heterogeneous multi-core data acquisition and backpressure transmission method, characterized in that, include: S1, Construct a time-sharing pipeline on the acquisition core, wherein the time-sharing pipeline cyclically executes acquisition tasks, capture tasks, packet assembly tasks and communication tasks; S2, In the acquisition task, samples are periodically collected from the data source and written to the intermediate message queue; S3, In the ingestion task, samples are read from the intermediate message queue and written to the circular buffer, and the occupancy of the circular buffer is monitored; S4, In the package assembly task, perform pressure-sensing package assembly; S5, in the communication task, maintain a credit value, when the credit value is greater than zero, a new data batch is allowed to be sent, and the credit value is deducted for each new data batch sent; S6, the processing core receives the data batch and, based on its current buffer level and processing latency index, generates a credit control message including a credit adjustment amount and a cumulative confirmation message, and sends them to the acquisition core; S7, in the communication task, receive and respond to the credit control message and the cumulative confirmation message, update the credit value accordingly, and release the confirmed data batch.

2. The heterogeneous multi-core data acquisition and backpressure transmission method according to claim 1, characterized in that, S3 further includes: when the occupancy of the annular buffer reaches or exceeds the first high water level threshold, the annular buffer enters the pressure mode; when the occupancy of the annular buffer drops to or below the first low water level threshold, the annular buffer exits the pressure mode.

3. The heterogeneous multi-core data acquisition and backpressure transmission method according to claim 2, characterized in that, S4, in the packet assembly task, pressure-aware packet assembly is performed, including: when the annular buffer is in the pressure mode, determining the number of packets based on the portion of the occupancy of the annular buffer that exceeds the first low water level threshold; when the annular buffer is not in the pressure mode, determining the number of packets based on the number of available samples and the link upper limit; and encapsulating the packets into a data batch including batch number, first sample timestamp, sample quantity, and sampling period.

4. The heterogeneous multi-core data acquisition and backpressure transmission method according to claim 1, characterized in that, S5 also includes: monitoring the timeout of data batches that have been sent but have not received confirmation, and prioritizing the retransmission of the earliest sent data batch that has timed out; the retransmission operation does not consume the credit value.

5. The heterogeneous multi-core data acquisition and backpressure transmission method according to claim 1, characterized in that, S7 also includes: dynamically adjusting the timeout threshold based on the cumulative confirmed round-trip delay.

6. The heterogeneous multi-core data acquisition and backpressure transmission method according to claim 1, characterized in that, S6, the processing core receives the data batch and, based on its current buffer level and processing latency index, generates a credit control message including a credit adjustment amount and a cumulative confirmation message, including: The processing core triggers the calculation of the credit adjustment amount based on a first preset period or based on an event that a preset number of data batches have been processed. The processing core sets a second high-water threshold and a second low-water threshold for its receiving buffer queue. The second high-water threshold is set to 3 / 4 of the queue capacity, and the second low-water threshold is set to 1 / 4 of the queue capacity to form hysteresis control. When the occupancy of the receiving buffer queue is lower than the second low watermark threshold, the processing core generates the credit control message, wherein the credit adjustment amount is a positive value; When the occupancy of the receiving buffer queue reaches or exceeds the second high water level threshold, the processing core generates the credit control message, wherein the credit adjustment amount is negative, and the absolute value of the negative credit adjustment amount does not exceed the current credit value of the acquisition core. The processing core calculates a timeout recommendation value based on the actual delay of the data batch it processes, and fills the timeout recommendation value into the credit control message.

7. The heterogeneous multi-core data acquisition and backpressure transmission method according to claim 6, characterized in that, The method further includes: the processing core sending a keep-alive probe message with type identifier 0x9F01 to the acquisition core through the control channel at a second preset period, and replying after receiving a response message with type identifier 0x9F02 to maintain session activity; The processing core generates the cumulative confirmation message when the number of data batches arriving reaches a preset threshold, a preset time period is reached, or the occupancy of the receiving buffer queue exceeds a preset alarm threshold. The cumulative confirmation message carries a cumulative confirmation sequence number.

8. The heterogeneous multi-core data acquisition and backpressure transmission method according to claim 7, characterized in that, In S7, receiving and responding to the credit control message and the cumulative confirmation message includes: Upon receiving the credit control message, proceed according to the following formula: Update local credit score, where This is a clamping function, where the preset maximum credit window value is a constant statically configured by the system during initialization. Upon receiving the timeout suggestion value in the credit control message, the timeout threshold maintained by the data collection core is updated to max(timeout threshold, timeout suggestion value); Upon receiving the cumulative confirmation message, release the data batches in the in-transit queue whose batch sequence number is not greater than the cumulative confirmation sequence number; Calculate the round-trip delay sample value based on the sending time of the data batch corresponding to the cumulative confirmation message.

9. The heterogeneous multi-core data acquisition and backpressure transmission method according to claim 1, characterized in that, The capacity of the annular buffer Estimated based on the following formula: Estimate the required number of bytes, then take a value not less than The smallest non-negative integer power of 2 is used to calculate the result, which is then clamped to a value not exceeding the maximum capacity of a preset circular buffer. ; in, The preset sampling rate per second, The preset length of a single sample byte. The preset worst-case link latency, The estimation result is based on a preset safety factor greater than 1. The unit is bytes.

10. A heterogeneous multi-core data acquisition and backpressure transmission system, wherein the system implements the heterogeneous multi-core data acquisition and backpressure transmission method as described in claim 1, characterized in that, include: The construction module is used to construct a time-sharing pipeline on the acquisition core, wherein the time-sharing pipeline cyclically executes acquisition tasks, capture tasks, packet assembly tasks and communication tasks; The sampling module is used to periodically sample from the data source and write the samples to an intermediate message queue during the acquisition task. The monitoring module is used to read samples from the intermediate message queue and write them into the circular buffer during the ingestion task, and to monitor the occupancy of the circular buffer. The sensing module is used to perform pressure sensing and packet assembly in the packet assembly task; A maintenance module is used to maintain a credit value in the communication task. When the credit value is greater than zero, a new data batch is allowed to be sent, and the credit value is deducted for each new data batch sent. The control module is used to process the data batch received by the core, and generate a credit control message including a credit adjustment amount and a cumulative confirmation message based on its current buffer level and processing latency index, and send them to the acquisition core. The response module is used in the communication task to receive and respond to the credit control message and the cumulative confirmation message, thereby updating the credit value and releasing the confirmed data batch.

Images