A method and apparatus for multi-channel data aggregation

Abstract

The application relates to the technical field of network communication, and provides a multi-channel data aggregation method and device, which comprises the following steps: in the case that a shared cache space is detected to be in an idle state, sending a write authorization signal to a target input channel in a polling sequence, so that the target input channel exclusively occupies the shared cache space to perform data writing; in the case that the target input channel completes data writing or no data writing action of the target input channel is detected within a preset timeout time, switching the polling sequence to a next input channel; in parallel with the write authorization and the channel switching, in the case that it is detected that there is cache data in the shared cache space and a back-end circuit is in a ready state, outputting the cache data and releasing the shared cache space. The application replaces independent buffer zones with a single shared cache, greatly reduces storage resource consumption, and guarantees system throughput and time delay performance.

Description

Technical Field

[0001] This application relates to the field of network communication technology, specifically to a multi-channel data aggregation method and apparatus. Background Technology

[0002] In network communication and data processing systems, multi-channel data aggregation is a key component of link convergence and traffic scheduling.

[0003] When designing based on FPGA (Field Programmable Gate Array), existing technologies typically employ an architecture of "independent buffer per channel + output arbitration," such as... Figure 2 As shown, the common multi-channel data aggregation structure includes N front-end circuits, internal buffer processing logic, and one back-end circuit. The internal buffer processing logic includes N independent data buffers and an output control module. Each front-end circuit corresponds to an independent buffer. Data is first written to its respective buffer, and then the output control module stores the data according to the data carried by each front-end circuit under the control of the ready signal in its respective buffer. Finally, the output control module reads the data from each buffer in turn according to a certain arbitration strategy and outputs it to the back-end circuit for aggregation. However, with the increase in communication speed and the number of channels, the shortcomings of this traditional architecture become increasingly apparent: on the one hand, storage resource consumption increases linearly with the number of channels. When the number of channels is large, it will occupy a large amount of hardware storage resources, leading to a significant increase in system cost. For example, 32-channel aggregation requires 32 independent FIFO buffers, occupying a large amount of FPGA Block RAM resources. On the other hand, the arbitration logic between multiple buffers is complex, easily causing timing convergence difficulties and data interleaving problems, increasing the difficulty of system debugging and maintenance.

[0004] Therefore, there is an urgent need for a multi-channel data aggregation solution that can significantly reduce storage resource consumption while ensuring system throughput and latency performance. Summary of the Invention

[0005] To address the problems of high storage resource consumption and complex arbitration logic in existing multi-channel data aggregation technologies, this application provides a multi-channel data aggregation method, apparatus, electronic device, and storage medium that achieves low-resource-consumption data aggregation within a single shared cache space.

[0006] In a first aspect, this application provides a multi-channel data aggregation method, comprising: a write authorization step, wherein when a shared buffer space is detected to be idle, a write authorization signal is sent to a target input channel in a polling order, so that the target input channel exclusively occupies the shared buffer space for data writing; a channel switching step, wherein when the target input channel completes data writing, or when no data writing action is detected in the target input channel within a preset timeout period, the polling order is switched to the next input channel; and a data output step, which is executed in parallel with the write authorization step and the channel switching step, wherein when cached data is detected in the shared buffer space and the subsequent circuit is in a ready state, the cached data is output and the shared buffer space is released.

[0007] As an optional implementation, the step of completing data writing in the target input channel includes: determining that the target input channel has completed data writing when the target input channel has completed writing a complete frame of data.

[0008] As an optional implementation, the step of switching the polling order to the next input channel when no data write action is detected in the target input channel within a preset timeout period includes: starting a timer after sending the write authorization signal to the target input channel; if no data write action is detected in the target input channel within the preset timeout period, canceling the write authorization signal and switching the polling order to the next input channel.

[0009] As an optional implementation, the preset timeout time is determined by: obtaining the configuration value of the internal register; and dynamically determining the preset timeout time based on the configuration value.

[0010] As an optional implementation, the storage capacity of the shared cache space is greater than or equal to the maximum length of a single frame of data, so as to support the complete storage of a frame of input data.

[0011] As an optional implementation, the step of outputting the cached data and releasing the shared cache space includes: when the subsequent circuit is detected to be in a ready state and the cached data exists in the shared cache space, outputting the cached data completely to the subsequent circuit; and gradually releasing the storage space of the shared cache space during the output of the cached data.

[0012] As an optional implementation, the shared cache space further includes a reserved storage area, and the method further includes: when the subsequent circuit generates a reverse voltage signal, using the reserved storage area to cache the data input by the preceding circuit.

[0013] Secondly, this application also provides a multi-channel data aggregation device, comprising: an authorization sending module, configured to send write authorization signals to a target input channel in a polling order when the shared buffer space is detected to be idle, so that the target input channel exclusively occupies the shared buffer space for data writing; a channel switching module, configured to switch the polling order to the next input channel when the target input channel completes data writing or when no data writing action is detected in the target input channel within a preset timeout period; and an output control module, which works in parallel with the authorization sending module and the channel switching module, configured to output the cached data and release the shared buffer space when cached data is detected in the shared buffer space and the subsequent circuit is in a ready state.

[0014] Thirdly, this application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the multi-channel data aggregation method as described above.

[0015] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-channel data aggregation method as described above.

[0016] Beneficial effects: The multi-channel data aggregation method, apparatus, electronic device, and storage medium provided in this application replace the traditional N independent buffers with a single shared cache space. Combined with a polling write authorization mechanism triggered by idle state, this ensures that storage resource consumption no longer increases linearly with the number of channels, but depends only on the maximum length of a single frame of data, thus achieving extreme compression of storage resources at the physical level. Simultaneously, the synergistic effect of the dual channel switching conditions of "write complete" and "timeout skip" guarantees complete data frame transmission and boundary alignment, avoiding data interleaving and reassembly overhead, and effectively prevents system congestion caused by single-channel silence, ensuring determinism and low latency in the aggregation process. Furthermore, the parallel pipeline design for data writing and output, along with the reserved backpressure buffer mechanism, further guarantees the stability and continuity of the system under complex operating conditions such as high throughput and burst traffic, achieving optimal overall resource consumption, processing performance, and system robustness. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a multi-channel data aggregation method provided by the present invention; Figure 2 This is a schematic diagram of a common multi-channel data aggregation structure in existing technologies; Figure 3 This is a schematic diagram of the low-resource-consumption multi-channel data aggregation structure provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0021] Figure 1 This is a flowchart illustrating a multi-channel data aggregation method provided by the present invention, as shown below. Figure 1 As shown, the method mainly includes a write authorization step, a channel switching step, and a data output step. Through the coordinated operation of these three steps, efficient data aggregation with low resource consumption is achieved.

[0022] In step S100, the write authorization step, when the shared buffer space is detected to be in an idle state, a write authorization signal is sent to the target input channel in a polling order so that the target input channel can exclusively occupy the shared buffer space for data writing.

[0023] Specifically, an "idle state" is a necessary prerequisite for triggering write authorization. Its core technical significance lies in physically ensuring that only one channel can access the shared buffer space at any given time, thus completely avoiding the interleaving and mixing of data from multiple channels within the buffer. For example, when the system contains N front-end circuits, the control logic will iterate through these N channels, but will only issue permission to the currently polled target input channel when there is no residual data in the shared buffer space and it is not occupied by other channels. This exclusive write mechanism eliminates the need for complex frame reassembly or deinterleaving operations when the subsequent circuits read data, greatly simplifying the data processing logic. It should be understood that the target input channel can be any front-end circuit capable of sending data; its specific number and type are not limited to a particular scenario, as long as the polling and idle triggering mechanism described above is followed.

[0024] In step S200, the channel switching step, if the target input channel completes data writing or if no data writing action is detected in the target input channel within a preset timeout period, the polling order will be switched to the next input channel.

[0025] This step establishes two parallel channel switching conditions, which together ensure the system's determinism and anti-blocking capabilities. On the one hand, "write complete" serves as a normal switching condition, ensuring that each authorization can carry a valid data transmission task; on the other hand, "timeout without write" serves as an anomaly protection condition, effectively identifying and skipping channels that, although polled, do not actually send data or respond abnormally. For example, if a front-end circuit fails to start writing after being authorized due to a fault or temporary silence, the system will not wait indefinitely but will immediately revoke the authorization and switch to the next channel after the preset time window closes. This dual-trigger mechanism avoids the problem of deadlock or increased latency in the entire aggregation link due to a single channel failure, as seen in traditional polling schemes, ensuring the continuity and real-time performance of the data stream.

[0026] In step S300, the data output step, it is executed in parallel with the write authorization step and the channel switching step. When cached data is detected in the shared cache space and the subsequent circuit is in a ready state, the cached data is output and the shared cache space is released.

[0027] It is important to emphasize that this step does not wait for the entire write process to complete before starting serially; rather, it operates in parallel with the write and switching processes on the timeline. Specifically, once valid data has been written to the shared cache and the subsequent circuitry is ready to receive it, the output logic immediately begins data transmission, regardless of which channel is currently writing or whether a channel switch has occurred. For example, while the current channel is writing data in the latter half of its phase, the previously written data may have already begun to be transmitted to the subsequent circuitry; or, during a channel switch, the output operation continues. This pipelined parallel processing mechanism fully utilizes the turnover efficiency of the shared cache, so that the throughput of data aggregation is no longer limited by the write speed or switching overhead of a single channel. This significantly reduces storage resource consumption while maintaining system performance comparable to or even better than traditional multi-cache architectures. Simultaneously, as data is gradually output, the shared cache is dynamically released and returns to an idle state, creating conditions for the next write authorization, forming an efficient closed loop.

[0028] Based on the above embodiments, as an optional embodiment, the multi-channel data aggregation method provided by the present invention determines that the target input channel has completed data writing when the target input channel has completed writing a complete frame of data.

[0029] Here, "a complete data frame" refers to the smallest transmission unit that conforms to the preset communication protocol specifications and has independent semantics. It typically includes complete structural information such as frame start identifier, payload, and frame end identifier. Strictly limiting the trigger condition for channel switching to "complete data storage of a single frame" is of core technical significance in ensuring that the data stored in the shared buffer space always exists with complete protocol frames as boundaries.

[0030] Based on the above embodiments, as an optional embodiment, the multi-channel data aggregation method provided by the present invention, in the case that no data write action is detected in the target input channel within a preset timeout period, switches the polling order to the next input channel, including: starting a timer after sending the write authorization signal to the target input channel; if no data write action is detected in the target input channel within the preset timeout period, canceling the write authorization signal and switching the polling order to the next input channel.

[0031] It is important to emphasize here that the timing starts "after the write authorization signal is sent," rather than at the start of the polling cycle or when a data request is detected on a channel. This design accurately distinguishes between "normal waiting without being polled" and "abnormal silence due to authorization but no response." For example, if a front-end circuit is unable to initiate data transmission after receiving a write authorization signal due to hardware failure, link interruption, or internal logic deadlock, the system will not enter an indefinite, invalid wait. Instead, it will immediately execute a cancellation action after the preset time window closes. This cancellation action can specifically manifest as setting the authorization signal to an invalid level, resetting the write pointer of the current channel, or clearing the relevant status flags, and then immediately incrementing the polling pointer to the next channel. This mechanism effectively solves the problem of the entire aggregation link being blocked or experiencing a sharp increase in latency due to a single channel "false death" in traditional polling schemes, ensuring the determinism and real-time response capability of the system under non-ideal operating conditions.

[0032] Furthermore, to enhance the adaptability of the solution to different application environments, the preset timeout is not a fixed constant in the hardware logic, but is dynamically determined through the following methods: obtaining the configuration value of the internal register; and dynamically determining the preset timeout based on the configuration value. In practical engineering applications, different front-end circuits may have different response speeds, data generation frequencies, or link transmission characteristics. For example, during the system debugging phase, engineers can flexibly adjust the timeout threshold by modifying the value of the internal register to quickly locate whether the slow response of the front-end is due to a slow response or a true channel failure; during normal operation, this value can be optimized according to the business load, minimizing the occupancy time of abnormal channels while ensuring that normal slow channels are not misjudged, thus achieving the best balance between waiting efficiency and the risk of misjudgment. This hardware and software co-configuration method enables the same hardware architecture to seamlessly adapt to front-end devices of various rate levels and protocol types, significantly improving the reusability and lifecycle value of the technical solution.

[0033] Based on the above embodiments, as an optional embodiment, the multi-channel data aggregation method provided by the present invention has a shared cache space storage capacity greater than or equal to the maximum length of a single frame of data, so as to support the complete storage of a frame of input data.

[0034] For example, if the maximum length of a standard Ethernet frame processed by the system is 1518 bytes, the physical capacity of the shared buffer space only needs to meet this length, without needing to multiply by the number of channels N. In contrast, existing technologies for 32-channel aggregation typically require 32 independent FIFO buffers, with storage resources increasing linearly with the number of channels. This application, however, reduces storage requirements from being related to the number of channels to being related to the length of a single frame, significantly reducing storage resource consumption. More importantly, only when the shared buffer space can hold at least one complete frame of data can it be guaranteed that the authorized target input channel can complete data transmission without overflow or fragmentation at any given time. This completely avoids cross-buffer reassembly logic caused by data truncation, ensuring the atomicity and determinism of data processing.

[0035] Furthermore, to improve the turnover efficiency of the shared buffer space, this application employs a dynamic space release mechanism. Specifically, this includes: when the subsequent circuit is detected to be in a ready state and buffered data exists in the shared buffer space, the buffered data is completely output to the subsequent circuit; during the output of buffered data, the storage space of the shared buffer space is gradually released. This "gradual release" differs from the traditional "complete clearing" mode; its core lies in the real-time updating of the space state. For example, this can be achieved by maintaining a read pointer and a write pointer: whenever the subsequent circuit successfully reads and confirms the reception of partial data, the read pointer immediately moves forward, and the storage unit traversed by the read pointer is immediately marked as free and available, without waiting for the entire frame of data to be transmitted. This pipelined operation of outputting and releasing simultaneously allows the area just vacated in the shared buffer space to be immediately reused by the next authorized target input channel. For example, while the latter half of the current frame data is still being transmitted to the subsequent stage, the space occupied by the first half may have already been overwritten by a new write operation. This mechanism significantly shortens the waiting window for subsequent channels, effectively alleviating the problem that a single shared cache may become a bottleneck in high-load scenarios, enabling the system to maintain throughput performance close to that of a multi-buffer architecture even with only one physical buffer.

[0036] Based on the above embodiments, as an optional embodiment, the present invention also provides a multi-channel data aggregation method, wherein the shared cache space further includes a reserved storage area, and the method further includes: when the subsequent circuit generates a reverse voltage signal, using the reserved storage area to cache the data input by the preceding circuit.

[0037] The reserved storage area can be an independent address range partitioned from the shared cache space, or it can be logically related but physically adjacent extended storage units. When the downstream circuit is temporarily unable to receive data and send a backpressure signal due to congestion, bus arbitration failure, or busy downstream links, if the upstream circuit is in the process of writing at this time, the system will not directly discard the data or forcibly block the upstream circuit, but will allow the data to continue to be written to the reserved storage area.

[0038] To better illustrate the solution of the present invention, a specific example is provided below.

[0039] like Figure 3 As shown, this embodiment provides a low-resource-consumption multi-channel data aggregation structure, including N front-end circuits, internal cache processing logic, and back-end circuits. The internal cache processing logic consists of a shared data buffer and a polling ready control module.

[0040] The specific workflow of this embodiment is as follows: (1) Initialization phase: The polling ready control module initializes the polling pointer to the first input channel, and the shared data buffer is in an idle state.

[0041] (2) Data receiving stage: a. The polling-based ready control module detects that the shared data buffer is free and sends a ready signal to the first input channel; b. If the first pre-amplifier circuit needs to output data, after detecting the ready signal, write a frame of data into the shared data buffer; after writing a frame of data, the polling pointer is automatically incremented by 1, pointing to the second input channel; c. Repeat the above process, iterating through all N input channels and then polling the pointer back to channel 1 to enter the next loop; d. If no data is written to a certain input channel within M clock cycles (M is the timeout period), skip that channel and poll the next channel.

[0042] (3) Data output stage: The output data stream logic detects the ready signal of the subsequent circuit in real time; when there is data in the shared data buffer and the subsequent circuit is ready, the data in the buffer is completely output to the subsequent circuit; during the data output process, the shared data buffer will be gradually released, waiting for the next data write.

[0043] In this embodiment, the shared data buffer has a data width of 64 bits and a depth of 512, with a maximum storage length of 2048 bytes for a single frame of data. An additional 2048 bytes are reserved for buffering the preceding circuit in case of sudden backpressure in the subsequent circuit, ensuring pipelined data processing. The polling order is 1→2→...→N→1, and the maximum waiting latency for each data stream is N×(T+M), where T is the transmission time of a single frame of data. In a 32-channel data aggregation scenario, this design consumes only 3.125% of storage resources compared to traditional solutions, while increasing logic resource consumption by approximately 5%. The system throughput and latency performance are comparable to traditional solutions, fully meeting the requirements of engineering applications.

[0044] As an optional embodiment, the present invention also provides a multi-channel data aggregation device, comprising: The authorization sending module is used to send write authorization signals to the target input channel in a polling order when the shared buffer space is detected to be idle, so that the target input channel can exclusively occupy the shared buffer space for data writing; The channel switching module is used to switch the polling order to the next input channel when the data writing to the target input channel is completed, or when no data writing action is detected in the target input channel within a preset timeout period. The output control module, which works in parallel with the authorization sending module and the channel switching module, is used to output the cached data and release the shared cache space when it is detected that there is cached data in the shared cache space and the subsequent circuit is in a ready state.

[0045] It should be noted that the multi-channel data aggregation device provided in this embodiment of the invention can execute the multi-channel data aggregation method described in any of the above embodiments during specific operation, and this embodiment will not elaborate on this.

[0046] As an optional embodiment, this embodiment provides an electronic device including at least one processor and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the multi-channel data aggregation method described in any of the foregoing embodiments.

[0047] As an optional embodiment, this embodiment also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the multi-channel data aggregation method as described in any of the foregoing embodiments. Specifically, the computer-readable storage medium can be a tangible, non-transitory storage medium, including but not limited to disks (such as floppy disks, hard disks), optical disks (such as CD-ROMs, DVDs), and semiconductor memories (such as ROMs, EEPROMs, USB flash drives, memory cards), etc. This storage medium, as an independent carrier of the technical solution, allows the logic of the multi-channel data aggregation method to exist independently of specific hardware design documents, facilitating its distribution, sale, and deployment in the form of software products, firmware update packages, or drivers.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-channel data aggregation method, characterized in that, include: In the write authorization step, if the shared cache space is detected to be idle, a write authorization signal is sent to the target input channel in a polling order so that the target input channel exclusively occupies the shared cache space for data writing; The channel switching step involves switching the polling order to the next input channel if the target input channel completes data writing or if no data writing action is detected in the target input channel within a preset timeout period. The data output step is executed in parallel with the write authorization step and the channel switching step. When cached data is detected in the shared cache space and the subsequent circuit is in a ready state, the cached data is output and the shared cache space is released.

2. The multi-channel data aggregation method according to claim 1, characterized in that, The process of writing data to the target input channel includes: If the target input channel completes the writing of a complete frame of data, it is determined that the target input channel has completed the data writing.

3. The multi-channel data aggregation method according to claim 2, characterized in that, If no data write operation is detected on the target input channel within a preset timeout period, the polling order is switched to the next input channel, including: The timing is started after the write authorization signal is sent to the target input channel; If no data write operation is detected on the target input channel within the preset timeout period, the write authorization signal is revoked and the polling order is switched to the next input channel.

4. The multi-channel data aggregation method according to claim 3, characterized in that, The preset timeout period is determined in the following way: Retrieve the configuration values ​​of internal registers; The preset timeout period is dynamically determined based on the configuration value.

5. The multi-channel data aggregation method according to claim 1, characterized in that, The storage capacity of the shared cache space is greater than or equal to the maximum length of a single frame of data, so as to support the complete storage of a frame of input data.

6. The multi-channel data aggregation method according to claim 5, characterized in that, The step of outputting the cached data and releasing the shared cache space includes: If the downstream circuit is detected to be in a ready state and the cached data exists in the shared cache space, the cached data will be completely output to the downstream circuit. During the process of outputting the cached data, the storage space of the shared cache space is gradually released.

7. The multi-channel data aggregation method according to claim 6, characterized in that, The shared cache space also includes a reserved storage area, and the method further includes: When the subsequent circuit generates a reverse voltage signal, the reserved storage area is used to buffer the data input from the previous circuit.

8. A multi-channel data aggregation device, characterized in that, include: The authorization sending module is used to send write authorization signals to the target input channel in a polling order when the shared buffer space is detected to be idle, so that the target input channel can exclusively occupy the shared buffer space for data writing; The channel switching module is used to switch the polling order to the next input channel when the data writing to the target input channel is completed, or when no data writing action is detected in the target input channel within a preset timeout period. The output control module, which works in parallel with the authorization sending module and the channel switching module, is used to output the cached data and release the shared cache space when it is detected that there is cached data in the shared cache space and the subsequent circuit is in a ready state.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the multichannel data aggregation method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the multi-channel data aggregation method as described in any one of claims 1 to 7.

Images