Timestamp-based fpga data slicing processing method and device
By introducing timestamp identifiers and two-dimensional cache arrays into FPGA data slicing processing, combined with parallel processing unit arrays and state trackers, the problems of data boundary continuity and result reorganization are solved, thereby improving the performance and reliability of FPGA data processing.
Patent Information
- Application Number
- CN202511136149.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-14
- Publication Date
- 2026-01-09
- Estimated Expiration
- 2045-08-14
AI Technical Summary
Existing FPGA data slicing methods have shortcomings in terms of data boundary continuity, parallel processing, and result reassembly, which affect the integrity and efficiency of the processing results.
A timestamp-based FPGA data slicing processing method is adopted. By allocating timestamp identifiers in dynamic random access memory, a two-dimensional cache array is constructed. Parallel processing unit arrays and state trackers are used to achieve secure data storage and efficient access, ensuring the integrity and accuracy of data processing.
It significantly improves the performance and reliability of FPGA data processing, solves the shortcomings in slicing, parallel computing and result reassembly, and achieves efficient data processing.
Smart Images

Figure CN120723797B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of data processing, specifically to a timestamp-based FPGA data slicing processing method and device. BACKGROUND
[0002] The existing FPGA data slicing processing method has obvious deficiencies. The traditional system lacks systematicness in data slicing and cannot effectively handle the continuity problem of data boundaries, affecting the integrity of the processing result.
[0003] In addition, the existing technology has bottlenecks in parallel processing. Most systems fail to fully utilize the hardware characteristics of FPGA, lack timestamp-based scheduling mechanisms and state tracking strategies, resulting in insufficient processing efficiency.
[0004] The existing system has technical shortcomings in result reorganization. It lacks the ability to arrange data sub-blocks in order and cannot accurately splice through position indexing, affecting the correctness of the output result. Solving these problems is of great significance to improving the performance of FPGA processing. SUMMARY
[0005] To solve the problems in the prior art, the present application provides a timestamp-based FPGA data slicing processing method and device, which can effectively solve the deficiencies of traditional technology in slicing processing, parallel computing and result reorganization, and significantly improve the performance and reliability of FPGA data processing.
[0006] To solve at least one of the above problems, the present application provides the following technical solutions:
[0007] In a first aspect, the present application provides a timestamp-based FPGA data slicing processing method, comprising:
[0008] Collecting data to be processed, transmitting the data to be processed through a PCIe link to a dynamic random access memory of a field programmable gate array, assigning a timestamp identifier to the data to be processed, dividing the data to be processed into blocks according to a preset slicing size, adding an overlap region at the boundary of each data block to maintain data continuity, constructing a two-dimensional cache array, writing each data block into the corresponding position of the two-dimensional cache array according to the timestamp identifier, and generating a plurality of data sub-blocks with position indexes;
[0009] Constructing a processing unit array, the processing unit array comprising a plurality of parallel processing units and a scheduling controller, a linear cache and a sliding window are provided in each parallel processing unit, the data sub-blocks are distributed to idle parallel processing units through the scheduling controller, the linear cache is used to cache adjacent data rows and construct the sliding window, convolution operation is performed and the processing result is outputted;
[0010] A state tracker is arranged to monitor the processing status of the data sub-blocks, the state tracker records a processing completion flag for each data sub-block, sorts the data sub-blocks that have completed processing according to the timestamp identifiers, detects whether there is a missing or misplacement of the data sub-blocks, writes the sorted data sub-blocks into a result cache area, reads out the data sub-blocks from the result cache area according to the position index and splices them to generate a complete processing result.
[0011] Further, the method further comprises: allocating a physically continuous memory region as a direct memory access buffer through a host end of a PCIe link, setting a direct memory access descriptor queue in a shared memory, mapping the shared memory to a user space, configuring a PCIe device driver to enable an interrupt response, loading PCIe intellectual property core configuration information, and establishing an AXI bus-based data transmission channel on the PCIe link.
[0012] The PCIe link is configured to be a third-generation 8-channel specification, the to-be-processed data is written into the direct memory access buffer, the to-be-processed data is transmitted to a dynamic random access memory of a field programmable gate array through the PCIe link, an incremental timestamp identifier is generated for the to-be-processed data based on a time counter, and the to-be-processed data is divided into a plurality of data blocks according to a preset slicing size.
[0013] Further, the method further comprises: creating a double buffering structure in a two-dimensional cache array, padding and extending the data block boundary, copying the boundary pixels outward to form an overlapping area, calculating a two-dimensional index value according to the row and column positions of the data block in the original data, the two-dimensional index value comprising a row index and a column index, and constructing a position index identifier based on the row index and the column index.
[0014] The data blocks are written into the two-dimensional cache array in the order of the timestamp identifiers, the storage position of each data block is determined by the position index identifier, a data block state flag bit is set in the double buffering structure to record the writing state of the data block, whether the data block has completed writing is determined according to the data block state flag bit, and a data sub-block with the position index identifier is generated.
[0015] Further, the method further comprises: dividing a processing unit region in a programmable logic unit of a field programmable gate array, constructing a plurality of parallel processing units in the processing unit region, allocating a block random access memory resource for each parallel processing unit, the block random access memory resource being used to construct a linear cache, and configuring the depth of the linear cache to match the row length of the data block.
[0016] A global scheduling controller is constructed, which includes a task allocation module and a state monitoring module, the task allocation module maintains a queue of idle processing units, and the state monitoring module records the working state of each parallel processing unit. A data window generation circuit is configured in each parallel processing unit, which constructs a sliding window with a fixed size based on the linear cache.
[0017] Further, a processing unit state table is established to record the working state of each parallel processing unit, and the scheduling controller reads the processing unit state table to obtain the identification information of idle parallel processing units, allocates the data sub-blocks to the corresponding idle parallel processing units, establishes a prefetch buffer for the data sub-blocks in the local fast cache, and preloads the next data sub-block to be processed into the prefetch buffer.
[0018] A plurality of data row buffer areas are allocated in the linear cache, adjacent data rows of the data sub-blocks are sequentially written into the data row buffer areas, an N×N size sliding window matrix is constructed based on the data row buffer areas, the sliding window matrix moves pixel by pixel on the data sub-blocks, convolution calculation is performed on the data in the sliding window matrix to obtain a processing result, and the processing result is written into a result cache.
[0019] Further, a state tracking table is constructed to record the processing state information of the data sub-blocks, the state tracking table includes data sub-block identification, timestamp identification, processing unit identification, and processing state flag, the state tracker periodically scans the state tracking table to obtain the processing progress of each data sub-block, updates the processing state flag when detecting that the data sub-block processing is completed, and counts the number of data sub-blocks that have completed processing.
[0020] A sorting buffer is established to store the data sub-blocks that have completed processing, the data sub-blocks in the sorting buffer are sorted in ascending order according to the timestamp identification, the sorted data sub-blocks are written into a ready queue, the state tracker maintains the read-write pointers of the ready queue, records the number of data sub-blocks available for reading in the queue, and sends a data ready signal to a data aggregation module.
[0021] Further, a data integrity check table is established based on the timestamp identification and position index, whether the timestamp identifications of adjacent data sub-blocks are continuous is checked, whether the position index of the data sub-blocks conforms to the expected arrangement order is verified, a reprocessing request is initiated to the corresponding processing unit when a data sub-block is detected to be missing, and the mispositioned data sub-blocks are reordered according to their position index.
[0022] A plurality of cache blocks are divided in the result cache area for temporarily storing the sorted data sub-blocks, the size of the cache blocks matches the size of the data sub-blocks, the storage positions of the data sub-blocks in the result cache area are determined according to the position indexes, the data sub-blocks are written into the corresponding cache blocks, the overlapping area at the boundary of the data sub-blocks is removed, and the effective data of the adjacent data sub-blocks are spliced to obtain the complete processing result.
[0023] In a second aspect, the application provides a timestamp-based FPGA data slicing processing device, comprising:
[0024] The data blocking module is configured to collect to-be-processed data, transmit the to-be-processed data to a dynamic random access memory of an FPGA through a PCIe link, assign a timestamp identifier to the to-be-processed data, block the to-be-processed data according to a preset slicing size, add an overlapping area at the boundary of each data block for maintaining data continuity, construct a two-dimensional cache array, write each data block into a corresponding position of the two-dimensional cache array according to the timestamp identifier, and generate a plurality of data sub-blocks with position indexes.
[0025] The array construction module is configured to construct a processing unit array, wherein the processing unit array comprises a plurality of parallel processing units and a scheduling controller, a linear cache and a sliding window are arranged in each parallel processing unit, the data sub-blocks are distributed to idle parallel processing units through the scheduling controller, the linear cache is used to cache adjacent data rows and construct the sliding window, convolution operation is performed, and a processing result is output.
[0026] The data cache module is configured to set a state tracker to monitor the processing state of the data sub-blocks, the state tracker records a processing completion flag of each data sub-block, sorts the data sub-blocks that have completed processing according to the timestamp identifier, detects whether the data sub-blocks are missing or misaligned, writes the sorted data sub-blocks into a result cache area, reads out the data sub-blocks from the result cache area according to the position indexes and splices them to generate a complete processing result.
[0027] In a third aspect, the application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the timestamp-based FPGA data slicing processing method when executing the program.
[0028] In a fourth aspect, the application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the steps of the timestamp-based FPGA data slicing processing method.
[0029] In a fifth aspect, the present application provides a computer program product comprising computer programs / instructions which, when executed by a processor, implement the steps of the timestamp-based FPGA data slicing processing method.
[0030] From the above technical solution, the present application provides a timestamp-based FPGA data slicing processing method and device. By innovatively constructing a timestamp identification mechanism, through the data organization of dynamic random access memory and a two-dimensional cache array, safe storage and efficient access of data are achieved. A parallel processing unit array architecture is designed, combined with linear caching and sliding window technology, to establish an efficient data processing strategy. A state tracking mechanism is introduced to ensure the integrity and accuracy of data processing through processing state monitoring and ordering splicing. Based on safe data storage, this method effectively solves the deficiencies of traditional technologies in slicing processing, parallel computing and result reorganization, significantly improving the performance and reliability of FPGA data processing. BRIEF DESCRIPTION OF DRAWINGS
[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.
[0032] Figure 1 A flowchart of the timestamp-based FPGA data slicing processing method in the embodiments of the present application;
[0033] Figure 2 A structure diagram of the timestamp-based FPGA data slicing processing device in the embodiments of the present application;
[0034] Figure 3 A structure diagram of the electronic device in the embodiments of the present application.
[0035] Reference signs:
[0036] Electronic device 9600, central processor 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION
[0037] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.
[0038] The acquisition, storage, use, processing and the like of data in the technical solutions of the present application comply with relevant provisions of laws and regulations.
[0039] In view of the problems in the prior art, the present application provides a timestamp-based FPGA data slicing processing method and device, which innovatively constructs a timestamp identification mechanism, organizes data through a dynamic random access memory and a two-dimensional cache array, realizes safe storage and efficient access of data, designs a parallel processing unit array architecture, combines linear caching and sliding window technology, establishes an efficient data processing strategy, introduces a state tracking mechanism, and ensures the integrity and accuracy of data processing through processing state monitoring and ordering splicing. On the basis of safe storage of data, the method effectively solves the deficiencies of traditional technologies in slicing processing, parallel computing and result recombination, and significantly improves the performance and reliability of FPGA data processing.
[0040] In order to effectively solve the deficiencies of traditional technologies in slicing processing, parallel computing and result recombination, and significantly improve the performance and reliability of FPGA data processing, an embodiment of a timestamp-based FPGA data slicing processing method is provided in the present application, as shown in Figure 1 , which specifically includes the following contents:
[0041] Step S101: collecting data to be processed, transmitting the data to be processed to a dynamic random access memory of a field programmable gate array through a PCIe link, allocating a timestamp identifier for the data to be processed, dividing the data to be processed according to a preset slicing size, adding an overlapping area at the boundary of each data block to maintain data continuity, constructing a two-dimensional cache array, writing each data block into the corresponding position of the two-dimensional cache array according to the timestamp identifier, and generating a plurality of data sub-blocks with position indexes;
[0042] Optionally, the embodiment is directed to the low efficiency of data transmission, complex cache management and discontinuous data segmentation in large-scale real-time data processing, and innovatively designs a set of efficient data preprocessing scheme based on FPGA. In the aspect of data slice evaluation, the embodiment designs a slice quality scoring formula: Slice_Quality = α × (Block_Size / Max_Size) + β × (Overlap_Ratio) + γ × (Memory_Efficiency), wherein α, β, γ are weight coefficients, respectively representing the influence of data block size, overlap area ratio and memory utilization efficiency. At the same time, the cache efficiency evaluation formula: Cache_Score= (Access_Speed × Data_Locality) / (Memory_Usage + Access_Conflicts) is introduced to optimize the data storage strategy.
[0043] The embodiment deeply optimizes the data transmission mechanism. In the PCIe link transmission process, the system adopts a multi-queue DMA transmission strategy, and by reasonably configuring the transmission descriptor and interrupt aggregation parameters, the bus bandwidth utilization is maximized. For different types of data to be processed, the system will dynamically adjust the transmission parameters according to the data characteristics. For example, for time series data with high continuity requirements, the system will use larger transmission packet size and more frequent synchronization checks; for discrete data, a more flexible transmission strategy will be selected. After the data arrives at the dynamic random access memory of the FPGA, the system will immediately perform data integrity verification to ensure that no data loss or errors occur during transmission. This optimized transmission mechanism significantly improves the efficiency of data preprocessing.
[0044] The embodiment innovatively realizes the timestamp allocation mechanism. The system uses a high-precision clock source to generate timestamps, and completes the identification allocation at the first time of data reception. The timestamp not only contains accurate time information, but also carries metadata such as data source identification and processing priority. This rich timestamp information provides an important basis for subsequent data organization and processing. Especially when processing multi-source data streams, the system can accurately reconstruct the time sequence relationship of the data according to the timestamp information, ensuring the time consistency of the processing result. For possible clock deviation, the system will correct it through regular clock synchronization mechanism to maintain the accuracy of the timestamp.
[0045] The embodiment realizes efficient data blocking through intelligent data slicing strategy. The system dynamically adjusts the slicing size according to the hardware resource characteristics of FPGA and the requirements of processing algorithms. At the data block boundary, the system innovatively introduces adaptive overlapping region design. The size of the overlapping region is optimized according to the continuity characteristics of the data and the requirements of the processing algorithms. For example, for data requiring convolution operation, the system sets appropriate overlapping region according to the convolution kernel size; for data requiring spectrum analysis, the frequency resolution requirement is considered. This flexible slicing strategy ensures the continuity and accuracy of data processing.
[0046] The embodiment establishes an efficient two-dimensional cache management framework. In the organization process of the cache array, the system adopts a double buffering mechanism to realize parallel operation of data reading and writing. The row and column division of the cache array fully considers the memory architecture characteristics of FPGA, and maximally reduces access conflicts through optimized access mode. For each data block, the system generates a unique location index for quick positioning and access. This structured cache management mechanism significantly improves the efficiency of data access. Especially when processing large-scale data, the system can maintain stable processing performance.
[0047] The embodiment provides an innovative solution for data preprocessing. Through optimized data transmission and cache management, the system can efficiently process various data streams. This intelligent preprocessing mechanism significantly improves the efficiency and reliability of subsequent algorithm processing.
[0048] The embodiment realizes parallel processing through sub-block generation. The system generates complete metadata information for each data sub-block, including location index, timestamp, and data characteristics. This detailed sub-block information provides a reliable data organization method for subsequent parallel processing.
[0049] The preprocessing method adopted in the embodiment has good scalability. By adjusting the slicing parameters and cache strategy, the system can adapt to different scales and types of data processing requirements. This flexible preprocessing framework provides a general technical solution for FPGA acceleration computing. This adaptive processing method not only improves the processing efficiency of the system, but also enhances the practical value of the solution.
[0050] Step S102: Construct a processing unit array, which includes a plurality of parallel processing units and a scheduling controller, a linear cache and a sliding window are arranged in each parallel processing unit, the data sub-blocks are distributed to idle parallel processing units through the scheduling controller, adjacent data rows are cached by the linear cache and the sliding window is constructed, convolution operation is performed and processing results are outputted;
[0051] Optionally, the embodiment is directed to the low efficiency of resource allocation, complex task scheduling and large data access delay in FPGA parallel computing, and innovatively designs a set of efficient computing framework based on processing unit array. In the aspect of processing unit evaluation, the embodiment designs a unit efficiency scoring formula: PE_Efficiency = α × (Computing_Power / Max_Power) + β × (Cache_Hit_Rate) + γ × (Resource_Usage), wherein α, β and γ are weight coefficients, respectively representing the influence of computing power, cache hit rate and resource utilization. At the same time, a scheduling optimization scoring formula is introduced: Schedule_Score = (Task_Balance × Processing_Speed) / (Waiting_Time + Switch_Overhead), which is used to optimize the task allocation strategy.
[0052] The embodiment deeply optimizes the construction mechanism of the processing unit array. In the programmable logic unit of the FPGA, the system adopts a regular grid structure to layout the processing unit, ensuring the symmetry of the data path and the consistency of the signal timing. Each processing unit is equipped with an independent linear cache and computing resources, and the high-speed data exchange between units is realized through the carefully designed interconnection network. For different scales of data processing tasks, the system can dynamically adjust the number and organization of processing units. For example, when processing large-scale matrix operations, the system will use more parallel units; when processing small-scale but complex operations, a more fine-grained resource allocation strategy will be selected. This flexible processing unit organization significantly improves the utilization efficiency of FPGA resources.
[0053] The embodiment innovatively realizes the scheduling control mechanism. The scheduling controller adopts a hierarchical design, including a global task allocator and a local resource manager. The global task allocator is responsible for maintaining the state table of the processing unit, recording the work load and available resources of each unit; the local resource manager is responsible for the data flow control and computing scheduling within a single processing unit. This hierarchical scheduling structure can ensure load balancing while minimizing scheduling overhead. Especially when dealing with dynamic changing workloads, the system can quickly respond and optimize resource allocation. By monitoring the performance indicators of each processing unit in real time, the scheduling controller can adjust the task allocation strategy in time to ensure the efficient operation of the entire processing array.
[0054] This embodiment realizes efficient data access through optimized cache design. Each processing unit is equipped with a linear cache that adopts a multi-level structure, including a fast access cache and a data prefetch cache. The size and organization of the cache are optimized specifically for sliding window operations, ensuring that frequently accessed data is always kept in the fastest storage hierarchy. During data row caching, the system uses an intelligent prefetching strategy to load the data rows that will be needed in advance based on the data access pattern. This predictive cache management mechanism significantly reduces data access latency and improves computing efficiency. For different sizes of convolution kernels, the system can dynamically adjust the configuration of the sliding window to ensure optimal access performance.
[0055] This embodiment establishes a complete convolution calculation framework. When performing convolution operations, the system uses a pipeline architecture to parallelize data loading, window construction, and computation operations. Each processing unit is equipped with a dedicated multiply-add operator array to support highly parallel convolution calculations. In particular, for large-scale convolution operations, the system minimizes repeated calculations through an optimized data reuse mechanism. The processing results are directly written to the output buffer through a high-speed cache channel, avoiding additional data movement overhead. This pipeline-style calculation architecture ensures that the processing units can continuously and efficiently perform convolution operations.
[0056] This embodiment provides an innovative solution for FPGA parallel computing. Through optimized processing unit design and scheduling strategies, the system can efficiently handle various compute-intensive tasks. This intelligent parallel processing mechanism significantly improves the throughput and real-time performance of data processing.
[0057] This embodiment realizes computing acceleration through full-process optimization. From task allocation to result output, each link is carefully designed and optimized to ensure efficient operation of the entire processing process. This systematic optimization strategy provides reliable technical support for FPGA acceleration computing.
[0058] The parallel processing method adopted in this embodiment has good scalability. By adjusting the processing unit configuration and scheduling strategy, the system can adapt to different scales and types of computing requirements. This flexible computing framework provides a general technical solution for FPGA application development. This adaptive processing method not only improves the computing efficiency of the system, but also enhances the practical value of the solution.
[0059] Step S103: Set up a state tracker to monitor the processing status of the data sub-blocks. The state tracker records the processing completion flag of each data sub-block, sorts the data sub-blocks that have completed processing according to the timestamp identifier, detects whether there are missing or misaligned data sub-blocks, writes the sorted data sub-blocks into the result cache area, reads out the data sub-blocks from the result cache area according to the position index and splices them to generate a complete processing result.
[0060] Optionally, the embodiment is directed to the key issues of data integrity verification, sequential reconstruction and result splicing in parallel processing, and a set of data management scheme based on state tracking is innovatively designed. In terms of data integrity evaluation, the embodiment designs a state monitoring score formula: Status_Score = a x (Completion_Rate) + β x (Order_Accuracy) + γ x (Missing_Rate), wherein a, β, γ are weight coefficients, respectively representing the influence of processing completion degree, sequence accuracy and data missing rate. At the same time, a splicing quality evaluation formula: Merge_Quality = (Boundary_Smoothness x Data_Continuity) / (Gap_Count + Overlap_Error) is introduced to evaluate the effect of data reconstruction.
[0061] The embodiment deeply optimizes the state tracking mechanism. The state tracker adopts a hierarchical state management structure, and realizes efficient state updating and querying at the hardware level. For each data sub-block, the system maintains multi-dimensional state information, including processing stage identifier, completion timestamp, processing unit number and data validity flag. These state information is compressed and stored through bitmap technology to ensure the efficiency of state query. In the state updating process, the system adopts atomic operation mechanism to avoid state inconsistency caused by concurrent access. Especially for high concurrency scenarios, the system dynamically adjusts the frequency of state checking according to the real-time load of processing units, while ensuring the timeliness of response and reducing system overhead.
[0062] The embodiment innovatively realizes the data integrity verification mechanism. Based on timestamp and position index information, the system constructs a multi-level data integrity verification framework. At the data sub-block level, the system verifies the integrity of data content through checksum technology; at the sequence level, the system detects data loss by analyzing the continuity of timestamp; at the spatial level, the system verifies the correctness of data arrangement through the corresponding relationship of position index. When an anomaly is detected, the system will immediately start the data recovery process. For the missing data sub-block, the system will initiate a reprocessing request to the corresponding processing unit; for the misplaced data sub-block, the system will reorder based on the position index. This multi-level verification mechanism ensures the reliability of the processing result.
[0063] The embodiment realizes data reorganization through an efficient sorting strategy. The system uses an improved merge sorting algorithm, fully utilizing the local ordered characteristics of timestamps. During the sorting process, the system considers both timestamp and position index information to ensure the temporal correctness and spatial continuity of data. To improve sorting efficiency, the system implements a parallel sorting unit based on a comparator array at the hardware level. For data sub-blocks that have completed processing, the system immediately writes them to the sorting buffer, implementing parallel execution of sorting operations and data writing through a pipeline mechanism. This optimized sorting mechanism significantly improves the efficiency of data reorganization.
[0064] The embodiment establishes a complete result cache management framework. During the organization of the result cache area, the system uses a partition management strategy, pre-allocating storage space according to the position index of data sub-blocks. The layout of the cache area fully considers data access patterns, reducing access conflicts through an optimized storage structure. During data splicing, the system intelligently processes overlapping areas, eliminating splicing marks through a smoothing transition algorithm. This fine cache management mechanism ensures the quality of the final processing results.
[0065] The embodiment provides an innovative solution for data integrity assurance. Through comprehensive state tracking and data verification, the system can timely discover and handle various abnormal situations. This reliable data management mechanism significantly improves the stability of parallel processing.
[0066] The embodiment realizes a closed loop of the processing flow through result reorganization. The system will standardize the storage of complete processing results and generate processing reports to record the execution of key links. This complete result management method provides a reliable basis for subsequent data analysis and system optimization.
[0067] The data management method used in the embodiment has good scalability. By adjusting state tracking parameters and cache strategies, the system can adapt to different scales and types of parallel processing requirements. This flexible management framework provides a general technical solution for FPGA data processing. This adaptive management method not only improves the reliability of the system, but also enhances the practical value of the solution.
[0068] From the above description, the FPGA data slicing processing method based on timestamp provided by the embodiment of the application can realize safe storage and efficient access of data by innovatively constructing a timestamp identification mechanism, through data organization of dynamic random access memory and a two-dimensional cache array. An array architecture of parallel processing units is designed, linear cache and sliding window technology are combined, and an efficient data processing strategy is established. A state tracking mechanism is introduced, processing state monitoring and sequencing splicing are performed, and the integrity and accuracy of data processing are ensured. On the basis of safe storage of data, the method effectively solves the deficiencies of traditional technologies in slicing processing, parallel computing and result recombination, and significantly improves the performance and reliability of FPGA data processing.
[0069] In an embodiment of the FPGA data slicing processing method based on timestamp of the application, the following content can also be specifically included:
[0070] Step S201: allocating a physically continuous memory region as a direct memory access buffer through a host end of a PCIe link, setting a direct memory access descriptor queue in shared memory, mapping the shared memory to a user space, configuring a PCIe device driver to enable interrupt response, loading PCIe intellectual property core configuration information, and establishing a data transmission channel based on an AXI bus on the PCIe link;
[0071] Step S202: configuring the PCIe link to be a third-generation 8-channel specification, writing to-be-processed data into the direct memory access buffer, transmitting the to-be-processed data to a dynamic random access memory of a field programmable gate array through the PCIe link, generating an incremental timestamp identifier for the to-be-processed data based on a time counter, and dividing the to-be-processed data into a plurality of data blocks according to a preset slicing size.
[0072] Optionally, the embodiment is directed to the low efficiency of memory management, complex DMA configuration and difficult to guarantee data consistency in PCIe data transmission, and innovatively designs a set of efficient transmission scheme based on continuous physical memory. In terms of transmission efficiency evaluation, the embodiment designs a channel efficiency scoring formula: Channel_Efficiency = α × (Bandwidth_Usage / Max_Bandwidth) + β × (Buffer_Utilization) + γ × (DMA_Completion_Rate), wherein α, β, γ are weight coefficients, respectively representing the influence of bandwidth utilization, buffer utilization and DMA completion rate. At the same time, the timing control scoring formula: Timing_Score = (Clock_Accuracy × Data_Stability) / (Latency+ Jitter) is introduced to optimize the timing characteristics of data transmission.
[0073] The embodiment deeply optimizes the PCIe memory management mechanism. On the host side, the system creates continuous physical memory blocks through the physical memory management unit, avoiding the performance loss caused by traditional dispersed page allocation. These continuous memory areas are organized into a ring buffer structure, supporting efficient circular read-write operations. When configuring DMA descriptors, the system uses a chain structure, and each descriptor contains source address, target address, transmission length and control flag information. The descriptor queue is placed in the shared memory area, and the efficient data exchange between user space and kernel space is realized through memory mapping technology. This optimized memory management strategy significantly improves the efficiency of data transmission.
[0074] The embodiment innovatively realizes the interrupt processing mechanism. The PCIe device driver uses a multi-level interrupt processing architecture, including Top Half and Bottom Half handlers. The Top Half is responsible for quickly responding to hardware interrupts, recording interrupt status and triggering Bottom Half processing; the Bottom Half is responsible for specific data processing and resource release work. This layered interrupt processing mechanism not only ensures the timeliness of interrupt response, but also avoids long-term occupation of interrupt context. By configuring the interrupt aggregation parameters, the system can achieve the best balance between high throughput and low latency.
[0075] The embodiment realizes high-speed data transmission through optimized PCIe configuration. The system loads a specially optimized PCIe IP core, supports PCIe Gen3 x8 specification, and the theoretical bandwidth can reach 7.8 GB / s per channel. During the PCIe link configuration process, the system dynamically adjusts the link parameters such as the maximum payload size and the packet completion timeout according to the actual workload. In particular, when establishing an AXI bus-based data transmission channel, the system uses burst transmission mode to maximize the utilization of bus bandwidth. This efficient transmission mechanism provides a reliable data path for subsequent data processing.
[0076] The embodiment establishes a complete data timing management framework. The system uses a high-precision time counter to generate timestamps to ensure that the timing relationship of the data blocks is accurately recorded. The generation of timestamps uses a monotonic increasing strategy to avoid the sorting problem caused by time wraparound. During data blocking, the system selects the optimal blocking strategy according to the preset slice size and data characteristics. This timestamp-based data organization method provides a reliable synchronization mechanism for subsequent parallel processing.
[0077] The embodiment provides an innovative solution for PCIe data transmission. Through optimized memory management and transmission control, the system can efficiently process large-scale data streams. This intelligent transmission mechanism significantly improves the data throughput capacity of the FPGA acceleration system.
[0078] The embodiment realizes data transmission acceleration through full-process optimization. From memory allocation to data slicing, each link is carefully designed and optimized to ensure efficient operation of the entire transmission process. This systematic optimization strategy provides a reliable data path for FPGA application development.
[0079] The transmission method used in the embodiment has good scalability. By adjusting the transmission parameters and slicing strategy, the system can adapt to different sizes and types of data transmission requirements. This flexible transmission framework provides a general technical solution for FPGA application development. This adaptive transmission method not only improves the transmission efficiency of the system, but also enhances the practical value of the solution.
[0080] In an embodiment of the timestamp-based FPGA data slicing processing method of the present application, the following contents can be specifically included:
[0081] Step S301: Create a double buffering structure in the two-dimensional cache array, pad and expand the data block boundary, copy the boundary pixels outward to form an overlap region, calculate a two-dimensional index value based on the row and column positions of the data block in the original data, the two-dimensional index value includes a row index and a column index, and construct a position index identifier based on the row index and the column index;
[0082] Step S302: write the data blocks in the order of the timestamp identifiers into the two-dimensional cache array, the storage position of each data block being determined by the position index identifier, set a data block state flag bit in the double buffering structure to record the writing state of the data block, determine whether the data block is written according to the data block state flag bit, and generate a data sub-block with the position index identifier.
[0083] Optionally, the embodiment innovatively designs a data organization scheme based on double buffering to solve the problems of low data boundary processing efficiency, complex spatial positioning, and frequent cache access conflicts in FPGA cache management. In terms of cache efficiency evaluation, the embodiment designs a cache performance scoring formula: Cache_Performance = α × (Access_Speed / Max_Speed) + β × (Buffer_Efficiency) + γ × (Boundary_Quality), where α, β, and γ are weight coefficients, respectively representing the influence of access speed, buffer utilization rate, and boundary processing quality. At the same time, an index position scoring formula: Index_Quality = (Location_Accuracy × Access_Speed) / (Search_Time + Index_Overhead) is introduced to optimize data positioning efficiency.
[0084] The embodiment deeply optimizes the implementation mechanism of the double buffering structure. In the Block RAM resources of the FPGA, the system divides two cache areas of the same size, which are used for data reading and writing operations, respectively. This double buffering structure adopts a ping-pong working mode, that is, when one buffer performs data writing, the other buffer can simultaneously perform data reading, effectively avoiding read-write conflicts. In the buffer switching process, the system uses a lock-free synchronization mechanism to realize the rapid switching of the buffer through atomic operations of the state flag bit. For data blocks of different sizes, the system dynamically adjusts the organization mode of the buffer to ensure efficient use of storage space. For example, for large data blocks, the system adopts a block storage strategy; for small data blocks, a more compact storage method is used. This flexible buffer management mechanism significantly improves the efficiency of data access.
[0085] The embodiment innovatively realizes the boundary data processing mechanism. In the data block boundary processing process, the system adopts an adaptive padding strategy. For different types of data, the system will select the most suitable boundary expansion method. For example, for image data, the system will select mirror padding or copy padding according to the local features of the image; for signal data, periodic continuation or linear interpolation method will be used. When forming the overlapping area, the system ensures the continuity and smoothness of the boundary data through an intelligent pixel replication algorithm. This fine boundary processing mechanism not only ensures the accuracy of data processing, but also improves the calculation efficiency.
[0086] The embodiment realizes efficient spatial positioning through optimized index construction. The system constructs a hierarchical index structure based on the positional relationship of data blocks in the original data. The calculation of row and column indexes adopts normalization processing to ensure the stability of the range of index values. When constructing the position index identifier, the system considers the spatial position and access mode, and reduces the index overhead through bit field compression technology. This efficient index mechanism provides fast positioning capability for data access.
[0087] The embodiment establishes a complete data write management framework. In the data block writing process, the system strictly follows the timestamp order to operate, ensuring the time sequence consistency of the data. The storage location of each data block is uniquely determined by the position index identifier, avoiding storage conflicts. The system maintains detailed state information in the double buffer structure, including write state, validity flag and access counter. These state information is efficiently stored through bitmap, supporting fast state query and update.
[0088] The embodiment provides an innovative solution for data sub-block generation. Through complete position index identifier and state management, the system can accurately track the life cycle of each data sub-block. This reliable data organization mechanism significantly improves the efficiency of subsequent processing.
[0089] The embodiment realizes data management acceleration through whole-process optimization. From buffer organization to data sub-block generation, each link is carefully designed and optimized to ensure efficient operation of the whole management process. This systematic optimization strategy provides a reliable data organization scheme for FPGA data processing.
[0090] The data organization method adopted in the embodiment has good scalability. By adjusting the buffer configuration and index strategy, the system can adapt to different scales and types of data processing requirements. This flexible organization framework provides a general technical solution for FPGA application development. This adaptive organization method not only improves the processing efficiency of the system, but also enhances the practical value of the scheme.
[0091] In an embodiment of the time-stamp-based FPGA data slicing processing method of the present application, the following can also be included:
[0092] Step S401: Divide a processing unit area in a programmable logic unit of a field programmable gate array, construct a plurality of parallel processing units in the processing unit area, and allocate a block random access memory resource to each parallel processing unit, the block random access memory resource being used to construct a linear cache, the depth of the linear cache being configured to match the row length of a data block.
[0093] Step S402: Construct a global scheduling controller, the global scheduling controller including a task allocation module and a state monitoring module, the task allocation module maintaining a free processing unit queue, and the state monitoring module recording the working state of each parallel processing unit, and a data window generation circuit is configured in each parallel processing unit, the data window generation circuit being used to construct a sliding window with a fixed size based on the linear cache.
[0094] Optionally, the embodiment innovatively designs a parallel processing architecture based on resource division to solve the problems of unreasonable FPGA resource allocation, low processing unit scheduling efficiency, and complex data window management. In terms of resource utilization evaluation, the embodiment designs a unit configuration scoring formula: Unit_Score = α × (LUT_Usage / Total_LUT) + β × (BRAM_Efficiency) + γ × (Routing_Quality), where α, β, and γ are weight coefficients, respectively representing the influence of lookup table usage, block random access memory efficiency, and routing quality. A scheduling efficiency scoring formula: Schedule_Efficiency = (Processing_Speed × Resource_Balance) / (Task_Waiting + Switch_Cost) is introduced to optimize the task allocation strategy.
[0095] The embodiment deeply optimizes the FPGA resource division mechanism. In the layout process of the programmable logic unit, the system adopts a hierarchical resource planning strategy to divide the entire FPGA chip into a plurality of functional areas. The processing unit area is arranged in the lookup table resource-intensive area to ensure sufficient computing resources, and the storage resource area is concentrated in the block random access memory periphery to reduce data access delay. When dividing the processing units, the system considers the distribution of clock domains and arranges the processing units in the same clock domain to simplify the design of the clock tree. For each processing unit, the system allocates independent block random access memory resources to construct a dedicated linear cache. This optimized resource allocation strategy significantly improves the resource utilization efficiency of the FPGA.
[0096] The embodiment innovatively realizes a linear cache configuration mechanism. The linear cache of each processing unit adopts a multi-port design, supporting simultaneous data read and write operations. The cache depth configuration is accurately matched with the row length of the data block, ensuring the continuity of data access. In the cache structure design, the system adopts a double buffering mechanism to support the parallel execution of data loading and processing. Especially when processing boundary data, the linear cache design considers the need for data overlap, and through intelligent cache management strategies, it ensures the continuity of data processing. This flexible cache configuration mechanism provides efficient data support for sliding window operations.
[0097] The embodiment realizes efficient task allocation through an innovative scheduling controller. The global scheduling controller adopts a distributed architecture, including a central control unit and multiple local schedulers. The task allocation module maintains a dynamic processing unit state table, recording information such as the workload, resource occupation, and processing capacity of each unit. By analyzing these state information in real time, the system can make optimal task allocation decisions. The state monitoring module is responsible for real-time tracking of the running status of each processing unit, including processing progress, resource utilization, and performance indicators. This fine scheduling mechanism ensures efficient use of processing resources.
[0098] The embodiment establishes a complete window generation framework. The data window generation circuit adopts a pipeline design, supporting continuous window sliding operations. During window construction, the system reorganizes data quickly through a shift register array. The window size configuration supports dynamic adjustment, which can be flexibly changed according to actual processing needs. This efficient window generation mechanism significantly improves the flexibility of data processing.
[0099] The embodiment provides an innovative solution for FPGA parallel processing. Through optimized resource allocation and scheduling strategies, the system can efficiently process various types of computationally intensive tasks. This intelligent parallel processing mechanism significantly improves the throughput and real-time performance of data processing.
[0100] The embodiment realizes processing acceleration through full-process optimization. From resource division to window generation, each link is carefully designed and optimized to ensure efficient operation of the entire processing process. This systematic optimization strategy provides reliable technical support for FPGA application development.
[0101] The parallel processing method adopted in the embodiment has good scalability. By adjusting the processing unit configuration and scheduling strategy, the system can adapt to different scales and types of computing needs. This flexible processing framework provides a general technical solution for FPGA application development. This adaptive processing method not only improves the computing efficiency of the system, but also enhances the practical value of the solution.
[0102] In an embodiment of the timestamp-based FPGA data slicing processing method of the present application, the following can also be included:
[0103] Step S501: Establish a processing unit state table to record the working states of each parallel processing unit. The scheduling controller reads the processing unit state table to obtain the identification information of idle parallel processing units, allocates the data sub-blocks to the corresponding idle parallel processing units, establishes a prefetch buffer for the data sub-blocks in the local fast cache, and loads the next data sub-block to be processed into the prefetch buffer.
[0104] Step S502: Allocate a plurality of data row buffer areas in the linear cache, write the adjacent data rows of the data sub-blocks into the data row buffer areas in sequence, construct an N×N size sliding window matrix based on the data row buffer areas, move the sliding window matrix on the data sub-blocks pixel by pixel, perform convolution calculation on the data in the sliding window matrix to obtain processing results, and write the processing results into a result cache.
[0105] Optionally, the embodiment innovatively designs a task scheduling scheme based on state tracking to solve the problems of unbalanced processing unit scheduling, low data prefetching efficiency, and limited sliding window calculation performance. In terms of scheduling efficiency evaluation, the embodiment designs a state score formula: Status_Score = α×(Processing_Load / Max_Load) + β×(Cache_Hit_Rate) + γ×(Task_Balance), where α, β, and γ are weight coefficients representing the influence of processing load, cache hit rate, and task balance, respectively. A window calculation score formula: Window_Efficiency=(Compute_Speed × Data_Reuse) / (Memory_Access + Computation_Cost) is also introduced to optimize the convolution calculation efficiency.
[0106] The embodiment deeply optimizes the processing unit state management mechanism. The state table adopts a multi-dimensional structure design, which not only records the basic working state of the processing unit, but also contains detailed performance index information. The state information of each processing unit includes the current load level, resource occupancy rate, processing speed, and task queue depth. These state information is stored through an efficient bitmap encoding method, supporting fast state query and update. In the state update process, the system adopts an asynchronous notification mechanism to ensure the real-time performance of the state information. When the state of the processing unit changes, the state table is immediately updated, and the scheduling controller is notified to perform task redistribution. This fine state management mechanism provides a reliable basis for task scheduling.
[0107] The embodiment innovatively realizes a data prefetching mechanism. The prefetching buffer adopts a multi-level structure design, including a main buffer and a preloading buffer. The system analyzes the task execution sequence, predicts the next data sub-block to be processed, and loads it into the prefetching buffer in advance. The prefetching strategy dynamically adjusts according to the data access mode, and uses sequential prefetching for continuously accessed data and intelligent prediction prefetching for jump accessed data. In particular, when processing a regular data stream, the system can accurately predict the data access sequence, significantly improving the cache hit rate. This flexible prefetching mechanism effectively reduces the data loading delay.
[0108] The embodiment realizes efficient data row storage through an optimized linear cache design. The data row cache area adopts a circular buffer structure, supporting continuous writing and sliding window access of data. The size of the cache area is dynamically configured according to the demand of the sliding window, ensuring sufficient data reuse. During data writing, the system adopts a dual-port design, supporting simultaneous data writing and window reading operations. This optimized cache design provides efficient data support for sliding window operations.
[0109] The embodiment establishes a complete sliding window computing framework. The construction of the N×N sliding window matrix adopts a pipeline design, supporting continuous movement and data update of the window. During window movement, the system realizes fast reorganization of data through a shift register array, minimizing data movement overhead. When performing convolution calculation, the system adopts an array of parallel multiply-add units, supporting parallel processing of data within the window. The calculation results are quickly stored into the result cache through a dedicated write-back channel, avoiding the bottleneck between calculation and storage. This efficient computing architecture significantly improves processing performance.
[0110] The embodiment provides an innovative solution for data processing. Through optimized scheduling strategies and computing architectures, the system can efficiently process various convolution operation tasks. This intelligent processing mechanism significantly improves the throughput and real-time performance of calculations.
[0111] The embodiment realizes processing acceleration through full-process optimization. From task scheduling to result output, each link is carefully designed and optimized to ensure efficient operation of the entire processing process. This systematic optimization strategy provides reliable technical support for FPGA acceleration calculations.
[0112] The processing method adopted in the embodiment has good scalability. By adjusting the window parameters and calculation strategies, the system can adapt to different scales and types of convolution calculation requirements. This flexible processing framework provides a general technical solution for FPGA application development. This adaptive processing method not only improves the calculation efficiency of the system, but also enhances the practical value of the solution.
[0113] In an embodiment of the timestamp-based FPGA data slicing processing method of the present application, the following can also be included:
[0114] Step S601: Construct a state tracking table to record processing state information of data sub-blocks, the state tracking table including data sub-block identification, timestamp identification, processing unit identification, and processing state flag, the state tracker periodically scans the state tracking table to obtain processing progress of each data sub-block, and the processing state flag is updated when it is detected that a data sub-block is processed, and the number of data sub-blocks that have been processed is counted.
[0115] Step S602: Establish a sorting buffer to store data sub-blocks that have been processed, sort the data sub-blocks in the sorting buffer in ascending order according to the timestamp identification, write the sorted data sub-blocks into a ready queue, the state tracker maintains read and write pointers of the ready queue to record the number of data sub-blocks available for reading in the queue, and sends a data ready signal to a data aggregation module.
[0116] Optionally, the embodiment innovatively designs a data management scheme based on state tracking to solve problems such as complex data state tracking in parallel processing, low result sorting efficiency, and difficulty in guaranteeing data synchronization. In the state tracking evaluation aspect, the embodiment designs a tracking efficiency score formula: Tracking_Score = α × (Update_Speed / Max_Speed) + β × (State_Accuracy) + γ × (Memory_Usage), where α, β, and γ are weight coefficients, respectively representing the influence of update speed, state accuracy, and memory usage. At the same time, a sorting quality score formula is introduced: Sort_Quality = (Ordering_Accuracy × Processing_Speed) / (Buffer_Usage + Sorting_Cost), which is used to optimize data sorting efficiency.
[0117] The embodiment deeply optimizes the implementation mechanism of the state tracking table. The state tracking table adopts a multi-dimensional hash structure design, supporting fast state query and update operations. The state information of each data sub-block includes multiple key fields: data sub-block identification for uniquely identifying processing objects, using a compact encoding method to reduce storage overhead; timestamp identification recording time sequence information of data generation and processing, ensuring the orderliness of data processing; processing unit identification indicating the current computing unit responsible for processing the data block, facilitating tracking of processing sources; and processing state flag using a bitmap to represent the current processing stage, supporting fine-grained state tracking. The state tracker periodically updates state information through an efficient scanning algorithm, processes multiple state update requests in a pipeline manner, and significantly improves the state management efficiency.
[0118] The embodiment innovatively realizes the state update mechanism. The state tracker adopts a distributed update strategy, which allocates state checking tasks to multiple parallel checking units. Each checking unit is responsible for monitoring the processing state of a group of data sub-blocks and regularly obtains the processing progress information through polling. When detecting that the data sub-block processing is completed, the system will immediately trigger the state update process, including updating the processing state flag, recording the completion timestamp, and counting the number of completions. Especially when processing large-scale parallel tasks, this distributed state management mechanism can effectively avoid state update from becoming a system bottleneck.
[0119] The embodiment realizes efficient data sorting through optimized sorting buffer design. The sorting buffer adopts a multi-level cache structure, including pre-sorting cache and final sorting cache. The system realizes ascending order sorting of data sub-blocks based on timestamp identification, and uses an improved merge sorting algorithm to fully utilize the local ordered characteristics of data. During the sorting process, the system reduces the sorting overhead through the pre-sorting mechanism and quickly merges data sub-blocks with consecutive timestamps. This optimized sorting mechanism significantly improves the efficiency of data reorganization.
[0120] The embodiment establishes a complete ready queue management framework. The ready queue adopts a circular buffer structure, supporting efficient data read and write operations. The state tracker realizes dynamic management of the queue by maintaining read and write pointers, accurately recording the number of data sub-blocks available for reading. When the data in the queue reaches the preset threshold, the system automatically sends a ready signal to the data aggregation module, triggering the subsequent data processing process. This fine queue management mechanism ensures the continuity and efficiency of data processing.
[0121] The embodiment provides an innovative solution for data synchronization. Through complete state tracking and sorting mechanisms, the system can accurately manage parallel processing data streams. This reliable synchronization mechanism significantly improves the reliability of data processing.
[0122] The embodiment realizes data management acceleration through full-process optimization. From state tracking to data sorting, each link is carefully designed and optimized to ensure efficient operation of the entire management process. This systematic optimization strategy provides a reliable data management solution for FPGA parallel processing.
[0123] The data management method adopted in the embodiment has good scalability. By adjusting the state tracking parameters and sorting strategy, the system can adapt to different scales and types of parallel processing requirements. This flexible management framework provides a general technical solution for FPGA application development. This adaptive management method not only improves the processing efficiency of the system, but also enhances the practical value of the solution.
[0124] In an embodiment of the timestamp-based FPGA data slicing processing method of the present application, the following content can also be specifically included:
[0125] Step S701: Establish a data integrity check table based on the timestamp identifier and the position index, check whether the timestamp identifiers of adjacent data sub-blocks are continuous, verify whether the position indexes of the data sub-blocks conform to the expected arrangement order, initiate a reprocessing request to the corresponding processing unit when a missing data sub-block is detected, and reorder the misplaced data sub-blocks according to their position indexes;
[0126] Step S702: Divide a plurality of cache blocks in the result cache area for temporarily storing the reordered data sub-blocks, the size of the cache blocks matches the size of the data sub-blocks, determine the storage positions of the data sub-blocks in the result cache area according to the position indexes, write the data sub-blocks into the corresponding cache blocks, remove the overlapping areas at the boundaries of the data sub-blocks, and combine the valid data of adjacent data sub-blocks to obtain a complete processing result.
[0127] Optionally, the present embodiment innovatively designs a data integration scheme based on multiple verifications to solve the problems of complex data integrity verification, low cache management efficiency, and low result splicing quality in parallel processing. In the aspect of integrity evaluation, the present embodiment designs a verification quality scoring formula: Validation_Score = α × (Continuity_Rate) + β × (Order_Accuracy) + γ × (Recovery_Speed), where α, β, and γ are weight coefficients, respectively representing the influence of data continuity, order accuracy, and recovery speed. Meanwhile, a splicing quality scoring formula: Merge_Quality = (Boundary_Smoothness × Data_Consistency) / (Overlap_Error + Seam_Visibility) is introduced to optimize the data splicing effect.
[0128] This embodiment deeply optimizes the data integrity verification mechanism. The integrity verification table adopts a multi-dimensional matrix structure, recording verification information in both time and space dimensions. For the time dimension, the system constructs a time continuity verification model by analyzing the timestamp sequence of adjacent data sub-blocks. This model not only checks the increasing nature of timestamps, but also considers the time deviation caused by processing delays, adjusting verification parameters through an adaptive time window mechanism. For the spatial dimension, the system establishes a spatial relationship graph based on location indexes, verifying whether the relative positions of data sub-blocks conform to the original data's spatial distribution. When anomalies are detected, the system initiates a hierarchical recovery strategy: for minor misplacement issues, it uses in-place reordering to quickly correct; for data loss situations, it triggers reprocessing through task redistribution mechanisms. This comprehensive verification mechanism ensures the reliability of data processing.
[0129] This embodiment innovatively implements a cache management mechanism. The result cache adopts a block management strategy, with each cache block size optimized according to data sub-block dimensions. The cache block organization fully considers data access patterns, using interleaved storage to reduce access conflicts. During cache allocation, the system establishes a mapping relationship through location index calculation, ensuring that each data sub-block can be quickly located to the corresponding storage location. Especially for large-scale data processing, the system adopts a hierarchical cache management strategy, establishing data synchronization mechanisms between different levels of cache to improve storage efficiency. This optimized cache management mechanism significantly improves data access performance.
[0130] This embodiment achieves high-quality data stitching through fine boundary processing. When processing data sub-block boundaries, the system first analyzes the data characteristics of the overlapping area and establishes a boundary feature model. Based on this model, the system uses an adaptive cropping strategy to determine the optimal cropping position according to the continuity characteristics of the data. For adjacent areas that need to be stitched, the system eliminates stitching marks through a smooth transition algorithm to ensure data continuity. This fine boundary processing mechanism significantly improves result quality.
[0131] This embodiment establishes a complete result combination framework. In the data stitching process, the system uses a pipeline to process multiple data sub-blocks, supporting parallel stitching operations. The stitching algorithm considers multiple feature dimensions of the data, including spatial position, data distribution, and boundary characteristics, to determine the optimal stitching strategy through comprehensive analysis. For special stitching scenarios such as irregular boundaries or data distortion, the system will start a special processing module for optimization. This comprehensive stitching mechanism ensures the integrity and accuracy of the processing results.
[0132] This embodiment provides an innovative solution for data integration. Through optimized verification mechanisms and stitching strategies, the system can complete data reorganization tasks with high quality. This reliable integration mechanism significantly improves the effectiveness of parallel processing.
[0133] This embodiment accelerates data integration through end-to-end optimization. From integrity verification to result stitching, each step has been carefully designed and optimized to ensure efficient operation of the entire integration process. This systematic optimization strategy provides a reliable integration solution for FPGA data processing.
[0134] The data integration method employed in this embodiment exhibits excellent scalability. By adjusting verification parameters and splicing strategies, the system can adapt to data processing needs of varying scales and types. This flexible integration framework provides a universal technical solution for FPGA application development. This adaptive integration approach not only improves system processing efficiency but also enhances the practical value of the solution.
[0135] To effectively address the shortcomings of traditional technologies in slicing, parallel computing, and result reassembly, and to significantly improve the performance and reliability of FPGA data processing, this application provides an embodiment of a timestamp-based FPGA data slicing processing apparatus for implementing all or part of the timestamp-based FPGA data slicing processing method. See [link to embodiment]. Figure 2 The timestamp-based FPGA data slicing processing device specifically includes the following components:
[0136] The data segmentation module 10 is used to collect data to be processed, transmit the data to be processed to the dynamic random access memory of the field programmable gate array via a PCIe link, assign timestamp identifiers to the data to be processed, segment the data to be processed into blocks according to a preset slice size, add overlapping regions to the boundaries of each data block to maintain data continuity, construct a two-dimensional cache array, write each data block to the corresponding position of the two-dimensional cache array according to the timestamp identifier, and generate multiple data sub-blocks with position indices.
[0137] The array construction module 20 is used to construct a processing unit array, which includes multiple parallel processing units and a scheduling controller. Each parallel processing unit is equipped with a linear cache and a sliding window. The scheduling controller allocates the data sub-blocks to idle parallel processing units, uses the linear cache to cache adjacent data rows and constructs the sliding window, performs convolution operations and outputs the processing results.
[0138] The data caching module 30 is used to set a status tracker to monitor the processing status of the data sub-blocks. The status tracker records the processing completion flag of each data sub-block, sorts the processed data sub-blocks according to the timestamp identifier, detects whether the data sub-blocks are missing or misaligned, writes the sorted data sub-blocks into the result cache area, reads the data sub-blocks from the result cache area according to the position index and concatenates them to generate a complete processing result.
[0139] From the above description, the FPGA data slicing processing device based on timestamp provided by the embodiment of the application can realize safe storage and efficient access of data by innovatively constructing a timestamp identification mechanism, through data organization of dynamic random access memory and a two-dimensional cache array. A parallel processing unit array architecture is designed, combined with linear cache and sliding window technology, to establish an efficient data processing strategy. A state tracking mechanism is introduced to ensure the integrity and accuracy of data processing through processing state monitoring and ordering splicing. On the basis of safe data storage, the method effectively solves the deficiencies of traditional technologies in slicing processing, parallel computing and result recombination, and significantly improves the performance and reliability of FPGA data processing.
[0140] From the hardware level, in order to effectively solve the deficiencies of traditional technologies in slicing processing, parallel computing and result recombination, and significantly improve the performance and reliability of FPGA data processing, the application provides an embodiment of an electronic device for implementing all or part of the contents of the FPGA data slicing processing method based on timestamp, which specifically includes the following contents:
[0141] A processor, a memory, a communications interface and a bus; wherein the processor, the memory and the communications interface complete mutual communication through the bus; the communications interface is used to realize information transmission between the FPGA data slicing processing device based on timestamp and related devices such as core business systems, user terminals and related databases; the logic controller can be a desktop computer, a tablet computer and a mobile terminal, and the embodiment is not limited thereto. In the embodiment, the logic controller can be implemented with reference to the embodiment of the FPGA data slicing processing method based on timestamp and the embodiment of the FPGA data slicing processing device based on timestamp, the contents of which are incorporated herein, and the repeated parts will not be described again.
[0142] It can be understood that the user terminal can include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. The smart wearable device can include smart glasses, a smart watch, a smart bracelet, etc.
[0143] In actual application, part of the FPGA data slicing processing method based on timestamp can be executed on the electronic device as described above, or all operations can be completed in the client device. Specifically, the processing capacity of the client device and the use scenario of the user can be selected. The present application does not limit this. If all operations are completed in the client device, the client device can also include a processor.
[0144] The client device described above can have a communication module (i.e., a communication unit) that can be in communication connection with a remote server to realize data transmission with the server. The server can include a server on the task scheduling center side, and other implementation scenarios can also include a server of an intermediate platform, such as a server of a third-party server platform that is in communication link with the task scheduling center server. The server can include a single computer device, a server cluster composed of multiple servers, or a server structure of a distributed device.
[0145] Figure 3 A schematic block diagram of the system configuration of the electronic device 9600 of the embodiment of the present application is shown in FIG. 9. As shown in the figure, the electronic device 9600 can include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that the structure shown in the figure is exemplary; other types of structures can also be used to supplement or replace the structure to realize telecommunication functions or other functions. Figure 3 Figure 3 The structure shown in the figure is exemplary; other types of structures can also be used to supplement or replace the structure to realize telecommunication functions or other functions.
[0146] In an embodiment, the FPGA data slicing processing method based on timestamp can be integrated into the central processor 9100. The central processor 9100 can be configured to control as follows:
[0147] Step S101: collecting data to be processed, transmitting the data to be processed to the dynamic random access memory of the field programmable gate array through the PCIe link, assigning a timestamp identifier to the data to be processed, dividing the data to be processed according to a preset slicing size, adding an overlapping area at the boundary of each data block to maintain data continuity, constructing a two-dimensional cache array, writing each data block to the corresponding position of the two-dimensional cache array according to the timestamp identifier, and generating a plurality of data sub-blocks with position indexes;
[0148] Step S102: constructing a processing unit array, the processing unit array comprising a plurality of parallel processing units and a scheduling controller, each parallel processing unit being provided with a linear cache and a sliding window, the data sub-blocks being allocated to idle parallel processing units by the scheduling controller, adjacent data rows being cached by the linear cache and the sliding window being constructed, a convolution operation being performed and a processing result being outputted;
[0149] Step S103: setting a state tracker to monitor processing states of the data sub-blocks, the state tracker recording a processing completion flag of each data sub-block, processing completed data sub-blocks being sorted according to the time stamp identifiers, whether there is a missing or misplaced data sub-block being detected, the sorted data sub-blocks being written into a result cache area, data sub-blocks being read out from the result cache area according to the position index and spliced to generate a complete processing result.
[0150] As can be known from the above description, the electronic device provided by the embodiment of the application realizes safe storage and efficient access of data by innovatively constructing a time stamp identifier mechanism, by data organization of a dynamic random access memory and a two-dimensional cache array. A parallel processing unit array architecture is designed, combined with linear cache and sliding window technology, to establish an efficient data processing strategy. A state tracking mechanism is introduced to ensure the integrity and accuracy of data processing through processing state monitoring and sorting splicing. On the basis of safe data storage, the method effectively solves the deficiencies of traditional technologies in terms of slicing processing, parallel computing and result recombination, and significantly improves the performance and reliability of FPGA data processing.
[0151] In another embodiment, the timestamp-based FPGA data slicing processing device can be configured separately from the central processor 9100, for example, the timestamp-based FPGA data slicing processing device can be configured as a chip connected with the central processor 9100 to realize the function of the timestamp-based FPGA data slicing processing method through the control of the central processor.
[0152] As shown in FIG. 9, the electronic device 9600 can further include a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It should be noted that the electronic device 9600 does not necessarily include all the components shown in FIG. 9; in addition, the electronic device 9600 can include components not shown in FIG. 9, which can be referred to the prior art. Figure 3 As shown in FIG. 9, the electronic device 9600 can further include a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It should be noted that the electronic device 9600 does not necessarily include all the components shown in FIG. 9; in addition, the electronic device 9600 can include components not shown in FIG. 9, which can be referred to the prior art. Figure 3 As shown in FIG. 9, the electronic device 9600 can further include a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It should be noted that the electronic device 9600 does not necessarily include all the components shown in FIG. 9; in addition, the electronic device 9600 can include components not shown in FIG. 9, which can be referred to the prior art. Figure 3 As shown in FIG. 9, the electronic device 9600 can further include a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It should be noted that the electronic device 9600 does not necessarily include all the components shown in FIG. 9; in addition, the electronic device 9600 can include components not shown in FIG. 9, which can be referred to the prior art.
[0153] As shown in FIG. 9, the electronic device 9600 can further include a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It should be noted that the electronic device 9600 does not necessarily include all the components shown in FIG. 9; in addition, the electronic device 9600 can include components not shown in FIG. 9, which can be referred to the prior art. Figure 3As shown, the central processing unit 9100, which is sometimes also referred to as a controller or operation control, can include a microprocessor or other processor device and / or logic device, which receives input and controls the operation of the various components of the electronic device 9600.
[0154] The memory 9140, for example, can be one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. Information relating to failures can be stored, and in addition, programs for executing the information can be stored. The central processing unit 9100 can execute the programs stored in the memory 9140 to achieve information storage or processing, etc.
[0155] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and text. The display can be, for example, an LCD display, but is not limited thereto.
[0156] The memory 9140 can be a solid state memory such as a read only memory (ROM), a random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROM, etc. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 can include an application / function storage section 9142 for storing application programs and function programs or for executing the flow of the operation of the electronic device 9600 by the central processing unit 9100.
[0157] The memory 9140 can also include a data storage section 9143 for storing data such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. A driver program storage section 9144 of the memory 9140 can include various driver programs of the electronic device for communication functions and / or for executing other functions of the electronic device such as a messaging application, an address book application, etc.
[0158] The communication module 9110 is a transmitter / receiver that transmits and receives signals via an antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processing unit 9100 to provide input signals and receive output signals, which can be the same as in the case of a conventional mobile communication terminal.
[0159] Based on different communication technologies, multiple communication modules 9110, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module, etc., can be provided in the same electronic device. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and to receive audio input from the microphone 9132 to implement the usual telecommunication functions. The audio processor 9130 can include any suitable buffers, decoders, amplifiers, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100 to enable recording on the local device via the microphone 9132 and to enable playing of stored sounds on the local device via the speaker 9131.
[0160] The embodiments of the present application also provide a computer readable storage medium capable of implementing all steps of the timestamp-based FPGA data slicing processing method with the execution subject being a server or a client in the above-mentioned embodiments, and the computer program is stored on the computer readable storage medium. When the processor executes the computer program, all steps of the timestamp-based FPGA data slicing processing method with the execution subject being a server or a client in the above-mentioned embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented:
[0161] Step S101: collecting to-be-processed data, transmitting the to-be-processed data to a dynamic random access memory of a field programmable gate array through a PCIe link, allocating a timestamp identifier to the to-be-processed data, dividing the to-be-processed data according to a preset slicing size, adding an overlapping area at the boundary of each data block for maintaining data continuity, constructing a two-dimensional cache array, writing each data block into a corresponding position of the two-dimensional cache array according to the timestamp identifier, and generating a plurality of data sub-blocks with position indexes;
[0162] Step S102: constructing a processing unit array, the processing unit array including a plurality of parallel processing units and a scheduling controller, a linear cache and a sliding window being arranged in each parallel processing unit, allocating the data sub-blocks to idle parallel processing units through the scheduling controller, using the linear cache to cache adjacent data rows and constructing the sliding window, performing convolution operation and outputting processing results;
[0163] Step S103: setting a state tracker to monitor the processing state of the data sub-blocks, the state tracker recording a processing completion flag of each data sub-block, sorting the data sub-blocks that have completed processing according to the timestamp identifier, detecting whether there is a missing or misaligned data sub-block, writing the sorted data sub-blocks into a result cache area, reading out the data sub-blocks from the result cache area according to the position indexes and splicing, and generating a complete processing result.
[0164] As can be seen from the above description, the computer readable storage medium provided by the embodiments of the application realizes safe storage and efficient access of data by innovatively constructing a timestamp identification mechanism, through data organization of a dynamic random access memory and a two-dimensional cache array. A parallel processing unit array architecture is designed, linear cache and sliding window technology are combined, and an efficient data processing strategy is established. A state tracking mechanism is introduced, processing state monitoring and sorting splicing are performed, and the integrity and accuracy of data processing are ensured. On the basis of safe storage of data, the method effectively solves the deficiencies of traditional technologies in slicing processing, parallel computing and result recombination, and significantly improves the performance and reliability of FPGA data processing.
[0165] Embodiments of the application also provide a computer program product capable of implementing all steps of the timestamp-based FPGA data slicing processing method in which the execution subject is a server or a client. The computer program / instructions are executed by a processor to implement the steps of the timestamp-based FPGA data slicing processing method, for example, the computer program / instructions implement the following steps:
[0166] Step S101: collecting data to be processed, transmitting the data to be processed to a dynamic random access memory of a field programmable gate array through a PCIe link, allocating a timestamp identifier to the data to be processed, dividing the data to be processed into blocks according to a preset slicing size, adding an overlapping area at the boundary of each data block to maintain data continuity, constructing a two-dimensional cache array, writing each data block into a corresponding position of the two-dimensional cache array according to the timestamp identifier, and generating a plurality of data sub-blocks with position indexes;
[0167] Step S102: constructing a processing unit array, the processing unit array including a plurality of parallel processing units and a scheduling controller, a linear cache and a sliding window being arranged in each parallel processing unit, the data sub-blocks being allocated to idle parallel processing units by the scheduling controller, adjacent data rows being cached by the linear cache and the sliding window being constructed, convolution operation being performed, and processing results being outputted;
[0168] Step S103: setting a state tracker to monitor the processing state of the data sub-blocks, the state tracker recording a processing completion flag of each data sub-block, sorting the data sub-blocks that have completed processing according to the timestamp identifier, detecting whether the data sub-blocks have missing or misplacement, writing the sorted data sub-blocks into a result cache area, reading out the data sub-blocks from the result cache area according to the position indexes and splicing, and generating complete processing results.
[0169] From the above description, the computer program product provided by the embodiment of the present application realizes the safe storage and efficient access of data by innovatively constructing a timestamp identification mechanism, and organizing data by means of a dynamic random access memory and a two-dimensional cache array. The parallel processing unit array architecture is designed, the linear cache and the sliding window technology are combined, and the efficient data processing strategy is established. The state tracking mechanism is introduced, the processing state monitoring and sequencing splicing are performed, and the integrity and accuracy of data processing are ensured. On the basis of the safe storage of data, the method effectively solves the deficiencies of the traditional technology in the aspects of slicing processing, parallel computing and result recombination, and significantly improves the performance and reliability of FPGA data processing.
[0170] Those skilled in the art will understand that embodiments of the present application can be provided as methods, apparatuses, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.
[0171] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (apparatuses), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks
[0172] These computer program instructions can also be stored in a computer-readable memory capable of guiding a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including instruction devices that implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks
[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 the steps of a function specified in one or more blocks.
[0174] The principles and implementation manners of the present application are described in the specific embodiments. The above description of the embodiments is only used to help understand the method and core idea of the present application. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application scope will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.
Claims
1. A time-stamp based FPGA data slicing processing method, characterized in that, The method comprises: Collecting to-be-processed data, transmitting the to-be-processed data to a dynamic random access memory of a field programmable gate array through a PCIe link, assigning a timestamp identifier to the to-be-processed data, dividing the to-be-processed data into blocks according to a preset slice size, adding an overlapping area at the boundary of each data block for maintaining data continuity, constructing a two-dimensional cache array, writing each data block into a corresponding position of the two-dimensional cache array according to the timestamp identifier, and generating a plurality of data sub-blocks with position indexes; Constructing a processing unit array, the processing unit array comprising a plurality of parallel processing units and a scheduling controller, a linear cache and a sliding window being arranged in each parallel processing unit, assigning the data sub-blocks to idle parallel processing units through the scheduling controller, using the linear cache to cache adjacent data rows and constructing the sliding window, performing convolution operation and outputting processing results; Setting a state tracker to monitor the processing state of the data sub-blocks, the state tracker recording a processing completion flag of each data sub-block, sorting the data sub-blocks that have completed processing according to the timestamp identifier, detecting whether the data sub-blocks are missing or misaligned, writing the sorted data sub-blocks into a result cache area, reading out the data sub-blocks from the result cache area according to the position indexes and splicing, and generating a complete processing result.
2. The timestamp-based FPGA data slice processing method of claim 1, wherein, The collecting to-be-processed data, transmitting the to-be-processed data to a dynamic random access memory of a field programmable gate array through a PCIe link, assigning a timestamp identifier to the to-be-processed data, dividing the to-be-processed data into blocks according to a preset slice size, comprises: Allocating a physically continuous memory area as a direct memory access buffer through a host end of a PCIe link, setting a direct memory access descriptor queue in a shared memory, mapping the shared memory to a user space, configuring a PCIe device driver to enable interrupt response, loading PCIe intellectual property core configuration information, and establishing an AXI bus-based data transmission channel on the PCIe link; Configuring the PCIe link to be a third-generation 8-channel specification, writing to-be-processed data into the direct memory access buffer, transmitting the to-be-processed data to a dynamic random access memory of a field programmable gate array through the PCIe link, generating an incrementing timestamp identifier for the to-be-processed data based on a time counter, and dividing the to-be-processed data into a plurality of data blocks according to a preset slice size.
3. The timestamp-based FPGA data slice processing method of claim 1, wherein, The adding an overlapping area at the boundary of each data block for maintaining data continuity, constructing a two-dimensional cache array, writing each data block into a corresponding position of the two-dimensional cache array according to the timestamp identifier, and generating a plurality of data sub-blocks with position indexes, comprises: Creating a double-buffering structure in a two-dimensional cache array, padding and expanding the boundary of the data block, copying the boundary pixels outward to form an overlapping area, calculating a two-dimensional index value according to the row and column positions of the data block in the original data, the two-dimensional index value comprising a row index and a column index, and constructing a position index identifier based on the row index and the column index; Write the data blocks in the two-dimensional cache array according to the order of the timestamp identifiers, and the storage position of each data block is determined by the position index identifier; set a data block state flag bit in the double buffering structure to record the writing state of the data block; judge whether the data block is completed writing according to the data block state flag bit; and generate a data sub-block with the position index identifier.
4. The timestamp-based FPGA data slice processing method of claim 1, wherein, The array of processing units includes a plurality of parallel processing units and a scheduling controller, and a linear cache and a sliding window are arranged in each parallel processing unit, including: A processing unit region is divided in a programmable logic unit of the field programmable gate array, a plurality of parallel processing units are constructed in the processing unit region, and a block random access memory resource is allocated to each parallel processing unit, the block random access memory resource being used to construct a linear cache, and the depth of the linear cache is configured to match the row length of the data block; A global scheduling controller is constructed, the global scheduling controller including a task allocation module and a state monitoring module, the task allocation module maintaining a free processing unit queue, and the state monitoring module recording the working state of each parallel processing unit, and a data window generation circuit is configured in each parallel processing unit, the data window generation circuit constructing a sliding window with a fixed size based on the linear cache.
5. The timestamp-based FPGA data slice processing method of claim 1, wherein, The data sub-block is allocated to a free parallel processing unit by the scheduling controller, adjacent data rows are cached by using the linear cache and the sliding window is constructed, a convolution operation is performed, and a processing result is output, including: A processing unit state table is established to record the working state of each parallel processing unit, the scheduling controller reads the processing unit state table to obtain identification information of a free parallel processing unit, the data sub-block is allocated to the corresponding free parallel processing unit, a prefetch buffer area is established for the data sub-block in a local fast cache, and a next data sub-block to be processed is preloaded into the prefetch buffer area; A plurality of data row buffer areas are allocated in the linear cache, adjacent data rows of the data sub-block are sequentially written into the data row buffer areas, an N×N size sliding window matrix is constructed based on the data row buffer areas, the sliding window matrix moves on the data sub-block pixel by pixel, a convolution calculation is performed on the data in the sliding window matrix to obtain a processing result, and the processing result is written into a result cache.
6. The timestamp-based FPGA data slice processing method of claim 1, wherein, The state tracker is set to monitor the processing state of the data sub-block, the state tracker records a processing completion flag of each data sub-block, and the data sub-blocks that have completed processing are sorted according to the timestamp identifier, including: A state tracking table is constructed to record the processing state information of the data sub-block, the state tracking table including a data sub-block identifier, a timestamp identifier, a processing unit identifier, and a processing state flag, the state tracker periodically scans the state tracking table to obtain the processing progress of each data sub-block, the processing state flag is updated when it is detected that a data sub-block has completed processing, and the number of data sub-blocks that have completed processing is counted. A sorting buffer is established to store the processed data sub-blocks, the data sub-blocks in the sorting buffer are sorted in ascending order according to the timestamp identifiers, the sorted data sub-blocks are written into a ready queue, the state tracker maintains read and write pointers of the ready queue, records the number of data sub-blocks available for reading in the queue, and sends a data ready signal to a data aggregation module.
7. The timestamp-based FPGA data slice processing method of claim 1, wherein, The sorted data sub-blocks are written into a result cache, and the data sub-blocks are read from the result cache according to the position index and spliced to generate a complete processing result, including: A data integrity check table is established based on the timestamp identifier and the position index, the timestamp identifiers of adjacent data sub-blocks are checked for continuity, the position index of the data sub-block is verified for the expected arrangement order, a reprocessing request is initiated to the corresponding processing unit when a missing data sub-block is detected, and the misaligned data sub-block is reordered according to its position index; A plurality of cache blocks are divided in the result cache for temporarily storing the sorted data sub-blocks, the size of the cache block matches the size of the data sub-block, the storage position of each data sub-block in the result cache is determined according to the position index, the data sub-block is written into the corresponding cache block, the overlapping area at the boundary of the data sub-block is removed, and the effective data of adjacent data sub-blocks is spliced to obtain a complete processing result.
8. A timestamp-based FPGA data slicing processing apparatus, characterized by, The device comprises: A data blocking module is configured to collect data to be processed, transmit the data to be processed to a dynamic random access memory of a field programmable gate array through a PCIe link, assign a timestamp identifier to the data to be processed, block the data to be processed according to a preset slicing size, add an overlapping area at the boundary of each data block to maintain data continuity, construct a two-dimensional cache array, write each data block into the corresponding position of the two-dimensional cache array according to the timestamp identifier, and generate a plurality of data sub-blocks with position indexes; An array construction module is configured to construct a processing unit array, the processing unit array comprising a plurality of parallel processing units and a scheduling controller, a linear cache and a sliding window are arranged in each parallel processing unit, the data sub-blocks are distributed to idle parallel processing units through the scheduling controller, the linear cache is used to cache adjacent data rows and construct the sliding window, convolution operation is performed, and a processing result is output. A data cache module is configured to set a state tracker to monitor the processing status of the data sub-blocks, the state tracker records a processing completion flag of each data sub-block, sorts the processed data sub-blocks according to the timestamp identifier, detects whether the data sub-blocks are missing or misaligned, writes the sorted data sub-blocks into a result cache, reads the data sub-blocks from the result cache according to the position index and splices them to generate a complete processing result.
9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the timestamp-based FPGA data slicing processing method of any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the timestamp-based FPGA data slicing processing method according to any one of claims 1 to 7.
Citation Information
Patent Citations
TOE data splicing system and method based on FPGA
CN111917780A
Storage and calculation integrated parallel processing system and method
CN120179606A