Data movement method, system and storage medium under fixed descriptor budget

Abstract

The application discloses a data carrying method and system under a fixed descriptor budget and a storage medium, and relates to the technical field of embedded system data transmission. The method sets a total number of reusable descriptor slots in a descriptor slot pool, and divides the descriptor slots into a current slot bank and a next slot bank. A plurality of physical segments are divided into carrying windows according to the number of descriptors that can be accommodated by a single slot bank, and native descriptors of a current window are written into the current slot bank. The cache maintenance range is determined according to the source address and the destination address of the window, the current slot bank is released after the cache maintenance before carrying is completed, and the native descriptors of a next window are written into the next slot bank during the execution of the current window. When the native descriptors in the current window are continuously completed and the cache maintenance after carrying is completed, the end position of the window is written into a checkpoint record and serves as a new verified water level, and then the slot bank is released for reuse by a subsequent window. The application reduces the processor intervention degree under the fixed descriptor budget, and the carrying progress can be confirmed and recovered.

Description

Technical Field

[0001] This application relates to the field of embedded system data transmission technology, and in particular to a data transfer method, system and storage medium under a fixed descriptor budget. Background Technology

[0002] In embedded devices, during data acquisition, image processing, and data exchange between memory and peripherals, the processor typically delegates the transfer of batches of data to a Direct Memory Access (DMA) controller. The processor first prepares several descriptors, each specifying the source address, destination address, transfer length, and transfer attributes. The DMA controller then uses these descriptors to access the memory or peripherals.

[0003] When the data to be transferred is continuous, only a small number of descriptors are needed to complete the description. When the data to be transferred is scattered across multiple non-contiguous physical segments, it is usually necessary to create scattered cluster descriptors for each physical segment and link the descriptors into a chain or queue them. If the total number of descriptors does not exceed the capacity of the hardware queue or on-chip storage, the processor can generate a complete descriptor chain before the transfer task begins, which is then continuously traversed and executed by the DMA controller.

[0004] However, in real-world systems, the number of descriptor slots is often fixed, while the number of physical segments in a transport task may far exceed the number of descriptor slots. If the entire descriptor chain is allowed to reside for an extended period during task execution, the descriptors themselves will consume significant storage space and read bandwidth. If the next descriptor is temporarily generated only after a descriptor slot becomes available, the processor will need to intervene frequently, and the DMA controller may also stall while waiting for descriptors.

[0005] Furthermore, descriptor slot reuse needs to be coordinated with cache maintenance: before DMA reads source data, the processor may need to write back the corresponding cache contents; after DMA writes destination data, the processor may need to invalidate the corresponding cache before reading that data. If the processor only confirms that the descriptor has been completed, but cannot confirm whether the cache maintenance and the corresponding transfer progress record corresponding to the descriptor have also been completed, it may prematurely overwrite descriptor slots that are still in use, or mistakenly assume that data that is not yet visible to the processor has been completed, leading to data inconsistency.

[0006] Therefore, there is an urgent need for a data movement method that can move data continuously and efficiently while ensuring cache consistency and reducing processor intervention under a fixed descriptor budget. Summary of the Invention

[0007] This application provides a data transfer method, system, and storage medium under a fixed descriptor budget.

[0008] To achieve the above objectives, this application adopts the following technical solution: In a first aspect, this application provides a data transfer method under a fixed descriptor budget, applied to a data transfer system including a processor, memory, and a DMA controller, comprising the following steps: S1. Obtain the task information to be transported, which includes multiple physical segments arranged in logical transport order, and each physical segment records the source address, destination address, length and transmission attributes. S2. Set up reusable descriptor slots in the descriptor slot pool. The total number of reusable descriptor slots remains unchanged during the execution of the transport task. Divide the reusable descriptor slots into the current slot pool and the next slot pool. S3. The multiple physical segments are truncated or combined according to the number of descriptors that a single slot can hold to divide them into multiple transport windows. The native descriptor of the current transport window is generated and written to the current slot. The native descriptor is a descriptor that the DMA controller can directly execute. S4. Determine the cache maintenance range of the current transfer window based on the source address range and destination address range of the current transfer window. After completing the pre-transfer cache maintenance of the cache maintenance range, publish the current slot library to the DMA controller. When there is a next transfer window, during the data transfer corresponding to the current transfer window executed by the DMA controller, write the native descriptor corresponding to the next transfer window into the next slot library. S5. Receive the completion status returned by the DMA controller, and perform post-transfer cache maintenance on the cache maintenance range corresponding to the current transfer window; when the native descriptors of the current transfer window are completed consecutively and the post-transfer cache maintenance corresponding to the current transfer window is completed, write the end position of the current transfer window into the window checkpoint record, and use the end position of the current transfer window as the new verified water level; when the DMA controller no longer references the current slot library and the window checkpoint record is written, release the descriptor slots in the current slot library for reuse by subsequent transfer windows; wherein, the verified water level represents the continuous transfer end position jointly confirmed by the completion status returned by the DMA controller and the completion status of the post-transfer cache maintenance.

[0009] In conjunction with the first aspect, in one possible implementation, the pre-transfer cache maintenance includes: writing cached data within the cache maintenance range back to the memory before the DMA controller performs the data transfer corresponding to the current transfer window; The post-transfer cache maintenance includes: invalidating cached data within the cache maintenance range after the DMA controller completes the data transfer corresponding to the current transfer window.

[0010] In conjunction with the first aspect, in one possible implementation, step S1, obtaining the task information to be transported includes: The task information to be transported is encoded into a compact transmission script; the compact transmission script includes a base record and a differential record, the base record records the source base address and the destination base address, and the differential record records the source address increment and the destination address increment relative to the previous record.

[0011] In conjunction with the first aspect, in one possible implementation, step S3, the step of truncating or combining the plurality of physical segments to divide them into transport windows according to the number of descriptors that a single slot can accommodate, includes: From the candidate end positions of the transport window, exclude those that would cause the number of native descriptors after the transport window is expanded to exceed the number of descriptors that a single slot library can accommodate, cause the transmission length indicated by a single native descriptor to exceed the single transmission length limit of the DMA controller, cause the address corresponding to the end position to not meet the preset alignment requirements, or cause the transport window to cross an indivisible data dependency boundary. Then, determine the end position of the transport window based on at least one of the number of native descriptors generated after the transport window is expanded and the amount of cache maintenance corresponding to the transport window.

[0012] In conjunction with the first aspect, in one possible implementation, when writing the native descriptor corresponding to the next transport window into the next slot library in step S4, the following steps are executed sequentially: Invalidate the descriptor slot to be written in the next slot library; write the field of the native descriptor to the descriptor slot; perform a synchronization operation to make the field visible to the DMA controller; and invalidate the descriptor slot, or issue a queue tail pointer indicating that the descriptor boundary can be read. After the data transfer corresponding to the current transfer window is completed, the roles of the current slot and the next slot are swapped, so that the next slot becomes the new current slot.

[0013] In conjunction with the first aspect, in one possible implementation, step S3 further includes: forming a window control record for each transport window, the window control record including the source start and end positions, destination start and end ranges, and cache maintenance range of the transport window; the cache maintenance range consists of a source cache maintenance range and a destination cache maintenance range, the source cache maintenance range being the range obtained by rounding the source address range involved in the transport window according to the cache line boundary, and the destination cache maintenance range being the range obtained by rounding the destination address range involved in the transport window according to the cache line boundary.

[0014] In conjunction with the first aspect, in one possible implementation, the window control record further includes a window summary, which is generated from the source start and end positions, destination start and end ranges, native descriptor sequence, cache maintenance range, and version information of the transport window; when writing the end position of the current transport window into the window checkpoint record, the checkpoint content and the window summary are written first, and the valid state is written after the storage barrier is executed.

[0015] In conjunction with the first aspect, in one possible implementation, when a transport anomaly is detected, for the data source that can re-acquire data corresponding to the task information to be transported, the native descriptor corresponding to the incomplete transport window is reconstructed from the most recent verified water level.

[0016] In a second aspect, this application provides a data transfer system under a fixed descriptor budget, including a processor, a memory, and a DMA controller; the processor is configured to perform the method as described in the first aspect or any of the implementations thereof.

[0017] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in the first aspect or any of the implementations thereof.

[0018] Compared with the prior art, the beneficial effects of this application are as follows: First, by keeping the total number of reusable descriptor slots constant during the transport task execution and dividing the transport window according to the number of descriptors that a single slot library can accommodate, the number of native descriptors residing on the DMA controller side at any given time can be limited to a fixed descriptor budget, thereby reducing the bus overhead of the DMA controller reading descriptors and improving the overall throughput.

[0019] Second, by forming a dual-slot library with the current slot library and the next slot library, and writing the native descriptor of the next transfer window into the next slot library during the execution of the current transfer window by the DMA controller, the transfer execution of the DMA controller and the pre-filling of the descriptor can overlap, which helps to reduce the processor intervention and hide the configuration delay between transfer windows.

[0020] Third, by determining the cache maintenance scope based on the source address range and destination address range of the current transport window, cache maintenance can be performed precisely at the granularity of the transport window, avoiding unnecessary cache synchronization operations on data areas unrelated to the window, which helps reduce the latency caused by cache consistency maintenance.

[0021] Fourth, by using the end position of the window as the new verified water level only when the native descriptors of the current transport window are completed consecutively and the cache maintenance after the transport is completed, the verified water level can represent the consecutive transport position where both DMA transport and cache synchronization are completed, thus ensuring that the transport progress can be confirmed. In case of an anomaly, the incomplete part can be reconstructed from the position corresponding to the verified water level, which can avoid a full retransmission and help save the computing and bandwidth resources of the embedded system.

[0022] Fifth, by releasing the descriptor slots after the window checkpoint record shows that the write is complete and the DMA controller no longer references the current slot library, it is possible to support any number of transport windows to reuse the same batch of descriptor slots in sequence within a fixed descriptor budget, which helps to reduce the descriptor storage resource consumption. Attached Figure Description

[0023] Figure 1 A structural block diagram of a data transfer system provided in an embodiment of this application; Figure 2 A schematic diagram illustrating the division of the current slot and the next slot in the descriptor slot pool provided in this application embodiment; Figure 3 A flowchart illustrating a data transfer method under a fixed descriptor budget, provided as an embodiment of this application; Figure 4 This is a schematic diagram illustrating the division of the handling window and the execution of the dual-tank conveyor line as provided in the embodiments of this application. Detailed Implementation

[0024] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] For ease of description, the relevant terms in this application are defined as follows.

[0026] Direct Memory Access (DMA) is a technique used to describe descriptors that a DMA controller can directly read and execute. A transport set, consisting of a subset of physical segments arranged in logical transport order, is provided to the DMA controller for execution in a single slot library. This transport set is called a transport window. Descriptor storage locations that remain constant in number during transport task execution and can be repeatedly used to carry different transport windows are called reusable descriptor slots. All reusable descriptor slots constitute a descriptor slot pool 250. A set of slots used by the current transport window is called the current slot library 251, and a set of slots pre-written for the next transport window is called the next slot library 252. The number of descriptors a single slot library can hold is equal to the number of reusable descriptor slots allocated to that library. A window checkpoint record records information about the confirmed end position of a transport window. The verified watermark indicates the consecutive transport end positions confirmed by the completion status returned by the DMA controller 130 and the completion status maintained by the post-transport cache. FENCE records are used to mark non-crossable boundaries, defining indivisible data dependency boundaries or complete transport unit boundaries.

[0027] Example 1 This embodiment provides a data transfer method under a fixed descriptor budget, which will be described below in conjunction with... Figures 1-4 The system architecture, descriptor slot pool, and overall process are described. For ease of description, this embodiment uses a single memory-to-memory data transfer task as an example: the data to be transferred is distributed across eight physical segments arranged in logical transfer order. The lengths of the eight physical segments are 4MB, 8MB, 2MB, 16MB, 1MB, 10MB, 3MB, and 20MB, respectively, totaling 64MB; the maximum single transfer length of the DMA controller 130 is, for example, 16MB. The above values ​​are only used to illustrate the implementation process and do not constitute a limitation on the scope of protection of this application. The physical segments are contiguous physical memory regions: physical segments whose length does not exceed the maximum single transfer length of the DMA controller 130 directly correspond to one native descriptor; physical segments whose length exceeds this limit, such as the 20MB physical segment in this example, are split into multiple contiguous sub-segments whose length does not exceed this limit, and each sub-segment corresponds to one native descriptor.

[0028] like Figure 1As shown, the data transfer system 100 includes a processor 110, a memory 120, and a DMA controller 130. The memory 120 stores information about tasks to be transferred, a descriptor slot pool 250, and window checkpoint records. The information about tasks to be transferred includes multiple physical segments arranged in logical transfer order, each physical segment recording a source address, destination address, length, and transfer attributes. The processor 110 is configured to execute the data transfer method under a fixed descriptor budget provided in this application. It is responsible for obtaining information about tasks to be transferred, dividing the transfer window, generating and writing native descriptors, initiating cache maintenance, publishing the slot pool to the DMA controller 130, receiving completion status, submitting window checkpoint records, and managing the release of descriptor slots. The DMA controller 130 is used to read native descriptors from the published slot pool, perform data transfer according to the source address, destination address, length, and transfer attributes recorded in the native descriptors, and return a completion status to the processor 110. The DMA controller 130 only reads and executes the published native descriptors, without needing to read task management information such as physical segment sequences and window control records. Thus, this application can be implemented without modifying the existing descriptor format and transport state machine of the DMA controller 130.

[0029] like Figure 2 As shown, the processor 110 sets up reusable descriptor slots in the descriptor slot pool 250. The total number of reusable descriptor slots remains constant during the execution of the transport task. This total number of slots is the maximum number of native descriptors that the task can simultaneously occupy on the DMA controller 130 side, and is called the fixed descriptor budget. All reusable descriptor slots are divided into two non-overlapping groups: one group used by the current transport window is called the current slot library 251, and the other group used for pre-writing by the next transport window is called the next slot library 252. The number of descriptors that a single slot library can hold is the number of descriptor slots allocated to that slot library. The initialization and grouping of the slot pool can be completed once before the task starts, or it can be completed during task preparation at the start of the task. The capacity of the two slot libraries can be flexibly allocated within the fixed budget according to the window expansion size. For example, this application sets the fixed descriptor budget to 8, where the current slot library 251 corresponds to A0 to A3 of slot library A, and the next slot library 252 corresponds to B0 to B3 of slot library B.

[0030] Each descriptor slot sequentially goes through the states of FREE, BUILDING, READY, ACTIVE, and RETIRED, before returning to the FREE state. Slot states are used to identify slot ownership: once the DMA controller 130 begins referencing a slot in the ACTIVE state, the processor 110 must not overwrite that slot; after a slot enters the RETIRED state, it must wait for the window checkpoint record of the corresponding transport window to be committed before returning to the FREE state for reuse by subsequent transport windows.

[0031] The following is combined Figure 3 , Figure 4 This paper provides a detailed description of a data transfer method under a fixed descriptor budget. For ease of description, the transfer windows will be referred to as W0, W1, and W2 respectively.

[0032] S1. Obtain the task information to be transported, which includes multiple physical segments arranged in logical transport order, and each physical segment records the source address, destination address, length and transmission attributes. Specifically, the processor 110 receives a data transfer request from the upper layer and obtains the task information to be transferred. The task information to be transferred consists of multiple physical segments arranged in logical transfer order: physical segments represent data ranges whose source addresses are not contiguous and need to be transferred separately; the logical transfer order represents the order in which each physical segment must maintain its data sequence. This order is not only the basis for arranging physical segments in the task information, but also the basis for subsequently generating native descriptors, determining whether the completion status is continuous, and advancing the verified watermark. Each physical segment records four elements: source address, which is a physical address accessible by the DMA controller 130 or a device address mapped from the input / output address; destination address, which is the destination address of the data to be transferred; length, which is the amount of data continuously transferred in this physical segment; and transfer attributes, which describe additional requirements for the transfer of this segment, such as whether synchronous write-back is required, the direction of cache maintenance, transfer priority, or whether the source address is fixed.

[0033] S2. Set up reusable descriptor slots in the descriptor slot pool 250. The total number of reusable descriptor slots remains unchanged during the execution of the transport task. Divide the reusable descriptor slots into the current slot pool 251 and the next slot pool 252.

[0034] Specifically, the processor 110 sets up reusable descriptor slots in the descriptor slot pool 250 to keep the total number of slots constant during the execution of the transport task, and divides the reusable descriptor slots into the current slot library 251 and the next slot library 252. After grouping, the current slot library 251 is used as the write target for the first transport window, and the next slot library 252 is used as the target for pre-writing the second transport window during the execution of the first transport window.

[0035] S3. The multiple physical segments are truncated or combined according to the number of descriptors that a single slot can hold to divide them into multiple transport windows. The native descriptor of the current transport window is generated and written to the current slot 251. The native descriptor is a descriptor that the DMA controller 130 can directly execute.

[0036] Specifically, the processor 110 truncates or combines multiple physical segments to form transport windows based on the number of descriptors a single slot can hold. Starting from the first undivided physical segment, candidate segments are accumulated sequentially according to the logical transport order, ensuring that the number of native descriptors generated after each transport window is expanded does not exceed the number of descriptors a single slot can hold. This guarantees that the native descriptors of a transport window can be written to a single slot. The term "expansion" here refers to the process of converting physical segments into descriptors that the DMA controller 130 can directly read and execute, according to the native descriptor format supported by the DMA controller 130. Physical segments and native descriptors are not one-to-one. When the length of a physical segment exceeds the single transfer length limit of the DMA controller 130, or is constrained by address alignment and data dependencies, a physical segment can be expanded into multiple native descriptors. Subsequently, the processor 110 generates the native descriptors for the current transport window and writes them sequentially to each descriptor slot in the current slot 251. The format of the native descriptors is consistent with the native descriptor format supported by the DMA controller 130. Figure 4 As shown, under the conditions of a single transmission length limit of 16MB and a single slot library accommodating 4 native descriptors, the processor 110 divides the 1st to 4th physical segments into the first transport window W0; divides the 5th to 7th physical segments, and the portion of the 8th physical segment before its internal length of 16MB, into the second transport window W1, which expands into 4 native descriptors; divides the remaining portion of the 8th physical segment after this position, with a length of 4MB, into the third transport window W2, which expands into 1 native descriptor. Although the 8th physical segment is divided into two parts by its internal length of 16MB, belonging to W1 and W2 respectively, this physical segment itself is still recorded as a single physical segment; the window boundaries only determine the generation batch of native descriptors and do not change the source address and data content of the physical segments.

[0037] S4. Determine the cache maintenance range of the current transport window based on the source address range and destination address range of the current transport window. After completing the pre-transport cache maintenance of the cache maintenance range, publish the current slot library 251 to the DMA controller 130. When there is a next transport window, during the data transport corresponding to the current transport window executed by the DMA controller 130, write the native descriptor corresponding to the next transport window into the next slot library 252.

[0038] Specifically, the processor 110 determines the cache maintenance range of the current transfer window W0 based on the source address range and destination address range of W0, and performs pre-transfer cache maintenance on this cache maintenance range before publishing W0, so that the DMA controller 130 can obtain the latest data when reading from the source address. After the pre-transfer cache maintenance is completed, the processor 110 publishes the current slot library 251 to the DMA controller 130, for example, by notifying the DMA controller 130 of the chain head address, queue tail pointer, or doorbell value of the current slot library 251. The DMA controller 130 then begins to execute the data transfer corresponding to W0. When a next transfer window W1 exists, while the DMA controller 130 is executing W0, the processor 110 writes the native descriptor corresponding to W1 to the next slot library 252. Since the DMA controller 130 currently only references the current slot library 251, the processor 110's write to the next slot library 252 will not interfere with the ongoing transfer, thereby achieving the overlap of the current window transfer execution and the next window descriptor preparation on different slot libraries.

[0039] It should be noted that, in order to avoid the DMA controller 130 idling and waiting because the native descriptor of the next transport window has not been prewritten, this application sets up a prewrite readiness detection mechanism: before publishing the slot library corresponding to any transport window, the processor 110 checks whether the slot library has been filled and is in a ready state; if not, it is published after the native descriptor writing and pre-transport cache maintenance are completed, so as to avoid publishing an unready slot library to the DMA controller 130.

[0040] S5. Receive the completion status returned by the DMA controller 130, and perform post-transfer cache maintenance on the cache maintenance range corresponding to the current transfer window; when the native descriptors of the current transfer window are completed consecutively and the post-transfer cache maintenance corresponding to the current transfer window is completed, write the end position of the current transfer window into the window checkpoint record, and use the end position of the current transfer window as the new verified water level; when the DMA controller 130 no longer references the current slot library 251 and the window checkpoint record is written, release the descriptor slots in the current slot library 251 for reuse by subsequent transfer windows; wherein, the verified water level represents the continuous transfer end position jointly confirmed by the completion status returned by the DMA controller 130 and the completion status of the post-transfer cache maintenance.

[0041] Specifically, after the DMA controller 130 completes the transfer of W0, it returns the completion status to the processor 110 through a completion pointer, descriptor status word, completion queue, or completion interrupt. The processor 110 checks the completion status according to the logical transfer order, determining whether the completed native descriptors of W0, starting from the beginning descriptor of the transfer window, have no incomplete intermediate descriptors in the logical transfer order, i.e., the completed native descriptors form a continuous sequence from the beginning of the window without gaps; simultaneously, it performs post-transfer cache maintenance on the cache maintenance range corresponding to the window, making the result of the DMA controller 130 writing to the destination address visible to the processor 110. Only when the native descriptors of W0 are completed consecutively and the post-transfer cache maintenance corresponding to W0 is completed, does the processor 110 write the end position of W0 into the window checkpoint record and use this end position as the new verified watermark; if there are gaps in the completion status, the post-transfer cache maintenance is not completed, or the window digest verification is inconsistent, even if the DMA controller 130 returns a partial completion status, the verified watermark will not be advanced. The processor 110 releases the descriptor slots in the current slot library 251 only after the DMA controller 130 no longer references the current slot library 251 (e.g., the DMA controller 130 has moved past all descriptors in the current slot library 251 or has returned slot ownership to the processor 110) and the window checkpoint record has been written. This restores the slots to an idle and writable state for reuse by subsequent transport windows. It has been verified here that the watermark represents the continuous transport end position jointly confirmed by the completion status returned by the DMA controller 130 and the completion status of the post-transport cache maintenance. It is only advanced after both the DMA completion and cache maintenance of the window are confirmed, thus providing a reliable rollback baseline for anomaly recovery.

[0042] Understandably, in this application, the processor 110 advances each transport window in a cyclical manner according to S1-S5: During the execution of W0, the processor 110 completes the pre-writing of the native descriptor of W1 and the pre-transport cache maintenance in the next slot 252; once the completion status of W0 and the post-transport cache maintenance are confirmed, the processor 110 submits the window checkpoint record corresponding to W0 and advances the verified watermark to the 30MB position; then, the ready next slot 252 is released to start the execution of W1, and at the same time, the current slot 251 occupied by W0 is released for pre-writing the native descriptor of W2; after W1 is completed and the checkpoint is submitted, the verified watermark advances to the 60MB position; finally, W2 is released and the checkpoint is submitted, the verified watermark advances to the task endpoint of 64MB position, and the processor 110 ends the task after confirming that there are no transport windows to be executed or released. During the above process, the number of native descriptors residing on the DMA controller 130 side at any given time does not exceed the fixed descriptor budget. The processor 110 does not need to maintain a complete chain of native descriptors for the entire task in the long term. Multiple transfer windows can be transferred continuously by alternating between two slots for preparation and release. Furthermore, the amount of data that has been reliably completed at any given time can be uniquely determined by the verified water level.

[0043] Furthermore, the pre-transfer cache maintenance in this application refers to writing the cached data within the cache maintenance range back to the memory 120 before the DMA controller 130 executes the data transfer corresponding to the current transfer window. Specifically, when the data corresponding to the source address is currently residing in the cache on the processor 110 side and may not have been written back to the memory 120, the processor 110 performs a write-back / clean operation on the cache maintenance range before releasing the current slot library 251, so that the DMA controller 130 can obtain the latest data when reading from the source address, avoiding the DMA reading outdated cached data.

[0044] Post-transfer cache maintenance refers to invalidating cached data within the cache maintenance range after the DMA controller 130 completes the data transfer corresponding to the current transfer window. Specifically, after the DMA controller 130 writes data to the destination address, the write result may not yet be reflected in the cache on the processor 110 side; after the DMA transfer is completed, the processor 110 performs an invalidation operation on the cache lines within the cache maintenance range, so that when the processor 110 subsequently accesses this range, it rereads from the memory 120 to obtain the data after the DMA write.

[0045] This application ensures consistency between the cache and memory before and after DMA migration through targeted maintenance of write-back before migration and invalidation after migration, and also ensures that the DMA write results are visible to the processor 110. The current slot library 251 is only released after the cache maintenance before migration is completed, and the completion of cache maintenance after migration is one of the conditions for advancing the verified water level, thereby binding cache maintenance and window completion status to the same window boundary.

[0046] In one possible implementation, when the task to be transferred contains a large number of physical segments, the processor 110 can encode the task information into a compact transfer script to reduce the occupation of task information in memory 120 and the overhead of the processor 110 and the DMA path reading task information. The compact transfer script includes a base record and a differential record: the base record records the source base address and destination base address of the base point of the first physical segment or the current segment sequence; each subsequent physical segment corresponds to a differential record, which records the source address increment of the segment relative to the source address of the previous record, and the destination address increment relative to the destination address of the segment.

[0047] When the task information to be transferred is encoded as a compact transfer script, in step S3, during the process of truncating or combining multiple physical segments into transfer windows according to the number of descriptors that a single slot can hold, the processor 110 sequentially parses the compact transfer script. From the source and destination reference addresses given by the reference record and the increments given by each differential record, it reconstructs the source address, destination address length, and transfer attributes of each physical segment one by one. Then, each physical segment is expanded into a raw descriptor according to the single transfer length limit of the DMA controller 130. Finally, based on the number of expanded raw descriptors, the physical segments are sequentially divided into transfer windows according to the number of descriptors that a single slot can hold, and a raw descriptor corresponding to each transfer window is generated. Since there is usually only an address offset difference between consecutive discrete segments, using a combination of reference records and differential records can completely represent the entire sequence of physical segments with a smaller storage requirement.

[0048] In one possible implementation, when dividing the transport window, the processor 110 sequentially examines each candidate position from the candidate transport window end positions (e.g., the positions corresponding to the boundaries of each physical segment) and eliminates candidate end positions that would lead to any of the following situations: This causes the number of native descriptors in the expanded transport window to exceed the number of descriptors a single slot library can hold. Since the native descriptors of a transport window are written to a single slot library as a whole, if the number exceeds the limit after expansion, the slot library cannot accommodate them all at once, so this should be eliminated.

[0049] This causes the transfer length indicated by a single native descriptor to exceed the single transfer length limit of the DMA controller 130. The DMA controller 130 has an upper limit constraint on the length of a single transfer, and native descriptors exceeding the limit cannot be executed by the DMA controller 130, so they should be excluded.

[0050] This can cause the address corresponding to the end position to not meet the preset alignment requirements. For example, the DMA controller 130 or the bus requires the source address, destination address, or transfer length to meet alignment constraints at the word, cache line, or other granularities. An end position that does not meet the alignment requirements will prevent the generated descriptor from being executed or from being efficiently cached, so it should be excluded.

[0051] This causes the transport window to cross an indivisible data dependency boundary. An indivisible data dependency boundary refers to a physical segment or group of physical segments that must be transported continuously as a whole, cannot be split by window boundaries, or has an insurmountable boundary marked by a FENCE record; the window's ending position must not fall within this boundary. It's important to note that indivisible data dependency boundaries are finer-grained data integrity constraints than the transport window granularity, and can be explicitly marked using FENCE records. When dividing a window, if a physical segment does not contain a FENCE marker, it can be split into multiple sub-segments based on the maximum length of a single transport, belonging to different transport windows; if a physical segment contains a FENCE marker, the window's ending position must not fall within the indivisible boundary defined by that FENCE marker. In other words, the physical segment is the default candidate boundary granularity for window division, and the FENCE record is a mandatory constraint that further restricts the window splitting position within the physical segment. Following the previous example, the 8th physical segment (20MB) is not marked with a FENCE, so it can be split into different transport windows at 16MB; if a physical segment is marked with a FENCE, the window boundary must avoid the FENCE boundary.

[0052] After completing the above elimination process, the processor 110 determines the ending position of the transport window based on at least one of the following: the number of native descriptors generated after the transport window is expanded and the corresponding cache maintenance amount. For example, among the candidate positions that satisfy the constraints, the position that results in a larger number of native descriptors after expansion can be prioritized; alternatively, when the number of generated native descriptors is the same, the position that results in a smaller cache maintenance range can be prioritized; or the position can be determined after a comprehensive trade-off between the two. In this way, the window boundary not only conforms to hard constraints such as slot capacity, single transfer length, address alignment, and data dependency, but also achieves a balance between descriptor generation overhead and cache maintenance overhead.

[0053] In one possible implementation, while the DMA controller 130 is performing data transfer for the current transfer window, the processor 110 writes the native descriptor corresponding to the next transfer window to the next slot library 252. To avoid the DMA controller 130 or the processor 110 observing descriptor slots with inconsistent states, the processor 110 sequentially performs the following write timing for each descriptor slot to be written in the next slot library 252: Slot invalidation: Before writing, the descriptor slot is invalidated so that the DMA controller 130 does not treat the old contents in the slot as a readable descriptor; Write fields: Write the fields of the native descriptor for the next transport window (such as source address, destination address, length, transport attributes, and necessary link information) to this descriptor slot. Synchronization operation: Perform synchronization operations to make the written field visible to the DMA controller 130, such as a storage barrier or a corresponding cache cleanup / synchronization instruction, to ensure that the field observed by the DMA controller 130 is the field after it has been fully written, rather than the intermediate state of a partially written field. Slot Validation or Queue Tail Pointer Release: Validate the descriptor slot to make it readable, or release a queue tail pointer indicating the boundary of a readable descriptor to inform the DMA controller 130 that it can begin reading the descriptor in the slot.

[0054] After the data transfer for the current transfer window is completed, the processor 110 swaps the roles of the current slot library 251 and the next slot library 252, making the next slot library 252 the new current slot library 251. After the role swap, the next slot library 252, which originally carried the native descriptor of the next transfer window, becomes the new current slot library and can be released to the DMA controller 130 for execution at an appropriate time; while the descriptor slots in the original current slot library 251, after being released, can be used as the new next slot library for pre-writing of subsequent transfer windows. By alternating between the two slot libraries, the pipelining of descriptor preparation and the cyclic reuse of slots are achieved.

[0055] In one possible implementation, the processor 110 forms a window control record for each transport window. The window control record includes: the source start and end positions, the destination start and end range, and the cache maintenance range of the transport window. The source start and end positions identify the start and end positions of the physical segments contained in the window on the source side; the destination start and end range identifies the start and end addresses of the window on the destination side; and the cache maintenance range identifies the address range for which cache maintenance is required before and after the transport.

[0056] The cache maintenance scope consists of the source cache maintenance scope and the destination cache maintenance scope. The source cache maintenance scope is the range obtained by rounding the source address range involved in the transport window to the cache line boundary; the destination cache maintenance scope is the range obtained by rounding the destination address range involved in the transport window to the cache line boundary.

[0057] The window control record binds the source ranges, destination ranges, and cache ranges that need to be maintained within the same transport window boundary. When the processor 110 generates the native descriptor, it does so based on the source start and end ranges and the destination start and end ranges. When it initiates cache maintenance, it does so based on the cache maintenance range. When it submits the window checkpoint record, it does so based on the same window control record, thereby ensuring that the native descriptors, cache maintenance, and window checkpoint records within the same transport window correspond to each other consistently.

[0058] In one possible implementation, the window control record also includes a window summary. The window summary is summary data (e.g., a checksum or summary value obtained by verifying the above content) generated based on the source and end positions, destination and end ranges, native descriptor sequence, cache maintenance range, and version information of the transport window. The version information includes, for example, the window sequence number, slot library version, or checkpoint version. The window summary is used to verify the consistency between the window content and the checkpoint content during checkpoint submission and exception recovery, preventing erroneous checkpoint submissions due to slot content, cache range, or version mismatches.

[0059] When submitting a window checkpoint record, the processor 110 executes the following sequence: first, it writes the checkpoint content and window summary; then, it executes the storage barrier; and finally, it writes the validity status. The checkpoint content includes information such as the verified end position of the window and the starting point of the next transport window. The validity status is the status information in the window checkpoint record used to identify whether the record is valid, such as a validity flag. By first writing the checkpoint content and window summary, and then ensuring their writing order and visibility through the storage barrier, the validity status is written, making the window checkpoint record appear valid externally. When the processor 110 reads the window checkpoint record, it uses the validity status as the basis for determining whether the record is valid. If an exception occurs during the submission process, since the validity status has not yet been written, the window checkpoint record will not be considered valid. The processor 110 will still use the verified watermark corresponding to the previously confirmed valid window checkpoint record for recovery, thereby avoiding reading incomplete checkpoint records.

[0060] In one possible implementation, when the processor 110 or the DMA controller 130 detects a transport anomaly, such as a DMA bus error, descriptor verification failure, a gap in the completion status, or a cache maintenance failure, different recovery strategies are adopted based on the nature of the data source.

[0061] For data sources where data can be retrieved again, such as data sources whose source data is still intact and can be read again, the processor 110 reconstructs the native descriptors corresponding to the incomplete transport windows, starting from the most recent verified watermark. Specifically, the processor 110 reads the source start and end positions, destination start and end ranges, and other information of the incomplete transport windows after the verified watermark, regenerates the native descriptors of these transport windows, writes them to the released descriptor slots, and re-executes the corresponding cache maintenance and transport. Since the data transport before the verified watermark has been confirmed as complete by the DMA completion status and the post-transport cache maintenance, the recovery process only needs to start again from after that watermark, without redoing the confirmed completed parts, which helps to reduce the computational and bandwidth overhead caused by abnormal recovery.

[0062] For peripheral input sources where data cannot be retrieved again, such as data generated unidirectionally by the peripheral, or data sources that are non-retrievable or non-replayable, the processor 110 uses FENCE records as boundary markers to mark the boundaries of complete transport units when organizing the information to be transported. This ensures that the complete transport unit defined by the FENCE record is transported as an indivisible whole. When a complete transport unit is not fully completed, the processor 110 invalidates the destination data corresponding to that incomplete unit to prevent incomplete data from being read as valid data by the processor 110 or subsequent processing. The destination data corresponding to a complete transport unit defined by the FENCE record is only considered valid after the entire unit is completed, thus ensuring the transport consistency of non-retrievable data.

[0063] It should be noted that for peripheral input sources, the source data does not pass through the processor 110 cache, and there is no source-side cache data that needs to be written back. Therefore, no source-side write-back operation is required, and cache maintenance after the transfer only requires a destination-side invalidation operation. For memory-to-memory transfer scenarios, cache maintenance before the transfer performs a source-side write-back operation, and cache maintenance after the transfer performs a destination-side invalidation operation. The processor 110 determines the actual direction of cache maintenance based on the transfer attributes of the physical segment.

[0064] Example 2 This embodiment provides a data transfer system under a fixed descriptor budget, including a processor 110, a memory 120, and a DMA controller 130. The memory 120 stores a computer program and data such as information about tasks to be transferred, a descriptor slot pool 250, and window checkpoint records. The processor 110 is configured to execute the computer program stored in the memory 120 to implement the aforementioned data transfer method under a fixed descriptor budget, including operations such as obtaining information about tasks to be transferred, dividing the transfer window, generating and writing native descriptors, initiating cache maintenance, publishing the slot pool, receiving completion status, advancing the verified watermark, and releasing descriptor slots. Under the control of the processor 110, the DMA controller 130 reads the native descriptors in the published slot pool and performs the corresponding data transfer, returning a completion status to the processor 110.

[0065] Example 3 This embodiment provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the data transfer method under the fixed descriptor budget described above. The computer-readable storage medium can be various media capable of storing program code, such as a USB flash drive, external hard drive, read-only memory, random access memory, magnetic disk, or optical disk.

[0066] It should be emphasized that the above are merely preferred embodiments of this application and are not intended to limit this application in any way. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A data transfer method under a fixed descriptor budget, applied to a data transfer system (100) including a processor (110), a memory (120), and a DMA controller (130), characterized in that, Includes the following steps: S1. Obtain the task information to be transported, which includes multiple physical segments arranged in logical transport order, and each physical segment records the source address, destination address, length and transmission attributes. S2. Set up reusable descriptor slots in the descriptor slot pool (250). The total number of reusable descriptor slots remains unchanged during the execution of the transport task. Divide the reusable descriptor slots into the current slot pool (251) and the next slot pool (252). S3. The multiple physical segments are truncated or combined according to the number of descriptors that a single slot can hold to divide them into multiple transport windows, and the native descriptor of the current transport window is generated and written into the current slot (251). The native descriptor is a descriptor that the DMA controller (130) can directly execute. S4. Determine the cache maintenance range of the current transport window based on the source address range and destination address range of the current transport window. After completing the pre-transport cache maintenance of the cache maintenance range, publish the current slot library (251) to the DMA controller (130). When there is a next transport window, during the data transport corresponding to the current transport window executed by the DMA controller (130), write the native descriptor corresponding to the next transport window into the next slot library (252). S5. Receive the completion status returned by the DMA controller (130), and perform post-transfer cache maintenance on the cache maintenance range corresponding to the current transfer window; When the native descriptors of the current transport window are completed consecutively and the post-transport cache maintenance corresponding to the current transport window is completed, the end position of the current transport window is written into the window checkpoint record, and the end position of the current transport window is used as the new verified water level; when the DMA controller (130) no longer references the current slot library (251) and the window checkpoint record is written, the descriptor slots in the current slot library (251) are released for reuse by subsequent transport windows; wherein, the verified water level represents the continuous transport end position jointly confirmed by the completion status returned by the DMA controller (130) and the completion status of the post-transport cache maintenance.

2. The data transfer method under a fixed descriptor budget according to claim 1, characterized in that, The pre-transfer cache maintenance includes: writing cached data within the cache maintenance range back to the memory (120) before the DMA controller (130) performs the data transfer corresponding to the current transfer window. The post-transfer cache maintenance includes: after the DMA controller (130) completes the data transfer corresponding to the current transfer window, invalidating the cached data within the cache maintenance range.

3. The data transfer method under a fixed descriptor budget according to claim 1, characterized in that, In step S1, obtaining the information of the task to be transported includes: The task information to be transported is encoded into a compact transmission script; the compact transmission script includes a base record and a differential record, the base record records the source base address and the destination base address, and the differential record records the source address increment and the destination address increment relative to the previous record.

4. The data transfer method under a fixed descriptor budget according to claim 1, characterized in that, Step S3, which involves truncating or combining the multiple physical segments to divide them into transport windows according to the number of descriptors that a single tank can hold, includes: From the candidate end positions of the transport window, exclude those that would cause the number of native descriptors after the transport window is expanded to exceed the number of descriptors that a single slot library can accommodate, cause the transmission length indicated by a single native descriptor to exceed the single transmission length limit of the DMA controller (130), cause the address corresponding to the end position to not meet the preset alignment requirements, or cause the transport window to cross an indivisible data dependency boundary. Then, determine the end position of the transport window based on at least one of the number of native descriptors generated after the transport window is expanded and the amount of cache maintenance corresponding to the transport window.

5. The data transfer method under a fixed descriptor budget according to claim 1, characterized in that, In step S4, when writing the native descriptor corresponding to the next transport window into the next slot library (252), the following steps are executed sequentially: The descriptor slot to be written in the next slot library (252) is invalidated; the field of the native descriptor is written to the descriptor slot; and a synchronization operation is performed to make the field visible to the DMA controller (130). And perform slot validation on the descriptor slot, or issue a queue tail pointer indicating that the descriptor boundary can be read; After the data transfer corresponding to the current transfer window is completed, the roles of the current slot (251) and the next slot (252) are swapped, so that the next slot (252) becomes the new current slot (251).

6. The data transfer method under a fixed descriptor budget according to claim 1, characterized in that, Step S3 further includes: forming a window control record for each transport window, the window control record including the source start and end positions, destination start and end ranges, and cache maintenance range of the transport window; the cache maintenance range consists of a source cache maintenance range and a destination cache maintenance range, the source cache maintenance range is the range obtained by rounding the source address range involved in the transport window according to the cache line boundary, and the destination cache maintenance range is the range obtained by rounding the destination address range involved in the transport window according to the cache line boundary.

7. The data transfer method under a fixed descriptor budget according to claim 6, characterized in that, The window control record also includes a window summary, which is generated from the source start and end positions, destination start and end ranges, native descriptor sequence, cache maintenance range, and version information of the transport window. When writing the end position of the current transport window into the window checkpoint record, the checkpoint content and the window summary are written first, and the valid status is written after the storage barrier is executed.

8. The data transfer method under a fixed descriptor budget according to claim 1, characterized in that, When a transport anomaly is detected, for the data source that can re-acquire data corresponding to the task information to be transported, the native descriptor corresponding to the incomplete transport window is reconstructed from the most recent verified water level.

9. A data transfer system under a fixed descriptor budget, characterized in that, It includes a processor (110), a memory (120), and a DMA controller (130); the processor (110) is configured to perform the data transfer method under a fixed descriptor budget as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the data transfer method under a fixed descriptor budget as described in any one of claims 1 to 8.

Images