DMA method and device for reducing resource occupation

Abstract

The invention provides a DMA (Direct Memory Access) method and device for reducing resource occupation. The method comprises the following steps: acquiring discontinuous data blocks or data streams in a local storage unit; a discrete ID is allocated to each discontinuous data block or data stream, and address and length information of the corresponding data block or data stream is stored in each discrete ID; mapping the discrete ID to an intermediate address space; when discontinuous data blocks or data streams need to be read, address and length information corresponding to the discontinuous data blocks or data streams is obtained through the discrete IDs; and carrying out data migration according to the address and length information of each discontinuous data block or data stream, and the method has the beneficial effects that the address of the discontinuous data block in the local storage unit is stored locally based on the discrete ID, so that the delay generated in the process of acquiring the address of the discontinuous data block is reduced, and the storage performance is further optimized; meanwhile, waste of address space is reduced, and the utilization rate of memory resources is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a DMA method and device for reducing resource occupation. Background Art

[0002] In existing DMA technology, a linked list approach is often used to support scatter gather. However, the linked list approach wastes storage resources (a pointer to the next address needs to be stored), and the linked list is often stored in DDR. The access path delay is large and there is no performance advantage.

[0003] A new DMA approach is needed. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a DMA method and device for reducing resource occupancy, which can effectively reduce the waste of address space, improve the utilization rate of memory resources and reduce transmission delay.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: a DMA method for reducing resource occupancy, comprising:

[0006] Obtaining discontinuous data blocks or data streams in a local storage unit;

[0007] A discrete ID is assigned to each discontinuous data block or data stream, and each discrete ID stores the address and length information of the corresponding data block or data stream;

[0008] Map discrete IDs to an intermediate address space;

[0009] When it is necessary to read discontinuous data blocks or data streams, the address and length information of the corresponding discontinuous data blocks or data streams are obtained through discrete IDs;

[0010] Data is moved based on the address and length information of each discontinuous data block or data stream.

[0011] Furthermore, in the step of moving data according to the address and length information of each discontinuous data block or data stream, it also includes obtaining data transmission status information and dynamically adjusting the transmission rate and priority according to the data transmission status information.

[0012] Furthermore, in the step of moving data according to the address and length information of each discontinuous data block or data stream, it also includes obtaining system load status information and network status information, and dynamically adjusting data flow according to the data transmission status information or network status information.

[0013] Furthermore, after the step of moving data according to the address and length information of each discontinuous data block or data stream, the method further includes releasing the discrete ID.

[0014] The present invention also relates to a DMA device for reducing resource occupation, comprising: an acquisition module, an allocation module, a mapping module and a transport module;

[0015] The acquisition module is used to acquire discontinuous data blocks or data streams in the local storage unit;

[0016] The allocation module is used to allocate a discrete ID to each discontinuous data block or data stream, and each discrete ID stores the address and length information of the corresponding data block or data stream;

[0017] The mapping module is used to map the discrete ID to the intermediate address space;

[0018] The acquisition module is also used to acquire the address and length information of the corresponding discontinuous data block or data stream;

[0019] The transport module is used to move data according to the address and length information of each discontinuous data block or data stream.

[0020] Further, it also includes an adjustment module;

[0021] The acquisition module is also used to acquire data transmission status information;

[0022] The adjustment module is used to dynamically adjust the transmission rate and priority according to the data transmission status information.

[0023] Further, it also includes an adjustment module;

[0024] The acquisition module is also used to acquire system load status information and network status information;

[0025] The adjustment module is used to dynamically adjust the data flow according to data transmission status information or network status information.

[0026] Furthermore, it also includes a releasing module, and the releasing module is used to release the discrete ID.

[0027] The present invention also relates to an electronic device, characterized in that it comprises:

[0028] one or more processors;

[0029] A storage device for storing one or more programs;

[0030] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-described methods.

[0031] The present invention also relates to a computer-readable storage medium having a computer program stored thereon, wherein the program implements the above-described method when executed by a processor.

[0032] The beneficial effect of the present invention is that by storing the addresses of discontinuous data blocks in the local storage unit locally based on discrete IDs, the delay generated in the process of obtaining the addresses of discontinuous data blocks is reduced, thereby optimizing storage performance, while reducing the waste of address space and effectively improving the utilization of memory resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and other objects, features and advantages of the present disclosure will become more apparent by describing in detail its exemplary embodiments with reference to the accompanying drawings. The accompanying drawings described below are only some embodiments of the present disclosure, and it is clear to a person skilled in the art that other accompanying drawings can be obtained from these accompanying drawings without creative work.

[0034] Figure 1 is a flow chart of the present invention;

[0035] Figure 2 is a block diagram of the device of the present invention;

[0036] Figure 3 The present invention is a block diagram of an electronic device. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0038] In existing DMA technology, a linked list approach is often used to support scatter gather. However, the linked list approach wastes storage resources (a pointer to the next address needs to be stored), and the linked list is often stored in DDR. The access path delay is large and there is no performance advantage.

[0039] Based on this, the present invention provides a DMA method for reducing resource occupancy by adopting secondary mapping based on discrete ID addressing.

[0040] See also Figure 1 In one embodiment, a DMA method for reducing resource usage includes:

[0041] Obtaining discontinuous data blocks or data streams in a local storage unit;

[0042] A discrete ID is assigned to each discontinuous data block or data stream, and each discrete ID stores the address and length information of the corresponding data block or data stream;

[0043] Map discrete IDs to an intermediate address space;

[0044] When it is necessary to read discontinuous data blocks or data streams, the address and length information of the corresponding discontinuous data blocks or data streams are obtained through discrete IDs;

[0045] Data is moved based on the address and length information of each discontinuous data block or data stream.

[0046] In traditional DMA design, addresses are usually continuous. In this embodiment, addresses are identified by discrete IDs. Each discrete ID corresponds to a specific data block or data stream, and each discrete ID is unique. The data structure corresponding to the discrete ID is stored in a local storage unit to facilitate fast access by hardware. The data structure corresponding to the discrete ID includes addr, length, and eop information of the corresponding data block or data stream, where addr represents the address corresponding to the target data, length represents the length corresponding to the target data, and eop is used to identify the last address segment of the descriptor.

[0047] Similarly, each discrete ID may also correspond to n discontinuous data blocks or data streams, and the data structure corresponding to each discrete ID includes addr, length, and eop information of the corresponding multiple data blocks or data streams.

[0048] Discrete ID addressing can flexibly manage discontinuous data blocks or data streams, thereby reducing the waste of address space and improving the efficiency of address resolution.

[0049] Map discrete IDs to intermediate address spaces. This level of mapping is mainly used to convert discontinuous IDs into continuous intermediate addresses for subsequent processing.

[0050] The intermediate address is further mapped to the actual physical address space. This level of mapping is responsible for converting the intermediate address into the specific physical address of the discontinuous data block or data stream to ensure that the data can be correctly transmitted to the target location.

[0051] A secondary mapping table is used to store the mapping relationship between the intermediate address and the physical address. The secondary mapping table can be used to store the physical address actually corresponding to each discontinuous data block or data stream, and can also correspond to the virtual address corresponding to each discontinuous data block or data stream. The secondary mapping table stores the starting address and corresponding data length of each discontinuous data block or data stream; the storage array can be an SRAM or a register array.

[0052] A hash table or other efficient data structures may be used to optimize the query speed and storage efficiency of the mapping table.

[0053] The secondary mapping table can effectively manage and optimize the address space, reduce address conflicts, and improve the speed and accuracy of address resolution.

[0054] When it is necessary to read discontinuous data blocks or data streams, the address and length information of the corresponding discontinuous data blocks or data streams are obtained by parsing the data structure corresponding to the discrete ID, and the data can be moved through the bus, where the bus includes but is not limited to the AXI bus or other types of buses.

[0055] After the step of moving data according to the address and length information of each discontinuous data block or data stream, the method further includes releasing the discrete ID.

[0056] After the movement of each discontinuous data block or data stream is completed, the discrete ID is released and recycled into the discrete ID resource pool to facilitate the allocation of subsequent discrete IDs for discontinuous data blocks or data streams.

[0057] In one embodiment, data transmission is optimized to prevent data congestion or loss. Specifically, in the step of moving data according to the address and length information of each discontinuous data block or data stream, it also includes obtaining data transmission status information and dynamically adjusting the transmission rate and priority according to the data transmission status information.

[0058] In this embodiment, by introducing a flow control mechanism, the status of data transmission can be monitored in real time, and the transmission rate and priority can be dynamically adjusted according to the data transmission status information, thereby significantly improving the system throughput, reducing latency, and improving overall performance.

[0059] Each discrete ID corresponds to a data transmission priority, and commands are scheduled in different command queues based on the priority. The priority corresponding to each discrete ID can be adjusted based on the queue execution status of different commands (queue fullness), and the adjustment can be automatically adjusted based on software or hardware. In addition, since multiple queues are scheduled, the scheduling weights of different queues can also be adjusted to ensure that the high-priority queue is transmitted first.

[0060] In one embodiment, data transmission is optimized to prevent data congestion or loss. Specifically, in the step of moving data according to the address and length information of each discontinuous data block or data stream, it also includes obtaining system load status information and network status information, and dynamically adjusting data flow according to the system load status information or network status information.

[0061] In this embodiment, by introducing a flow control mechanism, the status of data transmission can be monitored in real time, and the transmission rate and priority can be adjusted according to system load status information or network status information, thereby significantly improving the system throughput, reducing latency, and improving overall performance.

[0062] From the above description, it can be seen that the beneficial effect of the present invention is that by storing the addresses of discontinuous data blocks in the local storage unit locally based on discrete IDs, the delay generated in the process of obtaining the addresses of discontinuous data blocks is reduced, thereby optimizing storage performance, while reducing the waste of address space and effectively improving the utilization of memory resources.

[0063] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments of the present invention, please refer to the method embodiments of the present invention.

[0064] See also Figure 2 , this embodiment discloses a DMA device for reducing resource occupation, including: an acquisition module, an allocation module, a mapping module and a transport module;

[0065] The acquisition module is used to acquire discontinuous data blocks or data streams in the local storage unit;

[0066] The allocation module is used to allocate a discrete ID to each discontinuous data block or data stream, and each discrete ID stores the address and length information of the corresponding data block or data stream;

[0067] The mapping module is used to map the discrete ID to the intermediate address space;

[0068] The acquisition module is also used to acquire the address and length information of the corresponding discontinuous data block or data stream;

[0069] The transport module is used to move data according to the address and length information of each discontinuous data block or data stream.

[0070] Further, it also includes an adjustment module;

[0071] The acquisition module is also used to acquire data transmission status information;

[0072] The adjustment module is used to dynamically adjust the transmission rate and priority according to the data transmission status information.

[0073] Further, it also includes an adjustment module;

[0074] The acquisition module is also used to acquire system load status information and network status information;

[0075] The adjustment module is used to dynamically adjust the data flow according to data transmission status information or network status information.

[0076] Furthermore, it also includes a releasing module, and the releasing module is used to release the discrete ID.

[0077] According to the device provided by the embodiment of the present invention, the addresses of discontinuous data blocks in the local storage unit are stored locally based on discrete IDs, thereby reducing the delay generated in the process of obtaining the addresses of discontinuous data blocks, thereby optimizing storage performance, reducing the waste of address space, and effectively improving the utilization of memory resources.

[0078] Figure 3 It is a block diagram of an electronic device according to an exemplary embodiment.

[0079] Refer to the following Figure 3 The electronic device according to this embodiment of the present invention will be described. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0080] like Figure 3 As shown, the electronic device is in the form of a general computing device. The components of the electronic device may include, but are not limited to: at least one processing unit, at least one storage unit, a bus connecting different system components (including the storage unit and the processing unit), a display unit, etc.

[0081] The storage unit stores a program code, which can be executed by the processing unit, so that the processing unit performs the steps of various exemplary embodiments of the present invention described in the above-mentioned DMA method for reducing resource usage. For example, the processing unit can perform the following steps: Figure 1 Follow the steps shown in .

[0082] The storage unit may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) and / or a cache memory unit, and may further include a read-only memory unit (ROM).

[0083] The storage unit may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0084] The bus may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0085] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication may be performed through an input / output (I / O) interface. In addition, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter may communicate with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0086] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to perform the above method according to the implementation of the present invention.

[0087] A computer readable medium, a program product for implementing the above method according to an embodiment of the present invention, can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited to this. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0088] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0089] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0090] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0091] The computer-readable medium carries one or more programs. When the one or more programs are executed by a device, the computer-readable medium implements the following functions:

[0092] Obtaining discontinuous data blocks or data streams in a local storage unit;

[0093] A discrete ID is assigned to each discontinuous data block or data stream, and each discrete ID stores the address and length information of the corresponding data block or data stream;

[0094] Map discrete IDs to an intermediate address space;

[0095] When it is necessary to read discontinuous data blocks or data streams, the address and length information of the corresponding discontinuous data blocks or data streams are obtained through the discrete ID of the intermediate address space, and data is moved according to the address and length information of each discontinuous data block or data stream.

[0096] Those skilled in the art will appreciate that the above modules may be distributed in the device according to the description of the embodiment, or may be changed accordingly and used in one or more devices that are different from the present embodiment. The modules and / or units and / or sub-units of the above embodiments may be combined into one module and / or unit and / or sub-unit, or may be further divided into multiple modules and / or units and / or sub-units and / or sub-modules.

[0097] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiment of the present invention.

[0098] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structure, configuration or implementation method described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

[0099] In addition, the structures, proportions, sizes, etc. shown in this specification and the drawings are only used to match the contents disclosed in the specification for those skilled in the art to understand and read, and are not used to limit the limiting conditions for the implementation of the present invention, so they have no substantial technical significance. Any modification of the structure, change of the proportion relationship or adjustment of the size, without affecting the technical effects and purposes that can be achieved by the present invention, should still fall within the scope of the technical contents disclosed in the present invention. At the same time, the terms such as "on", "first", "second" and "one" quoted in this specification are only for the convenience of description, and are not used to limit the scope of the implementation of the present invention. The change or adjustment of their relative relationship should also be regarded as the scope of the implementation of the present invention without substantially changing the technical contents.

Claims

1. A DMA method for reducing resource usage, comprising: Obtaining discontinuous data blocks or data streams in a local storage unit; A discrete ID is assigned to each discontinuous data block or data stream, and each discrete ID stores the address and length information of the corresponding data block or data stream; Map discrete IDs to an intermediate address space; When it is necessary to read discontinuous data blocks or data streams, the address and length information of the corresponding discontinuous data blocks or data streams are obtained through discrete IDs; Data is moved based on the address and length information of each discontinuous data block or data stream.

2. The method according to claim 1, characterized in that: The step of moving data according to the address and length information of each discontinuous data block or data stream also includes obtaining data transmission status information and dynamically adjusting the transmission rate and priority according to the data transmission status information.

3. The method according to claim 1, characterized in that: The step of moving data according to the address and length information of each discontinuous data block or data stream also includes obtaining system load status information and network status information, and dynamically adjusting data flow according to the data transmission status information or network status information.

4. The method according to claim 1, characterized in that: After the step of moving data according to the address and length information of each discontinuous data block or data stream, the method further includes releasing the discrete ID.

5. A DMA device for reducing resource usage, characterized in that: include: Acquisition module, allocation module, mapping module and handling module; The acquisition module is used to acquire discontinuous data blocks or data streams in the local storage unit; The allocation module is used to allocate a discrete ID to each discontinuous data block or data stream, and each discrete ID stores the address and length information of the corresponding data block or data stream; The mapping module is used to map the discrete ID to the intermediate address space; The acquisition module is also used to acquire the address and length information of the corresponding discontinuous data block or data stream; The transport module is used to move data according to the address and length information of each discontinuous data block or data stream.

6. The device according to claim 5, characterized in that: Also includes adjustment modules; The acquisition module is also used to acquire data transmission status information; The adjustment module is used to dynamically adjust the transmission rate and priority according to the data transmission status information.

7. The device according to claim 5, characterized in that: Also includes adjustment modules; The acquisition module is also used to acquire system load status information and network status information; The adjustment module is used to dynamically adjust the data flow according to data transmission status information or network status information.

8. The device according to claim 5, characterized in that: The system also includes a releasing module, and the releasing module is used to release the discrete ID.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images