A data scheduling method, a chip and an electronic device

Abstract

The application discloses a data scheduling method, comprising the following steps: obtaining to-be-scheduled data by a first controller; in response to the destination address of the to-be-scheduled data being within a virtual address provided by a second controller, sequentially obtaining data of each data page in the to-be-scheduled data and determining the offset in a first data block in each data page according to the PRP address of each data page; sending a data processing request to the first controller by the second controller, so that the first controller sends data of a corresponding size in the current data page to the second controller according to the size of to-be-processed data corresponding to the data processing request; determining the page sequence number of a to-be-stored data page and the in-page offset in the to-be-stored data page according to the virtual address range corresponding to the destination address of the data of the corresponding size and the offset of the first data block in the current data page; and writing the data of the corresponding size into a corresponding position in the to-be-stored data page according to the page sequence number and the in-page offset.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, and in particular to a data scheduling method, a chip and an electronic device. BACKGROUND

[0002] The booming development of applications such as AI and cloud computing puts higher requirements on data processing. Servers need to have greater data storage capacity, higher data storage bandwidth and stronger data processing performance. The speed of CPU performance improvement has gradually failed to keep up with the improvement speed of storage bandwidth and network bandwidth, and is called a bottleneck. A storage acceleration device is a hardware with a special processing chip, which is used to offload CPU for storage-related data processing, such as data migration, data verification, storage protocol processing, RAID calculation, etc., so that the server system can break through the storage bottleneck caused by the CPU, and the relatively expensive general-purpose computing resources of the CPU can be replaced by relatively cheap storage-specific computing resources to reduce server costs.

[0003] Storage acceleration devices can be divided into many types according to application scenarios, and RAID acceleration is one of the widely used types. It is generally used to map the data read / write requests issued by the host to the corresponding multiple disks, and is responsible for calculating the redundant verification data to protect the data. For example, a typical RAID5 system can have four disks, of which three disks are used to store data and one disk is used to store verification data (only refers to the space, and the actual data and redundant verification are distributed in all disks). The upper application in the host generally sees a large space composed of three disk spaces. The disk read / write (hereinafter referred to as IO) command issued by the host is directed to this whole space. The acceleration card analyzes the IO command, maps the data to multiple physical disks according to the host read / write position, then moves the data from the host to the acceleration card, calculates the redundant verification data, and finally reorganizes the IO command for disk operation and issues it to the disk.

[0004] For other types of storage acceleration, there is also a similar process. The core steps of the acceleration card can be abstracted as analyzing the host command and moving the data, operating the data, mapping the data to the physical space of the disk and reorganizing the command. The specific business done by the above three steps may be different in different application scenarios, but the essence is the same. For example, for storage compression acceleration, the difference from RAID is only that the operation on the data is compression instead of calculating the redundant verification bit, and the mapping of the compressed data to the disk and the original data may be different.

[0005] There are many factors that affect the performance of a storage acceleration chip, but generally speaking, the scheduling of data is one of the key factors. Due to the involvement of data movement between the host, the acceleration board, and the disk, the temporary storage of data during the calculation and verification process, etc., data needs to be written / read to the DRAM of the acceleration card multiple times. In a high-performance storage system, due to the wide use of the PCIe interface between the host, the board, and the disk, which has a very high data bandwidth, very high requirements are put forward for the bandwidth of the DRAM, and it is therefore very likely to become a bottleneck of the system.

[0006] The NVMe protocol is increasingly widely used in storage systems due to its high performance, and therefore puts forward higher requirements for the data scheduling method of the acceleration device. Compared with traditional storage protocols such as SAS / SATA, the NVMe protocol realizes high performance mainly due to several key features. First, it supports a larger number of IO queues and a larger queue depth. Second, the NVMe device acts as the initiator of the DMA process, i.e., the device actively obtains data from the host memory rather than the host sending data to the device.

[0007] The device acting as the initiator of the DMA process puts forward new requirements for the NVMe storage system, i.e., the organization of data needs to meet the NVMe protocol standard to be recognized by the device. In the traditional SAS / SATA system, data can be organized in any form in the acceleration card, because as the transmission initiator, the acceleration card can reorganize the data during the data packaging and sending process. However, in the NVMe system, the disk directly obtains data from the acceleration card, and the acceleration card cannot perform additional operations during the data transmission process.

[0008] The NVMe protocol defines two data organization methods, called PRP and SGL. Among them, PRP is a format that the NVMe protocol requires to be supported compulsorily and is more widely used.

[0009] As shown in Figure 1 , PRP is composed of two addresses, PRP1 and PRP2. Among them, the PRP1 address points to a memory page for storing data. The PRP1 address contains 2 parts, the low part is used to record the starting address of the valid data in the page (related to the page size, for example, for a 4096-byte data page, the low 12 bits are used), also known as the offset of the valid data, and the high part records the page base address of the data page. PRP2 is determined as follows:

[0010] If the total data is stored in 1 data page, PRP2 is invalid;

[0011] If the total data is stored in 2 data pages, PRP2 points to a data page, but at this time the offset of PRP2 must be 0.

[0012] If the total data is stored across two or more data pages, then PRP2 points to a memory page used to store the PRP list. In this case, the offset of PRP2 can be non-zero, meaning that the valid data of the PRP list does not start from address 0 of this memory page. The PRP list consists of multiple PRP addresses, each pointing to a valid data page, but the offsets of these PRP addresses must be zero.

[0013] like Figure 2 As shown, a typical storage system consists of many disks, which, after being managed by an accelerator card, often appear as a unified storage device to the host. Therefore, a single large data block I / O operation on the host side requires the data to be mapped across multiple disks. Furthermore, because the data sent by the host includes offsets, each mapped data block will also have an offset. Since the PRP format does not allow two or more data blocks to use offset fields, it must be split into multiple I / O operations on the disks. This splitting leads to an increase in the number of I / O operations and a decrease in the amount of data per I / O operation, inevitably putting significant I / O pressure on the disks and thus reducing read / write efficiency.

[0014] There are also several traditional methods. One method is to place data belonging to the same disk contiguously on the local machine when moving data from the host via DMA. This can solve the problem of splitting I / O as mentioned above. However, this will cause the DMA operation of a data block to be split into two. When the offset is small, it means that the data load of one of the DMA operations is small, which will greatly reduce the bus bandwidth utilization.

[0015] Another approach is to reorganize the data in local memory, placing data belonging to the same disk consecutively or eliminating offsets. However, regardless of the method, it is inevitable to perform a read and a write operation on the DRAM, putting enormous pressure on the DRAM's bandwidth requirements. Summary of the Invention

[0016] In view of this, in order to overcome at least one aspect of the above problems, embodiments of the present invention propose a data scheduling method, comprising the following steps:

[0017] The data to be scheduled is obtained using the first controller;

[0018] In response to the destination address of the data to be scheduled being within the virtual address provided by the second controller, the data of each data page in the data to be scheduled is sequentially obtained and the offset in the first data block in each data page is determined according to the PRP address of each data page.

[0019] sending, by the second controller, a data processing request to the first controller, so that the first controller sends data of a corresponding size in a current data page to the second controller according to a size of data to be processed corresponding to the data processing request;

[0020] determining, by the second controller, a page sequence number of a data page to be stored and an offset in the data page to be stored according to a virtual address range corresponding to a destination address of the data of the corresponding size and an offset of a first data block in the current data page;

[0021] writing the data of the corresponding size into a corresponding position in the data page to be stored according to the page sequence number and the offset.

[0022] In some embodiments, further comprising:

[0023] initializing a preset region of a memory, wherein the preset region after initialization comprises a plurality of data pages to be stored;

[0024] generating a resource number according to an address of each data page to be stored and putting each resource number into a resource queue.

[0025] In some embodiments, further comprising:

[0026] determining a number of data pages to be stored required according to a size of the data to be scheduled;

[0027] obtaining a corresponding number of resource numbers in the resource queue and converting the corresponding number of resource numbers into addresses to form a PRP address chain table, and performing the following operations on each of the PRP address chain table:

[0028] feeding back the PRP address chain table to the second controller;

[0029] in response to writing data on a plurality of data pages to be stored in the PRP address chain table into a disk, re-adding resource numbers of the plurality of data pages to be stored into the resource queue.

[0030] In some embodiments, the second controller determines a page sequence number of a data page to be stored and an offset in the data page to be stored according to a virtual address range corresponding to a destination address of the data of the corresponding size and an offset of a first data block in the current data page, further comprising:

[0031] determining the page sequence number of the corresponding data page to be stored in the PRP address chain table according to the virtual address range corresponding to the destination address of the data of the corresponding size;

[0032] determining whether the data of the corresponding size belongs to the first data block according to an offset of the first data block in the current data page and a virtual address range corresponding to a destination address of the data of the corresponding size.

[0033] In some embodiments, writing the data of the corresponding size into the corresponding position in the to-be-stored data page according to the page number and the intra-page offset further comprises:

[0034] in response to belonging to the first data block, writing the data of the corresponding size into the corresponding position in the to-be-stored data page corresponding to the page number according to the intra-page offset;

[0035] in response to not belonging to the first data block, writing the data of the corresponding size into the corresponding position in the to-be-stored data page corresponding to the next page number or the previous page number of the page number according to the intra-page offset.

[0036] In some embodiments, writing the data of the corresponding size into the corresponding position in the to-be-stored data page according to the page number and the intra-page offset further comprises:

[0037] setting a cache smaller than the size corresponding to the virtual address in the second controller;

[0038] saving the data of the corresponding size into the cache before writing into the corresponding position in the to-be-stored data page.

[0039] In some embodiments, sending a data processing request to the first controller by the second controller to make the first controller send the data of the corresponding size in the current data page to the second controller according to the size of the to-be-processed data corresponding to the data processing request further comprises:

[0040] initializing a first counter and a second counter in the first controller and the second controller respectively;

[0041] setting an initial value of the second counter according to the size of the cache of the second controller and setting an initial value of the first counter as 0;

[0042] in response to sending the data processing request to the first controller by the second controller, subtracting the size of the to-be-processed data in the data processing request from the second counter and adding the size of the to-be-processed data in the data processing request to the first counter in the first controller, wherein the size of the to-be-processed data in the data processing request is not greater than the size of the corresponding cache space.

[0043] In some embodiments, further comprising:

[0044] decrement the first counter by a corresponding size in response to the first controller processing part of data in the data processing request;

[0045] increment the second counter by a corresponding size in response to the second controller writing part of data to the corresponding position of the data page to be stored.

[0046] Based on the same inventive concept, according to another aspect of the present application, embodiments of the present application further provide a chip comprising a digital logic circuit, wherein the digital logic circuit is configured to implement the steps of the data scheduling method according to any one of the above embodiments.

[0047] Based on the same inventive concept, according to another aspect of the present application, embodiments of the present application further provide an electronic device comprising the chip as described above.

[0048] The present application has the following beneficial technical effects: the scheme provided by the present application receives data and extracts corresponding information to complete the remapping of virtual addresses to real addresses, avoids the amplification effect of IO, and improves the performance utilization rate of a disk, so that better system performance can be achieved under the same conditions. The number of IOs of a bus is also reduced, and the problem of reduced bus utilization rate caused by small data load is avoided, so that the demand for the bus can be reduced, and the cost can be reduced. The number of data copying of a local DRAM is also reduced, the performance demand for the DRAM can be reduced, and the cost can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other embodiments can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0050] Figure 1 A schematic diagram of a typical PRP data organization method in the prior art;

[0051] Figure 2 A schematic diagram of data blocks with offsets being mapped to different disks;

[0052] Figure 3 A schematic diagram of the flow of the data scheduling method provided by the embodiments of the present application;

[0053] Figure 4 A schematic diagram of the overall circuit provided by the embodiments of the present application;

[0054] Figure 5 A schematic diagram of the structure of the DRAM page management module provided by the embodiments of the present application;

[0055] Figure 6 A functional diagram of a data mapper provided for an embodiment of the present application;

[0056] Figure 7 A structural diagram of a data mapper provided for an embodiment of the present application;

[0057] Figure 8 A disk RRP list regeneration diagram provided for an embodiment of the present application;

[0058] Figure 9 A structural diagram of a chip provided for an embodiment of the present application;

[0059] Figure 10 A structural diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the embodiments and the accompanying drawings.

[0061] It should be noted that all the expressions of "first" and "second" in the embodiments of the present application are used to distinguish two same name non-identical entities or non-identical parameters, and it can be seen that "first" and "second" are only for the convenience of description, and should not be understood as a limitation of the embodiments of the present application, and the subsequent embodiments will not be described one by one.

[0062] In the embodiments of the present application, DMA, Direct Memory Access, directly accesses the memory.

[0063] According to one aspect of the present application, an embodiment of the present application provides a data scheduling method, as shown in Figure 3 The method can include the following steps:

[0064] S1, obtaining the to-be-scheduled data by using a first controller;

[0065] S2, in response to the destination address of the to-be-scheduled data being within a virtual address provided by a second controller, sequentially obtaining data of each data page in the to-be-scheduled data and determining an offset in a first data block in each data page according to a PRP address of each data page;

[0066] S3, sending a data processing request to the first controller by using the second controller, so that the first controller sends data of a corresponding size in the current data page to the second controller according to a size of to-be-processed data corresponding to the data processing request;

[0067] S4. The second controller determines the page number of the data page to be stored and the page offset within the data page to be stored based on the virtual address range corresponding to the destination address of the data of the corresponding size and the offset of the first data block in the current data page.

[0068] S5. Write the data of the corresponding size into the corresponding position in the data page to be stored according to the page number and the page offset.

[0069] The proposed solution employs virtual space and virtual addresses to receive data and extract corresponding information, thus remapping virtual addresses to real addresses. This avoids the amplification effect of I / O and improves disk performance utilization, resulting in better system performance under the same conditions. It also reduces the number of bus I / O operations and avoids the problem of reduced bus utilization caused by small data loads, thereby reducing bus requirements and costs. Furthermore, it reduces the number of data copies to local DRAM, lowering DRAM performance requirements and reducing costs.

[0070] In some embodiments, such as Figure 4 The circuit shown implements data scheduling. The circuit includes several modules such as a PCIe bus controller, a DRAM controller, a DMA controller (first controller), a DRAM page manager, an accelerated computing unit, a data mapper (second controller), an array manager, and a disk controller. These modules are connected together through an interconnect bus.

[0071] The PCIe controller provides the PCIe interface for connection to the host, translating on-chip system-to-the-chip (SoC) bus read / write operations into and out of the host's PCIe bus domain, making the SoC appear as an endpoint device on the host's PCIe bus. The DMA controller handles data transfer between the host and the SoC. It controls the PCIe controller to send read / write requests to the host, allowing the PCIe controller to read data from the SoC and send it to the host, or retrieve data from the host and write it to a specified SoC address. Additionally, the DMA controller typically resolves the host's PRP (Programmable Request List). The DRAM controller controls the DRAM chips connected to the SoC and provides the corresponding bus read / write interface to the SoC. The RAID manager manages a disk array, primarily responsible for splitting and mapping host read / write I / O into read / write I / O for each disk, and reorganizing host data to conform to the data organization format of each disk. The RAID manager can be implemented using hardware circuitry or a combination of CPU and software. The disk controller is used to control external disks, mainly to enable the on-chip system to read and write to the disk via the bus. The disk controller can also be implemented using hardware circuits or a CPU.

[0072] The acceleration operation unit can have multiple units according to different use scenarios of the chip, each of which performs one or more operations, such as CRC operation for data verification, XOR operation for disk array redundancy check data, etc. After each operation, the unit can write data back to the DRAM.

[0073] The above modules are typical modules in a storage acceleration device. In actual implementation, the functional boundaries of the modules can be different from the above description, but the functions can generally correspond to one or more of the above modules. The embodiment adds a DRAM page management module and a data mapper to the above general modules. The DRAM page management module is used to provide data pages to be stored. The data mapper is mainly used to implement the storage of host-side data with offset into local-side data without offset, so as to simplify the processing of the data page linked list in the case of multiple disks.

[0074] In some embodiments, further comprising:

[0075] initializing a preset region of the memory, wherein the initialized preset region includes a plurality of data pages to be stored;

[0076] generating a resource number according to the address of each data page to be stored and putting each resource number into a resource queue.

[0077] In some embodiments, further comprising:

[0078] determining the number of data pages to be stored according to the size of the data to be scheduled;

[0079] obtaining a corresponding number of resource numbers in the resource queue and converting the corresponding number of resource numbers into addresses to form a PRP address linked list, and performing each of the PRP address linked list;

[0080] feeding back the PRP address linked list to the second controller;

[0081] in response to writing data on a plurality of data pages to be stored in the disk, re-adding the resource numbers of the plurality of data pages to be stored into the resource queue.

[0082] Specifically, as Figure 5As shown, the DRAM page management module is used to manage the space of the DRAM. First, a memory region is statically allocated in the DRAM for storing data, and the region is managed by the DRAM page management module. Then, the page management module divides the memory region into pages, each of which is a continuous memory, and the pages are adjacent to each other, and the size of the page is the same as the data exchange page size of the host and the storage acceleration device. That is, if the host builds a PRP list, the data page size pointed to by a PRP address is 4KB, and the page size of the DRAM page management module is also 4KB.

[0083] The DRAM page management module maintains a resource queue, and the depth of the queue is the same as the number of managed pages. For example, if the size of the DRAM managed by the page management module is 4GB, there are 1M pages in total, and therefore the depth of the queue is 1M. Depending on the size and cost requirements of the queue, the elements of the queue can be stored in the region of the DRAM other than the data storage region, only the head pointer and the tail pointer of the queue are maintained in the management module, or the elements of the queue are stored in the FIFO of the management module. Each element in the resource queue is a digital number (hereinafter referred to as a resource number), which represents an idle data page in the DRAM that can be allocated. The number can be represented by the high bits of the address of the page, or obtained by other simple operations.

[0084] The DRAM page management module also includes a linked list generation module. The module calculates the number of pages required according to the requested data size to be allocated, then obtains the corresponding number of resource numbers from the resource queue, and then converts the resource numbers into DRAM page addresses, and organizes the addresses into a linked list in the PRP format conforming to the NVMe protocol.

[0085] When a piece of data is written to the disk, the corresponding DRAM region can be released. The region is also a region described by a PRP linked list, and the linked list is parsed by an internal parser of the DRAM page management module to extract the data pages to be recycled and their corresponding resource numbers, and then put into the resource queue by the page resource recycling module.

[0086] In some embodiments, the second controller determines the page sequence number of the data page to be stored and the intra-page offset in the data page to be stored according to the virtual address range corresponding to the destination address of the data of the corresponding size and the offset of the first data block in the current data page, and further includes:

[0087] According to the virtual address range corresponding to the destination address of the data of the corresponding size, the page sequence number of the data page to be stored in the PRP address linked list is determined.

[0088] Whether the data of the corresponding size belongs to the first data block is determined based on the offset of the first data block in the current data page and the virtual address range corresponding to the destination address of the data of the corresponding size.

[0089] In some embodiments, writing data of the corresponding size to the corresponding position in the data page to be stored according to the page number and the page offset further includes:

[0090] In response to belonging to the first data block, the data of the corresponding size is written to the corresponding position of the data to be stored page corresponding to the page number according to the page offset;

[0091] In response to data that does not belong to the first data block, data of the corresponding size is written to the corresponding position of the data page to be stored, based on the page offset.

[0092] Specifically, such as Figure 6 As shown, when there is an offset in the host data storage, a data block is scattered across two data pages, and a data page contains the data of two data blocks. If the data is moved directly using ordinary DMA, since a host page needs to be moved to two different local pages, two DMA operations are required, one of which is for the offset. When the offset is small, the bandwidth utilization of the PCIe bus will be greatly reduced.

[0093] The core function of the data mapping module is to provide a contiguous "virtual space" for receiving data pages from the host as a whole and then distributing them to multiple local data pages. This requires only one DMA operation for each host data page, significantly improving DMA efficiency.

[0094] The functional structure of the data mapper is as follows Figure 7 As shown. Its internal task information module stores task parameters, the core of which are the host PRP and the local PRP, as well as other parameters such as page size. The PRP parser parses the host and local PRPs respectively to obtain the mapping relationship of data pages.

[0095] One of the core components of a data mapper is the data path, which consists of three modules: address decoding, data caching, and data distribution.

[0096] Unlike regular modules, a data mapper provides a "virtual address" much larger than its own cache to receive data from DMA. For example, a host I / O operation might require processing 4MB of data, while the data mapper's internal data cache might only be 4KB. However, the data mapper provides a 4MB virtual address range, and any data written to this range will be received by the mapper.

[0097] The address decoder is used to realize the decoding of the virtual address, which resolves the page number of the current data and the offset of the data in the page according to the virtual address, and temporarily stores the data in the specified position of the data cache. The specified position is generally determined according to the offset of the data in the page, and can be used to rearrange the possible out-of-order PCIe packets.

[0098] The data distribution module finds the local real page address corresponding to the current data according to the mapping relationship resolved by the page number and the PRP, and then writes the data into the corresponding position of the DRAM according to the offset of the data in the page.

[0099] For example, the current data page is data page 3 in Figure 6 For data page 3, the first data block is data block 3. The offset of the first data block in data page 3 can be determined according to the PRP of the host end. The size of each data page is a fixed value, for example, 4kb, so the virtual address corresponding to data page 3 is the range of 9-12kb, which corresponds to the to-be-stored data page 3. In this way, when the second controller obtains the data of the corresponding size in data page 3, it can determine the page number of the to-be-stored data page corresponding to the data to be written according to the virtual address to be written, and then determine the offset of the data in the page through the offset of the first data block and the virtual address to be written, so as to determine whether it belongs to data block 3. If it belongs to data block 3, the data of the corresponding size is written into the to-be-stored data page 3; if it does not belong to data block 3, it means that it belongs to data block 2, and it is written into the to-be-stored data page 2. In this way, only one data block is written into each to-be-stored data page in the DRAM, so that there is no offset when the data is stored locally.

[0100] It should be noted that the to-be-stored data page 2 and the to-be-stored data page 3 are arranged in order according to the order in the PRP chain table provided by the DARM controller. The to-be-stored data page corresponding to the first PRP address in the PRP chain table is the to-be-stored data page 1, the second is the to-be-stored data page 2, and so on.

[0101] In some embodiments, writing the data of the corresponding size into the corresponding position of the to-be-stored data page according to the page number and the offset in the page further comprises:

[0102] Setting a cache smaller than the size corresponding to the virtual address in the second controller;

[0103] Saving the data of the corresponding size in the cache and then writing it into the corresponding position of the to-be-stored data page.

[0104] Specifically, the data cache has two main functions. The first function is to temporarily store data to avoid data overflow when the data distribution module cannot write data into DRAM in time, for example, when other modules are accessing DRAM, or when DRAM refresh, pre-charge or other operations are encountered. The second function is to facilitate subsequent processing by rearranging the data in disorder, for example, when the next stage does not connect to DRAM but connects to other operation modules.

[0105] In some embodiments, the second controller sends a data processing request to the first controller, so that the first controller sends data of a corresponding size in the current data page to the second controller according to the size of the data to be processed corresponding to the data processing request. Further comprising:

[0106] Initializing a first counter and a second counter in the first controller and the second controller, respectively;

[0107] Setting the initial value of the second counter according to the size of the cache of the second controller, and setting the initial value of the first counter to 0;

[0108] In response to the second controller sending a data processing request to the first controller, subtracting the size of the data to be processed in the data processing request from the second counter, and adding the size of the data to be processed in the data processing request to the first counter in the first controller, wherein the size of the data to be processed in the data processing request is not greater than the size of the corresponding cache space.

[0109] In some embodiments, further comprising:

[0110] In response to the first controller processing part of the data in the data processing request, subtracting the corresponding size from the first counter;

[0111] In response to the second controller writing part of the data to the corresponding position of the data to be stored page, adding the corresponding size to the second counter.

[0112] Specifically, the flow control is another core module of the data mapper, which is used to prevent overflow due to too many data requests. Since the range of the virtual address is larger than the data cache, if the DMA controller sends too many data requests, but the mapper cannot consume them in time, data overflow will occur, especially when the data mapper is connected to other operation modules. The module sends a flow control message to the DMA controller module, requesting the DMA module to send data of a specified size. The DMA module can only send data not exceeding the size after receiving the request. The specific implementation is that a counter is used in the module to record the data requests that have been sent but not consumed, that is, the initial value of the counter is 0, and the value is added every time a certain number of data requests are sent to the DMA, and the counter is reduced by 1 every time data is sent to the next module. The control value should not exceed the cache size. A similar counter is implemented in the DMA controller, which increases the corresponding value when receiving the data request from the data mapper, and decreases the corresponding value when sending the data request to the PCIe controller. The DMA controller needs to ensure that the counter value is 0, and no longer sends a request to the PCIe controller.

[0113] The DMA controller also needs to be modified, that is, the original local PRP is changed to the virtual address space of the data mapper as the destination address.

[0114] As shown in Figure 8 , after the above processing, since there is no offset when the data is stored locally, only the local PRP needs to be extracted every N times to form the PRP list of each disk.

[0115] The scheme provided by the application manages the local data storage by pages, and the page size is equal to the page size of the host end or 1 / N of the host end, N being an integer. The virtual space and the virtual address are used to receive data and extract corresponding information to complete the remapping of the virtual address to the real address. After the data of the host end is moved to the local, it is stored without offset. In this way, the offset field of each entry in the PRP list is 0, so that the entry can be simply extracted by interleaving to correspond to each disk. Thus, the conversion between the offset page data and the non-offset page data can be realized without reducing the DMA efficiency.

[0116] Based on the same inventive concept, according to another aspect of the application, as shown in Figure 9 , the embodiment of the application also provides a chip 501, which comprises:

[0117] The digital logic circuit 510 is used to implement the steps of the data scheduling method according to any one of the above embodiments.

[0118] Based on the same inventive concept, according to another aspect of the present application, as shown in FIG. 6, an electronic device 601 is also provided, which comprises the chip 610 described above. Figure 10

[0119] Finally, it should be noted that those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by a computer program to instruct related hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of each method.

[0120] In addition, it should be understood that the computer-readable storage medium (for example, the memory) herein can be a volatile memory or a non-volatile memory, or can include both volatile memory and non-volatile memory.

[0121] Those skilled in the art will also appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends on the particular application and design constraints imposed on the overall system. Those skilled in the art can implement the functionality in various ways for each particular application, but such implementation decisions do not affect the scope of the present application embodiments disclosed.

[0122] The above is an exemplary embodiment disclosed by the present application, but it should be noted that various changes and modifications can be made without departing from the scope of the present application embodiments disclosed by the claims. The functions, steps and / or acts of the method claims described herein do not need to be performed in any particular order. Furthermore, although the elements of the present application embodiments disclosed can be described or claimed in individual forms, they can also be understood as plural unless explicitly limited to a single instance.

[0123] It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly dictates otherwise. It should also be understood that "and / or" as used herein refers to any and all possible combinations of one or more of the associated listed items.

[0124] The above-mentioned embodiment number of the present application embodiments disclosed is only for description, not representing the advantages and disadvantages of the embodiments.

[0125] ​Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by program instructing relevant hardware to complete, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0126] Those skilled in the art shall understand that the above discussion of any embodiment is only exemplary, and is not intended to imply that the scope (including claims) of the embodiments of the present application is limited to these examples; the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes of different aspects of the embodiments of the present application as above. In order to be brief, they are not provided in details. Therefore, any omission, modification, equivalent replacement, improvement, etc. made in the spirit and principle of the embodiments of the present application shall be included in the protection scope of the embodiments of the present application.

Claims

1. A data scheduling method, characterized by, The method comprises the following steps: acquiring the data to be scheduled by using a first controller; in response to the destination address of the data to be scheduled being within the virtual address provided by a second controller, acquiring the data of each data page in the data to be scheduled in sequence and determining the offset in the first data block in each data page according to the PRP address of each data page; sending a data processing request to the first controller by using the second controller, so that the first controller sends data of a corresponding size in the current data page to the second controller according to the size of the data to be processed corresponding to the data processing request; determining the page sequence number of the data page to be stored and the page offset in the data page to be stored according to the virtual address range corresponding to the destination address of the data of the corresponding size and the offset of the first data block in the current data page; writing the data of the corresponding size into the corresponding position in the data page to be stored according to the page sequence number and the page offset; writing the data of the corresponding size into the corresponding position in the data page to be stored according to the page sequence number and the page offset further comprises: in response to the data of the corresponding size belonging to the first data block, writing the data of the corresponding size into the corresponding position in the data page to be stored corresponding to the page sequence number according to the page offset; in response to the data of the corresponding size not belonging to the first data block, writing the data of the corresponding size into the corresponding position in the data page to be stored corresponding to the page sequence number or the page sequence number above or below the page sequence number according to the page offset.

2. The method of claim 1, wherein, Further comprising: initializing a preset area of a memory, wherein the preset area after initialization comprises a plurality of data pages to be stored; generating a resource number according to the address of each data page to be stored and putting each resource number into a resource queue.

3. The method of claim 2, wherein, Further comprising: determining the number of data pages to be stored required according to the size of the data to be scheduled; acquiring a corresponding number of resource numbers in the resource queue and converting the corresponding number of resource numbers into addresses to form a PRP address chain table, and processing each in the PRP address chain table; feeding back the PRP address chain table to the second controller; in response to the data on a plurality of data pages to be stored in the PRP address chain table being written into a disk, re-adding the resource numbers of the plurality of data pages to be stored into the resource queue.

4. The method of claim 3, wherein, The second controller determines the page sequence number of the data page to be stored and the page offset in the data page to be stored according to the virtual address range corresponding to the destination address of the data of the corresponding size and the offset of the first data block in the current data page further comprises: determining the page sequence number of the corresponding data page to be stored in the PRP address chain table according to the virtual address range corresponding to the destination address of the data of the corresponding size; determining whether the data of the corresponding size belongs to the first data block according to the offset of the first data block in the current data page and the virtual address range corresponding to the destination address of the data of the corresponding size.

5. The method of claim 1, wherein, The second controller determines the page sequence number of the data page to be stored and the page offset in the data page to be stored according to the virtual address range corresponding to the destination address of the data of the corresponding size and the offset of the first data block in the current data page further comprises: determining the page sequence number of the corresponding data page to be stored in the PRP address chain table according to the virtual address range corresponding to the destination address of the data of the corresponding size; determining whether the data of the corresponding size belongs to the first data block according to the offset of the first data block in the current data page and the virtual address range corresponding to the destination address of the data of the corresponding size. setting a cache smaller than the size corresponding to the virtual address in the second controller; saving the data of the corresponding size into the cache and then writing into the corresponding position of the data page to be stored.

6. The method of claim 5, wherein, sending a data processing request to the first controller by the second controller, so that the first controller sends the data of the corresponding size in the current data page to the second controller according to the size of the data to be processed corresponding to the data processing request, and further comprising: initializing a first counter and a second counter in the first controller and the second controller respectively; setting the initial value of the second counter according to the size of the cache of the second controller, and setting the initial value of the first counter as 0; in response to the second controller sending a data processing request to the first controller, subtracting the size of the data to be processed in the data processing request from the second counter, and adding the size of the data to be processed in the data processing request to the first counter in the first controller, wherein the size of the data to be processed in the data processing request is not greater than the size of the corresponding cache space.

7. The method of claim 6, wherein, further comprising: in response to the first controller processing part of the data in the data processing request, subtracting the corresponding size from the first counter; in response to the second controller writing part of the data into the corresponding position of the data page to be stored, adding the corresponding size to the second counter.

8. A chip, characterized by comprising a digital logic circuit, which realizes the steps of the method according to any one of claims 1-7 when working.

9. An electronic device, comprising: comprising the chip according to claim 8.

Images