Data transmission method and device, electronic equipment and storage medium

Abstract

The invention relates to a data transmission method and device, electronic equipment and a storage medium, and the method comprises the steps: according to transmission data determined by a direct memory access transmission request, locking M physical pages corresponding to the transmission data from a first memory of a first processor in a user mode, and according to a first physical address of each physical page, locking M physical pages corresponding to the transmission data from a second memory of the first processor; performing block recombination operation on the M physical pages to obtain N physical blocks and a second physical address of each physical block; and performing one-time direct memory access transmission by utilizing a direct memory access controller according to N second physical addresses corresponding to the N physical blocks, and transmitting transmission data stored in the N physical blocks in the first memory of the first processor to a second memory of the second processor. According to the embodiment of the invention, the direct memory access transmission performance can be improved, so that the direct memory access transmission rate between the first memory of the first processor and the second memory of the second processor is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data transmission method and device, an electronic device, and a storage medium. Background Art

[0002] In related technologies, direct memory access (DMA) can be used to transmit big data. However, with the rapid development of artificial intelligence computing, the volume of data has shown explosive growth. This growth not only brings a huge amount of information, but also gives rise to the demand for rapid transmission of massive data. In the application of moving large amounts of data using direct memory access, how to further improve the efficiency of data transmission is becoming increasingly important. Summary of the invention

[0003] The present disclosure proposes a technical solution for data transmission.

[0004] According to one aspect of the present disclosure, a data transmission method is provided, comprising: according to transmission data determined by a direct memory access transmission request, locking M physical pages corresponding to the transmission data from a first memory of a first processor in a user state, where M is an integer greater than 0; according to a first physical address of each physical page, performing a block reorganization operation on the M physical pages to obtain N physical blocks and a second physical address of each physical block, where N is less than or equal to M; and using a direct memory access controller to perform a direct memory access transmission according to N second physical addresses corresponding to the N physical blocks, and transmitting the transmission data stored in the N physical blocks in the first memory of the first processor to the second memory of the second processor.

[0005] In a possible implementation, the transmission data determined according to the direct memory access transmission request locks the M physical pages corresponding to the transmission data from the first memory of the first processor in the user state, including: initiating a memory request for the transmission data, and determining the user state address of the transmission data in the user space; calling the memory management interface in the user state, and determining the M physical pages corresponding to the user state address from the first memory of the first processor; and locking the M physical pages corresponding to the user state address from the first memory of the first processor.

[0006] In a possible implementation, the block reorganization operation is performed on M physical pages according to the first physical address of each physical page to obtain N physical blocks and the second physical address of each physical block, including: performing continuity judgment on M first physical addresses corresponding to the M physical pages to obtain a judgment result; when the judgment result indicates that there are any K physical pages with continuous first physical addresses among the M physical pages, the K physical pages are merged into a physical block, where K is an integer greater than or equal to 2 and less than or equal to M; and determining the second physical address of the physical block according to the K first physical addresses corresponding to the K physical pages.

[0007] In a possible implementation, the method uses a direct memory access controller to perform a direct memory access transmission according to N second physical addresses corresponding to N physical blocks, and transmits the transmission data stored in N physical blocks in the first memory of the first processor to the second memory of the second processor, including: applying for N third physical addresses for receiving the transmission data from the second memory of the second processor; using the direct memory access controller to establish a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports according to the N second physical addresses and the N third physical addresses; and performing direct memory access transmission according to the transmission channel, and transmitting the transmission data stored in N physical blocks in the first memory of the first processor to the second memory of the second processor.

[0008] In one possible implementation, applying for N third physical addresses for receiving the transmission data from the second memory of the second processor includes: determining the storage space of the physical block according to the second physical address corresponding to each physical block; and applying for a third physical address that satisfies the storage space from the second memory of the second processor according to the storage space.

[0009] In one possible implementation, the method further includes: in response to transferring the transmission data stored in N physical blocks in the first memory of the first processor to the second memory of the second processor, triggering a transmission completion interrupt, wherein the transmission completion interrupt is used to notify the first processor that the transmission data transmission is completed.

[0010] In a possible implementation, the method of using a direct memory access controller to establish a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports according to the N second physical addresses and the N third physical addresses includes: determining N descriptors according to the N second physical addresses and the N third physical addresses, each descriptor including a first head address, a first tail address, and a first address length determined by the second physical address of the corresponding physical block, and a second head address, a second tail address, and a second address length determined by a third physical address for receiving transmission data stored in the physical block; in response to writing the N descriptors into a descriptor controller register in a direct memory access controller, establishing a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports.

[0011] According to one aspect of the present disclosure, a data transmission device is provided, including: a locking module, used to lock M physical pages corresponding to the transmission data determined by a direct memory access transmission request from a first memory of a first processor in a user state, where M is an integer greater than 0; a reorganization module, used to perform a block reorganization operation on the M physical pages according to a first physical address of each physical page, so as to obtain N physical blocks and a second physical address of each physical block, where N is less than or equal to M; and a transmission module, used to perform a direct memory access transmission using a direct memory access controller according to N second physical addresses corresponding to the N physical blocks, so as to transmit the transmission data stored in the N physical blocks in the first memory of the first processor to the second memory of the second processor.

[0012] In one possible implementation, the locking module is used to: initiate a memory request for the transmission data, and determine the user state address of the transmission data in the user space; call the memory management interface in the user state, and determine the M physical pages corresponding to the user state address from the first memory of the first processor; and lock the M physical pages corresponding to the user state address from the first memory of the first processor.

[0013] In one possible implementation, the reorganization module is used to: perform continuity judgment on M first physical addresses corresponding to M physical pages to obtain a judgment result; when the judgment result indicates that there are any K physical pages with continuous first physical addresses among the M physical pages, merge the K physical pages into a physical block, where K is an integer greater than or equal to 2 and less than or equal to M; determine the second physical address of the physical block based on the K first physical addresses corresponding to the K physical pages.

[0014] In one possible implementation, the transmission module is used to: apply for N third physical addresses for receiving the transmission data from the second memory of the second processor; use a direct memory access controller to establish a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports according to the N second physical addresses and the N third physical addresses; perform direct memory access transmission according to the transmission channel, and transfer the transmission data stored in N physical blocks in the first memory of the first processor to the second memory of the second processor.

[0015] In one possible implementation, applying for N third physical addresses for receiving the transmission data from the second memory of the second processor includes: determining the storage space of the physical block according to the second physical address corresponding to each physical block; and applying for a third physical address that satisfies the storage space from the second memory of the second processor according to the storage space.

[0016] In one possible implementation, the device also includes a trigger module for: in response to transmitting the transmission data stored in N physical blocks in the first memory of the first processor to the second memory of the second processor, triggering a transmission completion interrupt, wherein the transmission completion interrupt is used to notify the first processor that the transmission data is completed.

[0017] In a possible implementation, the method of using a direct memory access controller to establish a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports according to the N second physical addresses and the N third physical addresses includes: determining N descriptors according to the N second physical addresses and the N third physical addresses, each descriptor including a first head address, a first tail address, and a first address length determined by the second physical address of the corresponding physical block, and a second head address, a second tail address, and a second address length determined by a third physical address for receiving transmission data stored in the physical block; in response to writing the N descriptors into a descriptor controller register in a direct memory access controller, establishing a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports.

[0018] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0019] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.

[0020] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program implements the steps of the above method when executed by a processor.

[0021] In the disclosed embodiment, according to the transmission data determined by the direct memory access transmission request, the M physical pages corresponding to the transmission data can be locked from the first memory of the first processor in the user state, and the M physical pages can be reorganized according to the first physical address of each physical page to obtain N physical blocks and the second physical address of each physical block; then, a direct memory access transmission is performed using the direct memory access controller according to the N second physical addresses corresponding to the N physical blocks, and the transmission data stored in the N physical blocks in the first memory of the first processor is transmitted to the second memory of the second processor. In this way, the direct memory access transmission performance can be improved, thereby improving the direct memory access transmission rate between the first memory of the first processor and the second memory of the second processor.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0024] Figure 1 A flow chart of a data transmission method according to an embodiment of the present disclosure is shown.

[0025] Figure 2 A schematic diagram showing a data transmission method according to an embodiment of the present disclosure is shown.

[0026] Figure 3 A schematic diagram showing a block reorganization operation according to an embodiment of the present disclosure.

[0027] Figure 4 A block diagram of a data transmission device according to an embodiment of the present disclosure is shown.

[0028] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0029] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0030] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0031] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0032] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0033] Figure 1 A flow chart of a data transmission method according to an embodiment of the present disclosure is shown as follows: Figure 1 As shown, the data transmission method includes:

[0034] In step S11, according to the transmission data determined by the direct memory access transmission request, M physical pages corresponding to the transmission data are locked from the first memory of the first processor in the user state, where M is an integer greater than 0;

[0035] In step S12, according to the first physical address of each physical page, a block reorganization operation is performed on the M physical pages to obtain N physical blocks and a second physical address of each physical block, where N is less than or equal to M;

[0036] In step S13, a direct memory access transmission is performed using a direct memory access controller according to N second physical addresses corresponding to the N physical blocks, and the transmission data stored in the N physical blocks in the first memory of the first processor is transmitted to the second memory of the second processor.

[0037] In a possible implementation, the data transmission method may be executed by an electronic device such as a terminal device or a server, and the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method may be implemented by a processor calling a computer-readable instruction stored in a memory. Alternatively, the method may be executed by a server.

[0038] Exemplarily, the data transmission method may be used to implement data transmission between a first processor and a second processor in an electronic device, wherein the second processor may be located in the same electronic device as the first processor, or the second processor may be located in a different electronic device from the first processor.

[0039] The first processor may work in conjunction with the second processor, the first processor may be responsible for the overall scheduling and logic control of the processing task, and the second processor may perform graphics processing or large-scale parallel computing. For example, the first processor may include, but is not limited to: a central processing unit (CPU), a digital signal processing unit (DSP), an application specific integrated circuit (ASIC), a tensor processing unit (TPU), a field programmable gate array (FPGA), etc. The second processor may include, but is not limited to, a graphics processing unit (GPU), a general-purpose computing on graphics processing units (GPGPU), etc. The embodiments of the present disclosure do not limit the types of the first processor and the second processor.

[0040] In a possible implementation, the first memory is used to store operation data in the first processor and transmission data exchanged between the first processor and other devices (such as the second processor), and the second memory is used to store operation data in the second processor and transmission data exchanged between the second processor and other devices (such as the first processor). The data transmission method may utilize a direct memory access controller to transfer the transmission data stored in the first memory of the first processor to the second memory of the second processor.

[0041] Among them, the memory may include read-only memory (ROM), random access memory (RAM), dynamic random access memory (DRAM), video random access memory (VRAM), cache, synchronous dynamic random access memory (SDRAM), etc. The embodiments of the present disclosure do not limit the type of memory.

[0042] For example, the first processor may be a central processing unit (CPU), and the first memory of the first processor may be a dynamic random access memory (DRAM); the second processor may be a graphics processing unit (GPU), and the second memory of the second processor may be a video random access memory (VRAM).

[0043] Compared with the direct memory access (DMA) transmission process in the related technology, when the operating system receives a direct memory access transmission request, it will enter the kernel state from the user state, and perform a memory locking operation on the transmission data determined by the direct memory access transmission request in the kernel state, that is, in the kernel state, the physical page of the transmission data is locked from the first memory of the first processor, and then the direct memory access transmission is performed. The memory locking process in the kernel state will take up data transmission time and have a great impact on the transmission performance of the direct memory access.

[0044] In order to reduce the processing time during the direct memory access transmission process and improve the direct memory access transmission performance, in step S11, the memory locking operation performed in the kernel state driver layer during the direct memory access transmission process can be moved to the user space memory application stage, and the operating system can lock the M physical pages corresponding to the transmission data from the first memory of the first processor in the user state according to the transmission data determined by the direct memory access transmission request, where M is an integer greater than 0, so that the memory locking operation is implemented in the user state and is separated from the direct memory access transmission, thereby reducing the time consumption of the direct memory access transmission.

[0045] Among them, locking the M physical pages corresponding to the transmission data from the first memory of the first processor during the direct memory access transmission phase is to fix the physical pages of the transmission data to prevent the transmission data from being swapped out when in use, thereby ensuring that the transmission data remains consistent during the direct memory access transmission process.

[0046] The direct memory access transfer request may include a user state address and data volume corresponding to the transfer data to determine the transfer data of the first processor. The direct memory access transfer request may be generated by software in the user space (such as a process or a driver), or may be triggered by a hardware interrupt. The embodiments of the present disclosure do not limit the source of the direct memory access transfer request.

[0047] In step S11, M physical pages corresponding to the transmission data are locked from the first memory of the first processor in the user state, and in step S12, a block reorganization operation is performed on the M physical pages according to the first physical address of each physical page to obtain N physical blocks and the second physical address of each physical block, where N is less than or equal to M. The addresses of the N physical blocks are not continuous.

[0048] For example, the M physical pages corresponding to the transmission data may include physical page 1 to physical page M, physical page 1 corresponds to the first physical address 1, physical page 2 corresponds to the first physical address 2, and so on, physical page M corresponds to the first physical address M. The first physical address 1 is used to indicate the location of the physical page 1 in the first memory of the first processor, the first physical address 2 is used to indicate the location of the physical page 2 in the first memory of the first processor, and so on, the first physical address M is used to indicate the location of the physical page M in the first memory of the first processor.

[0049] Since in the first memory of the first processor, some of the M physical pages may be continuous and some may be discontinuous and have blank locations, a block reorganization operation can be performed on physical pages 1 to M according to the first physical address 1 to the first physical address M, and the physical pages that are continuous in the first memory of the first processor are merged to obtain N physical blocks and the second physical address of each physical block, where N is less than or equal to M.

[0050] Wherein, for any physical page i among physical pages 1 to physical page M, i is any integer from 1 to M. If physical page i is not continuous with other M-1 physical pages, physical page i can be directly regarded as a physical block, and the first physical address i of physical page i can be regarded as the second physical address of the physical block. If physical page i is continuous with one or several physical pages among other M-1 physical pages, these continuous physical pages can be regarded as a physical block, and the second physical address of the physical block can be determined according to the first physical addresses of these continuous physical pages.

[0051] In step S12, N physical blocks and the second physical address of each physical block are obtained. In step S13, a direct memory access transmission can be performed using a direct memory access controller according to the N second physical addresses corresponding to the N physical blocks, and the transmission data stored in the N physical blocks in the first memory of the first processor can be transmitted to the second memory of the second processor. Wherein, the direct memory access controller can establish a transmission channel between the first memory and the second memory without the intervention of a processor (such as the first processor or the second processor), such as a transmission channel based on a high-speed serial computer expansion bus (Peripheral Component Interconnect Express, PCIE), to perform direct data transmission between the first memory and the second memory.

[0052] In this way, compared with the related art in which the direct memory access controller transmits only one continuous physical segment data each time and triggers a transmission completion interrupt each time, in step S13, each physical block is a continuous physical segment data, and the direct memory access controller can transmit multiple continuous physical segment data at a time, and trigger a transmission completion interrupt only after multiple physical blocks are transmitted, which is conducive to further improving data transmission efficiency.

[0053] According to the data transmission method of the embodiment of the present disclosure, according to the transmission data determined by the direct memory access transmission request, the M physical pages corresponding to the transmission data can be locked from the first memory of the first processor in the user state, and according to the first physical address of each physical page, the M physical pages are reorganized to obtain N physical blocks and the second physical address of each physical block; then, a direct memory access transmission is performed using the direct memory access controller according to the N second physical addresses corresponding to the N physical blocks, and the transmission data stored in the N physical blocks in the first memory of the first processor is transmitted to the second memory of the second processor. In this way, the direct memory access transmission performance can be improved, thereby improving the direct memory access transmission rate between the first memory of the first processor and the second memory of the second processor.

[0054] Figure 2A schematic diagram of a data transmission method according to an embodiment of the present disclosure is shown below. Figure 2 Take as an example to exemplarily illustrate the data transmission method of the embodiment of the present disclosure.

[0055] In step S11, according to the transmission data determined by the direct memory access transmission request, M physical pages corresponding to the transmission data may be locked in the first memory of the first processor in the user state.

[0056] In a possible implementation, step S11 may include: initiating a memory request for the transmission data, and determining the user state address of the transmission data in the user space; calling the memory management interface in the user state, and determining the M physical pages corresponding to the user state address from the first memory of the first processor; and locking the M physical pages corresponding to the user state address from the first memory of the first processor.

[0057] Compared with the direct memory access transmission method in the related art, taking the direction from the first memory of the first processor on the host side to the second memory of the second processor on the device side as an example, the first processor will pass the used user state address to the kernel state, and then perform a memory lock operation on the physical page corresponding to the user state address, and then perform direct memory access transmission. This memory locking operation in the kernel state will have a great impact on the direct memory access transmission performance.

[0058] With respect to the direct memory access transmission method in the related art, in step S11, the embodiment of the present disclosure moves the memory locking operation performed at the driver layer during the direct memory access transmission process to the user space memory application stage, so that the memory locking operation is implemented in the user state and separated from the direct memory access transmission, thereby reducing the time consumption of the direct memory access transmission.

[0059] like Figure 2 As shown, in the direct memory access transmission stage, the embodiment of the present disclosure can perform a memory locking operation in the first memory of the first processor according to the user state address requested by the user space for the transmission data.

[0060] For example, a memory request can be initiated in the user space for transmitting data, and the anonymous page generated by calling the dynamic memory allocation function malloc is used as the user state address of the transmitted data. The user state address is a virtual address, and according to the user state address, the memory management interface can be called in the user state to determine the M physical pages corresponding to the user state address from the first memory of the first processor, and lock the M physical pages. Among them, the M physical pages corresponding to the user state address are locked from the first memory of the first processor, and the user state address processed by the kernel driver layer becomes the first physical address, which can be used directly.

[0061] The above-mentioned memory locking operation is to fix one or more physical pages corresponding to the user state address to prevent them from being swapped out when in use, and obtain the physical page corresponding to the user state address. In this way, the kernel driver will obtain the physical page, and the kernel code can convert the physical page to obtain the first physical address of the physical page. Subsequently, the source port of the direct memory access controller can be filled in based on the first physical address of the physical page, and the destination port of the direct memory access controller can be filled in based on the third physical address in the second memory for receiving the transmission data, so as to move data between the first processor and the second processor without the need to waste time on the memory locking operation of the address in the kernel state.

[0062] Compared with the direct memory access transmission operation in the related art, which receives the user state address in the kernel driver layer and uses the kernel function interface get_user_pages_fast() in the kernel driver layer to perform the memory locking operation, it will take about 8%-10% of the entire direct memory access transmission stage. The data transmission method of the disclosed embodiment can find the physical page corresponding to the user state address generated by the user space by calling the corresponding memory management interface, so that the memory can be locked in the user state, thus saving this part of time in the kernel driver of direct memory access, and the transmission rate of direct memory access can be improved.

[0063] like Figure 2 As shown, in step S11, M physical pages corresponding to the transmission data are locked from the first memory of the first processor in the user state, and in step S12, a block reorganization operation is performed on the M physical pages according to the first physical address of each physical page to obtain N physical blocks and the second physical address of each physical block.

[0064] In a possible implementation, in some scenarios, especially in multi-concurrent server applications, the user space needs to apply for some large blocks of memory, so some preset interfaces will be used to allocate sparse memory, and the processing of sparse memory will affect the transmission rate of direct memory access. In order to further improve the transmission speed of direct memory access, step S12 may include: performing continuity judgment on the M first physical addresses corresponding to the M physical pages to obtain a judgment result; when the judgment result indicates that there are any K physical pages with continuous first physical addresses among the M physical pages, the K physical pages are merged into a physical block, where K is an integer greater than or equal to 2 and less than or equal to M; according to the K first physical addresses corresponding to the K physical pages, the second physical address of the physical block is determined.

[0065] For example, in some cases, especially when a large amount of memory needs to be applied for, the sparse memory allocated using the preset interface is likely to be discontinuous. At this time, if these discontinuous memory blocks can be processed more effectively, the rate of direct memory access can be improved.

[0066] For example, the user state may use some preset interfaces to allocate a large amount of first memory and second memory containing a sparse memory structure. After the optimization process of the memory locking operation is followed, the first physical address of each physical page is transmitted to the kernel driver layer. However, there are blank spaces between these physical pages, so valid physical pages (validpage) and invalid physical pages (invalidpage) appear. Since the invalid physical page is not mapped with the real physical page at this time, the invalid physical page has no corresponding first physical address.

[0067] Figure 3 A schematic diagram showing a block reorganization operation according to an embodiment of the present disclosure is shown as follows: Figure 3 As shown, it is assumed that there are 6 (M=6) physical pages, namely: physical page 1 to physical page 6. It should be understood that the embodiment of the present disclosure does not specifically limit the number M of physical pages.

[0068] In the example, the first physical address of physical page 1 is 0x1000, the first physical address of physical page 2 is 0x3000, and there is an invalid physical page between physical page 2 and physical page 1. In this case, physical page 1 can be extracted as physical block 1, and the first physical address 0x1000 of physical page 1 can be used as the second physical address of physical block 1.

[0069] In the example, the first physical address of physical page 3 is 0x4000. Although there is an invalid physical page between physical page 3 and physical page 2, the first physical addresses of physical page 3 and physical page 2 are continuous (corresponding to K=2), and can be recombined into physical block 2. The second physical address of physical block 2 can be determined based on the first physical address 0x3000 of physical page 2 and the address 0x4000 of physical page 3.

[0070] According to the same principle, the first physical addresses of physical page 4, physical page 5 and physical page 6 are also continuous (corresponding to K=3), and can be reorganized into physical block 3. The second physical address of physical block 3 can be determined based on the first physical address of physical page 4, the first physical address of physical page 5 and the first physical address of physical page 6.

[0071] By judging the continuity of physical pages and merging continuous physical pages into physical blocks, the first memory is optimized, which facilitates the subsequent use of a memory access controller to transmit multiple physical blocks as continuous physical segment data at one time, which is beneficial to improving subsequent data transmission efficiency.

[0072] In step S12, N physical blocks and the second physical address of each physical block are obtained. In step S13, a direct memory access transmission can be performed using a direct memory access controller according to the N second physical addresses corresponding to the N physical blocks to transfer the transmission data stored in the N physical blocks in the first memory of the first processor to the second memory of the second processor.

[0073] In one possible implementation, in order to further improve data transmission efficiency, step S13 may include: applying for N third physical addresses for receiving the transmission data from the second memory of the second processor; using a direct memory access controller to establish a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports according to the N second physical addresses and the N third physical addresses; performing direct memory access transmission according to the transmission channel, and transmitting the transmission data stored in N physical blocks in the first memory of the first processor to the second memory of the second processor.

[0074] Exemplarily, according to the direct memory access transmission request, a preset function interface may be called to apply for N third physical addresses for receiving transmission data from the second memory of the second processor. The third physical address may be obtained by performing a block reorganization operation on the physical page in the second memory. The specific operation may refer to the block reorganization operation of the physical page in the first memory above, which will not be described in detail here.

[0075] In one possible implementation, applying for N third physical addresses for receiving the transmission data from the second memory of the second processor includes: determining the storage space of the physical block according to the second physical address corresponding to each physical block; and applying for a third physical address that satisfies the storage space from the second memory of the second processor according to the storage space.

[0076] Exemplarily, the second physical address corresponding to each physical block can be parsed, information related to the size of each physical block can be extracted, and the storage space of the physical block can be determined based on the information. For example, the first address and the last address of each physical block can be obtained, and the storage space of the physical block can be determined based on the first address and the last address.

[0077] Then, according to the storage space of each physical block in the first memory, a third physical address satisfying the storage space can be applied for from the second memory of the second processor.

[0078] Assume that the transmission data is locked in N physical blocks of the first memory, the storage space of physical block 1 is S1, the storage space of physical block 2 is S2, and so on, the storage space of physical block N is SN. The N storage spaces can be sorted in order from large to small, and according to the order of the N physical blocks in the first memory from large to small, the third physical address of each physical block satisfying the storage space is determined from the continuous blank space of the second memory of the second processor.

[0079] Among them, when there are multiple continuous blank space segments in the second memory that meet the storage space of a physical block of the first memory, the smallest continuous blank space segment can be selected from the second memory, and the physical address of the smallest continuous blank space segment can be used as the third physical address to improve the utilization of the memory space.

[0080] In this way, N third physical addresses for receiving transmission data can be determined efficiently and accurately from the second memory.

[0081] After determining the N second physical addresses of the first memory and the N third physical addresses of the second memory, a direct memory access controller can be used to establish a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports according to the N second physical addresses and the N third physical addresses.

[0082] In a possible implementation, the method of using a direct memory access controller to establish a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports according to the N second physical addresses and the N third physical addresses includes: determining N descriptors according to the N second physical addresses and the N third physical addresses, each descriptor including a first head address, a first tail address, and a first address length determined by the second physical address of the corresponding physical block, and a second head address, a second tail address, and a second address length determined by a third physical address for receiving transmission data stored in the physical block; in response to writing the N descriptors into a descriptor controller register in a direct memory access controller, establishing a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports.

[0083] Exemplarily, the direct memory access controller can read the contents stored in the descriptor controller register and perform direct memory access transmission. By writing N descriptors into the descriptor controller register in the direct memory access controller, a transmission channel with N second physical addresses as source ports and N third physical addresses as destination ports can be established.

[0084] In the process of filling the descriptor into the descriptor controller register, the first beginning address, the first ending address, and the first address length of each physical block can be determined according to the second physical address of each physical block in the first memory, and the second beginning address, the second ending address, and the second address length of each physical block can be determined according to the third physical address in the second memory for receiving the transmission data stored in the corresponding physical block.

[0085] like Figure 3 As shown, the first first address, first last address, and first address length of physical block 1 can be written into descriptor 1 as the source port of data transmission in physical block 1, and the second first address, second last address, and second address length corresponding to physical block 1 can be written into descriptor 1 as the target port of data transmission in physical block 1; the first first address, first last address, and first address length of physical block 2 can be written into descriptor 2 as the source port of data transmission in physical block 2, and the second first address, second last address, and second address length corresponding to physical block 2 can be written into descriptor 2 as the target port of data transmission in physical block 2; the first first address, first last address, and first address length of physical block 3 can be written into descriptor 3 as the source port of data transmission in physical block 3, and the second first address, second last address, and second address length corresponding to physical block 3 can be written into descriptor 3 as the target port of data transmission in physical block 3.

[0086] In this way, a data transmission channel between the first memory and the second memory can be established efficiently and accurately.

[0087] A data transmission channel between the first memory and the second memory is determined, and direct memory access transmission can be used to transmit the transmission data stored in N physical blocks in the first memory of the first processor to the second memory of the second processor according to the data transmission channel between the first memory and the second memory.

[0088] In one possible implementation, the method further includes: in response to transferring the transmission data stored in N physical blocks in the first memory of the first processor to the second memory of the second processor, triggering a transmission completion interrupt, wherein the transmission completion interrupt is used to notify the first processor that the transmission data transmission is completed.

[0089] Compared with related technologies, direct memory access only transmits one continuous physical segment each time, and each transmission will trigger an interrupt. Figure 3 As shown, when using the block transfer method, in the related technology, transmitting physical page 1 requires one direct memory access transmission, transmitting physical page 2 and physical page 3 requires one direct memory access transmission, and transmitting physical page 4, physical page 5 and physical page 6 also requires one direct memory access transmission. In this way, three direct memory access transmissions are required, and the transmission process consumes a relatively long time.

[0090] In the embodiment of the present disclosure, a transmission completion interrupt is triggered only when the transmission data stored in multiple physical blocks in the first memory of the first processor is transferred to the second memory of the second processor, and the transmission process consumes a relatively short time.

[0091] In summary, compared with the time-consuming points in the data transmission process in the related art, such as the time of memory locking operation, and the time consumed in processing sparse memory structures and descriptors. The transmission method of the embodiment of the present disclosure can lock the M physical pages corresponding to the transmission data from the first memory of the first processor in the user state according to the transmission data determined by the direct memory access transmission request, and perform block reorganization operations on the M physical pages according to the first physical address of each physical page to obtain N physical blocks and the second physical address of each physical block; then, use the direct memory access controller to perform a direct memory access transmission according to the N second physical addresses corresponding to the N physical blocks, and transfer the transmission data stored in the N physical blocks in the first memory of the first processor to the second memory of the second processor.

[0092] In this way, the transmission method of the embodiment of the present disclosure moves the time-consuming memory locking operation upward, reduces the processing time in the direct memory access transmission process, and improves the direct memory access transmission performance. In addition, compared with the related art that uses an algorithm to find continuous blocks and compares multiple direct memory access transmission technologies for multiple discontinuous memory blocks, the transmission method of the embodiment of the present disclosure performs block reorganization processing on the sparse memory structure corresponding to the applied first memory and the second memory, and executes a direct memory access transmission once, so that the transmission data stored in the physical block can be transmitted.

[0093] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0094] In addition, the present disclosure also provides a data transmission device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any data transmission method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to in the corresponding records of the method part and will not be repeated here.

[0095] Figure 4 A block diagram of a data transmission device according to an embodiment of the present disclosure is shown as follows: Figure 4 As shown, the device comprises:

[0096] A locking module 41, configured to lock M physical pages corresponding to the transmission data determined by the direct memory access transmission request from the first memory of the first processor in a user state, where M is an integer greater than 0;

[0097] The reorganization module 42 is used to perform a block reorganization operation on the M physical pages according to the first physical address of each physical page to obtain N physical blocks and a second physical address of each physical block, where N is less than or equal to M;

[0098] The transmission module 43 is used to use the direct memory access controller to perform a direct memory access transmission according to the N second physical addresses corresponding to the N physical blocks, and transmit the transmission data stored in the N physical blocks in the first memory of the first processor to the second memory of the second processor.

[0099] In one possible implementation, the locking module 41 is used to: initiate a memory request for the transmission data, and determine the user state address of the transmission data in the user space; call the memory management interface in the user state, and determine the M physical pages corresponding to the user state address from the first memory of the first processor; and lock the M physical pages corresponding to the user state address from the first memory of the first processor.

[0100] In one possible implementation, the reorganization module 42 is used to: perform continuity judgment on M first physical addresses corresponding to M physical pages to obtain a judgment result; when the judgment result indicates that there are any K physical pages with continuous first physical addresses among the M physical pages, merge the K physical pages into a physical block, where K is an integer greater than or equal to 2 and less than or equal to M; determine the second physical address of the physical block based on the K first physical addresses corresponding to the K physical pages.

[0101] In one possible implementation, the transmission module 43 is used to: apply for N third physical addresses for receiving the transmission data from the second memory of the second processor; use a direct memory access controller to establish a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports according to the N second physical addresses and the N third physical addresses; perform direct memory access transmission according to the transmission channel, and transfer the transmission data stored in N physical blocks in the first memory of the first processor to the second memory of the second processor.

[0102] In one possible implementation, applying for N third physical addresses for receiving the transmission data from the second memory of the second processor includes: determining the storage space of the physical block according to the second physical address corresponding to each physical block; and applying for a third physical address that satisfies the storage space from the second memory of the second processor according to the storage space.

[0103] In one possible implementation, the device also includes a trigger module for: in response to transmitting the transmission data stored in N physical blocks in the first memory of the first processor to the second memory of the second processor, triggering a transmission completion interrupt, wherein the transmission completion interrupt is used to notify the first processor that the transmission data is completed.

[0104] In a possible implementation, the method of using a direct memory access controller to establish a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports according to the N second physical addresses and the N third physical addresses includes: determining N descriptors according to the N second physical addresses and the N third physical addresses, each descriptor including a first head address, a first tail address, and a first address length determined by the second physical address of the corresponding physical block, and a second head address, a second tail address, and a second address length determined by a third physical address for receiving transmission data stored in the physical block; in response to writing the N descriptors into a descriptor controller register in a direct memory access controller, establishing a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports.

[0105] This method has a specific technical connection with the internal structure of the computer system, and can solve the technical problem of how to improve the hardware computing efficiency or execution effect (including reducing the amount of data storage, reducing the amount of data transmission, increasing the hardware processing speed, etc.), thereby obtaining the technical effect of improving the internal performance of the computer system in accordance with the laws of nature.

[0106] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0107] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0108] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0109] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0110] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0111] Figure 5 1 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal device. Figure 5 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0112] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (Mac OS X TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or similar.

[0113] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0114] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0115] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0116] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0117] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0118] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0119] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0120] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0121] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0122] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0123] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0124] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0125] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0126] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A data transmission method, characterized in that: include: According to the transmission data determined by the direct memory access transmission request, locking M physical pages corresponding to the transmission data from the first memory of the first processor in the user state, where M is an integer greater than 0; According to the first physical address of each physical page, a block reorganization operation is performed on the M physical pages to obtain N physical blocks and a second physical address of each physical block, where N is less than or equal to M; A direct memory access controller is used to perform a direct memory access transmission according to the N second physical addresses corresponding to the N physical blocks, and the transmission data stored in the N physical blocks in the first memory of the first processor is transmitted to the second memory of the second processor.

2. The method according to claim 1, characterized in that The method of determining the transmission data according to the direct memory access transmission request and locking M physical pages corresponding to the transmission data from the first memory of the first processor in the user state includes: Initiate a memory request for the transmission data, and determine the user state address of the transmission data in the user space; Invoke a memory management interface in the user state to determine M physical pages corresponding to the user state address from the first memory of the first processor; The M physical pages corresponding to the user state address are locked from the first memory of the first processor.

3. The method according to claim 1, characterized in that The block reorganization operation is performed on the M physical pages according to the first physical address of each physical page to obtain N physical blocks and the second physical address of each physical block, including: Performing continuity judgment on the M first physical addresses corresponding to the M physical pages to obtain a judgment result; If the judgment result indicates that the first physical addresses of any K physical pages among the M physical pages are continuous, the K physical pages are merged into a physical block, where K is an integer greater than or equal to 2 and less than or equal to M; The second physical address of the physical block is determined according to the K first physical addresses corresponding to the K physical pages.

4. The method according to claim 1, characterized in that: The method of performing a direct memory access transmission using a direct memory access controller according to N second physical addresses corresponding to the N physical blocks, and transmitting the transmission data stored in the N physical blocks in the first memory of the first processor to the second memory of the second processor, includes: Applying for N third physical addresses for receiving the transmission data from the second memory of the second processor; Using a direct memory access controller to establish a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports according to the N second physical addresses and the N third physical addresses; Direct memory access transmission is performed according to the transmission channel to transmit the transmission data stored in the N physical blocks in the first memory of the first processor to the second memory of the second processor.

5. The method according to claim 4, characterized in that The applying for N third physical addresses for receiving the transmission data from the second memory of the second processor includes: Determine the storage space of each physical block according to the second physical address corresponding to the physical block; According to the storage space, a third physical address satisfying the storage space is applied for from the second memory of the second processor.

6. The method according to claim 4, characterized in that The method further comprises: In response to transmitting the transmission data stored in N physical blocks in the first memory of the first processor to the second memory of the second processor, a transmission completion interrupt is triggered, and the transmission completion interrupt is used to notify the first processor that the transmission data transmission is completed.

7. The method according to claim 4, characterized in that The method of using the direct memory access controller to establish a transmission channel with the N second physical addresses as source ports and the N third physical addresses as destination ports according to the N second physical addresses and the N third physical addresses includes: Determine N descriptors according to the N second physical addresses and the N third physical addresses, each descriptor including a first head address, a first tail address, and a first address length determined by the second physical address of the corresponding physical block, and a second head address, a second tail address, and a second address length determined by a third physical address for receiving transmission data stored in the physical block; In response to writing the N descriptors into a descriptor controller register in a direct memory access controller, a transmission channel is established with the N second physical addresses as source ports and the N third physical addresses as destination ports.

8. A data transmission device, characterized in that: include: A locking module, configured to lock M physical pages corresponding to the transmission data determined by the direct memory access transmission request from the first memory of the first processor in a user state, where M is an integer greater than 0; A reorganization module, used for performing a block reorganization operation on M physical pages according to a first physical address of each physical page, to obtain N physical blocks and a second physical address of each physical block, where N is less than or equal to M; The transmission module is used to use the direct memory access controller to perform a direct memory access transmission according to N second physical addresses corresponding to N physical blocks, and transmit the transmission data stored in the N physical blocks in the first memory of the first processor to the second memory of the second processor.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images