PCIe (Peripheral Component Interconnect Express) equipment capable of expanding base address register space and space parameter configuration method

Abstract

The invention discloses PCIe (Peripheral Component Interconnect Express) equipment capable of expanding a base address register space and a space parameter configuration method. The equipment comprises a PCIe equipment body internally provided with a configuration space and base address expansion equipment connected with the PCIe equipment body through a high-speed serial expansion interface, the PCIe device body is externally connected with memory devices through a high-speed serial interface, and the sum of the memory sizes of all the external memory devices is smaller than the space size of the base address extension device. By means of the PCIe device, decoupling of the base address register space and the local configuration space is achieved, meanwhile, dynamic expansion and configuration of memory resources of the base address register space are achieved, and the problem that application operation is limited due to the fact that the memory resources are insufficient is solved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a PCIe device and a method for configuring space parameters. Background Art

[0002] In the current PCIe technology, the device's configuration space (Configuration Space) and base address register space (BAR Space) are already fixed initial parameters in the device. Users can only access address data within this fixed space range through PCIe device access technology, and cannot access space outside the base address space.

[0003] In the current common PCIe architecture, after the system PCIe bus is scanned, the base address space of the PCIe device in the address range of the CPU domain has been determined, and the system cannot modify its address range. If the PCIe device is a memory device, the size of the only base address register space cannot meet the application requirements, but more memory resources are needed. The solution is to replace the PCIe device with a larger base address register space. If the hardware slots are limited, multiple devices cannot be used at the same time, and the existing devices will be unusable. Therefore, the base address space of the existing PCIe architecture cannot be dynamically adjusted in the address range of the CPU domain, resulting in the problem of limited operation due to insufficient memory resources when access content resources increase. Summary of the invention

[0004] Based on this, in order to solve the above technical problems, a PCIe device and space parameter configuration method with expandable base address register space are provided to solve the problem in the prior art that when the access content resources increase, the operation is limited due to insufficient memory resources.

[0005] In a first aspect, a PCIe device capable of expanding a base address register space comprises: a PCIe device body having a configuration space disposed therein, and a base address expansion device connected to the PCIe device body via a high-speed serial expansion interface;

[0006] The PCIe device body is connected to an external memory device via a high-speed serial interface, and the sum of the memory sizes of all external memory devices is smaller than the base address extension device space size.

[0007] In the above solution, optionally, the high-speed serial interface adopts SerDes technology to achieve high-speed data transmission with an external storage device.

[0008] In the above solution, optionally, the memory device is a DDR memory particle.

[0009] In the above solution, optionally, an address access protection module is provided inside the device to limit the address accessed by the host to exceed the base address to extend the device address range.

[0010] In a second aspect, a method for configuring space parameters of a PCIe device with an expandable base address register space is provided, wherein the PCIe device adopts the PCIe device with an expandable base address register space described in the first aspect; the method comprises:

[0011] Obtain the parameters stored in the FPGA's internal BRAM storage space so that the 0x00 offset of BAR0 is written; the parameters stored in the FPGA's internal BRAM storage space are the specific sizes of the memory particles in the base address extension space sent to the FPGA by the MCU after the MCU determines the specific sizes of the memory particles.

[0012] In the above scheme, optionally, the method also includes: when the space of the base address extension device changes, receiving and obtaining the re-modified parameters stored in the BRAM storage space of the FPGA, so as to rewrite the 0x00 offset of BAR0; the re-modified parameters stored in the BRAM storage space of the FPGA are after the space of the base address extension device is dynamically changed, and the MCU determines the specific size of the memory particles in the changed base address extension space and sends the specific size of the memory particles to the FPGA.

[0013] This application has at least the following beneficial effects:

[0014] This application separates the configuration space and base register space in the PCIe device, retains the configuration space locally, and uses the aurora protocol to separate the base register space of the PCIe device from the device itself, and flexibly expands the size of its base register space externally. This achieves the decoupling of the base register space from the local configuration space, while achieving dynamic expansion and configuration of the base register space memory resources, solving the problem of application operation restrictions due to insufficient memory resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a PCIe topology diagram of the prior art;

[0016] Figure 2 It is a structure diagram of PCIe configuration space and base address space in the prior art;

[0017] Figure 3 A structural diagram of a PCIe device with an expandable base address register space provided by one embodiment of the present application;

[0018] Figure 4 A schematic diagram of a logic circuit for base address space expansion provided by an embodiment of the present application;

[0019] Figure 5 An illustrative diagram of a spatial parameter configuration is provided for an embodiment of the present application;

[0020] Figure 6 This is a display result diagram of a PCIe device with an expandable base address register space using the present application and when the base address space is expanded to 256G;

[0021] Figure 7 A simplified structural diagram of a PCIe device with an expandable base address register space is provided for one embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0023] In the existing PCIe technology, the design of PCIe devices is to separate the configuration space from the base address space according to the standard PCIe framework. After the base address space of PCIe is loaded and mapped by the operating system, its address allocation in the system memory is fixed. The standard PCIe topology is as follows: Figure 1 shown.

[0024] For systems or applications that require large memory space, such as high-performance computing devices or storage systems, hardware design must consider how to effectively utilize limited base address space resources while ensuring system performance and reliability. This requires more complex hardware design and cost to support higher usage requirements. This will increase hardware costs and make the design more complex. Figure 2 Shown is a common PCIe configuration space and base address space structure diagram.

[0025] This application believes that based on the original design architecture of PCIe, the configuration space and base address space of the device are in a separate state, but the size of the base address space is immutable. From the perspective of the PCIe physical protocol, memory access is essentially the transmission of electrical signals based on physical media, and base address space access is also data packet transmission in high-speed serial signals, so a solution to expand the base address space using high-speed serial signals is proposed.

[0026] In one embodiment, Figure 3 As shown, a PCIe device capable of expanding base address register space is provided, the device comprising: a PCIe device body having a configuration space arranged therein, and a base address expansion device connected to the PCIe device body via a high-speed serial expansion interface;

[0027] The PCIe device body is connected to an external memory device via a high-speed serial interface, and the sum of the memory sizes of all external memory devices is smaller than the base address extension device space size.

[0028] Specifically, the base address space in the PCIe device is separated and interconnected with the extended base address space device, realizing a unique and novel design method in which the base address space of the PCIe device can be expanded and decoupled. The specific base address space device expansion method is as follows: First, the BAR (Base Address Space) address of the extended base address space PCIe device is configured as the base address space that a PCIe device can support at most, such as 256G. The host sees that the BAR size of the corresponding device Region4 is 256G through the lspci command (as can be seen in the invention effect diagram). In this design mode, external memory devices are connected through a high-speed serial interface, such as multiple DDR memory particles, but it is necessary to ensure that the sum of all DDR memory sizes cannot be greater than the initially set base address space size, that is, the initially set 256G size, otherwise some external memory devices will be inaccessible. The PCIe device and the base address expansion device are interconnected through a high-speed serial expansion interface, so that when the host accesses the extended base address space, it is through address routing, and the access on the address bus will be automatically routed to the expansion interface. Through the high-speed interconnection of the extended interface, data transmission stability and low latency are achieved.

[0029] like Figure 4 The figure shows the logic circuit diagram of base address space expansion. The logic design circuit diagram shows that a remote DDR is expanded, and different memory addresses accessed by the Host are routed to the local DDR or remote DDR through protocol conversion and axi-interconnect address routing. The extended access to the base address can be achieved by connecting the corresponding memory device at the corresponding routing end.

[0030] In the above-mentioned PCIe device with expandable base address register space, the configuration space and base address register space in the PCIe device are separated, the configuration space is retained locally, and the base address register space of the PCIe device is separated from the device itself through the aurora protocol, so that the size of its base address space can be flexibly expanded externally. The base address register space is decoupled from the local configuration space, and the memory resources of the base address register space are dynamically expanded and configured, solving the problem of application operation restrictions due to insufficient memory resources.

[0031] In one embodiment, the high-speed serial interface uses SerDes technology to achieve high-speed data transmission with an external storage device.

[0032] In one embodiment, an address access protection module is provided inside the device to limit the address accessed by the host to exceed the base address to extend the device address range.

[0033] Specifically, when using the driver, it should be noted that the maximum accessible base address space cannot exceed the memory size specified by the configuration parameters. If the address space outside the range is accessed (when the user accidentally operates beyond the address range provided by the logic), it may cause an abnormality in the PCIe bus, resulting in an abnormality or timeout initialization of the system. Therefore, it is necessary to add an address access protection strategy or a timeout mechanism to ensure that the host will not cause system downtime due to problems with the PCIe link, and avoid such abnormalities by limiting the address operation range.

[0034] In one embodiment, a method for configuring space parameters of a PCIe device with an expandable base address register space is provided, wherein the PCIe device adopts the above-mentioned PCIe device with an expandable base address register space; the method comprises:

[0035] Obtain the parameters stored in the FPGA's internal BRAM storage space so that the 0x00 offset of BAR0 is written; the parameters stored in the FPGA's internal BRAM storage space are the specific sizes of the memory particles in the base address extension space sent to the FPGA by the MCU after the MCU determines the specific sizes of the memory particles.

[0036] Specifically, Figure 5 The figure shows the parameter configuration diagram. An MCU (microcontroller) and a logic (logic) module are used to configure the base address space expansion. After the specific size of the memory particles in the base address expansion space is determined, the parameters of the corresponding memory size need to be configured through the MCU and logic. The MCU informs the FPGA through the network to modify its internal BRAM storage space, that is, to write the parameters to the 0x00 offset of BAR0 of the PCIe device. The detailed definition is that after determining the memory size of the base address space expansion, the 4K storage size is used as a unit and stored in the offset position starting from 0x00 in BAR0. The number of bytes occupied by storing this parameter size is 4 bytes. As long as the configuration parameter setting information at the corresponding position is modified, the size of the expanded memory can be known. For example, if an external 256G extended memory is connected and the size is 4K per unit, a parameter of 0x10000000 must be stored at the offset of BAR0. This parameter indicates that the extended base address space is connected to an external 256G memory size. If the size of the extended base address space is changed dynamically, this parameter needs to be modified through the MCU and logic module.

[0037] After loading the PCIe device driver, the driver needs to read the address data of BAR0 zero address offset and calculate the parameters of the first 4 bytes. After calculation, the total memory size corresponding to the base address extension space is determined, so that the driver knows how much address space it can access. After determining the size, the driver can read and write access operations to the extended base address space in the same way as the traditional PCIe design framework, without changing due to the specific implementation of external hardware, thus achieving the universality and good portability of the driver.

[0038] Based on high-speed SerDes interconnection, the configuration space and base address register space in the PCIe device are separated, realizing the redefinition of the device design architecture. Through innovative design, the stability of the device is ensured at the local level, while providing the scalability of base address space decoupling. It provides a reference design for the definition and design of future PCIe devices. It supports the requirements of high performance and high scalability, novel dynamic address allocation and management solutions, and an independent address space management system, which goes beyond the limitations of traditional static allocation. It also supports system-level expansion and integration to adapt to the management requirements of PCIe devices of different scales and complexities. Through this technical solution, the stability of the system is improved while the complexity of integration and maintenance is reduced.

[0039] Compared with the existing PCIe technology, the scalable and decoupled technical solution proposed by the present invention can realize the separation of the configuration space and the register space of the PCIe device, providing a valuable reference for the architecture of future PCIe devices. At the same time, this method realizes the dynamic expansion and efficient use of the base address register space of the PCIe device unprecedentedly.

[0040] When PCIe devices can dynamically load the base address register space, the driver can also adapt to the size of the external base address register space, solving the problem of insufficient base address register space at certain times. This design not only ensures the stability of the local space, but also provides decoupled scalability, decoupling the base address space as an efficient separation, and dynamically configuring its memory size, providing new references and ideas for future PCIe framework design.

[0041] The base address register space of general PCIe memory devices is fixed, but the design of this application can dynamically adjust and use the base address space, and the data transmission delay is at the nanosecond level. Not only is the delay comparable to that of common PCIe memory devices, but more importantly, the size of the (BAR4) base address register space can be made larger, such as Figure 6 The base address space is expanded to 256G to display the result.

[0042] like Figure 7This is a simplified diagram of the present application. The master and slave machines are interconnected through a protocol, and the externally interconnected memory device is used as an access space device with a base address extension, which can achieve local stability of the PCIe system and realize a decoupled design method based on high-speed serial signal extension. Among them, the host represents a device connected to the PCIe slot of the computer, and the slave is a storage device extended through the protocol (it can be a storage device, and the two are linked through a simple protocol and physical medium communication).

[0043] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0044] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A PCIe device capable of expanding base address register space, characterized in that: The device comprises: a PCIe device body having a configuration space therein, and a base address expansion device connected to the PCIe device body via a high-speed serial expansion interface; The PCIe device body is connected to an external memory device via a high-speed serial interface, and the sum of the memory sizes of all external memory devices is smaller than the base address extension device space size.

2. The PCIe device with expandable base address register space according to claim 1, characterized in that: The high-speed serial interface adopts SerDes technology to achieve high-speed data transmission with an external storage device.

3. The PCIe device with expandable base address register space according to claim 1, characterized in that: The memory device is a DDR memory chip.

4. The PCIe device with expandable base address register space according to claim 1, characterized in that: The device is internally provided with an address access protection module to limit the address accessed by the host from exceeding the base address to extend the device address range.

5. A method for configuring space parameters of a PCIe device capable of expanding base address register space, characterized in that: The PCIe device adopts the PCIe device with expandable base address register space as claimed in any one of claims 1 to 4; the method comprises: Obtain the parameters stored in the FPGA's internal BRAM storage space so that the 0x00 offset of BAR0 is written; the parameters stored in the FPGA's internal BRAM storage space are the specific sizes of the memory particles in the base address extension space sent to the FPGA by the MCU after the MCU determines the specific sizes of the memory particles.

6. The method for configuring space parameters of a PCIe device capable of expanding base address register space according to claim 5, characterized in that: The method also includes: when the space of the base address extension device changes, receiving and obtaining the re-modified parameters stored in the BRAM storage space of the FPGA so as to rewrite the 0x00 offset of BAR0; the re-modified parameters stored in the BRAM storage space of the FPGA are the specific size of the memory particles of the changed base address extension space after the space of the base address extension device is dynamically changed, and the MCU determines the specific size of the memory particles in the changed base address extension space and sends the specific size of the memory particles to the FPGA.

Images