Data processing system and method, and medium

Abstract

Disclosed in the present application are a data processing system and method, and a medium in the technical field of computers. In the present application, the data processing system comprises a plurality of memory devices and at least one host; after the host receives an access request, if the host determines that a cache component thereof does not store target data to be accessed by means of the access request, the current host transmits the access request to the plurality of memory devices, such that the plurality of memory devices respond to the access request, a cache pre-fetching decision maker is used to determine pre-cached data, and the pre-cached data is transmitted to the cache component in the current host for storage. Therefore, the cache pre-fetching decision maker is transferred from a host end to a memory end, thereby reducing the load of the host, and improving the efficiency of processing of the access request by the host; moreover, the memory end may directly transmit the pre-cached data to the cache component in the current host for storage without needing the host to request the pre-cached data from the memory end, and thus the pre-caching efficiency can be improved.

Description

Data processing system, method and medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on March 11, 2024, with application number 202410270572.5 and application name “A Data Processing System, Method and Medium”, all contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a data processing system, method, and medium. Background Art

[0004] Currently, the host can pre-read additional data when reading the back-end storage device and store the pre-read data in the host cache. This implementation process requires the host to determine the pre-read data. Determining the pre-read data on the host side will occupy host resources and affect the host's processing efficiency of front-end read and write requests. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a data processing system, method and medium to reduce the host load and improve the efficiency of the host in processing read and write requests. The specific solution is as follows:

[0006] In one aspect, the present application provides a data processing system comprising: a plurality of memory devices and at least one host; the plurality of memory devices are connected to the at least one host;

[0007] At least one host includes: a cache component;

[0008] At least one memory device among the plurality of memory devices stores a cache prefetch decision maker;

[0009] At least one host is used to: after receiving an access request, if it is determined that the target data to be accessed by the access request is not stored in its own cache component, the current host will transmit the access request to multiple memory devices so that the multiple memory devices respond to the access request, and use the cache prefetch decider to determine the pre-cached data, and transmit the pre-cached data to the cache component in the current host for storage.

[0010] On the other hand, at least one memory device is configured to: after determining the pre-cached data using the cache prefetch decision maker, forcibly store the pre-cached data into a cache component in the current host.

[0011] On the other hand, at least one memory device is configured to: after determining the pre-cached data using the cache prefetch decision maker, send the pre-cached data to the current host, so that the current host stores the pre-cached data in a cache component in the current host.

[0012] On the other hand, the plurality of memory devices communicate with at least one host through a multi-level interconnected CXL switch device.

[0013] In another aspect, at least one host is configured to: identify the plurality of memory devices and the CXL switch device to which the host is currently connected, and determine device numbers of the plurality of memory devices and the CXL switch device to which the host is currently connected.

[0014] On the other hand, a single upstream physical port of any CXL switch device is virtualized into multiple upstream virtual ports, and one upstream virtual port is connected to a host or a downstream physical port of another CXL switch device.

[0015] On the other hand, multiple memory devices form a memory pool, which further includes a memory controller. The memory controller is used to uniformly address the multiple memory devices and store an addressing table obtained by the addressing.

[0016] On the other hand, the memory controller is used to synchronize the addressing table to at least one host.

[0017] On the other hand, the memory controller is used to divide the memory area of ​​a single memory device in the memory pool into multiple memory slices.

[0018] On the other hand, the memory controller is used to divide the memory area of ​​a single memory device in the memory pool into multiple memory slices according to a preset application layer operation size.

[0019] On the other hand, the memory controller is used to receive multiple binding requests sent by different hosts, and bind different memory slices obtained by dividing the memory area of ​​a single memory device to different hosts according to the multiple binding requests.

[0020] On the other hand, the memory controller is used to: receive a single binding request sent by any host, and bind a memory area of ​​a single memory device in the memory pool to the current host according to the single binding request.

[0021] On the other hand, at least one host is configured to: generate an operation command based on an access request, and send the operation command to a memory controller in a memory pool formed by a plurality of memory devices via a CXL switch device to which the current host is connected, so that the memory controller determines a target memory device in the memory pool and executes the operation command on the target memory device.

[0022] On the other hand, at least one host is configured to: if it is determined that the target data is stored in its own cache component, then the current host responds to the access request based on the target data in its own cache component.

[0023] On the other hand, at least one host further includes a cache controller; the cache controller is configured to detect whether target data is stored in a cache component in the current host.

[0024] On the other hand, at least one memory device is used to: input the access request and the physical address of each memory device into the cache prefetch decision maker, so that the cache prefetch decision maker outputs a predicted physical address and determines the data stored in the predicted physical address as precache data.

[0025] On the other hand, at least one memory device is used to: determine the delay time and historical read time average corresponding to the predicted physical address, and determine the transmission timing of the pre-cached data according to the delay time and historical read time average.

[0026] On the other hand, at least one memory device is used to: determine the communication link between the memory device to which the predicted physical address belongs and the host that receives the access request, and count the link layer delay and link bandwidth delay of the communication link; and / or determine the device performance delay of the memory device to which the predicted physical address belongs; and / or determine the regional performance delay of the predicted physical address in the memory device to which it belongs; and obtain the delay time corresponding to the predicted physical address by integrating the link layer delay, link bandwidth delay, device performance delay and / or regional performance delay.

[0027] On the other hand, at least one memory device is used to: detect the hit rate of pre-cached data and / or whether the predicted physical address output by the cache prefetch decision maker is correct, and determine the accuracy of the cache prefetch decision maker based on the corresponding detection results; if the accuracy is lower than a preset threshold, optimize the cache prefetch decision maker based on the detection results.

[0028] On the other hand, at least one memory device is configured to: use a decision tree classifier to detect a hit rate of pre-cache data and / or whether a predicted physical address output by a cache prefetch decision maker is correct.

[0029] On the other hand, the present application provides a data processing method, applied to at least one host, wherein the at least one host is connected to a plurality of memory devices; the at least one host includes: a cache component; at least one memory device of the plurality of memory devices stores a cache prefetch decision maker;

[0030] After at least one host receives an access request, if it determines that the target data to be accessed by the access request is not stored in its own cache component, the current host transmits the access request to multiple memory devices so that the multiple memory devices respond to the access request, and uses the cache prefetch decider to determine the pre-cached data, and transmits the pre-cached data to the cache component in the current host for storage.

[0031] On the other hand, the present application also provides a data processing method, comprising:

[0032] After receiving the access request, if at least one host determines that the target data to be accessed by the access request is not stored in its own cache component, the current host transmits the access request to the multiple memory devices;

[0033] The plurality of memory devices respond to the access request and determine pre-cached data using a cache pre-fetch decider stored in at least one of the plurality of memory devices, and transmit the pre-cached data to a cache component in the current host for storage;

[0034] Wherein, at least one host is connected to a plurality of memory devices; at least one host includes a cache component; and at least one memory device among the plurality of memory devices stores a cache prefetch decision maker.

[0035] On the other hand, the present application provides a non-volatile readable storage medium for storing a computer program, wherein the computer program implements any of the aforementioned methods when executed by a processor.

[0036] It can be seen from the above scheme that the present application provides a data processing system, including: multiple memory devices and at least one host; multiple memory devices are connected to at least one host; at least one host includes: a cache component; at least one memory device among the multiple memory devices stores a cache prefetch decision maker; at least one host is used to: after receiving an access request, if it is determined that the target data to be accessed by the access request is not stored in its own cache component, the current host transmits the access request to multiple memory devices so that the multiple memory devices respond to the access request, and uses the cache prefetch decision maker to determine the pre-cached data, and transmits the pre-cached data to the cache component in the current host for storage.

[0037] The beneficial effects of the present application are as follows: a data processing system includes multiple memory devices and at least one host; after the host receives an access request, if it determines that the target data to be accessed by the access request is not stored in its own cache component, the current host transmits the access request to multiple memory devices, so that the multiple memory devices respond to the access request, and uses the cache prefetch decision maker to determine the pre-cached data, and transmits the pre-cached data to the cache component in the current host for storage, thereby transferring the cache prefetch decision maker from the host side to the memory side, reducing the host load and improving the efficiency of the host in processing access requests (including read requests and write requests); and the memory side can directly transmit the pre-cached data to the cache component in the current host for storage, without the host requesting the pre-cached data from the memory side. Compared with the related scheme in which the memory side only detects changes in the host cache, the pre-caching efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0039] FIG1 is a schematic diagram of a data processing system disclosed in some embodiments of the present application;

[0040] FIG2 is a schematic diagram of a memory interconnect architecture system consisting of a single switch disclosed in some embodiments of the present application;

[0041] FIG3 is a schematic diagram of a multi-level interconnection architecture system disclosed in some embodiments of the present application;

[0042] FIG4 is a schematic diagram of a memory virtual partition architecture system disclosed in some embodiments of the present application;

[0043] FIG5 is a schematic diagram of a memory mapping disclosed in some embodiments of the present application;

[0044] FIG6 is a schematic diagram showing the layout and functions of a host cache component and a cache prefetch decision maker disclosed in some embodiments of the present application;

[0045] FIG7 is a schematic diagram of a communication process disclosed in some embodiments of the present application;

[0046] FIG8 is a diagram of a server structure provided by some embodiments of the present application;

[0047] FIG9 is a diagram of a terminal structure provided in some embodiments of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the accompanying drawings of some embodiments of the present application to clearly and completely describe the technical solutions in some embodiments of the present application. Obviously, the embodiments described are only some embodiments of the present application, not all embodiments. Based on some embodiments of the present application, all other examples obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0049] Currently, when a host reads a back-end storage device, it can pre-read additional data and store the pre-read data in the host cache. This implementation requires the host to determine the pre-read data, which consumes host resources and affects the host's efficiency in processing front-end read and write requests. To this end, the present application provides a data processing system, method, and medium that can reduce host load and improve the host's efficiency in processing read and write requests.

[0050] As shown in FIG1 , some embodiments of the present application disclose a data processing system comprising: multiple memory devices and at least one host; the multiple memory devices are connected to the at least one host; the at least one host includes a cache component; and at least one of the multiple memory devices stores a cache prefetch decision maker. The host may be a server or other device. The cache prefetch decision maker is any model capable of predicting pre-cached data, such as a neural network model or a machine learning model.

[0051] At least one host is configured to: upon receiving an access request, if it determines that the target data to be accessed by the access request is not stored in its own cache component, transmit the access request to multiple memory devices, so that the multiple memory devices respond to the access request, and use a cache prefetch decision maker to determine pre-cached data, and transmit the pre-cached data to the cache component in the current host for storage. The multiple memory devices constitute a memory pool, which monitors the change status of data stored in the cache component in the host and can actively write the pre-cached data to the cache component in the host based on the monitored change status.

[0052] In one example, at least one memory device is configured to: after determining the pre-cached data using the cache prefetch decision maker, forcibly store the pre-cached data into a cache component of the current host; or after determining the pre-cached data using the cache prefetch decision maker, send the pre-cached data to the current host so that the current host stores the pre-cached data into the cache component of the current host.

[0053] In one example, multiple memory devices and at least one host communicate via a multi-layered CXL (Compute Express Link) switch. Accordingly, at least one host is configured to identify the CXL switch to which the multiple memory devices and the current host are connected, and determine the device number of the memory pool and the device number of the CXL switch to which the current host is connected. The CXL switch communicates with other devices, such as switches, via CXL.

[0054] In one example, a single upstream physical port of any CXL switch device is virtualized into multiple upstream virtual ports, where one upstream virtual port is connected to a host or a downstream physical port of another CXL switch device.

[0055] In one example, the memory pool further includes: a memory controller; the memory controller is used to: uniformly address the memory pool and store the addressing table obtained by the addressing. Accordingly, the memory controller is used to: synchronize the addressing table to at least one host. Accordingly, the memory controller is used to: divide the memory area of ​​a single memory device in the memory pool into multiple memory slices. Accordingly, the memory controller is used to: divide the memory area of ​​a single memory device in the memory pool into multiple memory slices according to a preset application layer operation size. Accordingly, the memory controller is used to: receive multiple binding requests sent by different hosts, and bind different memory slices obtained by dividing the memory area of ​​a single memory device to different hosts according to the multiple binding requests. Accordingly, the memory controller is used to: receive a single binding request sent by any host, and bind the memory area of ​​a single memory device in the memory pool to the current host according to the single binding request.

[0056] In one example, at least one host is configured to: generate an operation command based on an access request, and send the operation command to a memory controller in a memory pool through a CXL switch device to which the current host is connected, so that the memory controller determines a target memory device in the memory pool and executes the operation command on the target memory device.

[0057] In one example, at least one host is configured to, upon determining that the target data is stored in its cache component, respond to the access request based on the target data in its cache component. The at least one host further includes a cache controller configured to detect whether the target data is stored in the cache component of the current host.

[0058] In one example, at least one memory device is used to input an access request and the physical address of each memory device in a memory pool into a cache prefetch decision maker, so that the cache prefetch decision maker outputs a predicted physical address and determines the data stored in the predicted physical address as pre-cached data. In one example, the cache prefetch decision maker is a multimodal model, and its input data and output data can be in different forms. The input data includes: a program counter (i.e., an access request) and the physical addresses of all memories (i.e., the physical addresses of each memory device in the memory pool); the output data includes: the physical address of the memory to be pre-read. Specifically, the cache prefetch decision maker can be implemented using a Transformer (deep learning model) architecture, with the input being a program counter instruction and the output being the physical address of the memory to be pre-read. Because the program counter and the physical address of the memory to be pre-read are in different forms, it is called a multimodal model. The program counter is transformed into data that can be recognized by the multimodal model through an encoding operation, and is input into the multimodal model. The output result of the multimodal model is transformed into a predicted memory physical address through a decoding operation. The multimodal model is an artificial intelligence model that can accurately understand the read and write behavior of the program counter, significantly improving the accuracy of pre-caching and significantly increasing access speed. However, the cache prefetching decision maker requires a large amount of space and is difficult to place on the host side.

[0059] In one example, at least one memory device is used to: determine the delay time and historical read time average corresponding to the predicted physical address, and determine the transmission timing of the pre-cached data based on the delay time and historical read time average. Accordingly, at least one memory device is used to: determine the communication link between the memory device to which the predicted physical address belongs and the host that receives the access request, and calculate the link layer delay and link bandwidth delay of the communication link; and / or determine the device performance delay of the memory device to which the predicted physical address belongs; and / or determine the regional performance delay of the memory device to which the predicted physical address belongs; and comprehensively calculate the link layer delay, link bandwidth delay, device performance delay, and / or regional performance delay to obtain the delay time corresponding to the predicted physical address.

[0060] Link layer latency is determined by the length of the communication link between the memory device and the host receiving the access request, while link bandwidth latency is the total bandwidth latency on the communication link. Device performance latency is determined by the characteristics of the memory device itself. Regional performance latency depends on the predicted location of the physical address in the memory device, as shown in Figure 4: Region 1 (MLD1, Memory Latency Determination), Region 2 (MLD2), Region 3 (MLD3), and Region 4 (MLD4). Different regions correspond to different regional performance latencies.

[0061] In one example, a timing predictor can be used to determine the transmission timing of pre-cached data based on the delay time and the historical read time average. The timing predictor is simple to implement. The rule-based timing predictor uses a detection table to detect the timing access pattern in the workload. The detection table records the delay time and the historical read time average corresponding to the predicted physical address. If multiple pre-cached data exist at the same time, each pre-cached data is transmitted to the host in the order of the corresponding transmission timing. The timing predictor can be set in the cache prefetch decision maker.

[0062] In one example, at least one memory device is configured to detect the hit rate of pre-cached data and / or the accuracy of a predicted physical address output by a cache prefetch decision maker, and determine the accuracy of the cache prefetch decision maker based on the detection results; if the accuracy is below a preset threshold, the cache prefetch decision maker is optimized based on the detection results. Accordingly, at least one memory device is configured to use a decision tree classifier to detect the accuracy of the pre-cached data and / or the accuracy of the predicted physical address output by the cache prefetch decision maker.

[0063] In this embodiment, a data processing system includes a memory pool and at least one host; after receiving an access request, if the host determines that the target data to be accessed by the access request is not stored in its own cache component, the current host transmits the access request to multiple memory devices so that the multiple memory devices respond to the access request, and uses a cache prefetch decision maker to determine pre-cached data, and transmits the pre-cached data to the cache component in the current host for storage. In this way, the cache prefetch decision maker is transferred from the host side to the memory pool, reducing the host load and improving the efficiency of the host in processing access requests (including read requests and write requests); in addition, the memory pool can directly transmit the pre-cached data to the cache component in the current host for storage, without the host requesting the pre-cached data from the memory pool. Compared with the related scheme in which the memory side only detects changes in the host cache, the pre-caching efficiency can be improved.

[0064] A single host may include at least one switch device that supports the CXL protocol. The upstream port of the switch device is used to connect to the host, and the downstream port is used to connect to other switch devices, multiple acceleration devices, and a memory pool. After the downstream port of the switch device completes the configuration of the configuration space and the allocation of the BAR (Base Address Register) bus base address, each downstream device can periodically broadcast its own free memory information to each other through the PCIE (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard) interface, thereby achieving direct PCIE communication between downstream devices. The switch device can effectively realize the communication and interconnection between the host and its downstream devices. PCIE is a high-speed serial point-to-point dual-channel high-bandwidth transmission, and the connected devices are allocated exclusive channel bandwidth. PCI Express also has a variety of specifications, from PCI Express x1 to PCI Express x32, which can meet the needs of low-speed and high-speed devices that will appear in a certain period of time in the future. The PCI Express interface is the PCIE 3.0 interface, with an 8 Gbps bit rate, approximately double the bandwidth of the previous generation. It also incorporates a range of key features, such as transmitter and receiver equalization and clock data recovery, to improve data transmission and protection. PCI Express bus links support full-duplex communication between any two endpoints. A channel consists of two differential signal pairs—one for receiving data and the other for transmitting data—as well as a differential reference clock pair. Therefore, each channel consists of four data lines. Conceptually, each channel functions as a full-duplex byte stream, simultaneously transmitting 8-bit byte packets in both directions between link endpoints. A physical PCI Express link may contain from one to 32 lanes.

[0065] The CXL protocol, compatible with the PCIe standard, addresses cache and memory access coherency issues across heterogeneous devices, making the memory, memory pool, and host cache of accelerators globally accessible to all CXL-capable devices. CXL technology maintains memory coherency between the CPU memory space and the memory on connected devices, enabling resource sharing (or memory pooling) for higher performance, reduced software stack complexity, and lower overall system cost. Therefore, the CXL interface enables communication between the host CPU and accelerators such as GPUs and FPGAs, resulting in higher data access efficiency and lower local data access latency. CXL consists of three dynamically multiplexed subprotocols on a single link: a PCIe-like IO protocol (CXL.io), a cache protocol (CXL.cach), and a memory access protocol (CXL.memory). Depending on the specific accelerator device usage model, all or only one of the subprotocols can be enabled. The CXL.io protocol is required for operations such as discovery and enumeration, error reporting, and host physical address lookup. A key advantage of CXL is that it provides a low-latency, high-bandwidth path for accelerator devices to access the system. CXL's memory cache coherence allows sharing of memory resources between the host CPU and acceleration devices.

[0066] In one example, downstream acceleration devices of a switch device in a host can communicate directly with each other using the DMA method without going through the host. If any acceleration device needs to transmit data, it determines other acceleration devices in the current host that are used to process the data based on the free memory capacity and the function of the acceleration card, and queries the memory free information of other acceleration devices in itself; then, it uses its own memory direct access controller and memory free information to write the current data to be transmitted into the memory of other acceleration devices using memory direct access. The switch device is a multi-network port device whose main function is to forward data between different ports, and it also reserves a data interface for docking with other devices.

[0067] In one example, the host also includes a data transmission device. The data transmission device is used to: directly connect to a memory address of the corresponding remote device according to the memory request of at least one remote device, so that the host can directly access the model training task and related data stored in the remote device with the help of the data transmission device. Wherein, the data transmission device includes: an address resolution module and a plurality of memory access modules; each memory access module is used to directly connect to a memory address of the corresponding remote device according to the memory request of at least one remote device, and supports time-sharing multiplexing of different remote devices connected to it, and each remote device directly accessed by each memory access module shares the processor and acceleration device of the control host. The memory access module includes a (Remote Direct Memory Access, remote direct data access) unit.

[0068] Another data processing system provided by this embodiment includes multiple hosts, and any host includes: multiple acceleration devices and a memory pool. Acceleration devices include FPGA acceleration cards, GPU acceleration cards, etc. Among them, the multiple hosts include a control host, and the control host divides the same model training task into multiple subtasks, and distributes the multiple subtasks to multiple hosts. The multiple hosts use the multiple acceleration devices in themselves to execute the received subtasks in parallel, and use the memory pool in themselves to store the training data, intermediate results and weight data corresponding to the corresponding subtasks. The control host uses the memory pool in itself to collect and process the weight data stored in the memory pools of the multiple hosts, and writes the latest weight data obtained through processing back to the memory pools of the multiple hosts. The memory pool can be a memory expansion card implemented based on FPGA.

[0069] In this embodiment, model training tasks are used to train various intelligent models. These intelligent models can be of any structure and can be used to implement tasks such as data encryption, decryption, image recognition, and classification. Hosts can share resources, accelerating computation. This embodiment divides a model training task into several parallel subtasks and distributes these subtasks across different host nodes, allowing them to run simultaneously on these nodes, thereby accelerating computation. If a host node is overloaded, some jobs can be moved to other host nodes for execution, reducing the load on the previous host node. This job migration is called load balancing. If a node fails, the remaining nodes can continue to operate, preventing the entire system from collapsing due to the failure of one or a few nodes. Therefore, the system has excellent fault tolerance. The system can also detect node failures and take appropriate measures to recover from them. Once the system determines the faulty node, it will no longer use it for services until it resumes normal operation. The functions of the failed node can be performed by other nodes, and when the failed node is restored or repaired, it can be smoothly integrated into the system.

[0070] In one embodiment, the host's processor is used to: query an idle memory access module in the data transmission device based on a memory request sent by any remote device; if an idle memory access module is found, an address configuration operation is generated for the idle memory access module, and the address configuration operation is sent to the idle memory access module; accordingly, the idle memory access module is used to: configure the memory address range corresponding to any remote device carried by the address configuration operation in itself according to the address configuration operation, and establish a remote memory access connection with the current remote device; accordingly, the address resolution module is used to: record the mapping relationship between the memory address range, the current remote device, and the memory access module configured with the memory address range. The processor is used to: generate an address configuration operation for the idle memory access module in the data transmission device based on a memory request sent by any remote device, and send the address configuration operation to the data transmission device; accordingly, the data transmission device is used to: enable the idle memory access module to configure the memory address range corresponding to any remote device carried by the address configuration operation in itself according to the address configuration operation, and establish a remote memory access connection with the current remote device;

[0071] In one embodiment, the host processor is configured to: if no free memory access module is found, return a request failure message to the corresponding remote device. In another embodiment, the host processor is configured to: detect the memory space size of a memory address range based on a memory request sent by any remote device; determine a memory mode that matches the memory space size; and manage the corresponding memory space according to the memory mode. In another embodiment, the host processor is configured to: set a configurable address range size for each memory access module in the data transmission device.

[0072] CXL is a cache coherent interconnect protocol that can separate memory from computing to build large-scale extended memory. Persistent memory has the advantages of low cost and large capacity compared to dynamic random access memory, so byte-addressable solid-state drives with CXL protocol and persistent memory can be used as extended memory. For example, CXL can be integrated into a solid-state drive that supports Optane (an ultra-high-speed memory technology) to achieve hierarchical memory expansion. Optane can also be replaced with Z-NAND (a specific type of NAND flash memory technology) or XLFlash (a high-performance storage solution). Z-NAND and XLFlash are also a type of memory technology.

[0073] As can be seen, CXL combined with SSDs allows for extended access to large memory capacities, but at a slower speed than dynamic random access memory. Therefore, the SSD's DRAM (Dynamic Random Access Memory) can also be used as a cache, similar to high-performance NVMe (Non-Volatile Memory Express) disk storage with larger DRAM, which can effectively reduce write latency.

[0074] However, it should be noted that when CXL-enabled SSDs (Solid State Drives) are used as host memory, the following problems exist: the host CPU (Central Processing Unit) has a low cache hit rate for the memory architecture that integrates CXL and SSDs; and different CXL-SSDs (CXL-enabled SSDs) located at different locations in the CXL switching network have different latencies. To this end, pre-caching technology can be introduced to improve cache hit rates and reduce latency. Pre-caching technology can be implemented based on spatial prefetching and / or temporal prefetching methods. Generally, data is pre-cached in the host CPU's last level cache (LLC) based on pre-caching technology.

[0075] The interconnect network topology used in CXL-based memory separation enables the increase of memory capacity in a scalable manner. CXL introduces a multi-level switching architecture. Adjusting the location of memory devices in the network can result in different latency for the memory devices because the processing time required by CXL switches at different levels is different.

[0076] CXL consists of three subprotocols: CXL.io, CXL.cache, and CXL.mem. Because CXL is built on the existing PCIe physical layer, CXL.io is functionally equivalent to the PCIe protocol. CXL.cache enables efficient access to host memory. CXL.mem enables the host to access connected memory devices anywhere within the corresponding CXL network. Therefore, CXL.io and CXL.mem are used to connect multiple memory devices and create large-scale memory pools. This is done using the two subprotocols CXL.io and CXL.mem.

[0077] Specifically, the memory pool is a multi-level memory pool, and multiple CXL memory devices in the memory pool can be interconnected through one or more CXL switches (i.e., CXL switching devices). A CXL switch includes several upstream physical ports and downstream physical ports. At least one upstream physical port, at least one downstream physical port, and a manager form a group, and each group can connect to a host and a CXL memory device. It can be seen that a structure-oriented manager can configure the upstream physical ports and downstream physical ports of the CXL switch so that each host can access its own CXL memory device through a unique data path called a virtual hierarchy. Based on the virtual hierarchy, a physical upstream port can be virtualized into multiple virtual upstream ports. CXL also supports multi-level interconnection, so that the upstream physical port or downstream physical port of the CXL switch can be connected to other CXL switches, significantly increasing the number of memory devices in each virtual hierarchy and greatly improving the scalability of resource separation.

[0078] In some examples, there may be three different levels of memory interconnect architecture, namely, a single switch, a multi-level interconnect, and memory virtual partitioning.

[0079] As shown in Figure 2, a memory interconnect architecture system consisting of a single switch may include a host connected to an upstream physical port of the switch, and each downstream physical port of the switch connected to a memory device. The host, switch, and memory devices communicate over CXL.

[0080] As shown in Figure 3, a multi-tier interconnect architecture system may include a host and three switches, one of which is located at the first tier and the other two at the second tier. Each downstream physical port of the two switches at the second tier is connected to a memory device. Communication between the host, switches, and memory devices is based on CXL.

[0081] Referring to Figure 4 , a memory virtual partitioning architecture system may include two hosts and a switch device. The two hosts are connected to two virtual upstream ports virtualized from the same upstream physical port of the switch device. The switch device's three downstream physical ports are each connected to a memory device. The memory area of ​​memory device 1 is divided into four regions: Region 1 (MLD1), Region 2 (MLD2), Region 3 (MLD3), and Region 4 (MLD4). These four regions are visible to both hosts 1 and 2, but different regions are used by different processors and cannot be shared.

[0082] The next step is to implement the memory pool logical mapping, which specifically implements the mapping from the memory physical address to the CXL memory pool logical address.

[0083] First, the CXL switch hierarchy is identified. The host cache component is a module consisting of both hardware and software. The hardware includes the host cache, and the software includes the cache policy. During PCIe enumeration, the host cache component effectively identifies the PCIe switch hierarchy of each memory device to determine how many CXL switches exist between the host and the memory device. The host uses CXL.io to access the configuration space of all connected devices (including memory devices and CXL switches) and organize them within its system bus within the CXL network.

[0084] It's important to note that during PCIe enumeration, each new device is identified, with a device number and corresponding bus number assigned to that bus. The CXL switch operates as a PCIe bridge device with its own bus number, allowing the number of switches between the host CPU and the target memory device to be determined. The host cache component can include a processor at its RC (Root Complex) end, which sits between the CPU and the CXL switch and stores all the information obtained during the enumeration process. This information is used to more accurately estimate prefetch timing.

[0085] When the host boots up, it enumerates all memory devices and initializes the mapping of BAR, SSD, and DRAM memory segments to the host. BAR and SSD correspond to different memory segments; the CXL memory controller also stores this mapping information. The host can update the corresponding memory segment mapping using the memory information and configuration information segments of the memory devices. All memory devices are uniformly addressed, and when a program in the host requires memory, the corresponding memory address segment is allocated. As shown in Figure 5, the RC maps the physical memory (including DRAM and SSD), BAR (Base Address Register), memory capacity, and configuration information of the memory pool end and stores them in the RP (Root Point, the hardware in the switching device). The memory controller also stores this information.

[0086] After a CXL memory pool is established, the host transmits load / store instructions via CXL.mem and CXL.io, accessing the memory through CXL message packets generated by a data flow control unit (such as flit software). This flit-based communication method enables the integration of various memory and storage media into the memory pool. CXL, combined with flit communication, effectively decouples memory resources from the host. Furthermore, CXL enables the introduction of back invalidation (BI). The memory pool uses CXL.mem to perform back-snooping on the host's cache, allowing the memory side to autonomously invalidate data in the host's cache. Alternatively, using CXL.cache, the memory side can also autonomously invalidate data in the host's cache.

[0087] As shown in Figure 6, the host cache component and cache prefetch decision maker constitute the cache prefetcher. The host cache component is located on the host side and includes the host-side RC and cache controller (including cache software policy). The cache prefetch decision maker is located in a memory device in the memory pool with a capacity greater than a fixed value. The host cache component provides the cache prefetch decision maker with key information necessary for decision-making, including the program counter (i.e., the operation instruction) and the switch depth of the memory device. It also relays the cache prefetch results determined by the cache prefetch decision maker. To achieve this, the host cache component uses a small buffer (demarcated from the host's L3 cache, which can be set to 16KB) to record the cache line update status prefetched by the cache prefetch decision maker. It also ensures that the cache controller of each host in the CXL network first checks the cache area of ​​the host cache component. If the requested data already exists in the RC, the cache controller directly provides the data from the RC's cache area and completes the response processing without traversing the entire memory pool.

[0088] As shown in Figure 6, the cache prefetch decision maker can be implemented based on spatial prefetching and temporal prefetching. Spatial prefetching predicts the next memory address to read by adding an offset to the currently accessed address. At runtime, this offset is optimized to minimize the cache miss rate. Temporal prefetching records cache miss sequences and delivers data to the host cache line from locations in the cache miss sequence that are likely to recur.

[0089] To meet accuracy requirements, deep learning techniques can be used to enhance the performance of cache prefetch decision makers. Because prefetching inherently involves prediction, deep learning techniques offer improved performance. However, deploying cache prefetch decision makers within the host is challenging because model calculation and metadata management on the host require significant storage space, which reduces host load.

[0090] This embodiment offloads the cache prefetch decision maker from the host to the memory pool, which can implement complex prefetch strategies. In one example, the cache prefetch decision maker can be in a heterogeneous form, and the cache prefetch decision maker performs irregular memory access in a random pattern. After the cache prefetch decision maker determines the prefetch location (the memory address to be read), it then transfers the data at this location to the buffer of the host cache component. The cache prefetch decision maker also provides a recording function for the memory addresses predicted in the past, so that the cache prefetch decision maker can be fine-tuned online. Specifically, the cache hit behavior of the application is monitored through a decision tree classifier, and feedback is given to the cache prefetch decision maker based on the address prefetch accuracy. When the accuracy decreases, the cache prefetch decision maker itself will fine-tune the parameters to optimize the model and improve the accuracy of memory address prediction.

[0091] The timing predictor in the cache prefetch decider determines the historical read time corresponding to a single memory address by averaging past arrival times, which is used to estimate the future read time of the data at that address. To predict the next arrival time, all past arrival times in its history window need to be recorded in a buffer.

[0092] If the access request is responded to by the host, the host cache component transmits the cache hit event to the cache prefetch decision maker through CXL.io, and can also record the response time, so that the timing predictor can calculate the average past read time of a single memory address based on it.

[0093] It should be noted that prefetching data too early may pollute the host cache and reduce its hit rate, while prefetching data too late may excessively delay execution. Therefore, it is crucial to accurately locate the exact prefetch timing of the pre-cached data. The actual prefetch timing can be determined by comprehensively considering: the data object exchange capability of the memory device (i.e., device performance delay), link layer delay, link bandwidth delay and / or regional performance delay, that is, calculating the delay overhead generated between the RC and the target memory device based on this. The host cache component stores this delay overhead in the configuration space of the corresponding memory device.

[0094] As shown in Figure 7, the process of transmitting read and write instructions from the host to the memory pool includes the following: When the host performs a load / store operation on a memory device, the CXL RP generates a CXL operation information packet and transmits it to the memory device via CXL.mem. The memory controller parses the CXL operation information packet to obtain the command operator, the memory address to be read, and other information. The memory controller interacts with the storage firmware to execute the corresponding command operation. Based on CXL.mem, to accurately predict the address and transmit the access request in a timely manner, the host's transaction data from CXL.mem to the slave memory includes: requests without data, requests with data, and reverse invalidation responses. Requests without data are primarily used for memory read opcodes without a payload, while requests with data carry memory write opcodes. Requests with data allow for 13 custom opcodes specified by CXL, including one for carrying instructions for memory reads. Reverse invalidation responses are responses to reverse invalidation sniff commands from the memory device.

[0095] When the prefetch time arrives, the cache prefetch decider predicts the memory address and updates the data at that memory address to the host cache component buffer. The transaction message from the memory pool to the host using CXL.mem is similar to a CXL.mem request without data; it is a message without a payload and is used only to monitor the host cache status. Host cache states include: M (Modified), which indicates that the data exists only in the current cache and is different from the data in the next-level storage unit. In other words, the latest data is in the current cache, with no copy in other caches, and the contents of the cache line are inconsistent with the main memory. O (Owned), which indicates that a cache line is dirty and may exist in multiple caches (more than one). A cache line in the Owned state stores the latest and correct data. Only one core's cache can store data in the Owned state; the others are in the Shared state. E (Exclusive), which indicates that the data exists only in the current cache line and is clean. The data in the cache line in the cache is consistent with the main memory, and no other core's cache has a copy of the data at that address, so it exists in only one cache. The S (Shared) state indicates that the data in the cache line is not necessarily consistent with the main memory. Corresponding to the cache line in the Owned state, the data in the Owned state is copied to the Shared cache line, so the data in the Shared state is also the latest. The I (Invalid) state indicates invalid data.

[0096] This embodiment also introduces a new BI opcode, called BISnoopData, which allows the use of 10 custom opcodes. Using BISnoopData, the cache prefetch decision maker generates a payload accompanying this message, which contains data for updating the host cache. Alternatively, when the host cache component detects BISnoopData, it waits for the corresponding payload and inserts the waiting data into its buffer, enabling the cache controller to obtain the data for execution. In one example, when the host reads data 1 in the memory pool, data 1 is returned to the host and cached by the host. Before the next request arrives, data 2 and 3 are prefetched into the host cache based on the address predicted by the cache prefetch decision maker.

[0097] It can be seen that this embodiment provides a memory architecture that integrates CXL-SSD and DRAM, and unloads the pre-cache of the last level cache (LLC) from the host CPU to the memory pool side. A cache prefetch decider is used to pre-cache data, and the memory rows on the host side are reversely invalidated through CXL.mem BI to ensure data consistency. In addition, it can also accurately estimate the prefetch time, reduce the cache prefetch time from the host to CXL-SSD, and enable the host cache to directly access most data in the memory architecture that integrates DRAM and CXL-SSD. Since the cache prefetch decision process occurs on the CXL-SSD side, the storage capacity and computing power are stronger, and complex prefetch strategies can be implemented to improve the cache hit rate of the host side to CXL-SSD.

[0098] This embodiment can also prefetch data across multiple memory devices. The cache prefetch host logic ensures that the CXL-SSD maintains awareness of the host CPU's execution semantics, while its CXL-SSD-side logic maintains data consistency between the host and the CXL-SSD through the reverse invalidation mechanism of CXL.mem. This two-way collaborative approach allows user applications to access most memory data directly on the host, significantly reducing dependence on the CXL-SSD. On the other hand, to fully understand the prefetch latency of different CXL-SSDs, the underlying CXL network topology and device latency are identified during PCIe enumeration and device discovery. This information is used to calculate a more accurate end-to-end latency for the CXL-SSD in each network, and this value is communicated by writing it into the PCIe configuration space of each device. This determines the optimal time for data to move from the memory pool to the host cache, effectively reducing the long latency introduced by the CXL-SSD backend.

[0099] The following introduces a data processing method provided by some embodiments of the present application. The data processing method described below can be referenced with other embodiments described in this document.

[0100] Some embodiments of the present application disclose a data processing method, which is applied to a host in a data processing system. The method includes: after receiving an access request, if it is determined that the target data to be accessed by the access request is not stored in its own cache component, the current host transmits the access request to multiple memory devices so that the multiple memory devices respond to the access request, and uses a cache prefetch decider to determine the pre-cached data, and transmits the pre-cached data to the cache component in the current host for storage.

[0101] The data processing system comprises: a memory pool and at least one host; the memory pool is connected to the at least one host; any host comprises: a cache component; the memory pool comprises: a plurality of memory devices; a cache prefetch decision maker is stored in at least one memory device.

[0102] On the other hand, the memory pool and at least one host communicate via a multi-level interconnected CXL switch device.

[0103] On the other hand, at least one host is configured to: identify the memory pool and the CXL switch device currently connected to the host, and determine the device number of the memory pool and the device number of the CXL switch device currently connected to the host.

[0104] On the other hand, a single upstream physical port of any CXL switch device is virtualized into multiple upstream virtual ports, and one upstream virtual port is connected to a host or a downstream physical port of another CXL switch device.

[0105] On the other hand, the memory pool further includes a memory controller; the memory controller is used to uniformly address the memory pool and store an addressing table obtained by the addressing.

[0106] On the other hand, the memory controller is used to synchronize the addressing table to at least one host.

[0107] On the other hand, the memory controller is used to divide the memory area of ​​a single memory device in the memory pool into multiple memory slices.

[0108] On the other hand, the memory controller is used to divide the memory area of ​​a single memory device in the memory pool into multiple memory slices according to a preset application layer operation size.

[0109] On the other hand, the memory controller is used to receive multiple binding requests sent by different hosts, and bind different memory slices obtained by dividing the memory area of ​​a single memory device to different hosts according to the multiple binding requests.

[0110] On the other hand, the memory controller is used to: receive a single binding request sent by any host, and bind a memory area of ​​a single memory device in the memory pool to the current host according to the single binding request.

[0111] On the other hand, at least one host is configured to: generate an operation command based on the access request, and send the operation command to a memory controller in the memory pool via a CXL switch device to which the current host is connected, so that the memory controller determines a target memory device in the memory pool and executes the operation command on the target memory device.

[0112] On the other hand, at least one host is configured to: if it is determined that the target data is stored in its own cache component, then the current host responds to the access request based on the target data in its own cache component.

[0113] On the other hand, at least one host further includes a cache controller; the cache controller is configured to detect whether target data is stored in a cache component in the current host.

[0114] On the other hand, the memory pool can implement the following functions: inputting an access request and the physical addresses of each memory device in the memory pool into a cache prefetch decision maker, causing the cache prefetch decision maker to output a predicted physical address, and determining the data stored at the predicted physical address as precached data. Determining the delay time and historical read time average corresponding to the predicted physical address, and determining the transmission timing of the precached data based on the delay time and historical read time average. Determining the communication link between the memory device to which the predicted physical address belongs and the host receiving the access request, and calculating the link layer delay and link bandwidth delay of the communication link; and / or determining the device performance delay of the memory device to which the predicted physical address belongs; and / or determining the regional performance delay of the memory device to which the predicted physical address belongs; and combining the link layer delay, link bandwidth delay, device performance delay, and / or regional performance delay to obtain the delay time corresponding to the predicted physical address. Detecting the hit rate of the precached data and / or the accuracy of the predicted physical address output by the cache prefetch decision maker, and determining the accuracy of the cache prefetch decision maker based on the corresponding detection results; if the accuracy is below a preset threshold, optimizing the cache prefetch decision maker based on the detection results. A decision tree classifier is used to detect whether the hit rate of the pre-cache data and / or the predicted physical address output by the cache prefetch decision maker is correct.

[0115] Some embodiments of the present application also disclose another data processing method, which is applied to at least one host, wherein the at least one host is connected to multiple memory devices; the at least one host includes a cache component; and at least one of the multiple memory devices stores a cache prefetch decision maker. The method includes: upon receiving an access request, if the at least one host determines that the target data to be accessed by the access request is not stored in its own cache component, the current host transmits the access request to the multiple memory devices so that the multiple memory devices respond to the access request, uses the cache prefetch decision maker to determine pre-cached data, and transmits the pre-cached data to the cache component of the current host for storage.

[0116] Furthermore, some embodiments of the present application disclose another data processing method, which is applied to a data processing system, including: after at least one host receives an access request, if it determines that the target data to be accessed by the access request is not stored in its own cache component, the current host transmits the access request to multiple memory devices; the multiple memory devices respond to the access request and use a cache prefetch decision maker stored in at least one of the multiple memory devices to determine the pre-cached data, and transmit the pre-cached data to the cache component in the current host for storage. The at least one host and the multiple memory devices are connected to form a data processing system; the at least one host includes a cache component; and at least one of the multiple memory devices stores a cache prefetch decision maker.

[0117] The following describes an electronic device provided by some embodiments of the present application. The electronic device described below can be cross-referenced with other embodiments described herein. The electronic device provided by this embodiment can be any device or apparatus described in other embodiments, such as a memory pool, a memory device, a host, a memory controller, and a cache controller.

[0118] Some embodiments of the present application provide electronic devices that are used to: after receiving an access request, if it is determined that the target data to be accessed by the access request is not stored in its own cache component, the current electronic device will transmit the access request to multiple memory devices so that the multiple memory devices respond to the access request, and use a cache prefetch decider to determine the pre-cached data, and transmit the pre-cached data to the cache component in the current electronic device for storage.

[0119] Some embodiments of the present application provide electronic devices that, after determining pre-cached data using a cache prefetch decision maker, compulsorily store the pre-cached data in a cache component of the current electronic device. Alternatively, after determining pre-cached data using the cache prefetch decision maker, send the pre-cached data to the current electronic device so that the current electronic device stores the pre-cached data in the cache component of the current electronic device.

[0120] Some embodiments of the present application provide an electronic device for identifying multiple memory devices in a memory pool and a CXL switch device to which the electronic device is currently connected, and determining the device number of each memory device in the memory pool and the device number of the CXL switch device to which the electronic device is currently connected. The CXL switch device is a device such as a switch that communicates with other devices via CXL.

[0121] Some embodiments of the present application provide electronic devices for: uniformly addressing multiple memory devices in a memory pool, and storing an addressing table obtained by addressing. Accordingly, it is also used to: synchronize the addressing table to at least one electronic device. Accordingly, it is also used to: divide the memory area of ​​a single memory device in the memory pool into multiple memory slices. Accordingly, it is also used to: divide the memory area of ​​a single memory device in the memory pool into multiple memory slices according to a preset application layer operation size. Accordingly, it is also used to: receive multiple binding requests sent by different electronic devices, and bind different memory slices obtained by dividing the memory area of ​​a single memory device to different electronic devices according to the multiple binding requests. Accordingly, the electronic device provided in this embodiment is used to: receive a single binding request sent by any electronic device, and bind the memory area of ​​a single memory device in the memory pool to the current electronic device according to the single binding request.

[0122] Some embodiments of the present application provide electronic devices for: generating an operation command based on an access request, and sending the operation command to a memory controller in a memory pool through a CXL switch device to which the current electronic device is connected, so that the memory controller determines a target memory device in the memory pool and executes the operation command on the target memory device.

[0123] Some embodiments of the present application provide an electronic device configured to, upon determining that target data is stored in a cache component of the electronic device, respond to an access request based on the target data in the cache component of the electronic device. Any electronic device further includes a cache controller configured to detect whether the target data is stored in the cache component of the electronic device.

[0124] Some embodiments of the present application provide electronic devices for inputting an access request and the physical addresses of each memory device in a memory pool into a cache prefetch decision maker, so that the cache prefetch decision maker outputs a predicted physical address and determines the data stored at the predicted physical address as precached data. The cache prefetch decision maker is a multimodal model whose input and output data can be in different forms. The input data includes: a program counter (i.e., an access request) and the physical addresses of all memories (i.e., the physical addresses of each memory device in the memory pool); the output data includes: the physical addresses of the memories to be pre-read. Specifically, the cache prefetch decision maker can be implemented using a Transformer architecture, with the input being a program counter instruction and the output being the physical addresses of the memories to be pre-read. Because the program counter and the physical addresses of the memories to be pre-read are in different forms, it is called a multimodal model. The program counter is transformed into data recognizable by the multimodal model through an encoding operation, which is then input into the multimodal model. The output of the multimodal model is transformed into the predicted physical address of the memories through a decoding operation. The multimodal model is an artificial intelligence model that can accurately understand the read and write behavior of the program counter, significantly improving the accuracy of pre-caching and significantly increasing access speed. Of course, the cache prefetch decision maker takes up a large space and is difficult to place on the electronic device.

[0125] Some embodiments of the present application provide electronic devices for: determining the delay time and historical read time average corresponding to the predicted physical address, and determining the transmission timing of the pre-cached data based on the delay time and historical read time average. Accordingly, the memory pool is used to: determine the communication link between the memory device to which the predicted physical address belongs and the electronic device that receives the access request, and to calculate the link layer delay and link bandwidth delay of the communication link; and / or determine the device performance delay of the memory device to which the predicted physical address belongs; and / or determine the regional performance delay of the predicted physical address in the memory device to which it belongs; and obtain the delay time corresponding to the predicted physical address by integrating the link layer delay, link bandwidth delay, device performance delay and / or regional performance delay.

[0126] Link-layer latency is determined by the length of the communication link between the memory device and the electronic device receiving the access request, while link bandwidth latency is the total bandwidth latency on that communication link. Device performance latency is determined by the characteristics of the memory device itself. Regional performance latency depends on the predicted physical address location within the memory device, as shown in Figure 4: Region 1 (MLD1), Region 2 (MLD2), Region 3 (MLD3), and Region 4 (MLD4). Different regions correspond to different regional performance latencies.

[0127] Some embodiments of the present application provide electronic devices for detecting the hit rate of pre-cached data and / or whether the predicted physical address output by a cache prefetch decision maker is correct, and determining the accuracy of the cache prefetch decision maker based on the corresponding detection results; if the accuracy is lower than a preset threshold, optimizing the cache prefetch decision maker based on the detection results. Accordingly, the electronic devices are further configured to detect the hit rate of pre-cached data and / or whether the predicted physical address output by the cache prefetch decision maker is correct using a decision tree classifier.

[0128] Furthermore, some embodiments of the present application further provide an electronic device. The electronic device may be a server as shown in FIG8 or a terminal as shown in FIG9. FIG8 and FIG9 are both structural diagrams of electronic devices according to an exemplary embodiment, and the contents of the diagrams are not to be construed as limiting the scope of application of the present application.

[0129] Figure 8 is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server may include: at least one processor, at least one memory, a power supply, a communication interface, an input / output interface, and a communication bus. The memory is used to store a computer program, which is loaded and executed by the processor to implement the relevant steps of the data processing disclosed in any of the aforementioned embodiments.

[0130] In some embodiments of the present application, a power supply is used to provide operating voltage for various hardware devices on the server; the communication interface can create a data transmission channel between the server and external devices, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0131] In addition, the memory as a carrier for resource storage can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon include operating system, computer programs and data, etc. The storage method can be temporary storage or permanent storage.

[0132] The operating system is used to manage and control the hardware devices and computer programs on the server to enable the processor to calculate and process data in the memory. It can be Windows Server (a server operating system), NetWare (a network operating system), Unix (a multi-user, multi-tasking operating system), Linux (an open source Unix-like operating system), etc. In addition to computer programs that can be used to complete the relevant steps of any of the aforementioned embodiments, computer programs can further include computer programs that can be used to complete other specific tasks. In addition to data such as application update information, data can also include data such as application developer information.

[0133] FIG9 is a schematic structural diagram of a terminal provided in some embodiments of the present application. The terminal may specifically include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0134] Generally, the terminal in some embodiments of the present application includes: a processor and a memory.

[0135] Among them, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0136] The memory may include one or more computer non-volatile readable storage media, which may be non-transitory. The memory may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory is used to store at least the following computer program, wherein, after the computer program is loaded and executed by the processor, it can implement the relevant steps performed by the terminal side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory may also include an operating system and data, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system may include Windows, Unix, Linux, etc. The data may include but is not limited to update information of the application.

[0137] In some embodiments of the present application, the terminal may further include a display screen, an input and output interface, a communication interface, a sensor, a power supply, and a communication bus.

[0138] Those skilled in the art will appreciate that the structure shown in FIG9 does not limit the terminal and may include more or fewer components than shown in the figure.

[0139] The following describes a non-volatile readable storage medium provided in some embodiments of the present application. The non-volatile readable storage medium described below can be cross-referenced with other embodiments described herein. A non-volatile readable storage medium is provided for storing a computer program, wherein the computer program, when executed by a processor, implements the method of any of the aforementioned embodiments.

[0140] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0141] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of non-volatile readable storage medium known in the art.

[0142] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A data processing system, characterized in that: include: a plurality of memory devices and at least one host; The plurality of memory devices are connected to the at least one host; The at least one host comprises: a cache component; At least one of the plurality of memory devices stores a cache prefetch decision maker; The at least one host is configured to: after receiving an access request, if it is determined that the target data to be accessed by the access request is not stored in its own cache component, the current host transmits the access request to the multiple memory devices so that the multiple memory devices respond to the access request, and uses the cache prefetch decider to determine the pre-cached data, and transmits the pre-cached data to the cache component in the current host for storage.

2. The system according to claim 1, wherein: The at least one memory device is configured to: after determining the pre-cached data using the cache prefetch decision maker, forcibly store the pre-cached data into a cache component in the current host.

3. The system according to claim 1, wherein: The at least one memory device is configured to: after determining the pre-cached data using the cache prefetch decision maker, send the pre-cached data to the current host, so that the current host stores the pre-cached data in a cache component in the current host.

4. The system according to claim 1, wherein: The plurality of memory devices communicate with the at least one host via a multi-level interconnected CXL switch device.

5. The system according to claim 4, characterized in that The at least one host is configured to: identify the plurality of memory devices and the CXL switch device to which the host is currently connected, and determine device numbers of the plurality of memory devices and the device number of the CXL switch device to which the host is currently connected.

6. The system according to claim 4, characterized in that A single upstream physical port of any CXL switch device is virtualized into multiple upstream virtual ports. One upstream virtual port is connected to a host or a downstream physical port of another CXL switch device.

7. The system according to claim 4, wherein: The multiple memory devices form a memory pool, and the memory pool further includes a memory controller; the memory controller is configured to: uniformly address the multiple memory devices and store an addressing table obtained by the addressing.

8. The system according to claim 7, characterized in that The memory controller is configured to synchronize the addressing table to the at least one host.

9. The system according to claim 7, wherein: The memory controller is configured to divide the memory area of ​​a single memory device in the memory pool into a plurality of memory slices.

10. The system according to claim 9, characterized in that The memory controller is configured to divide the memory area of ​​a single memory device in the memory pool into a plurality of memory slices according to a preset application layer operation size.

11. The system according to claim 9, wherein: The memory controller is configured to receive multiple binding requests sent by different hosts, and bind different memory slices obtained by dividing the memory area of ​​a single memory device to different hosts according to the multiple binding requests.

12. The system according to claim 9, wherein: The memory controller is configured to: receive a single binding request sent by any host, and bind a memory area of ​​a single memory device in the memory pool to the current host according to the single binding request.

13. The system according to claim 4, wherein: The at least one host is configured to: generate an operation command based on the access request, and send the operation command to a memory controller in a memory pool formed by the multiple memory devices through a CXL switch device to which the current host is connected, so that the memory controller determines a target memory device in the memory pool and executes the operation command on the target memory device.

14. The system according to claim 1, wherein: The at least one host is configured to: if it is determined that the target data is stored in its own cache component, the current host responds to the access request based on the target data in its own cache component.

15. The system according to claim 1, wherein: The at least one host further includes a cache controller; the cache controller is configured to detect whether the target data is stored in a cache component in the current host.

16. The system according to any one of claims 1 to 15, characterized in that The at least one memory device is configured to: input the access request and the physical address of each memory device into the cache prefetch decision maker, so that the cache prefetch decision maker outputs a predicted physical address, and determines the data stored in the predicted physical address as the precache data.

17. The system according to claim 16, wherein: The at least one memory device is configured to: determine a delay time and a historical read time average corresponding to the predicted physical address, and determine a transmission timing of the pre-cached data according to the delay time and the historical read time average.

18. The system according to claim 17, wherein: The at least one memory device is configured to: determine the communication link between the memory device to which the predicted physical address belongs and the host that receives the access request, and count the link layer delay and link bandwidth delay of the communication link; and / or determine the device performance delay of the memory device to which the predicted physical address belongs; and / or determine the regional performance delay of the predicted physical address in the memory device to which it belongs; and obtain the delay time corresponding to the predicted physical address by integrating the link layer delay, link bandwidth delay, device performance delay and / or regional performance delay.

19. The system according to any one of claims 1 to 15, characterized in that The at least one memory device is configured to: detect whether the hit rate of the pre-cached data and / or the predicted physical address output by the cache prefetch decision maker is correct, and determine the accuracy of the cache prefetch decision maker based on the corresponding detection results; if the accuracy is lower than a preset threshold, optimize the cache prefetch decision maker based on the detection results.

20. The system according to claim 19, wherein: The at least one memory device is configured to: use a decision tree classifier to detect whether a hit rate of the pre-cache data and / or a predicted physical address output by the cache prefetch decision maker is correct.

21. A data processing method, characterized in that: The invention is applied to at least one host, wherein the at least one host is connected to a plurality of memory devices; the at least one host comprises: a cache component; at least one memory device among the plurality of memory devices stores a cache prefetch decision maker; After receiving the access request, if the at least one host determines that the target data to be accessed by the access request is not stored in its own cache component, the current host transmits the access request to the multiple memory devices so that the multiple memory devices respond to the access request, and uses the cache prefetch decider to determine the pre-cached data, and transmits the pre-cached data to the cache component in the current host for storage.

22. A non-volatile readable storage medium, characterized in that: The device is configured to store a computer program, wherein the computer program implements the method according to claim 21 when executed by a processor.

Images