423 results about "Host memory" patented technology

The invention provides an expert access prediction method and system suitable for a hybrid expert architecture large language model, and the method comprehensively considers the model structure and hardware feature information, constructs an expert access predictor based on a neural network for each layer, and carries out the training through the historical data. And experts needing to be activated in the model reasoning process can be predicted in advance. In the process, the predictor executes prediction on the CPU in parallel, and the prefetcher loads expert parameters from a host memory to a GPU video memory in advance according to a prediction result. By adjusting the prediction distance, the prediction accuracy and the prefetching advance are balanced, and the prefetching number can be dynamically adjusted according to the prediction accuracy. Meanwhile, parallel loading is adopted, system resources are utilized to the maximum extent, and the influence on GPU reasoning performance is reduced. And finally, a prediction error condition is processed through mechanisms such as high-priority re-prefetching, and normal model reasoning is ensured.

A Data Storage Device (DSD) includes a first memory storing reference files used to derive vector embeddings in a vector database. A query vector embedding is received from a host and one or more vector embeddings similar to the query vector embedding are identified in the vector database. One or more reference files from which the one or more vector embeddings were derived are identified and stored in a second memory for faster access. In another aspect, a query vector embedding is received by a host that identifies one or more vector embeddings that are similar to the query vector embedding and retrieves the one or more vector embeddings from a DSD to provide to an Artificial Intelligence (AI) model. One or more reference files are identified from which the one or more vector embeddings were derived and are prefetched from the DSD for storage in a host memory.

The invention discloses a cross-host memory sharing method, system and device and a medium, the memory sharing system comprises a master node and a plurality of slave nodes, the memory sharing method is applied to the master node, and the method comprises the steps that a memory application request of a target slave node is received; applying for a target memory in a target memory sharing area according to a memory size requirement corresponding to the request, feeding back memory information of the target memory to a target slave node, so that the target slave node judges a corresponding memory type according to the memory information, and if a process corresponding to the target slave node executes a memory mapping method, sending the target memory to the target slave node. If yes, the target memory can be mapped to the proceeding virtual address space according to the target type and the offset corresponding to the target memory, so that the process can use the distributed target memory. Therefore, it can be guaranteed that when one node writes the memory, other nodes cannot synchronously write the memory, and then the data consistency of the shared memory accessed by multiple hosts is guaranteed.

The invention provides an RDMA (Remote Direct Memory Access) message aggregation processing method and a network card device, which belong to the technical field of data communication, and comprise the following steps: S1, an RDMA message analysis step which comprises the links of hardware-level message capture, BTH header analysis and message classification and metadata generation; s2, an RDMA (Remote Direct Memory Access) caching step, wherein the step comprises a message descriptor recording link and a caching strategy link; s3, an RDMA message aggregation step, wherein the step comprises a context table aggregation step, an aggregation process step, a new aggregation process step and an existing context aggregation process step; s4, a zero-copy aggregation step, wherein the step comprises the links of metadata transmission, DMA delay processing and host memory writing; in order to solve the problem that stateless aggregation processing is not tried to be carried out on RDMA messages when an existing network card processes the RDMA messages, a plurality of received RDMA messages are aggregated into a larger RDMA message, and the larger RDMA message is sent to a downstream module for processing, so that the number of messages processed by a rear-stage module is reduced, and the bandwidth of the network card for processing the RDMA messages is improved.

The invention discloses a large language model semantic query acceleration method based on a sparse KV Cache index. The method comprises a KV Cache semantic pruning strategy based on an attention mechanism and a set of asynchronous pipeline reasoning architecture with overlapped calculation and I / O. According to the method, the attention sparsity characteristic of the large language model in the reasoning stage and the asynchronous transmission capacity between the Host memory and the GPU video memory are fully utilized, calculation redundancy and video memory occupation in repetitive semantic query are greatly reduced, and high-throughput, low-delay and high-performance batch semantic data processing service is provided. Through a mechanism for mapping a static text into a compressed semantic index and combining a prefix cache technology in a reasoning process to realize state multiplexing, high-performance reasoning acceleration and high-efficiency storage compression are provided for a data-intensive semantic analysis task in a resource-constrained environment.

Dynamic provisioning of resources in a datacenter improves the utilization efficiency of compute, memory, storage, and network resources, while maintaining flexibility to meet changing demands. Embodiments herein disclose protocol-aware provisioning of resources over Compute Express Link (CXL) fabrics, enabling multi-protocol pooling and disaggregation of resources, including memory and workload-specific accelerators such as GPUs and DSAs. In some embodiments, a Resource Provisioning Unit (RPU) facilitates intent-based protocol translations and mappings between address spaces, potentially enabling the creation of large-scale compute-memory fabrics that may utilize both coherent and non-coherent Non-Transparent Bridging (NTB) between multiple protocols in a single system, serving as the underlying infrastructure for executing workloads such as Large-Language Models (LLMs) and Deep Learning Recommendation Models (DLRMs). Some embodiments also optimize low-latency communication between processes running on different nodes by enabling host-to-host memory provisioning for libraries such as OpenMP, Pthreads, or CUDA, which can utilize shared memory.

Splitting an address translation cache (ATC) into two portions can reduce costs and maintain efficient retrieval of data. One portion can be disposed in a first location while a second portion can be disposed in a second location distinct from the first location. The first location can be in the controller. The second location can be in a host memory buffer (HMB) or in a memory device separate from the controller. To obtain translated addresses, untranslated addresses can be searched in the first portion and the corresponding translated addresses can be retrieved from the second portion. When invalidating untranslated addresses, the untranslated addresses of the first portion can be deleted without a need to delete corresponding translated addresses in the second portion. To improve ATC storage capacity, grouping of untranslated addresses is possible using most significant bytes (MSBs).

The present disclosure relates to a computing system, a PCI device manager and an initialization method thereof. The PCI device manager is used to realize peer-to-peer data transmission between PCI devices, and the PCI device manager includes: a DMA engine module that realizes data and command communication with a host and a PCI device through DMA; a device memory; a memory mapping module that maps a data cache space of the device memory to a physical address space of the host memory, so as to perform data communication and / or command communication with the PCI device via the DMA engine module in a running process; and a processor that sends a PCI device configuration instruction to the host through the DMA engine module, and creates PCI device objects respectively for PCI devices discovered by the host.

Instead of a system that does not address host buffer fragmentation or saturation, utilize a different metric for command prioritization. Commands are re-ordered and prioritized based on the number of outstanding host buffers that will be released on command completion, thereby limiting and / or reducing the physical address fragmentation and host memory overhead. Command processing priority will take the number of host memory page segments represented by the command into consideration.

The present application relates to the field of data processing, and disclosed are a network card device and a data transmission system, which are used for solving the problem whereby data transmission occupies the access bandwidth of a host memory. The network card device comprises a storage module; a descriptor queue address register; a calculation high-speed link hard core module, which is used for transmitting data sent by a host to a first data storage area, transmitting a descriptor sent by the host to a first descriptor storage area, and transmitting a trigger instruction sent by the host to a network data processing module; and the network data processing module, which is used for acquiring a base address from the descriptor queue address register when receiving the trigger instruction, reading a target descriptor in the first descriptor storage area based on the base address and the trigger instruction, reading target data corresponding to the target descriptor in the first data storage area, and outputting the target data to a network. The present application can reserve the access bandwidth of a host memory for other functions running on the host to use, so that the overall operation efficiency of the host is improved.

The invention discloses a garbage collection optimization method and device, equipment and a medium. The invention relates to the technical field of solid state disks, which comprises the following steps that: a global invalid data bit chart is established in a host memory, each table item in the global invalid data bit chart corresponds to a physical block, and a bitmap bit in each table item corresponds to a physical page in the physical block; bit values corresponding to the bitmap bits comprise a first bit value and a second bit value; responding to a data updating event through an access acceleration engine so as to dynamically update the bit value of the corresponding physical page in the global invalid data bit graph; and when garbage collection is triggered, screening the physical page of which the bit value is the second bit value in a to-be-collected source physical block according to the global invalid data bit chart and the access acceleration engine to perform validity verification and data migration, and skipping the physical page of which the bit value is the first bit value. According to the invention, the garbage recycling efficiency is improved.

Modem datacenters require flexible interconnect solutions that bridge diverse protocol domains while maintaining compatibility with existing infrastructure. Embodiments herein disclose semiconductor devices that implement protocol translations and physical address translations within IC packages conforming to the PCIe Retimer Supplemental Features and Standard BGA Footprint Specification. The devices comprise first and second interfaces communicating according to first and second protocols respectively, with an embedded computer that extracts physical addresses from messages received via the first interface, translates these addresses, and generates messages carrying the translated addresses for transmission via the second interface. This retimer-compatible form factor essentially enables drop-in deployment within existing PCIe and cabling infrastructures while providing protocol bridging and address translation capabilities. The standardized BGA layout provides high-speed differential signaling suitable for low-latency address translation, optionally supporting memory disaggregation, host-to-host memory sharing, accelerator integration, and protocol conversion between CXL, UALink, NVLink, and / or PCIe domains, addressing interoperability challenges.

Embodiments herein describe techniques for a host to provide TX packets to a shared adapter that then uses a NIC to forward the TX packet to a network. In one embodiment, the host creates or fills in one or more TX storage block page entries (SBPE) for the TX packet and stores an index of the TX SBPE in host memory. The host memory can also store an initiative state that tracks whether the shared adapter is currently processing TX packets. If not, the host can issue an interrupt to the shared adapter, along with the index associated with the TX SBPE. The shared adapter can then fetch the TX SBPE from host memory and program the NIC to transmit the TX packet.

The invention discloses a communication transmission method, a product, electronic equipment and a medium, and relates to the technical field of storage. In the method, a host end determines a nonvolatile memory host controller interface specification protocol command according to address information of equipment managed by the host end, and sends the command to the equipment, so that any equipment can obtain address information of other equipment, which is the premise for realizing direct data transmission between the equipment; further, the host end determines an end-to-end data transmission instruction according to the address information of the source device and the address information of the target device, and sends the instruction to the source device. According to the method, under the condition that address information of other devices is stored in any device, end-to-end data transmission instructions of the source device and the target device are added, so that data transmission can be directly carried out between the source device and the target device, namely, host memory transfer is bypassed, the data copying frequency is reduced, and the data transmission efficiency is improved. And the delay and the occupation of host resources are reduced.

The invention discloses a host memory buffer access method, a storage device and a storage medium, and relates to the technical field of memorizers, the method comprises the following steps: monitoring the running state of a bus, and confirming the busy and idle state of the bus, the busy and idle state comprising an idle state and a busy state; if the read-write request is received, acquiring and judging whether to start a transmission process or not according to the busy / idle state; if the bus is in an idle state, starting a transmission process according to the read-write request; and obtaining a to-be-transmitted data packet corresponding to the read-write request, and sending the to-be-transmitted data packet to a bus for transmission. According to the method and the device, the transmission time point of the data packet corresponding to the access request of the hmb can be dynamically adjusted according to the busy / idle state of the bus, so that rapid data transmission can be ensured, and transmission delay caused by data packet blockage is avoided.

A data processing method, a network interface card, an electronic device, and a storage medium. The method includes: determining a target dispatch queue to be dispatched from a host memory in response to a doorbell signal sent by a target host, where the doorbell signal indicates that there is a target message to be sent; determining a current target dispatch state of the target dispatch queue, where the target dispatch state is obtained based on an activity and a credit value corresponding to the target work queue element; determining whether a dispatch mechanism corresponding to the target dispatch queue is valid based on the target dispatch state, where the dispatch mechanism indicates whether the target dispatch queue is allowed to be dispatched; and performing a dispatch operation on the target work queue element and the target message in response to the dispatch mechanism being valid.

A storage device may simplify an ungraceful shutdown recovery process and reduce a time-to-ready (TTR) value associated with an ungraceful shutdown bootup sequence. The storage device may include a cache to store data structures associated with host data and meta data. The storage device may also include a controller to store the data structures in a host memory buffer. After an ungraceful shutdown, the controller may execute a bootup sequence and access the host memory buffer during the bootup sequence. The controller may use the data structures stored in the host memory buffer to recover the host data and meta data. The controller applies the host data and meta data to the bootup sequence to simplify the ungraceful shutdown recovery process and reduce the TTR value associated with the ungraceful shutdown bootup sequence.

A module identifier and a request address associated with an access request are received at a host memory buffer (HMB) access module in a solid-state drive (SSD) system. A translated address is determined based at least in part on the module identifier and the request address, including by accessing at least one translation table that stores address mappings between (1) a plurality of processors in the SSD system and (2) a host memory buffer; each processor in the plurality of processors has a non-overlapping memory space in the host memory buffer. The access request is performed at a host interface, including by accessing the host memory buffer using the translated address, wherein the host memory buffer is located in host memory that is external to the SSD system.

Secure and efficient access to host memory for guests is disclosed. For example, a system includes (i) a processor, (ii) a host memory, (iii) a hypervisor including a filesystem daemon, (iv) a guest including a storage controller, and (v) a first filesystem queue. The storage controller is configured to receive a file request, translate the file request, and add the translated file request to a filesystem queue. The filesystem daemon is configured to retrieve the translated file request from the filesystem queue and provide access to a file in the host memory identified in the file request to the guest.

A system receives, by a network interface card (NIC), inputs including an instruction to read or write a payload of a message, a tracker state indicating a round of processing, and a datatype descriptor defining organization of the message payload. The system identifies a current context and a processing state for the instruction. If the datatype descriptor indicates a first type, the system: obtains the current context associated with the first type from a host memory or a cache of the NIC; and creates direct memory access (DMA) instructions corresponding to the received instruction by executing operations in a nested loop. If the datatype descriptor indicates a second type, the system: obtains the current context associated with the second type by fetching vector entries from a buffer of the NIC; and creates the DMA instructions corresponding to the received instruction based on addresses and lengths in the vector entries.

Distributed fiber optic sensing (DFOS) / distributed acoustic sensing (DAS) that illustrate inventive techniques—applicable to all embedded systems—that deliver high-bandwidth traffic from a DFOS system to a cloud or other processing resources in real-time, employ firmware that operates at a register-transfer level and writes data repeatedly into the embedded host memory directly—without software intervention after initial configuration. This technique advantageously enables the use of the relatively large host memory to compensate for any software jitter. Preferably configured, the firmware employs only a small buffer to compensate for a transaction and host-memory arbitration latency. A second dedicated thread sends out data from the user space buffer to a cloud, or a connected server. Remote procedures that are executed on a remote system (a computer, network of computers, or cloud), receive data transmitted from the embedded system, and perform further processing as necessary.

A memory sub-system controller includes a host queue interface circuit to interact with submission queues of a host system, an address translation circuit to handle address translation requests sent to the host system from the host queue interface circuit, and a cache to store address translations associated with the address translation requests. The host queue interface circuit pauses command fetch arbitration on a submission queue of the host system that is targeted by an address translation request that has missed at the cache and causes a page request interface (PRI) handler to send a page miss request to a translation agent of the host system. The host queue interface circuit further receives a restart message from the PRI handler upon the PRI handler receiving a page miss response from the translation agent and restarts command arbitration on the submission queue.

A device includes an address translation circuit of host interface circuitry to handle address translation requests to a host system from a host queue interface circuit. The address translation circuit includes cache to store address translations associated with the address translation requests. The host queue interface circuit, coupled to the address translation circuit, is to: pause command fetch arbitration on a submission queue of the host system that is targeted by an address translation request that missed at the cache; trigger a page request interface (PRI) handler to send a page miss request to a translation agent (TA) of the host, the page miss request including a virtual address of the address translation request; receive a restart message from the PRI handler upon the PRI handler receiving a page miss response from the TA; and restart command arbitration on the submission queue that was paused responsive to the restart message.

Apparatuses, systems, and techniques of using one or more circuits (e.g., of a network interface) to obtain assembly code for one or more machine code segments loaded and / or injected into a process, and determine whether the assembly code is likely to perform at least one unauthorized task.

A bridge receives a first memory access command from a host computer, the first memory access command including an indication of one or more blocks of memory locations in a host memory of the host computer. The bridge device stores the first memory access command in a queue of the bridge device and determines one or more virtual addresses to be used by the solid state memory for the first memory access command. The bridge generates a second memory access command that is a revised copy of the first memory access command so that the indication of the one or more blocks of memory locations in the host memory is replaced with an indication of the one or more virtual addresses. The bridge sends the second memory access command to the solid state memory while keeping the first memory access command in the queue.

The invention discloses a data transmission method and device, a medium and a computer program product, and relates to the computer technology. A memory area hardware entry memory table and a memory conversion area table of a host memory are stored in a first device local; when a first device executes a data transmission task of remote direct memory access of a host to carry out data migration between a host memory and a local memory, the first device does not need to look up a table in the host to determine a physical address of a target memory, but identifies an identifier of the target memory and a target virtual address of the target memory; the memory conversion area table is queried according to the target virtual address to obtain the target physical address, and at the moment, the first bus needs to be called to access the host memory so as to execute data migration between the host memory and the local memory of the first device, so that the communication bottleneck in the current remote direct memory access process can be solved; the technical effect of improving the communication efficiency of remote direct memory access is achieved.

Modern datacenters require flexible interconnect solutions that bridge diverse protocol domains while maintaining compatibility with existing infrastructure. Embodiments herein disclose semiconductor devices that implement protocol translations and physical address translations within IC packages conforming to the PCIe Retimer Supplemental Features and Standard BGA Footprint Specification. The devices comprise first and second interfaces communicating according to first and second protocols respectively, with an embedded computer that extracts physical addresses from messages received via the first interface, translates these addresses, and generates messages carrying the translated addresses for transmission via the second interface. This retimer-compatible form factor essentially enables drop-in deployment within existing PCIe and cabling infrastructures while providing protocol bridging and address translation capabilities. The standardized BGA layout provides high-speed differential signaling suitable for low-latency address translation, optionally supporting memory disaggregation, host-to-host memory sharing, accelerator integration, and protocol conversion between CXL, UALink, NVLink, and / or PCIe domains, addressing interoperability challenges.

The embodiment of the invention provides a memory deletion method and device, equipment and a medium, and relates to the field of virtual machine management. The method comprises the following steps: in response to a deletion instruction for a virtual machine deployed on a host, creating a memory snapshot of the host, then deleting the virtual machine deployed on the host, scanning the memory of the host, comparing a scanning result with the memory snapshot, identifying residual data of the virtual machine, and sending the residual data to the host. And the residual data is cleared, and finally the memory state of the memory page related to the virtual machine is recovered to an available state, so that the host distributes the memory page related to the virtual machine to other virtual machines. Through the mode, the problem that residual data exists after the virtual machine is deleted is efficiently and thoroughly solved, more effective storage space is released, the safety of the system can be improved, user information is prevented from being leaked by the residual data after deletion, effective cyclic utilization of resources is facilitated, and the use experience of a user is improved.

Techniques for providing secure cross-host memory sharing are described herein. A memory buffer device having processing circuitry is to receive a first request from a first initiator to share a region of memory associated with the memory buffer device with a second initiator. The processing circuitry may identify a first passcode associated with the first initiator. The processing circuitry may receive a second request from the second initiator to access the region of memory. The second request includes a second passcode. The processing circuitry may authenticate the second request using the first passcode and the second passcode. Responsive to authentication of the second request, the processing circuitry may generate a mapping between a host physical address space associated with the second initiator and a physical memory address space associated with the region of memory to enable the second initiator to access the region of memory.

The invention relates to the technical field of key value databases, and discloses a key value database near-storage computing system based on computing storage equipment, the system comprises a host, a memory and a computing memory, a decoder in the computing memory is used for reading files needing to be merged from the memory, analyzing all key value pairs from the files, and storing the key value pairs in the computing memory; placing the key value pairs in a buffer area connected with a sequencer; the sequencer is used for comparing the size relationship of the key value pairs, taking out the key value pair from the corresponding decoder buffer area according to the comparison result of each time, selecting the key value pair with the minimum key value according to the size of the key in the key value pair, and transmitting the corresponding key value pair to the buffer area connected with the encoder; and the encoder is used for extracting the key value pair from the buffer area connected with the sequencer, generating an SS-Table file according to a file encoding rule and storing the SS-Table file into the memory. Deep fusion of calculation and storage can be realized, calculation resources in a host are fully utilized, and the overall system performance is improved.