520 results about "Host memory" patented technology

The invention discloses a large model reasoning method for a user request and a server, and relates to the technical field of large model application, and the method comprises the steps: inputting an obtained request text and a target statement into a preset language model for reasoning processing, if it is judged that the request text can be matched with the historical prefix key-value cache in the host memory, determining the historical prefix key-value cache as a preliminary pre-filling reasoning result; performing pre-filling reasoning processing according to the request text, the preliminary pre-filling reasoning result and the target statement to obtain a pre-filling reasoning result; performing recursive reasoning processing according to the pre-filling reasoning result to obtain a recursive reasoning result; the recursive reasoning result is output to the user side, the historical prefix key-value cache is stored through the host memory, and the corresponding historical prefix key-value cache does not need to be reasoned repeatedly, so that the reasoning efficiency of the request text is improved, and the user experience is improved.

The embodiment of the invention provides an NVMe write IO initiating circuit of an NVMe controller, and relates to the technical field of storage. The write initiation circuit includes a DMA command generation circuit coupled with the DMA transfer circuit, generating, in response to receiving addressing information, a DMA command indicative of moving data from the host memory to the storage device cache based on the addressing information, and providing the DMA command to the DMA transfer circuit; and the storage command generation circuit is coupled with the DMA transmission circuit and the storage command processing unit, responds to a first message which is sent by the DMA transmission circuit and indicates that the DMA command is processed completely, generates a storage command which indicates that the data is migrated from the storage device to the flash memory of the storage device, and provides the storage command to the storage command processing unit. The write-in initiating circuit is a hardware circuit independent of the CPU and unloads a task for generating the DMA command from the CPU, so that the burden of the CPU is reduced.

The invention provides an LDPC code parallel GPU decoding method and system, and relates to the technical field of communication, and the method comprises the steps: reading an LDPC code through a CPU; distributing the check matrix parameters to a constant memory space of the GPU, and initializing the constant memory space; allocating a global memory space for the channel information, the decoding hard decision result and the C2V through the GPU; quantizing the channel information into signed character type data; setting a GPU thread block based on the sub-block dimension of the check matrix; performing layered updating on the LDPC code through floating point operation by using intra-frame parallelism; storing the layered information into a shared memory space of the GPU, and initializing the layered information through channel information; iteratively updating the initialized hierarchical information to generate a decoding result of the to-be-transmitted data; transmitting a decoding result to a host memory space of the CPU; and after code word decoding is completed, resource release is carried out.

Apparatus for data communication includes a network interface for connection to a packet data network and a host interface for connection to a host computer, which includes a central processing unit (CPU) and a host memory. Packet processing circuitry receives, via the host interface, from a kernel running on the CPU, associations between multiple remote direct memory access (RDMA) sessions and multiple different User Datagram Protocol (UDP) 5-tuple, which are assigned respectively to the RDMA sessions, and receives from an application running on the CPU a request to send an RDMA message, using a selected group of one or more of the RDMA sessions, to a peer application over the packet data network, and in response to the request, transmits, via the network interface, one or more data packets using a UDP 5-tuple that is assigned to one of the RDMA sessions in the selected group.

The invention provides an expert access prediction method and system suitable for a hybrid expert architecture large language model, and the method comprehensively considers the model structure and hardware feature information, constructs an expert access predictor based on a neural network for each layer, and carries out the training through the historical data. And experts needing to be activated in the model reasoning process can be predicted in advance. In the process, the predictor executes prediction on the CPU in parallel, and the prefetcher loads expert parameters from a host memory to a GPU video memory in advance according to a prediction result. By adjusting the prediction distance, the prediction accuracy and the prefetching advance are balanced, and the prefetching number can be dynamically adjusted according to the prediction accuracy. Meanwhile, parallel loading is adopted, system resources are utilized to the maximum extent, and the influence on GPU reasoning performance is reduced. And finally, a prediction error condition is processed through mechanisms such as high-priority re-prefetching, and normal model reasoning is ensured.

A Data Storage Device (DSD) includes a first memory storing reference files used to derive vector embeddings in a vector database. A query vector embedding is received from a host and one or more vector embeddings similar to the query vector embedding are identified in the vector database. One or more reference files from which the one or more vector embeddings were derived are identified and stored in a second memory for faster access. In another aspect, a query vector embedding is received by a host that identifies one or more vector embeddings that are similar to the query vector embedding and retrieves the one or more vector embeddings from a DSD to provide to an Artificial Intelligence (AI) model. One or more reference files are identified from which the one or more vector embeddings were derived and are prefetched from the DSD for storage in a host memory.

The invention discloses a cross-host memory sharing method, system and device and a medium, the memory sharing system comprises a master node and a plurality of slave nodes, the memory sharing method is applied to the master node, and the method comprises the steps that a memory application request of a target slave node is received; applying for a target memory in a target memory sharing area according to a memory size requirement corresponding to the request, feeding back memory information of the target memory to a target slave node, so that the target slave node judges a corresponding memory type according to the memory information, and if a process corresponding to the target slave node executes a memory mapping method, sending the target memory to the target slave node. If yes, the target memory can be mapped to the proceeding virtual address space according to the target type and the offset corresponding to the target memory, so that the process can use the distributed target memory. Therefore, it can be guaranteed that when one node writes the memory, other nodes cannot synchronously write the memory, and then the data consistency of the shared memory accessed by multiple hosts is guaranteed.

A system and method for real-time monitoring of personal protective equipment (PPE) compliance at a worksite. The system includes a processor, a host memory, a graphical processing unit (GPU), a GPU memory, a storage device, an input device, a plurality of cameras, and a display device. The system creates multiple inference engines and loads input frames from the storage device for pre-processing. The pre-processed frames are processed to obtain annotations and generate an output file. The system also includes a preparatory area and an operations area with cameras configured to capture and transmit video frames. The system can process input frames at a rate of at least 15 frames per second, providing real-time PPE compliance information.

The invention provides an RDMA (Remote Direct Memory Access) message aggregation processing method and a network card device, which belong to the technical field of data communication, and comprise the following steps: S1, an RDMA message analysis step which comprises the links of hardware-level message capture, BTH header analysis and message classification and metadata generation; s2, an RDMA (Remote Direct Memory Access) caching step, wherein the step comprises a message descriptor recording link and a caching strategy link; s3, an RDMA message aggregation step, wherein the step comprises a context table aggregation step, an aggregation process step, a new aggregation process step and an existing context aggregation process step; s4, a zero-copy aggregation step, wherein the step comprises the links of metadata transmission, DMA delay processing and host memory writing; in order to solve the problem that stateless aggregation processing is not tried to be carried out on RDMA messages when an existing network card processes the RDMA messages, a plurality of received RDMA messages are aggregated into a larger RDMA message, and the larger RDMA message is sent to a downstream module for processing, so that the number of messages processed by a rear-stage module is reduced, and the bandwidth of the network card for processing the RDMA messages is improved.

The invention relates to a request device and a response device based on RDMA (Remote Direct Memory Access). The request device comprises a sending processor and a receiving processor; the sending processor is used for receiving a work queue element issued by a driving program, and if the work queue element is an RDMA read operation, the RDMA read operation is converted to obtain a request side write request; the sending processor is also used for sending the request side write request to the response device; and the receiving processor is used for receiving and analyzing a response side write request sent by the response device to obtain target data and a destination address corresponding to the RDMA read operation, and writing the target data into a storage space corresponding to the destination address. Through the application, the QPC data structure domain segment and the design difficulty are reduced, the message disorder processing complexity is reduced, and the consumption of a large amount of host memory and cache resources is avoided.

The invention discloses a large language model semantic query acceleration method based on a sparse KV Cache index. The method comprises a KV Cache semantic pruning strategy based on an attention mechanism and a set of asynchronous pipeline reasoning architecture with overlapped calculation and I / O. According to the method, the attention sparsity characteristic of the large language model in the reasoning stage and the asynchronous transmission capacity between the Host memory and the GPU video memory are fully utilized, calculation redundancy and video memory occupation in repetitive semantic query are greatly reduced, and high-throughput, low-delay and high-performance batch semantic data processing service is provided. Through a mechanism for mapping a static text into a compressed semantic index and combining a prefix cache technology in a reasoning process to realize state multiplexing, high-performance reasoning acceleration and high-efficiency storage compression are provided for a data-intensive semantic analysis task in a resource-constrained environment.

Dynamic provisioning of resources in a datacenter improves the utilization efficiency of compute, memory, storage, and network resources, while maintaining flexibility to meet changing demands. Embodiments herein disclose protocol-aware provisioning of resources over Compute Express Link (CXL) fabrics, enabling multi-protocol pooling and disaggregation of resources, including memory and workload-specific accelerators such as GPUs and DSAs. In some embodiments, a Resource Provisioning Unit (RPU) facilitates intent-based protocol translations and mappings between address spaces, potentially enabling the creation of large-scale compute-memory fabrics that may utilize both coherent and non-coherent Non-Transparent Bridging (NTB) between multiple protocols in a single system, serving as the underlying infrastructure for executing workloads such as Large-Language Models (LLMs) and Deep Learning Recommendation Models (DLRMs). Some embodiments also optimize low-latency communication between processes running on different nodes by enabling host-to-host memory provisioning for libraries such as OpenMP, Pthreads, or CUDA, which can utilize shared memory.

The invention provides a Turbo product code-oriented decoding method and system, and relates to the technical field of communication, and the method comprises the steps: initializing a Turbo product code decoder; obtaining channel information to be decoded; quantizing the channel information into signed character type data; transmitting the quantized channel information to a GPU (Graphics Processing Unit) in an asynchronous transmission mode; respectively updating the channel information according to rows and columns through a Chase algorithm by utilizing an intra-frame parallelism degree consistent with the size of a GPU thread block until the maximum number of iterations is reached, and finishing the updating of the soft output information; performing hard decision according to the channel information and the updated soft output information; transmitting the hard decision result to a host memory of the CPU in an asynchronous transmission mode; and after decoding of all code words is completed, resource release is carried out. According to the invention, the decoding process with low delay, high throughput rate and high reliability can be realized, and the resource utilization rate and the decoding efficiency are improved.

Splitting an address translation cache (ATC) into two portions can reduce costs and maintain efficient retrieval of data. One portion can be disposed in a first location while a second portion can be disposed in a second location distinct from the first location. The first location can be in the controller. The second location can be in a host memory buffer (HMB) or in a memory device separate from the controller. To obtain translated addresses, untranslated addresses can be searched in the first portion and the corresponding translated addresses can be retrieved from the second portion. When invalidating untranslated addresses, the untranslated addresses of the first portion can be deleted without a need to delete corresponding translated addresses in the second portion. To improve ATC storage capacity, grouping of untranslated addresses is possible using most significant bytes (MSBs).

The invention discloses a DMA data transmission method and device, and relates to the technical field of computers. The method comprises the steps of obtaining a DMA descriptor from a memory of a first PCIe device; reading the data to be transmitted from the memory of the first PCIe device based on the DMA descriptor; and transmitting the to-be-transmitted data to a target address corresponding to the DMA descriptor through the PCIe message so as to realize data transmission from the first PCIe device to the second PCIe device. According to the data transmission method and device, the DMA descriptor is stored in the memory of the first PCIe device, a PCIe message does not need to be sent to the host to obtain / access the DMA descriptor stored in the memory of the host, the bandwidth from the PCIe devices to the host is not occupied, 100% data transmission from the PCIe devices to the PCIe devices is achieved, and therefore the performance of the whole system is improved.

The present disclosure relates to a computing system, a PCI device manager and an initialization method thereof. The PCI device manager is used to realize peer-to-peer data transmission between PCI devices, and the PCI device manager includes: a DMA engine module that realizes data and command communication with a host and a PCI device through DMA; a device memory; a memory mapping module that maps a data cache space of the device memory to a physical address space of the host memory, so as to perform data communication and / or command communication with the PCI device via the DMA engine module in a running process; and a processor that sends a PCI device configuration instruction to the host through the DMA engine module, and creates PCI device objects respectively for PCI devices discovered by the host.

Instead of a system that does not address host buffer fragmentation or saturation, utilize a different metric for command prioritization. Commands are re-ordered and prioritized based on the number of outstanding host buffers that will be released on command completion, thereby limiting and / or reducing the physical address fragmentation and host memory overhead. Command processing priority will take the number of host memory page segments represented by the command into consideration.

The invention relates to the technical field of computer networks, in particular to an implementation method and system for an RDMA network card request queue, and the method comprises the steps: checking whether a residual space exists in an RDMA Cache module or not when a write request is received; if yes, writing the writing information of the writing request into an RDMA Cache module, updating a free bit, and recording a queue pair number and an unfinished pointer; if not, writing the writing information of the writing request into a host memory module; when a read request is received, checking whether the queue pair number and the unfinished pointer carried by the read request are matched with historical records in the RDMA Cache module or not; if yes, reading from the RDMA Cache module; and if not, reading from the host memory module. According to the invention, multiple QPs in the RDMA network card share the internal resources of the Cache, the resource consumption is reduced, and the utilization rate of the cache is improved.

A storage system includes one or more data storage devices, a PCIe switch coupled to the one or more data storage devices, and a controller unit coupled to the PCIe switch. The one or more data storage devices are DRAM-less. The controller unit includes a dynamic random access memory (DRAM) host memory buffer (HMB) controller and a DRAM pool or a controller memory buffer (CMB) controller, a root complex / port, and the DRAM pool. The DRAM pool includes one or more DRAM devices. The one or more data storage devices are configured to interact with the controller unit and store data to a DRAM of the DRAM pool of the controller unit.

A method and device for data access, electronic apparatus and storage medium are provided. The method for data access includes: writing a key of a first key-value pair to a host memory in response to a write request to write the first key-value pair to the memory database; determining whether to write a value of the first key-value pair to the host memory or a Computing Express Link (CXL) memory based on a length of the value of the first key-value pair; and writing the value of the first key-value pair to the host memory or the CXL memory, based on a result of the determination.

The present application relates to the field of data processing, and disclosed are a network card device and a data transmission system, which are used for solving the problem whereby data transmission occupies the access bandwidth of a host memory. The network card device comprises a storage module; a descriptor queue address register; a calculation high-speed link hard core module, which is used for transmitting data sent by a host to a first data storage area, transmitting a descriptor sent by the host to a first descriptor storage area, and transmitting a trigger instruction sent by the host to a network data processing module; and the network data processing module, which is used for acquiring a base address from the descriptor queue address register when receiving the trigger instruction, reading a target descriptor in the first descriptor storage area based on the base address and the trigger instruction, reading target data corresponding to the target descriptor in the first data storage area, and outputting the target data to a network. The present application can reserve the access bandwidth of a host memory for other functions running on the host to use, so that the overall operation efficiency of the host is improved.

The invention discloses a garbage collection optimization method and device, equipment and a medium. The invention relates to the technical field of solid state disks, which comprises the following steps that: a global invalid data bit chart is established in a host memory, each table item in the global invalid data bit chart corresponds to a physical block, and a bitmap bit in each table item corresponds to a physical page in the physical block; bit values corresponding to the bitmap bits comprise a first bit value and a second bit value; responding to a data updating event through an access acceleration engine so as to dynamically update the bit value of the corresponding physical page in the global invalid data bit graph; and when garbage collection is triggered, screening the physical page of which the bit value is the second bit value in a to-be-collected source physical block according to the global invalid data bit chart and the access acceleration engine to perform validity verification and data migration, and skipping the physical page of which the bit value is the first bit value. According to the invention, the garbage recycling efficiency is improved.

Modem datacenters require flexible interconnect solutions that bridge diverse protocol domains while maintaining compatibility with existing infrastructure. Embodiments herein disclose semiconductor devices that implement protocol translations and physical address translations within IC packages conforming to the PCIe Retimer Supplemental Features and Standard BGA Footprint Specification. The devices comprise first and second interfaces communicating according to first and second protocols respectively, with an embedded computer that extracts physical addresses from messages received via the first interface, translates these addresses, and generates messages carrying the translated addresses for transmission via the second interface. This retimer-compatible form factor essentially enables drop-in deployment within existing PCIe and cabling infrastructures while providing protocol bridging and address translation capabilities. The standardized BGA layout provides high-speed differential signaling suitable for low-latency address translation, optionally supporting memory disaggregation, host-to-host memory sharing, accelerator integration, and protocol conversion between CXL, UALink, NVLink, and / or PCIe domains, addressing interoperability challenges.

A storage device may read a plurality of data units from a host memory buffer prior to entering a low power mode, release a communication link with the host after reading the plurality of data unit, execute a background operation based on at least one of the plurality of data units, and enter into the low power mode after executing the background operation.

The invention relates to the technical field of computers and provides a flow table deployment method, computer equipment and a medium. According to the flow table deployment method, three different types of storage resources, namely a hardware memory, a host memory and a host cache, are utilized, a proper flow table deployment mode is selected according to the specification of the flow table, the storage resources of the hardware memory are fully utilized, and the forwarding performance of the flow table is improved; the flow table index of the host memory and the flow table index of the hardware memory are globally addressed, so that the flow table can be preferentially unloaded into the hardware memory, and the flow table is unloaded into the host memory after items of the hardware memory are full of specifications; a corresponding flow table item adding mechanism is designed, the highest priority of a hardware memory is kept, and the flow table items in a host cache are updated in time.

Embodiments herein describe techniques for a host to provide TX packets to a shared adapter that then uses a NIC to forward the TX packet to a network. In one embodiment, the host creates or fills in one or more TX storage block page entries (SBPE) for the TX packet and stores an index of the TX SBPE in host memory. The host memory can also store an initiative state that tracks whether the shared adapter is currently processing TX packets. If not, the host can issue an interrupt to the shared adapter, along with the index associated with the TX SBPE. The shared adapter can then fetch the TX SBPE from host memory and program the NIC to transmit the TX packet.

The embodiment of the invention relates to the field of computers, in particular to a construction method and system of an annular cache region, a storage medium and electronic device.The construction method of the annular cache region comprises the steps that whether an RAID card supports the host memory buffering characteristic or not is determined; under the condition of determining that the RAID card supports the host memory buffer characteristic, allocating a memory space for the RAID card; and sending an enabling instruction to the RAID card, so that the RAID card constructs an annular cache region in the memory space according to the enabling instruction and the host memory buffer characteristic, and the annular cache region is used for storing data written into the host by the RAID card. Through the embodiment of the invention, the problem of insufficient data storage capacity caused by limited hardware resources for running of RAID card firmware in related technologies can be solved.

The invention discloses a communication transmission method, a product, electronic equipment and a medium, and relates to the technical field of storage. In the method, a host end determines a nonvolatile memory host controller interface specification protocol command according to address information of equipment managed by the host end, and sends the command to the equipment, so that any equipment can obtain address information of other equipment, which is the premise for realizing direct data transmission between the equipment; further, the host end determines an end-to-end data transmission instruction according to the address information of the source device and the address information of the target device, and sends the instruction to the source device. According to the method, under the condition that address information of other devices is stored in any device, end-to-end data transmission instructions of the source device and the target device are added, so that data transmission can be directly carried out between the source device and the target device, namely, host memory transfer is bypassed, the data copying frequency is reduced, and the data transmission efficiency is improved. And the delay and the occupation of host resources are reduced.

The invention discloses a host memory buffer access method, a storage device and a storage medium, and relates to the technical field of memorizers, the method comprises the following steps: monitoring the running state of a bus, and confirming the busy and idle state of the bus, the busy and idle state comprising an idle state and a busy state; if the read-write request is received, acquiring and judging whether to start a transmission process or not according to the busy / idle state; if the bus is in an idle state, starting a transmission process according to the read-write request; and obtaining a to-be-transmitted data packet corresponding to the read-write request, and sending the to-be-transmitted data packet to a bus for transmission. According to the method and the device, the transmission time point of the data packet corresponding to the access request of the hmb can be dynamically adjusted according to the busy / idle state of the bus, so that rapid data transmission can be ensured, and transmission delay caused by data packet blockage is avoided.

A data processing method, a network interface card, an electronic device, and a storage medium. The method includes: determining a target dispatch queue to be dispatched from a host memory in response to a doorbell signal sent by a target host, where the doorbell signal indicates that there is a target message to be sent; determining a current target dispatch state of the target dispatch queue, where the target dispatch state is obtained based on an activity and a credit value corresponding to the target work queue element; determining whether a dispatch mechanism corresponding to the target dispatch queue is valid based on the target dispatch state, where the dispatch mechanism indicates whether the target dispatch queue is allowed to be dispatched; and performing a dispatch operation on the target work queue element and the target message in response to the dispatch mechanism being valid.