50 results about "Host memory" patented technology

A computer system executes a machine learning model having multiple layers, and includes host and work accelerator processors. A window size representing a number of layers to be loaded into accelerator memory is determined. Model data associated with a subset of layers the same size as the window is loaded to the accelerator. The model is iteratively executed by processing a current layer to provide output data and storing this in accelerator memory, offloading the output data to host memory, replacing model data for an already processed layer with model data corresponding to a next layer, and moving to a next layer. The window size is updated during execution based on a transfer duration for loading into accelerator memory, a transfer duration for loading into host memory, or a processing duration of a layer.

The application discloses a kind of application layer bucket level local host memory weight cache management method, equipment and medium, it is related to big language model inference field, method includes: obtaining the bucket level cache index table established by application layer, wherein, bucket level cache index record at least one bucket corresponding to big language model in local cache state, bucket is formed by the independent unit of all weight tensor of big language model according to preset bucketing rule division;The cached bucket is fixed in local host memory by memory locking system call, can solve the defect that operating system autonomously expels weight data in existing scheme, application layer cannot intervene;And when target big language model is loaded, based on bucket level cache index table, cache state is inquired bucket by bucket, and the cached bucket is loaded from local host memory, the expelled bucket is supplemented from other remote storage source, can accurately identify missing weight and only supplement from remote storage source, without full load, finally reduce cold start time consumption.

In one embodiment, a peripheral device includes an oscillator, a counter to be driven by the oscillator and provide a peripheral device counter value, and processing circuitry to receive a host device counter value from a host device, read host device clock translation parameters from a host memory of the host device, the host device clock translation parameters providing translation between the host device counter value and a host device clock time, read peripheral device clock translation parameters providing a translation between the peripheral device counter value and a peripheral device clock time, read the peripheral device counter value, compute a clock correction as a function of a difference between the host device clock time and the peripheral clock time, based on the host device and peripheral device counter values and clock translation parameters, and correct the host device or peripheral device clock translation parameters based on the clock correction.

This invention provides a data storage device and a data reading method. The data storage device is coupled to a host computer, which includes a host memory buffer. The data storage device includes a controller, non-volatile memory, and a buffer. The data reading method includes: downloading firmware data to be updated from the data storage device via the host computer; determining that the host memory buffer contains the firmware data to be updated; reading the firmware data to be updated from the host memory buffer into the buffer; determining that the firmware data to be updated in the buffer is correct through an error correction mechanism; and updating the firmware data of the data storage device. This reduces read / write operations on the data storage device and improves stability, while also increasing data transmission speed without affecting read / write operations.

An exemplary procedure involves a host operating system (host OS) receiving an initial request to allocate memory pages in virtual host memory. In response, the host OS allocates a first guest memory region of the virtual host memory to a virtual machine by at least pinning the first guest memory region. The procedure further involves the host OS receiving a second and a third request to allocate memory pages. In response, the host OS allocates a second and a third guest memory region to the virtual machine. The second guest memory region overlaps a first subregion of the first guest memory region, and the third guest memory region overlaps a second subregion of the first guest memory region.The procedure further involves unpinning, by the host OS, a third subregion of the first guest storage region, with the third subregion not overlapping either the first or the second subregion.

This application provides a data transmission method, a network interface card (NIC) for a data receiving end, an electronic device, and a storage medium. The data transmission method is applied to the NIC of the data receiving end. The NIC writes data to the host memory of the data receiving end via a bus. The data transmission method includes: if the bus bandwidth resources required for the NIC to write the target data to the host memory exceed the bus bandwidth resources available to the NIC, then writing a portion of the target data to the NIC's NIC buffer; the NIC buffer includes a first memory buffer configured for the NIC based on its on-chip memory and a second memory buffer configured for the NIC based on its onboard memory; the data sending end sends the target data to the data receiving end via remote direct memory access; and a portion of the data is written from the NIC buffer to the host memory. The solution of this application can better avoid data packet loss problems when facing sudden traffic surges.

The application discloses a stage-decoupling-based large language model online inference service delay optimization method and system, belongs to the technical field of artificial intelligence, and solves the large language model online inference service delay problem under the premise of ensuring model accuracy. The method comprises the following steps: a pre-padding scheduling is an adaptive pre-padding task scheduling strategy, the batch processing scale is dynamically adjusted according to the current load condition, the input prompt length and the GPU resource occupation; a dynamic batch size mechanism adaptively adjusts the batch processing scale according to the length distribution of input requests, dynamically combines and allocates tasks; a decoding scheduling is a decoding scheduling algorithm based on a multi-level feedback queue (MLFQ), and the task priority and the time slice allocation are dynamically adjusted; and a KV cache management mechanism is used for automatically exchanging the KV cache to the host memory when the task is idle or waiting for scheduling, and the GPU is used for pre-fetching back when the task is about to be executed. The application is suitable for performance improvement of a large language model for inference service.

This application discloses a buffer-based NAND programming control method, device, and storage medium. Relating to the field of data technology, the buffer-based NAND programming control method includes: if the current programming page is a secondary programming page determined by preset programming requirements, generating backup data of the page data of the current programming page from the main controller's internal cache; allocating buffer units for the backup data in the host memory buffer; storing the backup data in the buffer units, and associating the storage address of the buffer units with the programming address or host logical address of the page data of the current programming page as a mapping record. This application can achieve the technical effect of expanding the compatibility range of storage devices with different NAND Flash chips.

The application provides a RDMA packet level multi-path distribution method and system based on PSN granularity adjustment, which is executed by a sending end and makes the network side forward based on an equal multi-path routing mechanism, and comprises the following steps: generating a work queue element in response to a work request submitted by an upper layer application, copying a to-be-sent message from a host memory to a network card memory by an RDMA network card; segmenting the to-be-sent message, allocating a packet sequence number, a message sequence number and an intra-segment sequence number to each data segment; determining a path identifier according to the packet sequence number, a preset granularity parameter and the number of available equal paths, so that data segments with a continuous number equal to the preset granularity parameter correspond to the same path identifier; encoding the path identifier to a UDP port field, and writing the message sequence number and the intra-segment sequence number into an optional extended transmission header. The application takes the packet sequence number as a scheduling index, adjusts the path switching frequency through the granularity parameter, and can realize the adjustable trade-off between the load balancing benefit and the out-of-order overhead without modifying the network side device.

This invention relates to the field of data processing technology, and in particular to a data request processing method, apparatus, device, and medium. The method includes: adopting a scheme of separating cached metadata and cached data; using a redundant independent disk array card to uniformly manage read cached metadata; and utilizing a large-capacity host memory as a cached data storage area to decouple cached metadata and cached data. When a user's data processing request hits a cache line node in the cache area of ​​the redundant independent disk array card, the redundant independent disk array card sends the cached data address corresponding to the hit target cache line node to the host. The host processes the cached data corresponding to the cached data address on the host. Data processing can be performed directly from the host memory. By using the host memory as a cache pool, the cache capacity is greatly increased, thereby improving the data hit rate and data access speed.

This application discloses a flow table unloading method, system, apparatus, and readable storage medium, relating to the field of computer networks. In this scheme, the template type of the target flow table is identified; the target field is determined based on the identified template type and a preset template type-field correspondence; the target flow table is completed using the target field to obtain a precise target flow table; upon receiving unloading information, the precise target flow table is unloaded into the DDR memory of the target hardware. This application can convert a masked target flow table into a precise target flow table through field completion, thereby enabling the unloading of the target flow table into DDR memory, which does not support masks. Compared to tcam memory and / or host memory, DDR memory has lower cost and power consumption.

A computing system having non-volatile memory cells configured to provide a storage space accessible via logical block addressing addresses. A processing of the computing system is configured to: determine workload statistics of storage access commands configured to access the storage space; predict, using a machine learning model, a first performance level of processing storage access commands having the workload statistics using a host memory buffer allocated according to first allocation parameters; predict, using the machine learning model, a second performance level of processing the storage access commands having the workload statistics using a host memory buffer allocated according to second allocation parameters; and decide, based on the first performance level and the second performance level, to allocate a host memory buffer according to third allocation parameters.

The present invention relates to a memory management apparatus and a memory management method. The memory management method according to one embodiment may comprise steps in which: an I / O (input / output) classifier determines whether I / O requests are latency-sensitive; a queue management module classifies the latency-sensitive I / O requests into a fast-queue, so as to copy a memory to a host memory; the queue management module classifies, into a batch-queue, the latency-insensitive I / O requests; a queue processing module generates a scatter-gather list (SGL) for RDMA if the I / O requests classified into the batch-queue is accumulated and satisfies a predetermined flushing condition; the queue processing module transmits a request for registration to a byte-addressable storage on the basis of the generated SGL for the I / O requests; and the queue processing module generates the SGL for other I / O requests classified into the batch-queue until a response to the request for registration is received.

A storage system, a storage server, and an operating method of the storage server are provided. The storage server includes a storage device including a storage controller configured to provide a virtual function, a non-volatile memory, and a memory, a host memory buffer managed by the storage device, a shared memory device, a CXL memory device, and a storage management system configured to receive service level agreement (SLA) information from a virtual machine, receive attribute information from the storage device, analyze the SLA information and the attribute information and generate an analysis result, allocate an internal memory resource or an external memory resource to the virtual function based on the analysis result, and monitor SLA violation of the virtual function, wherein the internal memory resource includes the memory of the storage device, and the external memory resource includes the CXL memory device, the shared memory device, and the host memory buffer of the storage device.

The present application relates to an RDMA-based requester and an RDMA-based responder. The requester comprises a transmit processor and a receive processor, wherein the transmit processor is used for receiving a work queue element issued by a driver program, and if the work queue element is an RDMA read operation, converting the RDMA read operation, so as to obtain a request-side write request; the transmit processor is further used for transmitting the request-side write request to a responder; and the receive processor is used for receiving and parsing a response-side write request sent by the responder, so as to obtain target data and a destination address which correspond to the RDMA read operation, and writing the target data into a storage space corresponding to the destination address. The present application reduces QPC data structure fields and design difficulty, reduces the complexity of out-of-order processing of messages, and avoids the consumption of a large number of host memory and cache resources.

A method of operating a storage device using a host memory buffer of a host includes: recording information on HMB addresses on the host memory buffer, respectively corresponding to a plurality of read requests for the host memory buffer, in an address information table when the plurality of read requests are required; transmitting the plurality of read requests to the host memory buffer, regardless of a response of the host memory buffer; receiving a plurality of pieces of read data, respectively corresponding to the plurality of read requests, from the host memory buffer; detecting an error of each of the plurality of pieces of read data; and updating the address information table based on a result of the error detection.

The embodiment of the application relates to the computer or network technology application field, and discloses a loading method, system, host and storage medium of an operating system, the loading method of the operating system uses the mode of a pre-boot execution environment to load a system image file into a host memory, then loads the system image file to a read-only file system of a virtual file system, and hangs a storage device of the host to a writable file system of the virtual file system, so that the writable file system is automatically loaded when next power-on starting, thereby solving the problem that the mode of the pre-boot execution environment cannot save system data, and improving the loading flexibility of the operating system.

A memory sub-system having host interface configured to communicate with a host system, non-volatile memory cells configured to provide a storage space accessible to the host system via logical block addressing addresses, and a controller. The controller is configured to: receive, via the host interface, a first set feature command from the host system, the first set feature command configured to identify a first host memory buffer; configure caching of at least a portion of a logical to physical translation table in the first host memory buffer; receive, via the host interface, a second set feature command from the host system, the second set feature command configured to identify a second host memory buffer; and reconfigure the caching in the second host memory buffer.

A storage system and an operating method of a storage system are provided. The storage system includes a memory device including a host memory buffer and a storage device including a non-volatile memory device and a storage controller configured to manage the host memory buffer and the non-volatile memory device. The storage controller is configured to read a codeword including first data from the host memory buffer, perform an error detection operation and an error correction operation on the first data, read a plurality of first codewords each including data included in subset data and a codeword including first RAID parity data from the host memory buffer based on the error of the first data not being corrected, and recover the first data based on the first RAID parity data and the subset data, the subset data including data used with the first data to generate the first RAID parity data.

The application relates to a general-purpose GPU-oriented JPEG decoding batch processing and double-buffer pipeline scheduling method, belongs to the field of heterogeneous computing and image processing, and comprises the following steps: obtaining code stream data of a to-be-decoded JPEG image; performing entropy decoding and inverse quantization on the code stream data by a CPU to obtain a plurality of DCT coefficient blocks; if the size of the to-be-decoded JPEG image is not smaller than a preset size threshold, the DCT coefficient blocks are cached in a ring buffer located in a host memory; the state of the ring buffer is continuously monitored; when a preset batch triggering condition is met and a GPU is detected to be available, the DCT coefficient blocks in the ring buffer are batch copied to GPU display memory; based on the DCT coefficient blocks in the GPU display memory, integer IDCT transformation and color space conversion calculation are performed by the GPU to obtain an original image corresponding to the to-be-decoded JPEG image, and the original image is read back to the host memory; wherein the double-buffer pipeline is adopted to perform batch copying of the DCT coefficient blocks, integer IDCT transformation, color space conversion calculation and reading back of the original image. Efficient real-time decoding is realized.

This application is directed to managing memory resources for data caching in a memory device. A memory device has a memory controller, a data processor, a volatile memory including a first memory portion, and a non-volatile memory. The memory device stores a first paging structure in the first memory portion, and the first paging structure maps logical addresses of a plurality of first pages to physical addresses in one or more of the first memory portion, a host memory buffer (HMB), and the non-volatile memory. The memory device executes a hypervisor based on the first paging structure. A data request for a target page is received from the data processor. The memory device searches the first paging structure to identify a target location of the target page and fetches the target page from one of the first memory portion, the HMB, and the non-volatile memory based on the target location.

The application discloses a model training method, a distributed training system, equipment, a medium and a product. The model training method is applied to a first node in a distributed training system, the first node comprises a first training framework and a first memory management service, and the method comprises the following steps: in response to obtaining that a second node in the distributed training system fails, performing initialization of the first training framework, and acquiring checkpoint data of a training target model from a host memory of the second node through the first memory management service; and in response to the first training framework being initialized, continuing to train the training target model based on the checkpoint data. The application can shorten the time delay of model recovery training.

A memory sub-system having a host interface, non-volatile memory cells configured to provide a storage space accessible using logical block addressing addresses, and a controller. The controller is configured to: execute, during a first time period of operations, first storage access commands to access the storage space; receive, via the host interface and after the first time period, a first command identifying a first host memory buffer; configure, based on the first command, caching of a portion of a logical to physical translation table of the memory sub-system in the first host memory buffer; and execute, during a second time period following the first time period and without restarting between the first time period and the second time period, storage access commands based on address translation performed using the portion of the logical to physical translation table cached in the first host memory buffer.

Embodiments of the present application disclose a memory access method and an electronic device. The method initiates a first translation request to an input-output memory management unit including a cache unit, an IO strategy management unit and a control unit in response to a memory access request initiated by an IO device using a virtual address; if a physical address corresponding to the virtual address is not queried from the cache unit, the first translation request is preliminarily screened by the IO strategy management unit; if the preliminary screening result is a preset first result, a second translation request is sent to the control unit to initiate a page table traversal to the memory system to obtain the physical address; and the physical address is transmitted to a PCIe root complex or a CXL host to perform DMA read-write access of the host memory. The present application can intercept illegal requests before address translation, avoid invalid page table traversal, reduce system delay and improve security.

Disclosed is a storage controller which includes a host interface and a storage processor. The host interface receives a write request, a write logical address range, and write data from a first host processor and receives first address translation information between the write logical address range and a storage region of a second host memory device associated with a second host processor. The storage processor transmits the write data to at least one non-volatile memory of a storage device and transmits the write data to the second host memory device based on receiving a read request for the write data from the first host processor and the first address translation information.

A storage controller is disclosed that includes a host interface and a storage processor. The host interface receives a write request, a write logical address range, and write data from a first host processor and receives first address translation information between the write logical address range and a storage region of a second host memory device associated with a second host processor. The storage processor sends the write data to at least one non-volatile memory of a storage device and sends the write data to the second host memory device based on receiving a read request for the write data and the first address translation information from the first host processor.

This invention discloses a method and apparatus for verifying access to a host, an access verification system, and a storage medium, relating to the field of information security technology. By obtaining the host memory address and address length of the object to be measured, a pre-compiled kernel program is used to simulate a trusted software base, and the host memory address and address length are passed to the target software stack. The target software stack interfaces with a Trusted Platform Control Module (TPCM). The TPCM dynamically measures the host memory segment indicated by the host memory address and address length, obtaining a measurement log. The measurement log includes at least a digest value of the object to be measured. The digest value of the object to be measured is compared with a pre-recorded host digest value to obtain a comparison result. If the comparison result indicates that the digest values ​​are consistent, it is confirmed that the TPCM can obtain host resources. Verification of the TPCM's access to host resources can be completed without changing the product interface of the TPCM.

A present invention embodiment performs memory management and shadow queueing in a hypervisor environment. A queue is created in a hypervisor system that includes a host and a guest, wherein the queue is created in response to the guest issuing an I / O instruction, and wherein creating the queue comprises providing guest memory tables and corresponding shadow tables in the host that are accessed by an adapter. Data of the I / O instruction is stored to the guest memory tables, which are synchronized with the shadow memory tables, wherein the synchronizing comprises translating guest memory addresses into host memory addresses and pinning the guest memory addresses. A host unpin structure is provided with a copy of translated and pinned host memory addresses. In response to the adapter executing the I / O instruction, the host unpin structure is utilized, by the host, to locate and unpin the translated and pinned host memory addresses.

The application discloses a cloud host memory optimization allocation method based on capacity calculation and dynamic migration. The method receives a large-specification cloud host creation request and obtains the required memory size. The resource distribution is detected. When the global resources are sufficient but no single machine meets the request, it is judged whether the total free memory meets the preset redundancy condition. If yes, the request is suspended and a fragment consolidation trigger instruction is generated. The host machine value is calculated according to the migration cost function. The target host machine is selected by comprehensively considering the free memory, the number of cloud hosts and the value. The migration path mapping of the cloud host to be migrated out is generated. The migration engine executes the migration through the hot migration interface by calling the automatic script. The memory release of the target host machine is monitored. After the available memory meets the standard, the suspended request is issued for deployment, and the opening is completed. The application realizes active rescue and stable migration when the large-specification instance creation fails, and solves the technical problem that the real-time opening fails due to memory fragmentation although the total resources are sufficient.