9889results about "Memory architecture accessing/allocation" patented technology

An advanced model management platform for optimizing and securing generative artificial intelligence systems such as large language models (LLMs) and diffusion models. The platform incorporates various techniques to address the limitations of current generative AI systems, such as hallucination, lack of validation, security vulnerabilities, and inadequate model management. The system employs reinforcement learning algorithms for model optimization, retrieval augmented generation (RAG) for hallucination mitigation, domain-specific validation against expert knowledge, model distillation and similarity scoring for security, adversarial training for robustness, and attention mechanism search and model blending for advanced management and neuro symbolic AI routine combinations. By integrating these techniques, the platform significantly improves the performance, reliability, and security of generative AI across a wide range of tasks and domains leveraging the best elements of symbolic and connectionist techniques alongside automated planning and modeling simulation.

A method of generating outputs in LLMs including receiving including textual content, defining a context for the documents including identifying a topic or a category, segmenting the textual content into content chunks associated with the topic or category, assigning a tag to each content chunk, identifying selected chunks, adding metadata to the selected chunks indexing the selected chunks, receiving a query, and performing a response generation process including determining if a cache includes information for the query and either retrieving the information from the cache or performing a search of the index to retrieve the information, generating an augmented query by augmenting the query with retrieved information, generating a response from the augmented query, evaluating the response for compliance with criteria, and one of generating a final response and transmitting the final response to the user or performing a fine-tuning process comprising redefining the of the one or more contexts.

A computer system implements a unified framework integrating an adaptive elastic funnel (AEF) with a convergent intelligence fabric (CIF) for multi-agent AI collaboration. The system provides a universal multi-modal key-value subsystem for sharing partial computations, implements hybrid placement strategies for dynamic memory management, and incorporates quantum-resistant secure enclaves. The architecture integrates hardware acceleration through GPU-FPGA hybrid caching and neuromorphic processors, applies adaptive energy and thermal management across hardware generations, and implements autonomous flash resource orchestration with multi-dimensional wear management. The system orchestrates tensor workflows using hierarchical scheduling, enables cross-agent collaboration with privacy preservation, and supports continuous learning without catastrophic forgetting. This integration delivers unprecedented computational efficiency and security in high-dimensional decision-making environments while supporting incremental adoption through modular interfaces.

A nonvolatile memory device includes a plurality of latch groups, an address controller, an encoder, and a buffer. The address controller controls an input address and an output address to indicate one of the plurality of latch groups. The encoder receives sector data from a latch group corresponding to the output address among the plurality of latch groups and also compresses the received sector data. The buffer stores the compressed sector data. Among the plurality of latch groups, the compressed sector data stored in the buffer is overwritten in a latch group corresponding to the input address.

Apparatuses, systems, and methods for half-page modes. A memory device may be operated in a full-page mode where all the memory cells along a word line are used for data or a half-page mode where less than all of the memory cells are used for data. In some memory devices, each half of the memory cells may be separately activated by different word line portions. In some half-page modes, data may be stored along a selected portion of the memory cells and additional information such as metadata or module parity may be stored along the non-selected portion of the memory cells. The additional information may be provided along additional data terminals so as not to increase the data burst length.

A system and method of improving performance of LLMs including receiving a query, retrieving relevant documents for the query to form a combined context, partitioning the combined context in context partitions, generating intermediate results from the context partitions using a mapper prompt, and generating a final result from the intermediate results using a reducer prompt. The final result is transmitted to a user.

Key-value (KV) caching accelerates inference in large language models (LLMs) by allowing the attention operation to scale linearly rather than quadratically with the total sequence length. Due to large context lengths in modern LLMs, KV cache size can exceed the model size, which can negatively impact throughput. To address this issue, KVCrush, which stands for KEY-VALUE CACHE SIZE REDUCTION USING SIMILARITY IN HEAD-BEHAVIOR, is implemented. KVCrush involves using binary vectors to represent tokens, where the vector indicates which attention heads attend to the token and which attention heads disregard the token. The binary vectors are used in a hardware-efficient, low-overhead process to produce representatives for unimportant tokens to be pruned, without having to implement k-means clustering techniques.

A system for adaptively caching network communication protocols enhances efficiency across heterogeneous device environments through a multi-level cache architecture with device-capability-based tiers. The system collects endpoint telemetry data including device capabilities and operational constraints to classify endpoints and generate context-aware protocol variants optimized for specific device types. Protocol optimization opportunities are determined through structural analysis of message patterns and state transitions. The system performs protocol deduplication by identifying functionally equivalent variants and maintaining canonical representations to reduce cache redundancy. Cache synchronization across distributed nodes uses enhanced Merkle tree structures with protocol normalization processing. The system predicts communication needs based on historical patterns, network context, and endpoint constraints, enabling proactive cache management tailored to device capabilities. Integration with event-driven data communication systems enables seamless protocol selection and translation while maintaining compatibility between diverse endpoint types, from high-performance servers to resource-constrained IoT devices.

A storage device manages memory use data, such as map data which the storage device manages using a host memory, as journal data in a buffer memory located inside the storage device in a low power mode. The memory use data may be managed by updating journal data during the low power mode, and a command of a host device may be processed by updated journal data after switching from a low power mode to an active mode.

Embodiments are directed to a file system that include object stores. An object store for write requests may be provided. Write ahead log (WAL) entries that include data blocks may be generated. A WAL object may be generated based on the WAL entries and stored in the object store. An in-memory overlay may be updated to associate the data blocks with the WAL object. A checkpoint operation may be executed to: generate an index object that includes index entries that associate other data blocks with data objects stored in the object store; update the index object to include index entries that associate the data blocks with the WAL object; store the updated index object in the object store; update the in-memory overlay to remove the association of the data blocks and the WAL object and update the in memory WAL to remove records of successfully checkpointed WAL objects.

A parallel processing unit comprises a plurality of processors each being coupled to a memory access hardware circuitry. Each memory access hardware circuitry is configured to receive, from the coupled processor, a memory access request specifying a coordinate of a multidimensional data structure, wherein the memory access hardware circuit is one of a plurality of memory access circuitry each coupled to a respective one of the processors; and, in response to the memory access request, translate the coordinate of the multidimensional data structure into plural memory addresses for the multidimensional data structure and using the plural memory addresses, asynchronously transfer at least a portion of the multidimensional data structure for processing by at least the coupled processor. The memory locations may be in the shared memory of the coupled processor and / or an external memory.

A key-value cache management system processes inference requests of a generative model. An inference request requests a starting virtual memory address and a number of layers in the generative model. In response to receiving an inference request, contiguous virtual memory space is reserved for the key-value cache in accordance with the inference request by assigning the starting virtual memory address in the key-value cache and calculating memory pointers for each layer in the model and each block so that the one or more blocks are written sequentially. The generated memory pointers are outputted to the generative model so that each self-attention layer writes computed key-value pairs at physical addresses in the key-value cache specified by the memory pointers.

A method, computer program product, and computing system for receiving a metadata page for storage within a log-structured metadata store. An entry may be generated within a translation table page of a translation table within the log-structured metadata store, wherein each entry of a plurality of entries within the translation table page is associated with a separate metadata page. One or more metadata page deltas for the metadata page may be recorded in the log-structured metadata store. The one or more metadata page deltas for the metadata page may be flushed to a persistent memory storage. In response to flushing the one or more metadata page deltas for the metadata page, the entry within the translation table page that is associated with the metadata page may be updated without accessing any other entry within the translation table page associated with other metadata pages.

A system and method for real-time team intent modeling using persistent cognitive machines with federated human profiles which processes individual team member behavioral signals through geometric intent analyzers that generate high-dimensional vector representations of individual objectives and preferences. A team intent orchestrator aggregates individual vectors into collective representations within a dynamic geometric manifold that evolves based on team coordination patterns. Federated human profiles enable privacy-preserving knowledge sharing across teams through geometric abstraction techniques that preserve coordination utility while protecting individual privacy. The system implements proactive conflict detection through trajectory analysis that identifies potential coordination issues before performance impact, and provides real-time synchronization mechanisms that maintain team coordination coherence despite individual behavioral changes. Cross-team learning capabilities enable organizational intelligence development through pattern abstraction and context-aware adaptation of successful coordination strategies. The persistent cognitive architecture maintains coordination patterns across sessions and team composition changes, enabling continuous improvement through accumulated team experience.

Systems and methods for dynamic knowledge integration in LLM systems including receiving multimodal input data comprising text, image, audio, video, and / or code data, extracting information by processing the multimodal input data through a document processor, storing the extracted information in a dynamic knowledge base, receiving a user query at a query processor, identifying knowledge domains related to the user query using domain-specific agents, retrieving real-time information from the dynamic knowledge base responsive to the identified knowledge domains, integrating the real-time information into the processing of an LLM by a dynamic knowledge integrator, and generating a response using the LLM with the real-time information.

Embodiments are directed to a file system that include object stores. An object store for write requests may be provided. Write ahead log (WAL) entries that include data blocks may be generated. A WAL object may be generated based on the WAL entries and stored in the object store. An in-memory overlay may be updated to associate the data blocks with the WAL object. A checkpoint operation may be executed to: generate an index object that includes index entries that associate other data blocks with data objects stored in the object store; update the index object to include index entries that associate the data blocks with the WAL object; store the updated index object in the object store; update the in-memory overlay to remove the association of the data blocks and the WAL object and update the in memory WAL to remove records of successfully checkpointed WAL objects.

The embodiment of the invention provides a memory access method and device, electronic equipment and a readable storage medium. The method comprises the following steps: receiving a cache line access request; extracting label information from a target memory address corresponding to the cache line access request, wherein the label information comprises a shared label, a private label, a group index, inter-block offset and intra-block offset; querying whether the cache line access request hits the L1 cache or not based on the label information; the group index is used for determining a target group, the shared tag and the private tag are used for determining a target cache line in the target group, and the inter-block offset is used for determining a target cache block in the target cache line; and if the cache line access request hits the L1 cache, determining a target data position in a hit target cache block based on the in-block offset, and executing a corresponding memory access request based on the target data position. According to the embodiment of the invention, on the basis of ensuring the data access efficiency, the storage space of the cache line is fully utilized, and the waste of the cache space is reduced.

Embodiments are directed to a file system that include object stores. An object store for write requests may be provided. Write ahead log (WAL) entries that include data blocks may be generated. A WAL object may be generated based on the WAL entries and stored in the object store. An in-memory overlay may be updated to associate the data blocks with the WAL object. A checkpoint operation may be executed to: generate an index object that includes index entries that associate other data blocks with data objects stored in the object store; update the index object to include index entries that associate the data blocks with the WAL object; store the updated index object in the object store; update the in-memory overlay to remove the association of the data blocks and the WAL object and update the in memory WAL to remove records of successfully checkpointed WAL objects.

This invention pertains to a system optimized for reutilizing allocated underutilized or unused allocated DRAM, comprising a first host, a second host, and a resource composer, interconnected via CXL. Both hosts run packaged computing environments (PCEs), which may be containers or virtual machines, and are equipped to handle respective processes, P1 and P2. The resource composer is tasked with receiving data related to P1's memory usage from a kernel module on the first host, identifying underutilized DRAM mapped to P1, and subsequently remapping it to P2's address space on the second host. This process involves the use of CXL.mem commands, which are then translated into appropriate CXL.cache or CXL.io commands for DRAM access based on the mapping.

A Data Storage Device (DSD) includes a first memory storing reference files used to derive vector embeddings in a vector database. A query vector embedding is received from a host and one or more vector embeddings similar to the query vector embedding are identified in the vector database. One or more reference files from which the one or more vector embeddings were derived are identified and stored in a second memory for faster access. In another aspect, a query vector embedding is received by a host that identifies one or more vector embeddings that are similar to the query vector embedding and retrieves the one or more vector embeddings from a DSD to provide to an Artificial Intelligence (AI) model. One or more reference files are identified from which the one or more vector embeddings were derived and are prefetched from the DSD for storage in a host memory.

An apparatus is provided that includes a network interface to transmit and receive data packets over a network; a memory including one or more buffers; an arithmetic logic unit to perform arithmetic operations for organizing and combining the data packets; and a circuitry to receive, via the network interface, data packets from the network; aggregate, via the arithmetic logic unit, the received data packets in the one or more buffers at a network rate; and transmit, via the network interface, the aggregated data packets to one or more compute nodes in the network, thereby optimizing latency incurred in combining the received data packets and transmitting the aggregated data packets, and hence accelerating a bulk data allreduce operation. One embodiment provides a system and method for performing the allreduce operation. During operation, the system performs the allreduce operation by pacing network operations for enhancing performance of the allreduce operation.

Methods and apparatus for application aware memory patrol scrubbing techniques. The method may be performed on a computing system including one or more memory devices and running multiple applications with associated processes. The computer system may be implemented in a multi-tenant environment, where virtual instances of physical resources provided by the system are allocated to separate tenants, such as through virtualization schemes employing virtual machines or containers. Quality of Service (QoS) scrubbing logic and novel interfaces are provided to enable memory scrubbing QoS policies to be applied at the tenant, application, and / or process level. This QoS policies may include memory ranges for which specific policies are applied, as well as bandwidth allocations for performing scrubbing operations. A pattern generator is also provided for generating scrubbing patterns based on observed or predicted memory access patterns and / or predefined patterns.

In response to instruction decoding circuitry decoding a conditional write instruction, processing circuitry determines whether a predetermined condition is satisfied for a target cache line corresponding to a target address specified by the conditional write instruction. If the predetermined condition is satisfied for the target cache line, a write request is issued to update the target cache line. If the predetermined condition is not satisfied for the target cache line, a failure indication is returned. The processing circuitry selects, depending on whether the sequence of instructions specifies cache-line-retention hint information applicable to the conditional write instruction, whether to prevent a unique coherency state of the target cache line being relinquished by a local cache associated with the processing circuitry for a retention period following processing of the conditional write instruction. The unique coherency state comprises a coherency state in which the processing circuitry has exclusive right to update the target cache line.

Apparatuses, systems, and techniques to cause information to be invalidated in a second cache location after information is stored in a first cache location. In at least one embodiment, one or more circuits are to perform an application programming interface (API) to cause information to be invalidated in a second cache location after information is stored in a first cache location.

Examples described herein include one or more processors; a network interface; and a direct memory access (DMA) engine communicatively coupled to the one or more processors. In some examples, the DMA engine is to receive a DMA data access request and based on an address in the DMA data access request corresponding to a remote memory device, the DMA engine is to cause the network interface to generate at least one packet for transmission to the remote memory device. In some examples, if the source address corresponds to a local memory device and the destination address corresponds to a remote memory device, the DMA engine is to cause the network interface to generate at least one packet for transmission to the remote memory device.

A method and apparatus for cache warming in a distributed storage system is described. The method can include detecting a destructive change to one or more nodes of an existing cluster of cache data nodes. The method can also include allocating a new cluster of cache data nodes in a write-only mode, and streaming data from each cache data node of the existing cluster to cache data nodes of the new cluster. The method can further include servicing a data access request from a selected cache data node of the existing cluster while writing data from the data access request to a selected cache data node of the new cluster. Furthermore, the method can include in response to a determination that data from the cache data nodes of the existing cluster has been successfully streamed to the new cluster, servicing new data access requests with the new cluster.

Systems, devices, methods, and circuits for managing data security in storage devices. In one aspect, a storage device includes at least one memory device and a controller coupled to the at least one memory device. The controller is configured to: encrypt first data with a first type of cryptographic algorithm and encrypt second data with a second type of cryptographic algorithm. The first data is associated with a first security level, and the second data is associated with a second security level that is higher than the first security level. The second type of cryptographic algorithm has a greater encryption strength than the first type of cryptographic algorithm.

This disclosure provides techniques for managing memory which match per-data metrics to those of other data or to memory destination. In one embodiment, wear data is tracked for at least one tier of nonvolatile memory (e.g., flash memory) and a measure of data persistence (e.g., age, write frequency, etc.) is generated or tracked for each data item. Memory wear management based on these individually-generated or tracked metrics is enhanced by storing or migrating data in a manner where persistent data is stored in relatively worn memory locations (e.g., relatively more-worn flash memory) while temporary data is stored in memory that is less worn or is less susceptible to wear. Other data placement or migration techniques are also disclosed.