296 results about "Data placement" patented technology

The present invention discloses an artificial intelligence-based adaptive big data storage and retrieval optimization system and method designed to intelligently manage and optimize large-scale distributed data environments. The system integrates data acquisition, distributed storage, metadata processing, adaptive learning, and retrieval optimization units configured to work collaboratively for continuous self-optimization. The invention employs deep reinforcement learning and predictive neural network techniques to dynamically analyze system telemetry, workload behavior, and data access patterns in real time, enabling proactive adjustment of data placement, caching, replication, and compression parameters across distributed nodes. The metadata processing framework utilizes graph-based dependency modeling to maintain semantic and contextual relationships among datasets, facilitating intelligent and context-aware data retrieval. The retrieval optimization unit interprets user queries semantically and computes the optimal retrieval route using latency prediction models and dynamic routing techniques.

The invention relates to the technical field of data processing, and discloses a cold and hot data exchange method and system based on optical storage and a storage medium. The method comprises the following steps: performing access feature acquisition and analysis on data in the optomagnetic hybrid storage system, calculating an access life cycle factor, generating a logic interval heat evaluation result, performing temperature grading, formulating a cold and hot data placement strategy, generating a data placement mapping table, executing batch migration according to priority, and updating a data position record. According to the method, the problem of how to realize logic interval hot degree accurate identification, temperature grade dynamic grading, storage mapping optimal matching and batch migration task efficient organization under the optomagnetic hybrid architecture is solved.

Provided is a method for data storage, the method including receiving a first request at a storage device, the first request being associated with a first protocol that is a memory protocol, converting address information associated with the first request to logical block address (LBA) information, determining, by the storage device, a first group of contiguous physical block addresses associated with the first request and associated with a first application, based on a data placement configuration of the storage device, and performing a first memory operation at the first group, based on the first request.

This disclosure provides techniques for managing memory which match per-data metrics to those of other data or to memory destination. In one embodiment, wear data is tracked for at least one tier of nonvolatile memory (e.g., flash memory) and a measure of data persistence (e.g., age, write frequency, etc.) is generated or tracked for each data item. Memory wear management based on these individually-generated or tracked metrics is enhanced by storing or migrating data in a manner where persistent data is stored in relatively worn memory locations (e.g., relatively more-worn flash memory) while temporary data is stored in memory that is less worn or is less susceptible to wear. Other data placement or migration techniques are also disclosed.

Improved placement of memory and functional modules, ‘tiles’, within a tiled processor architecture are disclosed for linear algebra calculations involving vectors and matrices comprising large amounts of data. The improved placement places the data in close proximity to the functional modules performing calculations using the data. These modules enable these calculations to be performed more quickly while using less energy. These modules, in particular, improve the efficiency of the training and application of deep learning and artificial neural network systems. This Abstract and the independent Claims are concise signifiers of embodiments of the claimed inventions. The Abstract does not limit the scope of the claimed inventions.

The invention relates to a method implemented by a computer for selecting a storage configuration defining at least one storage medium to use during the execution of an application to be executed by the high-performance computer. The method including obtaining input / output data of the application, decomposing the obtained input / output data into storage phases and computational phases, obtaining expected storage performances comprising providing the storage phases to a trained performance regression model, simulating the execution of the first application for each storage configuration of a plurality of storage configurations based on the plurality of storage mediums, wherein the simulation includes providing a performance score of each storage configuration of the plurality of storage configurations, and selecting a storage configuration having a highest performance score for the execution of the application.

Decisions related to need to service selection in the multi-cloud environment be taken by the enterprise at the cloud adoption phase as well as post cloud adoption phase. Existing approaches addressing the cloud selection and data placement are not capable of various requirements / parameters that are to be considered for optimum selection of cloud data centers. Method and system disclosed herein facilitate carbon-aware storage service selection in the multi-cloud environment. The system facilitates selection of storage in the multi-cloud environment by generating a plurality of migration solutions. The migration solutions are generated by the system by selectively prioritizing a) a migration cost, b) an operational cost, and c) a carbon footprint. After migrating the non-compliant UCs by selective prioritization, a list of feasible solutions is obtained. These solutions are used as input to obtain optimal solutions with minimal migration cost, operational cost and carbon footprint lower than the new threshold.

Methods and systems for managing service deployment are disclosed. To deploy services, dependencies and other characteristics (such as context data, capability data, placement rules, and workflow data) of services and the infrastructure boundaries in which the services are running may be dynamically collected, analyzed and updated. By dynamically analyzing dependencies and other characteristics (such as context data, capability data, placement rules, and workflow data) of services and the infrastructure boundaries, efficiency of use of computing resources may be improved by recycling existing workflow; statically defined operations and hard-coded conditional logic of services may also be avoided each time an environment in which the services are running has changed.

System-level garbage collection is performed in an SSD-based data storage system employing log-structured writing of data. Respective reclaim unit sizes of the storage devices are identified, wherein each reclaim unit is a unit of programming and erasure used by device-level garbage collection to generate erased reclaim units for allocation for physical data storage. The data is arranged across the storage devices in first and second data units, each first data unit being of a first size U equal to the product of R and a stripe width W, each second data unit including a respective set of first data units which occupy a maximum-size reclaim unit of a storage device among those storing the first data units. The system-level garbage collection is operated with second-data-unit granularity to regularly generate empty second data units for allocation in the log-structured writing of data, reducing write amplification and device wear.

A storage device may carve out redundant array of independent disks (RAID) and reduce write amplification when writing data to the RAID. The storage device includes multiple flexible data placement drives that are configured to execute RAID techniques. A controller on the storage device may receive data from a host. The controller obtains reclaim unit handles from the data to determine locations where the data is to be stored on the flexible data placement drives. During storage, the controller stripes the data in parallel across reclaim units within different reclaim groups and endurance groups across a single flexible data placement drive or multiple flexible data placement drives. The storage device leverages the write amplification of the flexible data placement drives to reduce a write amplification factor on the RAID.

Techniques manage data within storage equipment having a Flexible Data Placement (FDP) drive. The techniques involve configuring the storage equipment to support storing data in reclaim units of the FDP drive according to an initial group of reclaim unit handles. The techniques further involve detecting that a number of currently available reclaim units of the FDP drive has fallen below a first predefined threshold. The techniques further involve, in response to detecting that the number of currently available reclaim units of the FDP drive has fallen below the first predefined threshold, reconfiguring the storage equipment to support storing data in reclaim units of the FDP drive according to a smaller group of reclaim unit handles. The smaller group of reclaim unit handles has fewer reclaim unit handles than the initial group of reclaim unit handles.

A system and method for data storage and a storage device are provided, the method comprising: receiving, at the storage device, a first request associated with a first protocol as a memory protocol; converting address information associated with the first request into logical block address (LBA) information; determining, by the storage device, a first group of contiguous physical block addresses associated with the first request and associated with the first application based on a data placement configuration of the storage device; and performing a first memory operation at the first group based on the first request.

Techniques manage data within storage equipment having a Flexible Data Placement (FDP) drive. The techniques involve configuring the storage equipment to support storing data in reclaim units of the FDP drive according to an initial group of reclaim unit handles. The techniques further involve detecting that a number of currently available reclaim units of the FDP drive has fallen below a first predefined threshold. The techniques further involve, in response to detecting that the number of currently available reclaim units of the FDP drive has fallen below the first predefined threshold, reconfiguring the storage equipment to support storing data in reclaim units of the FDP drive according to a smaller group of reclaim unit handles. The smaller group of reclaim unit handles has fewer reclaim unit handles than the initial group of reclaim unit handles.

Provided are systems, methods, and apparatuses for assisted cache data placement. In one or more examples, systems, methods, and apparatuses include assigning a first identifier to first data based on an aspect of the first data, the first data being data moved from a cache and assigning a second identifier to second data based on an aspect of the second data, the second data being moved to a storage device based on a policy of the cache. In one or more examples, systems, methods, and apparatuses include storing the first data in a first storage location of the storage device based on the first identifier and storing the second data in a second storage location of the storage device based on the second identifier.

System-level garbage collection is performed in an SSD-based data storage system employing log-structured writing of data. Respective reclaim unit sizes of the storage devices are identified, wherein each reclaim unit is a unit of programming and erasure used by device-level garbage collection to generate erased reclaim units for allocation for physical data storage. The data is arranged across the storage devices in first and second data units, each first data unit being of a first size U equal to the product of R and a stripe width W, each second data unit including a respective set of first data units which occupy a maximum-size reclaim unit of a storage device among those storing the first data units. The system-level garbage collection is operated with second-data-unit granularity to regularly generate empty second data units for allocation in the log-structured writing of data, reducing write amplification and device wear.

The invention provides a storage device, an electronic device, a data storage method and a storage medium, and relates to the technical field of data processing, the storage device comprises a CXL interface module, a dynamic memory module, a flash memory module and a flash memory controller module; wherein the dynamic memory module comprises a plurality of device side periodic regions, the flash memory module comprises a plurality of data placement units, and the device side periodic regions are in one-to-one correspondence with the data placement units; the CXL interface module is used for receiving host data from the host side periodic region and temporarily storing the host data in the equipment side periodic region corresponding to the host side periodic region in the dynamic memory module; the dynamic memory module is used for packaging the host data in the equipment side periodic region after preset time; and the flash memory controller module is used for storing the encapsulated host data into a data placement unit corresponding to the equipment side periodic region in the flash memory module. The storage device effectively improves the write amplification phenomenon.

Methods and systems for managing service deployment are disclosed. To deploy services, dependencies and other characteristics (such as context data, capability data, placement rules, and workflow data) of services and the infrastructure boundaries in which the services are running may be dynamically collected, analyzed and updated. By dynamically analyzing dependencies and other characteristics (such as context data, capability data, placement rules, and workflow data) of services and the infrastructure boundaries, efficiency of use of computing resources may be improved by recycling existing workflow; statically defined operations and hard-coded conditional logic of services may also be avoided each time an environment in which the services are running has changed.

The invention discloses an intelligent storage integrated memory storage architecture. The intelligent storage integrated memory storage architecture comprises a computing type memory semantic solid state disk which is used for providing a unified memory address space based on a computing expression link protocol and supporting a host to perform byte granularity access through a loading / storage instruction; an internal intelligent layering and granularity management mechanism is arranged; a flexible data placement mechanism capable of perceiving semantics is set; the host operating system is connected with the computing type memory semantic solid state disk and is used for receiving the long-delay prompt sent by the computing type memory semantic solid state disk and carrying out context switching; and a uniform abstract interface layer is arranged, and is used for uniformly managing byte granularity memory access to the computing type memory semantic solid state disk and transmission of data placement semantic prompts, and submitting a task unloading request to the embedded computing engine. By means of the mode, the problem of multi-impedance mismatch of traditional hierarchical storage in performance, cost, energy efficiency and intelligent processing can be solved.

According to one embodiment, a memory module includes: a memory die including a dynamic random access memory (DRAM) banks, each including: an array of DRAM cells arranged in pages; a row buffer to store values of one of the pages; an input / output (IO) module; and an in-memory compute (IMC) module including: an arithmetic logic unit (ALU) to receive operands from the row buffer or the IO module and to compute an output based on the operands and one of a plurality of ALU operations; and a result register to store the output of the ALU; and a controller to: receive, from a host processor, operands and an instruction; determine, based on the instruction, a data layout; supply the operands to the DRAM banks in accordance with the data layout; and control an IMC module to perform one of the ALU operations on the operands in accordance with the instruction.SPL

A memory sub-system having a first memory configured as a non-volatile storage medium of the memory sub-system; and a second memory having an access speed faster than the first memory. A processing device of the memory sub-system is configured to: run a plurality of data placement handlers concurrently; allocate a plurality of data buffers from the second memory to the plurality of data placement handlers respectively; reserve a plurality of backup spaces from the first memory for the plurality of data placement handlers respectively; allocate accumulation buffers from the second memory; and arrange the data placement handlers to time share the accumulation buffers via usages of the backup spaces.

A ship docking assistance device includes a position azimuth information acquisition unit; a LIDAR; a map generation updating unit; a high-point acquisition unit; and a position azimuth estimation unit. The LIDAR acquires point-group data three-dimensionally indicating the environment around a ship. The map generation updating unit generates a map around the ship based on the point-group data. The high-point acquisition unit acquires, from within the point-group data, a high point having a prescribed height or more. The position azimuth estimation unit estimates the position and the azimuth of the ship through matching between the position of the high point acquired by the high-point acquisition unit and the position of the high point in the map. The map generation updating unit updates the map by placing the point-group data in the map using, as references, the position and the azimuth of the ship estimated by the position azimuth estimation unit.

Systems and methods of deduplication aware scalable content placement are described. A method may include receiving data to be stored on one or more nodes of a storage array and calculating a plurality of hashes corresponding to the data. The method further includes determining a first subset of the plurality of hashes, determining a second subset of the plurality of hashes of the first subset, and generating a node candidate placement list. The method may further include sending the first subset to one or more nodes represented on the node candidate placement list and receiving, from the nodes represented on the node candidate placement list, characteristics corresponding to the nodes represented on the candidate placement list. The method may further include identifying one of the one or more nodes represented on the candidate placement list in view of the characteristic and sending the data to the identified node.

The invention relates to the technical field of wear leveling, and particularly discloses a solid state disk wear leveling algorithm based on a log structure merging tree. Comprising the following steps: S1, dividing the hierarchy of LSM-Tree, and triggering a merging operation when write-in data is increased and reaches a threshold value; s2, distinguishing cold and hot data according to the data layering characteristic of the LSM-Tree, modifying a corresponding interface to transmit layering information to an FTL of an underlying SSD, and confirming a data placement area through the layering information transmitted by an upper layer; s3, storing the erasing times of each flash memory block by adopting a cold and hot block table, and presetting a threshold value; s4, judging and distinguishing cold and hot blocks for the flash memory blocks which are subjected to read-write operation and cannot be updated on site; and S5, organizing all the blocks into a double linked list according to the number of wear times, distributing different hierarchical data storage positions according to hierarchical information of the LSM-Tree transmitted by an upper layer, and performing data exchange. According to the method, resource waste caused by the fact that cold data occupies a low-erasure-frequency block for a long time is avoided, and the problem of unbalanced abrasion of the storage block is reduced.

The invention discloses a tracking type scanner and laser tracker combined measurement method and system and an electronic device. The system comprises a laser tracker, a tracking type scanner, a tracking adapter and a software system. An effective marking area is arranged in a view field overlapping area of the laser tracker and a tracker of the tracking type scanner; a reflection ball used in cooperation with the laser tracker and an identification mark used in cooperation with the tracker of the tracking type scanner are arranged in the tracking adapter. A tracking adapter of a pre-input system is arranged in an effective mark area, and at least three groups of point location data are obtained. And based on the three groups of point location data, calculating a joint transformation relation between a laser tracker coordinate system and a tracker coordinate system of the tracking scanner. Dividing the view field of the laser tracker into a plurality of working areas according to the size of the view field of a tracker of the tracking type scanner; each working area corresponds to a joint transformation relation, scanning data of each working area is converted to a coordinate system of the laser tracker through the joint transformation relation, the scanning data of all the working areas are placed in the same coordinate system, and the purpose of quickly splicing the scanning data of the large workpieces is achieved. And the scanning efficiency of the large workpiece is improved.

The invention relates to a hardware-based SENT peripheral module data processing method and a controller, and the method comprises the steps: configuring a data placement circuit at a SENT peripheral module through software, and carrying out the sequential placement of data positions realized by hardware before frame data received by the SENT peripheral module is placed into an FIFO storage unit in a hardware mode; frame data received by a SENT peripheral module is stored in a data position control register according to a data position sequence, so that the frame data received by the SENT peripheral module can be re-placed according to any required data position sequence according to the data position sequence requirement when actual sensor data is used; according to the method, the frame data are stored in a configurable data control register in the SENT peripheral module, so that the process of adjusting the sequence of the frame data in actual use is omitted, extra software resources are prevented from being occupied, the data processing speed is increased, and the efficiency of data management in the SENT peripheral module is improved.

The invention discloses a data processing method and device, electronic equipment and a readable storage medium, and relates to the technical field of data read-write.The method comprises the steps that after target data is determined, the data type and the operation frequency of the target data are obtained; determining a target data block according to the data type and the operation frequency, and distributing the target data block to a corresponding performance area in the target hard disk in advance according to the access frequency of the target data block; the data processing operation is executed based on the target data and the target data block, so that the problem that a write amplification effect exists in the data processing process can be solved, the data processing is executed based on the performance advantages of the target data block, the frequently written data is placed in an area with a smaller write amplification effect, and the data processing efficiency is improved. Therefore, the technical effects of reducing the write amplification effect based on flexible placement of the data, improving the data processing efficiency and prolonging the service life of the SSD are achieved.

In some embodiments, a method for die-level monitoring is provided. The method includes distributing user data throughout a plurality of storage nodes through erasure coding, wherein the plurality of storage nodes are housed within a chassis that couples the storage nodes. Each of the storage nodes has a non-volatile solid-state storage with non-volatile memory and the user data is accessible via the erasure coding from a remainder of the storage nodes in event of two of the storage nodes being unreachable. The method includes producing diagnostic information that diagnoses the non-volatile memory on a basis of per package, per die, per plane, per block, or per page, the producing performed by each of the plurality of storage nodes. The method includes writing the diagnostic information to a memory in the storage cluster.

Provided are systems, methods, and apparatuses for assisted cache data placement. In one or more examples, systems, methods, and apparatuses include assigning a first identifier to first data based on an aspect of the first data, the first data being data moved from a cache and assigning a second identifier to second data based on an aspect of the second data, the second data being moved to a storage device based on a policy of the cache. In one or more examples, systems, methods, and apparatuses include storing the first data in a first storage location of the storage device based on the first identifier and storing the second data in a second storage location of the storage device based on the second identifier.

The invention discloses a dynamic hierarchical architecture oriented to a partition namespace solid-state disk, which is characterized in that when a compression scheduling engine assigns a compression task, a capacity sensing module acquires the free space capacity of equipment, and determines weights of A, B and C categories for different hierarchies; then, in a priority judgment module, the priorities of the hierarchies are calculated according to the free space capacity and the scores of the hierarchies, and the hierarchies are enqueued according to the priorities of the hierarchies to form a priority queue; the hierarchies are sequentially taken out from the priority queue through a Victim selection module to serve as input of compression, and after the input hierarchies are determined, SST data, overlapped with the lower layer, of the input hierarchies are sent into a merging compression engine to be merged and sorted; and finally, outputting a result to a specified position through a data placement module. According to the dynamic hierarchical architecture oriented to the partition namespace solid-state disk, the effective data volume in the zone can be reduced by dynamically selecting a compact input hierarchy, so that the effective data migration overhead during garbage collection is reduced.