120 results about "Data layout" patented technology

The present invention discloses an artificial intelligence-based adaptive big data storage and retrieval optimization system and method designed to intelligently manage and optimize large-scale distributed data environments. The system integrates data acquisition, distributed storage, metadata processing, adaptive learning, and retrieval optimization units configured to work collaboratively for continuous self-optimization. The invention employs deep reinforcement learning and predictive neural network techniques to dynamically analyze system telemetry, workload behavior, and data access patterns in real time, enabling proactive adjustment of data placement, caching, replication, and compression parameters across distributed nodes. The metadata processing framework utilizes graph-based dependency modeling to maintain semantic and contextual relationships among datasets, facilitating intelligent and context-aware data retrieval. The retrieval optimization unit interprets user queries semantically and computes the optimal retrieval route using latency prediction models and dynamic routing techniques.

The invention relates to the field of data storage, in particular to a storage and calculation separation multi-dimensional data block indexing method based on adaptive learning. Comprising the steps of receiving original multi-dimensional data and a query load, and outputting a low-dimensional key set through a first model; the low-dimensional key set and the historical query load generate a partitioning scheme through a second model, and the partitioning scheme performs physical recombination on the original multi-dimensional data to output a plurality of physical data blocks; constructing a learning type index through the physical data blocks and the corresponding partitioning schemes; identifying hot spot and cold spot areas through the query frequency of each node of the learning type index and the real-time query load continuous monitoring index; when the access mode changes, adjusting the index structure, and outputting a self-adaptive index; and quickly positioning and reading the target data block through the self-adaptive index to complete access. According to the method, the index of adaptive learning is provided, and the index structure and the physical data layout are adjusted, so that the query accuracy and efficiency in a complex scene are improved, and the data storage overhead is reduced.

The invention discloses a data layout optimization method and device applied to NPU code compilation and a medium. The method comprises the steps that an intermediate representation IR input by an upper layer is split into an operation type OP and an operand, and the operand input by the upper layer is divided into logic data and a logic mask; converting the logic data of the upper layer into data vector representation of the abstraction layer, and converting the logic mask of the upper layer into mask vector representation of the abstraction layer; obtaining a corresponding bottom hardware instruction capability according to an operation type OP input by an upper layer, and performing legalization operation on the operation type OP; and performing legalization processing on the converted abstract data vector type representation and mask vector representation according to the target hardware capability, performing instruction mapping on the processed abstract instruction according to the underlying hardware pair operation type OP, and generating a target LLVM instruction compatible with the underlying hardware. Register layout abstraction and conversion can be carried out on the NPU code under mixed precision input, and the utilization efficiency of the NPU bottom layer register is improved.

The invention provides a graph data communication intensity mapping method and system based on optical transport network topology. According to the method, an adjacent matrix is established by extracting graph data structure information, vertex centrality and a clustering coefficient are calculated, and a graph data communication feature matrix is generated; oTN network equipment information and optical link topology are collected, link parameters and wavelength resource states are measured, and an OTN physical network feature matrix is obtained; establishing a double-layer mapping relation matrix through a resource mapping algorithm; integrating the communication frequency, the time delay sensitivity, the bandwidth requirement, the wavelength resource and the topological matching degree, and calculating a communication strength weight matrix; generating a candidate layout scheme, performing multi-dimensional evaluation, and determining an optimized data layout scheme; and dynamically adjusting the mapping relation by adopting an incremental updating algorithm. According to the invention, the efficient collaboration of the graph computing task and the optical transport network resource is realized, and the performance and resource utilization efficiency of the distributed graph computing system are obviously improved.

The invention relates to a mask generation method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: after obtaining first matrix data, loading the first matrix data to a first memory in response to a target access instruction to adjust a data arrangement mode of the first matrix data to obtain second matrix data; determining a multiplication result matrix of the second matrix data, and statically distributing the multiplication result matrix to a second memory corresponding to the corresponding calculation thread; and determining a mask register according to the thread identifier and the mask boundary of the calculation thread. In the process of loading the first matrix data to the first memory, the complex high-dimensional tensor first matrix data is adjusted into a regularly linearly arranged two-dimensional matrix, so that optimization and recombination of data layout are realized, a large number of access instructions across calculation units are prevented from being generated during data access in the mask calculation process, and the data access efficiency is improved. The utilization efficiency of the memory broadband is improved, and the overall calculation performance of the chip is improved.

The invention discloses a generation method and device for a mixed precision quantization operator, and relates to the field of deep learning, and the method comprises the steps: converting a quantization program inputted by a user into an initial Qtile calculation graph based on Qtile tensor representation; performing equivalent transformation on the initial Qtile calculation graph to generate a plurality of candidate Qtile calculation graphs which are equivalent to the initial Qtile calculation graph; according to the reuse frequency of operator input data in each candidate Qtile calculation graph, determining an optimal data layout position and an optimal data loading strategy of each candidate Qtile calculation graph; on the basis of the optimal data layout, the optimal data loading strategy and a code generation technology based on a template, codes of all the candidate Qtile computational diagrams are generated, and the codes of the candidate Qtile computational diagrams are achieved through codes of mixed precision quantization operators. According to the invention, an optimized high-performance mixing precision quantization operator can be output.

The invention discloses a solid state disk-based data transmission method and device, a medium and a product, and relates to the field of solid state disks. According to the method, a data layout description header is obtained from a host memory in a direct memory access mode according to a target memory initial address; analyzing the data layout description header through a flash translation layer of the solid state disk, identifying each data segment entry, and extracting length information and life cycle attribute tags of corresponding data segments from each data segment entry; based on the life cycle attribute tag, planning a corresponding physical flash memory page for each data segment as a target write address of the data segment; and based on the target write-in address, controlling the solid state disk to execute DMA operation segment by segment according to the sequence of data segment entries in the data layout description head until the write-in of the total data length is completed. By implementing the technical scheme provided by the invention, the data storage efficiency of the solid state disk is improved.

The invention provides a memory management method and a memory controller, and aims to solve the problem of storage mismatch caused by dynamic change of a data access mode. The method comprises the following steps: acquiring target data stored in a target entity page in a plurality of entity pages; determining a data attribute of the target data and an offset attribute of the target entity page; and if the data attribute of the target data is not matched with the offset attribute of the target entity page, selecting a matched entity page of which the offset attribute is matched with the data attribute from the plurality of entity pages, and moving the target data from the target entity page to the matched entity page. Through the technical scheme of the invention, the memory controller can actively and periodically identify and correct the attribute mismatch between the stored data and the physical position thereof, and the continuous dynamic optimization of the data layout is realized, so that the data reliability and the overall performance of the storage device in the long-term use process are improved.

This invention discloses a method, system, and storage medium for verifying instrument data records. Based on a received data verification command, each record in the instrument data record table is sequentially retrieved. The corresponding screen layout information is retrieved from the instrument information database according to the instrument model. Then, the display features in the screen image are identified and analyzed, and compared with the screen area shape and measurement data layout information stored in the instrument information database. If they are not similar, an instrument model entry error message is triggered. If they are similar, the identified instrument measurement parameters are compared with the recorded parameters in the data record. If they are not the same, a measurement parameter entry error message is triggered. By using screen image content to automatically verify and verify the instrument model and measurement parameters in electronic experimental records, the inefficiency and error-prone nature of existing manual verification methods are overcome.

The invention discloses an observable method and system for a unified log index and tracking data, and the method comprises the steps: collecting various types of original observation data, carrying out the unified formatting, constructing a column type storage structure, and carrying out the hierarchical storage; then optimizing data layout by utilizing column-level metadata and a query mode feature library, and implementing a multi-level index strategy; meanwhile, through data cleaning, structured enhancement and contextual information association, a causal atlas between heterogeneous observation data is established, and observation data with rich contexts is generated; and finally, a high-performance query processing system is constructed based on a unified query engine and a sub-linear algorithm, and progressive result return and visual analysis are realized. According to the method, the data retrieval efficiency and the analysis depth are remarkably improved, operation and maintenance personnel can more quickly position the root cause of the system problem, and the average time of fault diagnosis is effectively shortened.

The invention relates to the technical field of heterogeneous environment task migration, and particularly provides a heterogeneous environment task migration method and system and a medium. According to the method, a uniform data layout and a virtual address space structure are constructed to unify state representation modes among different instruction set architectures; the extensible executable file is expanded on the basis of a fat binary file, and a symbol table is split into a basic symbol table shared across instruction set architectures and an independent extended symbol table of each instruction set architecture; and executing task migration across the instruction set architecture by the runtime environment in combination with the basic symbol table and the extended symbol table based on the migration points and the trigger pages embedded in the construction process. According to the method, the mobility, the cross-platform compatibility and the system analyzability are balanced, and systematic support is provided for task migration in a heterogeneous environment.

The present disclosure proposes a communication method and a distributed system for distributed training of mixed expert models. The method comprises: collecting gating network routing decision information corresponding to each input data; generating mapping information for each input data to be transmitted according to the deployment information of the expert module and the collected gating network routing decision information, wherein the mapping information is used to indicate the reading position of the input data in the sending end buffer and the writing position in the receiving end buffer; and performing a communication operation based on the mapping information to send the input data to be transmitted to the corresponding expert module. By introducing the mapping information for indicating the address relationship of the input data in the memory in the communication operation, the data rearrangement is synchronized in the communication process, thereby realizing the cooperative execution of the communication path and the data layout. Further, a two-stage mapping structure across machines and within machines can be constructed to realize efficient training based on topology hierarchical communication.

The invention discloses a graph structure data layout optimization method for a Qt tree layering and dynamic force oriented graph, which comprises the following steps of: loading graph structure data and carrying out data preprocessing, converting original data into a structured data table containing node attributes and connection relationships, dividing node levels through a depth-first traversal algorithm, and carrying out layout optimization on the structured data table; and dynamically generating a fan-shaped initial layout according to the degrees of the nodes, carrying out layout iterative optimization calculation by adopting a force-oriented layout algorithm, determining the position information and topological connection relationship of each primitive after optimization, and carrying out visual implementation and interaction on the optimized graph structure data layout. According to the scheme of the invention, through cooperation of hierarchical and elastic constraint mechanisms, the readability of a graph containing circulation / multiple father nodes is improved, a dynamic updating strategy enables layout to maintain overall stability during interaction, the structure confusion degree is reduced, and pre-layout processing and a parameter dynamic attenuation strategy effectively improve the convergence speed of a force steering algorithm.

The application discloses a kind of dynamic resource adaptation multilayer cache architecture systems, it is related to computer system architecture and data storage cache technical field.The system includes: resource state perception module, for real-time acquisition system performance index data and load characteristics in running, performance index data includes the access delay of each cache level, cache hit rate, data access frequency, cache medium read-write speed and system concurrent connection quantity.The application is through resource state perception module comprehensive acquisition each cache level performance index and load characteristics, combined with the deep reinforcement learning algorithm of cache strategy decision module dynamically generates cache strategy parameter, solves the problem that static configuration is difficult to adapt complex load in prior art;Data layout optimization module is through hot data identification, hot and cold data asynchronous migration and cross-level data block size adaptation, accurately match the characteristics of heterogeneous cache medium, make up the defect that traditional architecture is not enough to perceive medium difference.

The application provides a convolution calculation method, an electronic device, a storage medium and a program product, and belongs to the technical field of artificial intelligence, and comprises the following steps: performing feature extraction on input information to obtain an input tensor; folding the input tensor along a target dimension direction to a batch dimension to obtain a plurality of batch target tensors; performing complete convolution calculation on each target tensor to obtain a first convolution result; and obtaining a convolution result of the input information according to the first convolution result; wherein the target dimension of the input tensor is greater than a first threshold under the limitation of a hardware size, the target dimension of the target tensor is less than or equal to the first threshold, and the target dimension is height or width. When the target dimension of the input tensor exceeds the limitation of the hardware size, the input tensor is folded along the target dimension to the batch dimension, the convolution calculation of the over-limit tensor can be realized based on the native hardware and the native data layout, the data layout does not need to be replaced, and the performance of the convolution calculation is improved.

Embodiments of the present disclosure relate to a neural processor circuit configured to switch between a width-last mode and a channel-last mode of input data for more efficient processing of tasks. A compiler may determine whether the neural processor circuit is likely to perform a task more efficiently by using the input data in a width-last format or the channel-last format and compiles instructions to enable or disable a transposer circuit in the neural processor circuit. When the neural processor circuit is in a mode that uses the channel-last format, the input data in the width-last format is transposed into transposed input data in the channel-last format before being fed into one or more neural engines of the neural processor circuit, and output data generated by the one or more neural engines are also transposed back into the width-last format.

A computing system includes a memory and a processor to detect a connection of one or more medical devices to the computing system, each of the one or more medical devices configured to provide image or video data to the computing system, cause display of a user interface associated with generation of a custom layout for presenting the image or video data, receive one or more inputs via the user interface that define the custom layout, including one or more of: a number of the one or more image or video tiles, a size of the one or more image or video tiles, or an association of the one or more medical devices with the one or more image or video tiles, and store the custom layout such that the image or video data is provided for display in accordance with the custom layout to one or more display devices.

The present invention provides a method for generating an object and a data layout in a digital twin. The invention may include the steps of: generating and placing an object according to a user's command; generating an endpoint for the object; requesting that the endpoint be connected to raw data of a real-world asset; defining a visualization of the raw data or custom data; generating and connecting a layout for the visualization; and displaying the result of the visualization of the raw data or custom data using the layout.

Embodiments of the present application provide a tensor loading and storing method, device and storage medium, and relate to the technical field of artificial intelligence chips. The method comprises: based on a pre-packaged basic processing unit, splitting a loading and storing process of a target tensor into a plurality of sub-loading and storing processes corresponding to respective sub-tensors; a first memory data layout of the basic processing unit is adapted to hardware attributes of the artificial intelligence chip, that is, the first memory data layout of the basic processing unit meets hardware alignment granularity of the artificial intelligence chip, and meanwhile, when loading and storing data according to the first memory data layout of the basic processing unit for processing, resource usage of the artificial intelligence chip reaches an upper limit value. Therefore, when loading and storing tensor data by taking the basic processing unit as a minimum data unit, the hardware memory structure of the artificial intelligence chip is more fitted, so that not only the hardware resources of the chip are fully utilized, the utilization rate of the hardware resources is improved, but also the performance of tensor loading and storing is improved.

This application provides an inference optimization method, apparatus, and storage medium for a generative diffusion model. The optimization method includes one or more of the following: first optimization of the inference parts of the U-Net and VAE in the generative diffusion model through code recompilation; second optimization of the performance bottleneck region in the VAE through data layout transformation; and multi-level operator optimization of key computational operations in the generative diffusion model based on performance bottleneck analysis. Through the above technical solutions, this invention can improve the generative diffusion model on a heterogeneous acceleration platform, aiming to locate the performance bottleneck part in the model through performance profiling, improve the inference efficiency on the heterogeneous acceleration platform through data layout transformation, and systematically improve the model's inference performance by utilizing multi-level optimization schemes from the graph level, data level, to the operator level.

The invention discloses a security management method and system for transmission, transformation and distribution sensing data of the Internet of Things, and relates to the field of transmission, transformation and distribution, and the method comprises the steps: building an edge node, and carrying out the receiving and analysis of the transmission, transformation and distribution sensing data; performing optimization processing on the sensing data of each edge node to obtain a forward-inverse optimal layout of a single edge node; a multi-island genetic algorithm forward optimization model is established, and the overall transmission, transformation and distribution perception data forward optimal layout is obtained; establishing a multi-island genetic algorithm reverse optimization model, and obtaining a reverse optimal layout of the overall transmission, transformation and distribution perception data; establishing a transmission, transformation and distribution sensing data security management model, and screening out the layout of the optimal transmission, transformation and distribution sensing data; and receiving, storing and displaying transmission and transformation distribution sensing data information and the optimal layout based on the Internet of Things technology. Through the transmission, transformation and distribution sensing data security management model, the efficient distribution of transmission, transformation and distribution is effectively improved by using a positive-inverse optimal layout mode, and the operation security of transmission, transformation and distribution is ensured.

The invention discloses a data layout method and system, and the method comprises the steps: firstly obtaining original data, and decomposing the original data into a plurality of sub-data sets; then, with maximization of data processing benefits as a target, a data distribution scheme about the sub-data sets is determined, and data distribution operation is executed on the sub-data sets based on the data distribution scheme, so that the sub-data sets are arranged on a plurality of server grid units of a data center, and preliminary distribution of data in the data center is achieved; according to the invention, after the preliminary distribution of the data is completed, the current data layout of the data center is optimized by obtaining the data distribution information of the data center, so that the reasonability of the final data layout of the data center is improved, and the effective improvement of data processing benefits is facilitated.

The application relates to the technical field of high-performance computing, and specifically discloses a CUDA Core and Tensor Core cooperative parallel computing method for irregular matrix multiplication, which cooperatively schedules CUDA Core and Tensor Core through a hybrid core fusion method, optimizes the allocation of computing resources based on a thread division coefficient model, and improves the computing efficiency of irregular matrix multiplication. The cooperative parallel computing of CUDA Core and Tensor Core at the SM granularity is realized for the first time, the alternating idle state between cores is eliminated through a hybrid core fusion algorithm, and the utilization rate of GPU hardware resources is significantly improved; through shared memory partitioning, a double-buffering mechanism and data layout optimization, memory bank conflicts are avoided, and memory access delays are hidden; thread-level load balancing is realized based on a performance model, and the overall computing efficiency is further improved.

The application provides a non-privacy and privacy reasoning scheduling method based on a reconfigurable chip, which comprises the following steps: determining whether the privacy requirement of a neural network reasoning task is privacy reasoning; if yes, performing a first scheduling step; otherwise, performing a second scheduling step; in the first scheduling step, analyzing an arithmetic logic unit required by the privacy reasoning, combining a reconfigurable multiplier and an adder by a data layout converter to form the arithmetic logic unit, and reconfiguring an interconnection network of a processing unit array into a butterfly network or a SIMD data path to obtain a reconfigurable chip and perform a reasoning step; in the second scheduling step, setting a slice as a tensor mode by the data layout converter, independently and parallelly working all the slices, performing a multiply-accumulate operation by each slice, transmitting data between processing units PEs through horizontal and vertical links to form a pulsating data stream, forming a pulsating array by the processing unit array, obtaining the reconfigurable chip, and performing the reasoning step; and in the reasoning step, performing the neural network reasoning task by the reconfigurable chip to obtain a reasoning result.

The invention relates to the technical field of charging piles, in particular to a direct current charging pile safety management method and system, and the method comprises the following steps: determining the operation parameter information of a to-be-monitored direct current charging pile, and constructing a multi-dimensional data layout based on the operation parameter information; acquiring operation parameter information of the to-be-monitored direct current charging pile in different operation states, and drawing on the corresponding data layout based on the operation parameter information to obtain a plurality of track graph streams; setting a calling rule for the plurality of track graph streams; current operation parameter information of the charging pile is collected, a real-time graph flow formed by the current operation parameter information in the data layout is obtained, and the real-time graph flow and the multiple track graph flows are compared and analyzed based on the calling rule; identifying an information point position deviating from the standard track graph in the current real-time graph stream to obtain a deviation symptom point; the accuracy of abnormality diagnosis of the charging pile can be improved, and the effect of safety management of the charging pile is improved.

This application discloses a method, apparatus, device, and storage medium for generating SCADA configuration screens based on generative AI, relating to the field of industrial control technology. The method includes: collecting natural language design requirements using an intent understanding module and the perception layer in an AI Agent cognitive architecture; obtaining state acquisition data and integrating it before inputting it to the cognitive layer; the inference engine and large language model in the cognitive layer decompose the configuration design task of the integrated data, generating a configuration design operation instruction set and inputting it to the execution layer; the execution layer processes the operation instruction set to obtain each component to be laid out in the configuration screen and its configuration data; the layout engine determines the component layout result in the configuration canvas; the variable binding engine binds and maps each component to the real-time data points of the SCADA system; and the target configuration screen is generated by combining the Shell command security module and the security layered architecture to improve the efficiency of generating SCADA configuration screens.

The invention discloses a data layout optimization method and device, electronic equipment and a storage medium, and the method comprises the steps: adding a layout state identifier for target data in an identified computational graph, and processing the computational graph based on the layout state identifier to obtain a target computational graph, the layout state identifier is used for indicating that the target data is converted into a reference data format during operation; judging whether a data format corresponding to the target data is matched with a reference data format or not according to the layout state identifier in the execution process of the target calculation graph; skipping a data layout conversion operation on the target data under the condition of determining that the data format is matched with the reference data format; according to the data layout optimization method and device, the reference data format of the target data is indicated through the layout state identifier, the format matching judgment is performed, the data layout conversion operation is skipped when the target data format is matched with the reference data format, the execution frequency of repeated data layout conversion operation is reduced, and the overall execution efficiency of the data layout optimization process is improved.

The invention belongs to the technical field of artificial intelligence, can be applied to financial science and technology or medical health related business scenes, and relates to an image generation method based on depth perception circulation attention and related equipment thereof. The image generation model comprises an image preprocessing layer, a depth feature coding layer, a depth feature layout processing layer and a visual rendering layer. The depth feature layout processing layer is adopted to carry out hierarchical attention calculation on depth level feature embedded representation, and a data layout result is generated according to a hierarchical attention calculation result, so that image information of different depth levels is more fully integrated; and high robustness and high accuracy of the image finally generated by the image generation model are ensured.

A device includes processing circuitry and memory, the memory including multiple dies. Each die accommodates multiple memory units. Each memory unit corresponds to a first read size and to a respective memory plane. Each respective memory plane spans a die of the multiple dies. An ordered sequence of memory units is arranged across the multiple dies based on a second read size and is further arranged across the respective memory planes according to a third read size. The third read size may be greater than the second read size, which may be greater than the first read size. Processing circuitry may simultaneously read data corresponding to the third read size based on reading data from multiple memory planes and from multiple dies of the multiple dies.

Methods and memory modules for performing in-memory computation are provided. The memory module includes a memory die including a plurality of dynamic random access memory (DRAM) banks, each DRAM bank including an array of DRAM cells arranged in pages, a row buffer storing a value of one of the pages, an input / output (IO) module, and an in-memory computation (IMC) module including an arithmetic logic unit (ALU) receiving operands from the row buffer or the IO module and computing an output based on the operands and one of a plurality of ALU operations, and a result register storing the output of the ALU, and a controller receiving operands and instructions from a host processor, determining a data layout based on the instructions, supplying the operands to the DRAM banks according to the data layout, and controlling the IMC module to perform one of the ALU operations on the operands according to the instructions.