70 results about "Data layout" patented technology

The present invention discloses an artificial intelligence-based adaptive big data storage and retrieval optimization system and method designed to intelligently manage and optimize large-scale distributed data environments. The system integrates data acquisition, distributed storage, metadata processing, adaptive learning, and retrieval optimization units configured to work collaboratively for continuous self-optimization. The invention employs deep reinforcement learning and predictive neural network techniques to dynamically analyze system telemetry, workload behavior, and data access patterns in real time, enabling proactive adjustment of data placement, caching, replication, and compression parameters across distributed nodes. The metadata processing framework utilizes graph-based dependency modeling to maintain semantic and contextual relationships among datasets, facilitating intelligent and context-aware data retrieval. The retrieval optimization unit interprets user queries semantically and computes the optimal retrieval route using latency prediction models and dynamic routing techniques.

The invention provides a graph data communication intensity mapping method and system based on optical transport network topology. According to the method, an adjacent matrix is established by extracting graph data structure information, vertex centrality and a clustering coefficient are calculated, and a graph data communication feature matrix is generated; oTN network equipment information and optical link topology are collected, link parameters and wavelength resource states are measured, and an OTN physical network feature matrix is obtained; establishing a double-layer mapping relation matrix through a resource mapping algorithm; integrating the communication frequency, the time delay sensitivity, the bandwidth requirement, the wavelength resource and the topological matching degree, and calculating a communication strength weight matrix; generating a candidate layout scheme, performing multi-dimensional evaluation, and determining an optimized data layout scheme; and dynamically adjusting the mapping relation by adopting an incremental updating algorithm. According to the invention, the efficient collaboration of the graph computing task and the optical transport network resource is realized, and the performance and resource utilization efficiency of the distributed graph computing system are obviously improved.

The invention discloses a solid state disk-based data transmission method and device, a medium and a product, and relates to the field of solid state disks. According to the method, a data layout description header is obtained from a host memory in a direct memory access mode according to a target memory initial address; analyzing the data layout description header through a flash translation layer of the solid state disk, identifying each data segment entry, and extracting length information and life cycle attribute tags of corresponding data segments from each data segment entry; based on the life cycle attribute tag, planning a corresponding physical flash memory page for each data segment as a target write address of the data segment; and based on the target write-in address, controlling the solid state disk to execute DMA operation segment by segment according to the sequence of data segment entries in the data layout description head until the write-in of the total data length is completed. By implementing the technical scheme provided by the invention, the data storage efficiency of the solid state disk is improved.

The invention provides a memory management method and a memory controller, and aims to solve the problem of storage mismatch caused by dynamic change of a data access mode. The method comprises the following steps: acquiring target data stored in a target entity page in a plurality of entity pages; determining a data attribute of the target data and an offset attribute of the target entity page; and if the data attribute of the target data is not matched with the offset attribute of the target entity page, selecting a matched entity page of which the offset attribute is matched with the data attribute from the plurality of entity pages, and moving the target data from the target entity page to the matched entity page. Through the technical scheme of the invention, the memory controller can actively and periodically identify and correct the attribute mismatch between the stored data and the physical position thereof, and the continuous dynamic optimization of the data layout is realized, so that the data reliability and the overall performance of the storage device in the long-term use process are improved.

This invention discloses a method, system, and storage medium for verifying instrument data records. Based on a received data verification command, each record in the instrument data record table is sequentially retrieved. The corresponding screen layout information is retrieved from the instrument information database according to the instrument model. Then, the display features in the screen image are identified and analyzed, and compared with the screen area shape and measurement data layout information stored in the instrument information database. If they are not similar, an instrument model entry error message is triggered. If they are similar, the identified instrument measurement parameters are compared with the recorded parameters in the data record. If they are not the same, a measurement parameter entry error message is triggered. By using screen image content to automatically verify and verify the instrument model and measurement parameters in electronic experimental records, the inefficiency and error-prone nature of existing manual verification methods are overcome.

The invention discloses an observable method and system for a unified log index and tracking data, and the method comprises the steps: collecting various types of original observation data, carrying out the unified formatting, constructing a column type storage structure, and carrying out the hierarchical storage; then optimizing data layout by utilizing column-level metadata and a query mode feature library, and implementing a multi-level index strategy; meanwhile, through data cleaning, structured enhancement and contextual information association, a causal atlas between heterogeneous observation data is established, and observation data with rich contexts is generated; and finally, a high-performance query processing system is constructed based on a unified query engine and a sub-linear algorithm, and progressive result return and visual analysis are realized. According to the method, the data retrieval efficiency and the analysis depth are remarkably improved, operation and maintenance personnel can more quickly position the root cause of the system problem, and the average time of fault diagnosis is effectively shortened.

The invention relates to the technical field of heterogeneous environment task migration, and particularly provides a heterogeneous environment task migration method and system and a medium. According to the method, a uniform data layout and a virtual address space structure are constructed to unify state representation modes among different instruction set architectures; the extensible executable file is expanded on the basis of a fat binary file, and a symbol table is split into a basic symbol table shared across instruction set architectures and an independent extended symbol table of each instruction set architecture; and executing task migration across the instruction set architecture by the runtime environment in combination with the basic symbol table and the extended symbol table based on the migration points and the trigger pages embedded in the construction process. According to the method, the mobility, the cross-platform compatibility and the system analyzability are balanced, and systematic support is provided for task migration in a heterogeneous environment.

The present disclosure proposes a communication method and a distributed system for distributed training of mixed expert models. The method comprises: collecting gating network routing decision information corresponding to each input data; generating mapping information for each input data to be transmitted according to the deployment information of the expert module and the collected gating network routing decision information, wherein the mapping information is used to indicate the reading position of the input data in the sending end buffer and the writing position in the receiving end buffer; and performing a communication operation based on the mapping information to send the input data to be transmitted to the corresponding expert module. By introducing the mapping information for indicating the address relationship of the input data in the memory in the communication operation, the data rearrangement is synchronized in the communication process, thereby realizing the cooperative execution of the communication path and the data layout. Further, a two-stage mapping structure across machines and within machines can be constructed to realize efficient training based on topology hierarchical communication.

The invention discloses a graph structure data layout optimization method for a Qt tree layering and dynamic force oriented graph, which comprises the following steps of: loading graph structure data and carrying out data preprocessing, converting original data into a structured data table containing node attributes and connection relationships, dividing node levels through a depth-first traversal algorithm, and carrying out layout optimization on the structured data table; and dynamically generating a fan-shaped initial layout according to the degrees of the nodes, carrying out layout iterative optimization calculation by adopting a force-oriented layout algorithm, determining the position information and topological connection relationship of each primitive after optimization, and carrying out visual implementation and interaction on the optimized graph structure data layout. According to the scheme of the invention, through cooperation of hierarchical and elastic constraint mechanisms, the readability of a graph containing circulation / multiple father nodes is improved, a dynamic updating strategy enables layout to maintain overall stability during interaction, the structure confusion degree is reduced, and pre-layout processing and a parameter dynamic attenuation strategy effectively improve the convergence speed of a force steering algorithm.

The application discloses a kind of dynamic resource adaptation multilayer cache architecture systems, it is related to computer system architecture and data storage cache technical field.The system includes: resource state perception module, for real-time acquisition system performance index data and load characteristics in running, performance index data includes the access delay of each cache level, cache hit rate, data access frequency, cache medium read-write speed and system concurrent connection quantity.The application is through resource state perception module comprehensive acquisition each cache level performance index and load characteristics, combined with the deep reinforcement learning algorithm of cache strategy decision module dynamically generates cache strategy parameter, solves the problem that static configuration is difficult to adapt complex load in prior art;Data layout optimization module is through hot data identification, hot and cold data asynchronous migration and cross-level data block size adaptation, accurately match the characteristics of heterogeneous cache medium, make up the defect that traditional architecture is not enough to perceive medium difference.

The application provides a convolution calculation method, an electronic device, a storage medium and a program product, and belongs to the technical field of artificial intelligence, and comprises the following steps: performing feature extraction on input information to obtain an input tensor; folding the input tensor along a target dimension direction to a batch dimension to obtain a plurality of batch target tensors; performing complete convolution calculation on each target tensor to obtain a first convolution result; and obtaining a convolution result of the input information according to the first convolution result; wherein the target dimension of the input tensor is greater than a first threshold under the limitation of a hardware size, the target dimension of the target tensor is less than or equal to the first threshold, and the target dimension is height or width. When the target dimension of the input tensor exceeds the limitation of the hardware size, the input tensor is folded along the target dimension to the batch dimension, the convolution calculation of the over-limit tensor can be realized based on the native hardware and the native data layout, the data layout does not need to be replaced, and the performance of the convolution calculation is improved.

Embodiments of the present disclosure relate to a neural processor circuit configured to switch between a width-last mode and a channel-last mode of input data for more efficient processing of tasks. A compiler may determine whether the neural processor circuit is likely to perform a task more efficiently by using the input data in a width-last format or the channel-last format and compiles instructions to enable or disable a transposer circuit in the neural processor circuit. When the neural processor circuit is in a mode that uses the channel-last format, the input data in the width-last format is transposed into transposed input data in the channel-last format before being fed into one or more neural engines of the neural processor circuit, and output data generated by the one or more neural engines are also transposed back into the width-last format.

The present invention provides a method for generating an object and a data layout in a digital twin. The invention may include the steps of: generating and placing an object according to a user's command; generating an endpoint for the object; requesting that the endpoint be connected to raw data of a real-world asset; defining a visualization of the raw data or custom data; generating and connecting a layout for the visualization; and displaying the result of the visualization of the raw data or custom data using the layout.

This application provides an inference optimization method, apparatus, and storage medium for a generative diffusion model. The optimization method includes one or more of the following: first optimization of the inference parts of the U-Net and VAE in the generative diffusion model through code recompilation; second optimization of the performance bottleneck region in the VAE through data layout transformation; and multi-level operator optimization of key computational operations in the generative diffusion model based on performance bottleneck analysis. Through the above technical solutions, this invention can improve the generative diffusion model on a heterogeneous acceleration platform, aiming to locate the performance bottleneck part in the model through performance profiling, improve the inference efficiency on the heterogeneous acceleration platform through data layout transformation, and systematically improve the model's inference performance by utilizing multi-level optimization schemes from the graph level, data level, to the operator level.

The application provides a non-privacy and privacy reasoning scheduling method based on a reconfigurable chip, which comprises the following steps: determining whether the privacy requirement of a neural network reasoning task is privacy reasoning; if yes, performing a first scheduling step; otherwise, performing a second scheduling step; in the first scheduling step, analyzing an arithmetic logic unit required by the privacy reasoning, combining a reconfigurable multiplier and an adder by a data layout converter to form the arithmetic logic unit, and reconfiguring an interconnection network of a processing unit array into a butterfly network or a SIMD data path to obtain a reconfigurable chip and perform a reasoning step; in the second scheduling step, setting a slice as a tensor mode by the data layout converter, independently and parallelly working all the slices, performing a multiply-accumulate operation by each slice, transmitting data between processing units PEs through horizontal and vertical links to form a pulsating data stream, forming a pulsating array by the processing unit array, obtaining the reconfigurable chip, and performing the reasoning step; and in the reasoning step, performing the neural network reasoning task by the reconfigurable chip to obtain a reasoning result.

The invention relates to the technical field of charging piles, in particular to a direct current charging pile safety management method and system, and the method comprises the following steps: determining the operation parameter information of a to-be-monitored direct current charging pile, and constructing a multi-dimensional data layout based on the operation parameter information; acquiring operation parameter information of the to-be-monitored direct current charging pile in different operation states, and drawing on the corresponding data layout based on the operation parameter information to obtain a plurality of track graph streams; setting a calling rule for the plurality of track graph streams; current operation parameter information of the charging pile is collected, a real-time graph flow formed by the current operation parameter information in the data layout is obtained, and the real-time graph flow and the multiple track graph flows are compared and analyzed based on the calling rule; identifying an information point position deviating from the standard track graph in the current real-time graph stream to obtain a deviation symptom point; the accuracy of abnormality diagnosis of the charging pile can be improved, and the effect of safety management of the charging pile is improved.

This application discloses a method, apparatus, device, and storage medium for generating SCADA configuration screens based on generative AI, relating to the field of industrial control technology. The method includes: collecting natural language design requirements using an intent understanding module and the perception layer in an AI Agent cognitive architecture; obtaining state acquisition data and integrating it before inputting it to the cognitive layer; the inference engine and large language model in the cognitive layer decompose the configuration design task of the integrated data, generating a configuration design operation instruction set and inputting it to the execution layer; the execution layer processes the operation instruction set to obtain each component to be laid out in the configuration screen and its configuration data; the layout engine determines the component layout result in the configuration canvas; the variable binding engine binds and maps each component to the real-time data points of the SCADA system; and the target configuration screen is generated by combining the Shell command security module and the security layered architecture to improve the efficiency of generating SCADA configuration screens.

The invention discloses a data layout optimization method and device, electronic equipment and a storage medium, and the method comprises the steps: adding a layout state identifier for target data in an identified computational graph, and processing the computational graph based on the layout state identifier to obtain a target computational graph, the layout state identifier is used for indicating that the target data is converted into a reference data format during operation; judging whether a data format corresponding to the target data is matched with a reference data format or not according to the layout state identifier in the execution process of the target calculation graph; skipping a data layout conversion operation on the target data under the condition of determining that the data format is matched with the reference data format; according to the data layout optimization method and device, the reference data format of the target data is indicated through the layout state identifier, the format matching judgment is performed, the data layout conversion operation is skipped when the target data format is matched with the reference data format, the execution frequency of repeated data layout conversion operation is reduced, and the overall execution efficiency of the data layout optimization process is improved.

A device includes processing circuitry and memory, the memory including multiple dies. Each die accommodates multiple memory units. Each memory unit corresponds to a first read size and to a respective memory plane. Each respective memory plane spans a die of the multiple dies. An ordered sequence of memory units is arranged across the multiple dies based on a second read size and is further arranged across the respective memory planes according to a third read size. The third read size may be greater than the second read size, which may be greater than the first read size. Processing circuitry may simultaneously read data corresponding to the third read size based on reading data from multiple memory planes and from multiple dies of the multiple dies.

The invention relates to the technical field of data access speed control, and discloses a control method and device for improving the data access speed of a DDR chip, and the method comprises the steps: obtaining a memory data fragment, reading, analyzing and generating a visual graph, and obtaining data repeatability and a spatial distribution diagram; calculating the similarity of fields in the graph, and screening out a high-repetition field concentration region; if the continuity of the area field address is lower than a threshold value, rearranging the field to a continuous memory block to form a data layout structure; extracting a target field position of the access request from the layout, judging whether the target field position is in a continuous memory block and planning an access path to obtain a path planning result; clustering and grouping access sequences of related fields in the path, optimizing a skipping and reading sequence, and determining a skipping and reading sequence; if the sequence has a dynamic change, updating the clustering group to obtain an adjusted access sequence; and loading the memory block data according to the sequence, and judging a loading time optimization state to obtain an optimized access response. The method can solve the problem of low access response speed.

The invention provides a non-privacy and privacy reasoning scheduling method based on a reconfigurable chip, which comprises the following steps of: judging whether a privacy demand of a neural network reasoning task is privacy reasoning, if so, executing a first scheduling step, otherwise, executing a second scheduling step; a first scheduling step: analyzing an arithmetic logic unit required by privacy reasoning, combining and reconstructing a multiplier and an adder by a data layout converter to form the arithmetic logic unit, reconstructing an interconnection network of a processing unit array into a butterfly network or an SIMD (Single Instruction Multiple Data) data path to obtain a reconstructed chip, and executing a reasoning step; in the second scheduling step, the data layout converter sets the slices into a tensor mode, all the slices independently work in parallel, each slice executes multiplication and addition accumulation operation, data are transmitted between processing units PE through horizontal and vertical links, a pulsation data stream is formed, a pulsation array is formed by a processing unit array, a reconstructed chip is obtained, and the reasoning step is executed; and a reasoning step: executing the neural network reasoning task by the reconstruction chip to obtain a reasoning result.

The invention discloses a data layout adaptive adjustment method of an inference model and a related device, and belongs to the technical field of power equipment operation and maintenance and edge computing. According to the method, fusible attributes of all operators are obtained from a TVM framework, an original model calculation graph is traversed in a post-order traversal mode, all node operators are recursively processed from output nodes, and an initial default input data layout is an original layout DL0. In the traversing process, according to the data layout of the new input parameters, the operator fusible attribute and the operation type, the continuous layout is dynamically decided or converted into the target layout through transposition and deformation operation, and a corresponding node operator is newly established. After traversal is finished, a new model calculation graph is constructed, and if the output node layout is DL1, the output node layout is converted into DL0. According to the method, the automatic adjustment of the data layout is realized, the adaptive program can be reused, the automatic deployment after online updating of the model is supported, the processing efficiency of the reasoning accelerator is improved, and the problems of complexity and poor reusability of traditional manual adjustment are solved.

The invention relates to the field of distributed computing, and particularly provides a dynamic task unloading method, device and equipment and a medium, the method runs on a three-layer architecture composed of cloud, edges and ends, and the method comprises the following steps: collecting system state information including computing resources, network resources, task attributes and data positions in real time; on the basis of system state information, a preset decision model is used for online reasoning, an optimal task unloading strategy is generated, and task delay, system energy consumption and the resource utilization rate are comprehensively considered in the optimization target of the optimal task unloading strategy; the total task completion time delay comprises data transmission time delay, and the total system energy consumption comprises energy consumption generated in the data transmission process; and in response to the unloading strategy, executing a data prefetching operation, and migrating the data required by the task to the target node in advance. According to the method, comprehensive optimization of task processing time delay, system overall energy consumption and multi-dimensional resource utilization rate is realized through cooperative scheduling of calculation tasks and data layout.

An object of a system according to an embodiment is to generate an optimal layout based on an image of a room desired by a user and real environment data.SOLUTION: A system according to an embodiment includes an image input unit, a real environment data input unit, a layout generation unit, and a product information output unit. The image input unit inputs an image of a room desired by a user. The real environment data input unit inputs real environment data of a target room. The layout generation unit generates a layout based on the data input by the image input unit and the real environment data input unit. The product information output unit outputs specific product information based on the layout generated by the layout generation unit.SELECTED DRAWING: Figure 1

The present disclosure discloses a data layout optimization method and device, an electronic device and a storage medium, comprising: adding a layout state identifier to target data in a recognized calculation graph, and processing the calculation graph based on the layout state identifier to obtain a target calculation graph, wherein the layout state identifier is used to indicate that the target data is converted into a reference data format at runtime; during execution of the target calculation graph, it is judged whether the data format corresponding to the target data matches the reference data format according to the layout state identifier; in the case where it is determined that the data format matches the reference data format, the data layout conversion operation of the target data is skipped; through the indication of the reference data format of the target data by the layout state identifier and the format matching judgment, the data layout conversion operation is skipped when the target data format matches the reference data format, the number of repeated data layout conversion operation executions is reduced, and the overall execution efficiency of the data layout optimization process is improved.

Disclosed herein are techniques related to load-aware selection of query execution databases. Techniques may include receiving a query to be executed on a database selected from a plurality of databases. Each database of the plurality of databases may have a different data layout. The techniques may also include generating, for the received query, one or more query features and one or more current load features. The one or more current load features may indicate availability of one or more computing resources for query execution on least one database of the plurality of databases. Additionally, the techniques may include generating an output based on applying a model to the one or more generated query features. The techniques may further include selecting, from the plurality of databases and based on the output, a database for executing the query. Thus, the query may be executed on the selected database.

The application discloses an executable file format setting method, a data stream program migration method and device, wherein the executable file format setting method creates a node program topology graph formed based on a starting address list and a topology information list of each data stream node in the executable file, and when an operating system participates in data scheduling, data layout and data optimization in a data stream execution mode, the topology information and the starting address of each data stream node can be quickly obtained by the operating system, and the starting address of each data stream node is aligned at a page boundary, so that the subsequent node dynamic migration is facilitated, and the system running speed is improved. The data stream program migration method is based on the node program topology graph (DAG graph), and the mapping and unmapping of virtual and real pages can be performed with reference, so that the system running speed can be significantly improved.

Tensor multiplication operations in a deep neural network (DNN) can be performed using mixed-data layout based on weight dequantization. A tensor multiplication operation can involve a weight tensor containing weights in a floating-point data point. The weight tensor can be quantized to create an integer weight block with a smaller storage size than the weight tensor itself. The integer weight tensor can be partitioned into weight blocks. A weight block can further be partitioned into weight groups based on a predefined group size. A weight group can be packed by right-shifting bits. The data layout within the packed weight group can be modified by shuffling the data elements in the weight group. The packed and mixed weight group can be dequantized to produce a floating-point weight block.The tensor multiplication operation can be performed on the floating-point weighting block.

A method for training a Boolean deep learning model (308, 408, 608), performed by a processor device (302, 402, 510, 602), is provided. The method involves receiving a request from a Boolean deep learning application (306, 406, 512, 606) to instantiate a multi-dimensional Boolean matrix array (414, 502) comprising binary values representing the model's weights or biases. A sequence of matrix arithmetic operations is then received, detailing tasks to be performed on this Boolean matrix array. For each operation in the sequence, the method determines whether the bit data's memory layout (310, 410, 520) needs reordering into a different bit vectorization dimension. If reordering is required, the method converts the current bit data layout accordingly. Following each conversion, the method updates the indicative bit vectorization parameters to reflect the new layout. This dynamic adjustment ensures optimal memory utilization and computational efficiency throughout the training process, improving the performance of the Boolean deep learning model.

The invention discloses a storage adaptive optimization method based on a lake and warehouse integrated data management system, and relates to the technical field of data processing. According to the system, an intelligent storage optimization module is introduced, so that self-adaptive collaborative optimization of a storage format and a data layout is realized. The module comprises a dynamic storage format conversion unit and a fine-grained data layout optimization unit which are used for realizing dynamic switching of row and column storage formats and subtable-level data organization optimization based on LSTM load prediction and access mode perception respectively. According to the method, the problems of low resource utilization rate and unbalanced performance caused by a static storage strategy in a traditional lake-cabin architecture are effectively solved, and the writing throughput and the query performance of the system in a hybrid load scene are remarkably improved.