120 results about "Distributed memory" patented technology

The invention discloses a large model training fault recovery method based on distributed memory management, and the method comprises the steps: obtaining a model state at a current moment when judging that a check point storage service needs to be carried out in a current training round, serializing the obtained model state into a storable format, and generating check point data; then transmitting the check point data to a host side memory from a device side memory in the node; after the transmission is finished, continuing to train the task, and storing asynchronous check points at the same time; the asynchronous check point storage comprises data distributed memory backup and data persistence to a disk; and when the training task goes wrong, task recovery is carried out through check point data. When the fault is recovered, the backup data is preferentially read from the memory, and the delay caused by the I / O bottleneck of the disk can be obviously reduced, so that the training pause or the efficiency reduction caused by the operation of storing or loading the check point data is reduced, the efficient utilization of the GPU or other computing resources is kept, and the continuity and the stability of large-scale model training are improved.

This application generally relates to memory provisioning and particularly describes systems and methods for distributed memory pools. For example, a computing / memory device in a distributed system may be configured to execute application logics using local cache for data maintained in non-local memory in another computing / memory device of the distributed system. The computing / memory device may further be configured to provide memory for other applications executed on other computing / memory devices of the distributed system. The local and non-local memory allocation for the applications in the computing / memory devices of may be dynamically and adaptively adjusted according to various system parameters and indicators.

The invention provides a concurrent debugging method and system based on sandbox-LLM (Language Model) session bidirectional binding. The method comprises the following steps: distributing a global unique session identifier for each LLM session; writing the globally unique session identifier and the sandbox instance identifier into a distributed memory database to form a bidirectional binding table, and marking the initial state as a binding state; realizing unvarnished transmission of a full-link global unique session identifier through an environment variable and an HTTP / gRPC header field; during sandbox thermal migration, the old instance atomically updates a binding table entry state into a migration state through a distributed mutual exclusion lock, and the new instance is written into the binding state after being preheated within a preset time; the log agent identifies an aggregation index according to a sandbox instance, collects average generation delay according to a preset frequency, accumulates a data collection number and GPU video memory occupation; and periodically polling the binding table, and cleaning entries which exceed a preset threshold value of the distributed mutual exclusion lock and are in a recovery state.

The invention relates to a remote sensing image data processing method and system based on a Spark architecture, and belongs to the technical field of remote sensing image data processing. The method comprises the following steps: converting a collected original file into a COG format, constructing a hybrid index through a spatial dimension reduction technology, extracting metadata and establishing a metadatabase; based on the hybrid index, quickly positioning a target tile through spatial query, reading a data block, and constructing a TileLayerRDD of a spatial partition in a Spark cluster; and dynamically assembling and calculating the directed acyclic graph according to the algorithm warehouse template, and submitting the directed acyclic graph to a Spark cluster for distributed memory calculation. According to the method, rapid retrieval is achieved through the COG format and the mixed index, pre-slicing is replaced with Spark distributed computing, the storage pressure is greatly reduced, and on-demand real-time analysis and high-concurrency online service of mass remote sensing data are finally achieved through micro-service and elastic resource scheduling.

A computer-implemented method implemented in a blockchain network is described. Validation nodes receive data regarding a newly mined block comprising a plurality of transactions and send a delete request to a distributed memory pool to delete the plurality of transactions from the distributed memory pool. Nodes storing the distributed memory pool store a plurality of transactions, the plurality of transactions forming at least part of a distributed memory pool of transactions waiting to be mined into a block of a blockchain. The computer-implemented method further comprises receiving a delete request from a validation node of the blockchain network, the delete request identifying one or more transactions which have been included in a newly mined block, the delete request indicating that the one or more transactions should be deleted from the distributed memory pool.

Systems and techniques include identifying a network layer for performing a memory operation, identifying a subset of a plurality of configuration bit clusters of a non-volatile distributed memory that are mapped to the identified network layer using a cluster mapping, in response to identifying the subset of the plurality of configuration bit clusters, activating the subset of the plurality of configuration bit clusters, loading network component data from the subset of the plurality of configuration bit clusters into a local buffer, and applying the network component data to the network layer for performing the memory operation.

The invention discloses a memory access device, a processor and a memory access method, and belongs to the technical field of computers. The memory access device comprises a scheduling module which is configured to distribute memory access instructions in an instruction queue to a plurality of assembly lines of the memory access device; the first cache module is configured to cache the first mapping relation; the control module is configured to query the page table and update the second mapping relation in the second cache module according to the memory access instruction of the virtual machine monitor if the memory access instruction currently processed by the plurality of assembly lines does not hit the first mapping relation and the memory access instruction is the memory access instruction of the virtual machine monitor; and the second cache module is configured to cache the second mapping relation. On the basis of not increasing the complexity of hardware design, the resource utilization rate in the execution process of the memory access instruction of the virtual machine monitor can be improved.

The invention discloses a data processing method and device, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps: receiving real-time data streams sent by a plurality of clients through a target server cluster; wherein the target server cluster comprises an asynchronous socket server based on a non-blocking network, and the target server cluster realizes event-driven network data processing based on a reactor mode; according to the real-time load of each partition in a target message queue, distributing the received real-time data flow to the corresponding partition in the target message queue; cleaning the real-time data flow of each partition in the target message queue by using a target rule engine with a hot loading function to obtain cleaned data; and storing the cleaned data to a distributed memory database cluster. Therefore, resource tension and response delay caused by connection congestion are reduced, and data consistency and service continuity are guaranteed.

The invention discloses a database distributed memory management method and device, equipment and a storage medium, and relates to the field of databases, and the method comprises the following steps: in a target database, constructing a distributed memory pooling framework by taking a database instance as a node, and under the distributed memory pooling framework, creating a mapping table by utilizing a memory mapping technology; accessing the shared cache page based on the mapping table among the nodes, and dynamically adjusting the distribution of the private cache page through a preset adaptive load monitoring algorithm and a preset load weight determination formula; and when a data write-in request of the node is obtained, controlling the request flow based on a token bucket algorithm, and executing a write operation on to-be-modified data located in the shared cache page in the node by using a timestamp-based optimistic lock mechanism. Therefore, the management efficiency of the distributed memory in the database can be improved.

Intelligent secure methods, processes, systems, and apparatus are disclosed for controlling and monitoring volatile memories in third-party vendor systems, such as an ATM or PoS, providing a solution to track and integrate memory components into the monitoring system of a customer's financial institution using scanning procedures, homomorphic encryption, and blockchain transactions to establish trust and ensure confidentiality by orchestrating the coupling of distributed memory storage located on the third-party vendor systems and / or hardware by leveraging PCCO (Parity Check Control Object) and DIP (Dependency Inversion Protocol).

The invention belongs to the technical field of stream computing data processing, and particularly discloses a stream computing data processing system based on kappa architecture construction, which comprises an event acquisition unit, an event processing unit, an event storage unit, an event computing unit and a management platform, through cooperation of the event collection unit, the event processing unit, the event storage unit and the event calculation unit, collection and cleaning of events are completed through a kafka + flink component, real-time calculation of the events is completed through a streaming big data calculation engine, real-time index storage is performed through a distributed memory technology, and real-time calculation of the events is completed. The use of components is ensured through a distributed cluster mechanism, and real-time calculation data is timely returned to a batch library, so that a Kappa architecture can be applied to data platform establishment, the streaming data value is fully mined, the business operation efficiency and the risk monitoring timeliness are effectively improved, and by integrating the streaming data and the batch data, the risk monitoring efficiency is improved. And the batch processing operation efficiency of the data platform is improved.

The invention relates to a bus track offset storage and query method based on a distributed memory index, and belongs to the technical field of track semantic storage and track search. The method comprises the following steps: firstly, acquiring urban line information and bus GPS track data, and respectively calculating grid sequences corresponding to a bus predetermined line and a bus actual GPS track; and further calculating the similarity between the actual GPS track of each bus and the predetermined route, and obtaining the route offset information of each bus. All the offset information is re-partitioned according to a time-grid mode by using an RDD elastic distributed data set of the Spark, and a global index is formed. In each partition, multi-level division is carried out according to attributes such as dates, lines and license plate numbers, and an auxiliary filtering vector is established in each level, so that a local index is formed. And finally, defining a spatio-temporal query interface, and supporting multi-query condition combination for query. According to the method, line deviation retrieval with semantic information can be quickly and efficiently executed in a super-large-scale GPS data set.

Systems and techniques include identifying a network layer for performing a memory operation, identifying a subset of a plurality of configuration bit clusters of a non-volatile distributed memory that are mapped to the identified network layer using a cluster mapping, in response to identifying the subset of the plurality of configuration bit clusters, activating the subset of the plurality of configuration bit clusters, loading network component data from the subset of the plurality of configuration bit clusters into a local buffer, and applying the network component data to the network layer for performing the memory operation.

The invention discloses a method, a system and a medium for expanding and accelerating communication between microkernel processes by using memory labels, and the method comprises the following steps: distributing memories for application programs and system services, and labeling the distributed memories; realizing an isolated memory domain and an execution domain based on the label; inter-process communication is carried out in a library operating system or kernel module form by utilizing memory mark extension characteristics; and cross-service interaction is realized through the fast channel. According to the invention, the delay of inter-process communication of the microkernel is obviously reduced, and meanwhile, effective isolation and security are ensured.

The invention belongs to the technical field of chip algorithms, and particularly relates to a circular array beam forming method and system based on a domestic FPGA. Comprising the following steps: receiving multiple paths of array element signals, and performing frequency mixing and filtering to obtain baseband I / Q data; the base band I / Q data is unpacked to be distributed and stored in a plurality of distributed memories in an FPGA (Field Programmable Gate Array) processing unit; the method comprises the following steps: reading baseband I / Q data of a plurality of channels and corresponding phase shift coefficients in parallel, and carrying out multiplication, accumulation, summation and modulo operation to generate data of a beam point; repeating the calculation step of generating the beam point data by updating the phase shift coefficient so as to generate beam domain data; reordering the calculated sequence of the beam domain data, and adjusting the output sequence of the beam domain data; the method has the advantages of being high in processing speed, high in anti-noise capacity and the like on the basis of meeting domestic substitution.

The invention provides a two-dimensional code production method and system, electronic equipment and a medium, and relates to the technical field of two-dimensional code production, and the method comprises the steps: obtaining a character string constraint, and obtaining a character string set through a random character string generation plan under the character string constraint; a MySQL horizontal segmentation strategy is introduced to distribute and store the character string set to a distributed memory, and the distributed memory comprises a first memory and a second memory; writing the first character string group in the first memory and the second character string group in the second memory into a MySQL database in parallel; and calling a two-dimensional code generation library to convert the target character string in the MySQL database into a target two-dimensional code. According to the method and the device, the problem of low generation speed of an existing traditional two-dimensional code production scheme can be solved, and the technical effect of remarkably improving the generation and storage efficiency of the two-dimensional code is achieved.

An electric vehicle charging and discharging operation regulation platform, comprising: a data collection cluster, which is configured to receive and parse collected data sent by an external system, wherein the external system comprises multiple terminals, an Internet-of-Vehicles platform and a power grid dispatching platform; a data processing cluster, which is configured to perform processing analysis and periodic sampling storage on real-time remote signaling / telemetering data; a data storage cluster, which has a distributed memory database constructed therein, and is configured to implement the large-capacity storage of historical data and high-concurrency access to the historical data; and a micro-service cluster, which is configured to implement, on the basis of data stored in the data storage cluster, a service application micro-service and a platform micro-service by means of a micro-service architecture.

A high-speed parallel GRAND-CRC decoding method for functional safety communication is presented. M A round-robin noise error mode allocation scheme for each memory cell, without affecting error correction performance, provides... N The system adds an all-zero error mode to the set E of noise error modes and allocates noise error modes to each memory unit in turn. A high-speed parallel decoder architecture design scheme for GRAND-CRC is proposed, including a distributed memory design strategy and the advantage of unrestricted parallelism. The parallelism can be easily set according to hardware resources, and the decoder throughput increases linearly with the parallelism. The system uses a corrector as the basis for logical judgment, and the corrector signal processing unit triggers the correct decoding output. Its features include different processing methods in three cases, especially when contention occurs, selecting the guessed sequence corresponding to the corrector with the smallest sequence number as the decoding output.

A device for implementing sparse distributed memory may include first circuitry having a plurality of cells arranged in first subsets. The first circuitry may be configured to receive a first input vector for a write operation and calculate a similarity between the first subsets of the first circuitry and the first input vector. The device may include second circuitry coupled to the first circuitry and configured to output a first activation signal. The device may include third circuitry coupled to the second circuitry and having cells arranged in second subsets. The third circuitry may be configured to receive the first activation signal and selectively activate one or more first selected subsets of the second subsets in response to the first activation signal by incrementing or decrementing corresponding values of the cells of the one or more first selected subsets based on the first input vector.

The invention discloses a bidirectional replication consistency checking and repairing method based on a record source identifier, particularly relates to the technical field of data consistency of a distributed memory database, and is used for solving the problem that a bidirectional replication cluster is difficult to accurately identify a record source, missing and conflicts in a non-stop scene, so that the reliability of the bidirectional replication cluster is improved. Therefore, the problems of difficulty in consistency verification, high possibility of false difference and repeated modification are solved. A deletion mark, a lock mark and a source mark are coded in a record control area in a centralized manner, a primary key field and a source pseudo column are used as anchor points, and an idempotent identifier constructed based on a primary key and a version field, a fragment comparison process divided according to a verification window and a primary key range and an automatic repair decision oriented to missing records and records with inconsistent fields are combined; therefore, the real difference can still be accurately recognized under the condition of bidirectional replication continuous operation, replication loopback and false inconsistency are avoided, and the consistency of the business data of the memory database clusters on the two sides is kept on the premise that business writing is not stopped.

A system and method for configuring bits using distributed memory in an artificial neural network is provided. Systems and techniques include identifying a network layer for performing memory operations; identifying a subset of a plurality of configuration bit clusters of the volatile distributed memory, the configuration bit clusters of the subset being mapped to the identified network layer using a cluster mapping; in response to identifying the subset of the plurality of configuration bit clusters, initiating the subset of the plurality of configuration bit clusters; loading network component data from the subset of the plurality of configuration bit clusters into a local buffer; and applying the network component data to the network layer for performing the memory operation.

A system for administering regionally-based data storage and control is provided. The system may include a computer processor and a distributed memory. The distributed memory may be distributed across a plurality of regions. The distributed memory may provide data storage. The system may build a plurality of artificial intelligence (AI) models. Each of the models may correspond to a set of data rules. Each of the data rules governs the storage of data in each of the plurality of regions. The system may further create, or otherwise access, a plurality of data keys. Each of the data keys preferably corresponds to one of the regions. The system preferably centralizes the location of the plurality of data keys in a central location such that control of the data in the plurality of regions can be administered by the processor, via the plurality of data keys, from the central location.

The invention relates to the technical field of communication, and discloses a memory management method and device for data forwarding, equipment and a storage medium. The method comprises the following steps: dividing a memory pool into a memory common area and a memory expansion area; wherein the memory common area is of a fixed size, and the memory expansion area is of a dynamic size; when a message is forwarded, distributing memory resources of the memory common area or the memory expansion area to the outside according to a current memory resource usage amount pointer so as to map and store the message; wherein each memory resource distributed to the outside by the memory common area and the memory expansion area has a corresponding mark; and when forwarding of the message is finished, returning the memory resource storing the message to a region corresponding to a memory pool based on the mark corresponding to the memory resource. Through the method and the device, the technical problem that in the prior art, in order to ensure that each resource in the memory pool is sequential and continuous, only a complex algorithm can be added is solved.

The invention provides a system and a method for deploying a model based on a cluster. The resource utilization rate of the model after being deployed in the cluster can be improved. The system based on the cluster deployment model comprises a scheduling controller which is used for receiving a reasoning request, adjusting resource distribution of a multi-model shared container group and distributing the reasoning request to the multi-model shared container group to which a corresponding model belongs; the multi-model sharing container group is in communication connection with the dispatching controller, and the multi-model sharing container group is used for simultaneously operating a plurality of models of the same type; and the distributed memory is in communication connection with the multi-model sharing container group, and the distributed memory is used for storing model file data and model metadata of the multi-model sharing container group.

Multiple examples are disclosed of a multi-port SRAM system for a distributed memory pool and associated methods. In one example, a system comprises a memory bus switch coupled to a plurality of memory buses; a memory pool switch coupled to one or more memory arrays; and one or more SRAM blocks coupled to the memory bus switch and the memory pool switch to convert a first transaction received on one of the plurality of memory buses to a second transaction performed on the one or more memory arrays.

The invention discloses a method and system for improving RPC access efficiency and a storage medium, and relates to the technical field of remote procedure call, and the method for improving the RPC access efficiency comprises the following steps: a client and a server respectively distribute memories through a CPU, and generate handles for marking the memories; the client writes to-be-processed input data into a memory allocated by the client, sends an RPC control flow to the server by using a CPU, and exchanges handle information through a kernel module of a kernel layer to perform data transmission; the server accesses the input data based on the handle information to carry out heterogeneous calculation, writes a calculation result into a memory allocated by the server, sends an RPC control flow to the client by utilizing a CPU, and exchanges the handle information through a kernel module of a kernel layer to carry out data transmission; and the client accesses the calculation result based on the handle information. According to the method, the control flow and the data flow in the RPC access process are separated, the CPU only participates in transmission of the control flow data, and the transmission efficiency of effective data in the RPC access process is greatly improved.

The application provides a distributed memory pool memory batch allocation method and system. The method comprises the following steps: receiving a batch allocation request containing a number of slot positions, obtaining a CPU core identifier of a current thread; performing an atomic addition operation on a tail pointer of a corresponding logical shard in a single allocator according to the CPU core identifier, locking a continuous slot position interval, the single allocator being divided into a plurality of logical shards, each logical shard corresponding to a CPU core, each logical shard maintaining an independent tail pointer, the tail pointer being an atomic variable recording the highest bit offset of the allocated logical shard; updating the bit state of the corresponding slot position interval in the bitmap of the single allocator, completing the batch continuous slot position allocation and returning the continuous physical address corresponding to the slot position interval. The method realizes lock-free allocation to reduce the allocation delay under high concurrency, guarantees the global reuse of free slot positions, and maintains a high memory utilization rate.

The present application relates to a data sampling method based on a distributed memory database, which uses the distributed memory database as a filtering container, and the data filtering rule is a filtering condition. The filtering container attributes include distributed cluster servers, data cache size, and data cache policy. The filtering conditions include calculating a 128-bit HASH value based on a rule according to the MD5 algorithm and constructing a data storage memory database organization rule based on the HASH value, so as to quickly and accurately extract data according to custom rules in the face of large amounts of big data. It can meet the data sampling effect at various PB levels and can meet the effect of obtaining the required sampled result data in a short time without affecting the business efficiency during business use. Through sampling analysis of large amounts of data in various industries, the situation can be quickly grasped, various early warnings and judgments can be made in advance, and it has great significance in actual combat in various fields such as life production, event development, and disaster prediction.

The present application provides a MRAM chip, including one or more arrays, each array including row-column distributed memory cells and a plurality of word lines, a plurality of bit lines and a plurality of source lines connected with the memory cells, and each array further including a defect detection circuit for controlling the connection mode of the tested bit lines, comparing the leakage current between the tested bit lines with a reference current, or comparing the leakage voltage between the tested bit lines with a reference voltage, so as to detect whether there is a short circuit fault between each bit line in the array.

Consistent with the present disclosure, a switch is provided that has a memory which temporarily stores packets to be selectively directed toward one or more destination egress ports. A method and apparatus are disclosed for efficiently managing the memory to support high data rate throughput by parallel processing and distributing memory management tasks across multiple packet buffer manager circuits. In one example, the apparatus includes a memory system divided into portions, with each portion managed by individual packet buffer managers operating in parallel. The packet buffer managers maintain a first plurality of memory addresses as a dedicated free list for storing incoming packets. As the free list of a particular packet buffer manager drops below a certain threshold, a weighted randomizer circuit adjusts the allocation of memory addresses among other packet buffer managers based calculated weights, which are inversely related to the number of memory addresses available to receive packets.