Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

122 results about "Reference counting" patented technology

In computer science, reference counting is a programming technique of storing the number of references, pointers, or handles to a resource, such as an object, a block of memory, disk space, and others.

Data processing method and system based on cloud computing

The invention relates to the field of data processing, and discloses a cloud computing-based data processing method, which comprises the following steps of: constructing a reference count management micro-service cluster at a cloud end, receiving a reference count request carrying a count type through an API (Application Program Interface) gateway, and analyzing the reference count request into a structured operation instruction; reading a first state identifier of the referenced object and a second state identifier of the reference object from the distributed cache system according to the metadata snapshot; according to the method, through dynamic scheduling of a reference counting management micro-service cluster and a container arrangement engine, efficient allocation and scheduling of computing resources can be realized, maximization of resource utilization is ensured, resource waste is avoided, and high-speed query capability of a distributed cache system and a memory database is utilized; the state identification of the object can be retrieved at a millisecond level, the system response speed is improved, and the performance requirement under a high-concurrency scene is met.
Owner:SHENYANG WARD INTELLIGENT EQUIPMENT TECHNOLOGY CO LTD

Distributed storage access method initiated by GPU (Graphics Processing Unit) in AI (Artificial Intelligence) scene

The invention relates to a distributed storage access method initiated by a GPU (Graphics Processing Unit) in an AI (Artificial Intelligence) scene. The method comprises the following steps: S100: constructing a software cache module and an I / O (Input / Output) queue module in a GPU memory; s200, the GPU thread queries the target data through the software cache module, reads the target data if the target data is hit, and enters the next step if the target data is not hit; s300, when the I / O request is not hit, the repeated I / O requests initiated by multiple threads are automatically combined to generate a single effective I / O request, and the request is written into an I / O queue module; s400, the user mode distributed storage client monitors the written submission queue, reads the target data and then transmits the target data to the software cache module, and a completion queue of the I / O queue module is updated; and S500, the GPU thread polls the updated completion queue, and updates the reference count of the software cache module after reading the data. The main purpose of the invention is to realize high-throughput and low-delay data transmission without the participation of a CPU (Central Processing Unit).
Owner:KYLIN CORP

ZNS SSD-based repeated data deletion method and system

The invention discloses a repeated data deletion method and system based on ZNS SSD, and belongs to the field of repeated data deletion.The method comprises the steps that a fingerprint index table and a fingerprint updating cache are maintained, three-time opportunity repeated data deletion is achieved, and repeated data search is conducted in the fingerprint index table, the fingerprint updating cache and an address mapping table respectively; an address mapping table, an address reverse mapping table and a reference counting table are maintained, physical space garbage collection / defragmentation and write request merging are achieved, the address mapping table, the address reverse mapping table and the reference counting table conduct garbage collection with partition as granularity when free space is insufficient, and a fingerprint index table is not updated after garbage collection data migration; and instead, the mapping of fingerprints to migration-in physical pages is stored in a fingerprint updating cache, the physical pages are sequentially distributed in partitions by the fingerprint updating cache, normal data write requests and garbage collection data migration write requests are respectively cached, and the write requests which are continuous in sequence are merged and then issued. According to the method, the high-performance and low-cost efficient storage system can be constructed on the basis of the ZNS SSD at relatively low overhead.
Owner:HUAZHONG UNIV OF SCI & TECH

Reference-based backup

In some implementations, a computer system includes one or more processors and a non-transitory computer-readable medium storing programming for execution by the one or more processors. The programming includes instructions to receive a request to create a second air-gapped backup corresponding to a second transient backup. The second transient backup is subsequent to a first transient backup corresponding to a first air-gapped backup. The programming includes instructions to determine, in response to the request, whether the second transient backup matches the first transient backup, and to create, in response to determining that the second transient backup matches the first transient backup, a database entry for the second air-gapped backup that references the first air-gapped backup. The programming includes instructions to increment, in response to determining that the second transient backup matches the first transient backup, a reference counter for the first air-gapped backup.
Owner:HEWLETT PACKARD ENTERPRISE DEV LP

Method and system for realizing compatible file hard link and S3 shallow copy based on distributed key-value pair

The invention discloses a method and a system for realizing compatible file hard link and S3 shallow copy based on a distributed key-value pair, a file metadata key-value pair is used as a metadata storage main body of a file or a directory, a key consists of a file system ID (Identity), a parent directory ID and a file or directory name, and a value contains metadata information; a reverse index file handle key value pair is introduced to serve as a unique invariant identifier, a key is composed of a file system ID, an initial parent directory ID, an initial file or first N characters of a directory name and a file ID, and a value contains a current file metadata key and a reference count. When files or directories are created and deleted, the files or the directories are located at the same one-key value pair node in a large probability, and one-stage submission is supported to improve performance; hard linking and S3 shallow copying are achieved at the same time, independent file metadata key-value pairs and reverse index file handle key-value pairs are created for the hard linking and the S3 shallow copying, reverse index file handle keys of the target file are recorded by file metadata values of the file metadata key-value pairs, reference counts of the reverse index file handle values of the target file are updated, and performance and compatibility are both considered.
Owner:SANDSTONE DATA TECH CO LTD +1

Information storage sharing method and device, equipment and medium

The invention is suitable for the technical field of display, and particularly relates to an information storage sharing method and device, equipment and a medium. The method comprises the following steps: when detecting that a producer applies for a shared cache, determining an available shared buffer from a shared memory, returning the available shared buffer to the producer, after detecting that the available shared buffer is filled by the producer, acquiring the number of all consumers as a reference count, and storing the reference count in a count management memory, after a data request of any consumer is obtained, the available shared buffer area is returned to the consumer to read production data, one is subtracted from the reference count in the count management memory, all consumers are traversed, and the available shared buffer area is marked as idle when the reference count is zero. According to the method, one producer can correspond to a plurality of consumers, and one-time production can correspond to a plurality of consumers for synchronous consumption, so that more resources and memories are prevented from being occupied, data copying is not needed, and the distribution efficiency is higher.
Owner:SHENZHEN HOLLYLAND TECH CO LTD

Caching a memory descriptor for plural input / output requests

Caching a memory descriptor across input / output (I / O) requests. Based at least on receiving a request from a buffer consumer to register a buffer, a memory descriptor describing a layout of a set of physical memory pages assigned to the buffer is created. Creating the memory descriptor includes assigning and locking the set of physical memory pages to the memory descriptor. For each I / O request of a plurality of I / O requests on the buffer, at least a subset of the memory descriptor for use by the I / O request is returned, and a reference count is used to track the I / O request as pending on the memory descriptor. Later, the memory descriptor is destroyed, including unlocking the set of physical memory pages, based at least on the reference count indicating that there is no pending I / O request on the memory descriptor.
Owner:MICROSOFT TECHNOLOGY LICENSING LLC

Reference management method and garbage collection method

ActiveCN119088561Aprevent misidentificationAvoid nulling exceptionsResource allocationError detection/correctionSoftware engineeringDatabase
The invention discloses reference management and garbage collection of application data. The application data reference management method comprises the steps that when it is detected that target application data is referenced by a target variable, and the target variable belongs to a target variable type, a logic address of the target application data is written into a physical address of the target variable; and taking the logic address corresponding to the physical address of the target variable as a reference address, and writing the reference address into a reference address table. According to the technical scheme, the reference address table is used for managing application data reference which is not recorded in the reference array, the application data are prevented from being mistakenly regarded as junk data and deleted, null exception caused by reference of the inexistent application data is avoided, the reference address table records the reference address instead of the logic address of the application data, and the application data can be stored in the application data. Content in an application data reference counter or a reference address table is prevented from being synchronously modified when the reference relation is cleared, and application execution efficiency is guaranteed.
Owner:BEIJING TONGFANG MICROELECTRONICS

Adaptive tracking of data patterns for deduplication

In some implementations, a memory apparatus may receive a command to write data to volatile memory of the memory system. The memory apparatus may compare the data to one or more data patterns to identify whether the data matches a data pattern of the one or more data patterns. The memory apparatus may increment a reference counter associated with the data pattern based on identifying that the data matches the data pattern. The memory apparatus may write, based on identifying that a measure flag associated with the data pattern is set, the data to the volatile memory.
Owner:MICRON TECHNOLOGY INC

UNLOADING FROM INTERDEPENDENT SHARED LIBRARIES

This describes the unloading of shared resources. A shared library correlation table (SLCT) is generated in a simulated resource. During a close function, selecting an entry in the SLCT decrements the reference counter of the selected entry, checks the state of the selected entry based on the reference counter, and removes its associated container from memory if the state of the selected entry indicates that the container is not a shared resource, thus preventing segmentation errors.
Owner:INTERNATIONAL BUSINESS MACHINE CORPORATION

GPU dynamic memory resource management method and system based on time sequence historical analysis

The invention discloses a GPU dynamic memory resource management method and system based on time sequence historical analysis. According to the method, a CUDAAPI function during running of a deep learning model is intercepted, and an execution history table containing a kernel execution sequence, a memory block size and an address is generated. By comparing two latest execution history tables, calculating a consistency ratio to judge whether a memory access mode is stable or not, so that a memory prefetch strategy is started; and when the GPU memory is insufficient, the memory blocks are exchanged to the CPU memory based on an expelling strategy of reference counting, so that sufficient GPU memory can be available during kernel execution. The system comprises a memory extraction module, a prefetching module and a dynamic decision module, supports dynamic asynchronous memory exchange, and optimizes GPU memory utilization efficiency. The method is suitable for deep learning model training in a GPU memory resource limited environment, a larger-scale deep learning model and training batches can be supported on the GPU with the same configuration, memory waste is reduced, and operation performance is improved.
Owner:TIANJIN UNIVERSITY OF TECHNOLOGY

Read lock operation method, write lock operation method and system

The embodiments of the present application disclose a read lock operation method, a write lock operation method and a system. The read lock operation method includes: setting a private reference count corresponding to each core; when threads of different cores read the same data, performing read lock and read lock operations with the private reference counts corresponding to the different cores. Utilizing the embodiments of the present application, when threads of different cores read the same data, the private reference counts corresponding to the core can be independently operated. The private reference counts corresponding to these different cores do not need to be synchronized between the cores, so the execution efficiency is improved. Moreover, the scalability of the read lock is also improved, that is, no matter how many core threads concurrently add and read locks, the time for adding and reading locks will hardly increase, thereby improving the execution efficiency.
Owner:BEIJING OCEANBASE TECHNOLOGY CO LTD

Handling memory accounting when suspending and resuming virtual machines to / from volatile memory

Techniques that enable a hypervisor to (1) maintain shared memory pages and (2) handle memory accounting for VMs that are suspended to and resumed from the volatile memory of a host system are provided. Regarding (1), the hypervisor can maintain shared memory pages in volatile memory across the suspend-to-memory and resume-from-memory operations, without having to save their reference counts. Regarding (2), the hypervisor can keep track of the volatile memory reserved and consumed by VMs as they are suspended and resumed, without erroneously double counting that memory.
Owner:VMWARE INC

Data transfer system and image processing device

The embodiment of the invention provides a data transfer system and an image processing device. According to the data circulation system, the public cache module is arranged, so that the upstream module can call the cache blocks which are divided in advance from the public cache module to cache the target data. And the downstream module can search the cache block through the preset identifier issued by the upstream module to obtain the target data. By introducing a reference counting mechanism, write protection can be set for the called cache block, and the public cache module can release the cache block only when the reference times of the cache block return to zero from a non-zero number. Therefore, one-to-many transmission between the upstream module and the downstream module can be realized, the control logic of the target data acquisition process is reduced, and the data circulation efficiency is improved.
Owner:AXERA SEMICON (SHANGHAI) CO LTD

Request-level consistency control method and system for dynamic configuration hot update

The present disclosure relates to a request-level consistency control method and system for dynamic configuration hot update. The method comprises: organizing configuration content into multiple versioned configuration snapshots, establishing a corresponding reference count for each versioned configuration snapshot, and setting a candidate version; enhancing a configuration reading path in a service node by plugging so as to be able to perform a configuration reading operation according to a bound version; in response to receiving a service request, creating an activation token for the service request, obtaining a current candidate version as the bound version, writing the bound version into the activation token, and increasing the reference count of the bound version; during processing of the service request, when configuration needs to be read, reading a configuration value from a corresponding versioned configuration snapshot according to the bound version in the activation token; and in response to the processing of the service request being completed, decreasing the reference count of the bound version by one.
Owner:HEBEI HAPPY CONSUMPTION FINANCE CO LTD

System and method for expanding the lifetime of virtual entries in data storage systems

A method, computer program product, and computing system for expanding the lifetime of virtual entries in a data storage system. A deduplication function of a virtual layer of a data storage system is monitored, a reference count of a virtual entry is incremented when a data page is written to a physical layer of the data storage system and the reference count of the virtual entry is decremented when a data page is deleted from the virtual layer. The virtual entry in the virtual layer is maintained when the reference count reaches a predetermined value and the virtual entry is reclaimed when a predetermined action of the data storage system is to be performed.
Owner:DELL PROD LP

Data recovery method and electronic device

PendingCN123030138ATime delaysData Corruption
The application discloses a data recycling method and an electronic device, and relates to the technical field of storage, which comprises marking reference levels of data blocks according to reference counting and identifying sharing degrees of the data blocks; generating level distribution by counting the number and / or space occupation of the data blocks in each reference level in a group to obtain the sharing degree composition in the group; determining a comprehensive score based on the level distribution, space usage rate and running load, wherein the comprehensive score is positively correlated with the space usage rate, negatively correlated with the load, positively correlated with the proportion of low levels and negatively correlated with the proportion of high levels, so that the score of the group with low levels, tight space and low load is higher; and determining whether to recycle by comparing the comprehensive score with a score threshold, so that the data blocks with high scores are recycled preferentially, unnecessary migration of high sharing hotspot blocks is avoided, the probability of conflict between recycling and host access is reduced, the increase of time delay and the risk of data damage and loss are reduced, and the stability and data security of the storage system are improved.
Owner:INSPUR SUZHOU INTELLIGENT TECH CO LTD

Value log memory mapping management method and system supporting concurrent index construction

The invention discloses a value log memory mapping management method and system supporting concurrent index construction, and the method comprises the steps: carrying out the partitioning of a value log file, and obtaining a plurality of blocks with a fixed size; defining a block mapping array, wherein each element in the block mapping array records the mapping state of each block of the value log file; introducing a lock array corresponding to the block mapping array, wherein each lock in the lock array corresponds to the entry indexes of the plurality of blocks; when a task needs to read data of a certain offset of the value log file, calculating a value log block id according to the offset; searching whether a slot position corresponding to the block mapping array is empty or not according to the value log block id, and locking a corresponding position of the lock array only when the slot position is empty; after locking succeeds, file mapping operation is executed; when the block is not subjected to the offset, the data of the offset is not accessed within a period of time, that is, the state that the reference count of the block is 0 exceeds the preset duration, the timed cleaning mechanism can actively remove memory mapping at the corresponding position of the block mapping array, and related system resources are released.
Owner:JIANGSU DAMENG DATABASE CO LTD

An optimization method for large model systems

The present application is applicable to the field of artificial intelligence technology, and provides an optimization method for large model systems, including: in the process of target model reasoning, reference counting is performed on the data involved in the calculation in the target model to obtain a reference count value; model data with a reference count value less than a first preset value is determined as low-frequency reference data; low-quality corpus is determined based on the low-frequency reference data; low-quality corpus is eliminated from the pre-training corpus to obtain training corpus, and the training corpus is used to train the optimized model. This method determines the actual application of the model data by reference counting the model data involved in the FFN calculation in the model, and then determines the low-quality corpus based on the application of the model data, and eliminates the low-quality corpus, thereby reducing the model parameters obtained by training, reducing the model data storage and calculation costs, and optimizing the model while ensuring the reasoning service quality of the model.
Owner:GUANGDONG LEAPFIVE TECH CO LTD

Memory management using build identifications

An apparatus, a method, and computer program product are provided that track build identifications of a project to determine when their associated files should be deallocated by a garbage collection system. The method includes receiving, by a continuous integration server, a payload from a source control management (SCM) system. The method also includes detecting at least one build identification (ID) within the payload and analyzing metadata contained within the payload to determine an adjustment to the reference counter for the build ID. The method further includes adjusting, based on the analysis, the reference counter relating to the build ID in a reference table, and passing the table to a garbage collection system for memory management purposes of the continuous integration server. The garbage collection system can then delete files associated with the build IDs that have reference counters at or below zero.
Owner:INTERNATIONAL BUSINESS MACHINE CORPORATION

Transaction closed-loop processing method and system based on double-end collaborative hot update and containerization

The invention discloses a transaction closed-loop processing method and system based on double-end collaborative hot update and containerization, and the method comprises the steps: carrying out the encryption storage of a script based on a management end, and pushing the encrypted script to a server; after receiving the script based on the server side, executing decryption and real-time compiling, generating an executable instance and storing the executable instance into a pre-compilation instance pool; the instance is automatically released by the JVM when the reference count returns to zero; the server side is deployed in a Kubernetes cluster in a containerization manner, and the capacity is dynamically expanded and shrunk; the method comprises the following steps of: performing closed-loop processing on a service request by the same server instance: completing multi-step operation through a local database transaction; realizing cross-method state transfer by using JVM memory sharing; the call link depth is constantly a preset value and is directly referenced and executed through a memory. According to the method, the management end pushes the script to the server end, the server end compiles in real time, and Kubernetes containerization capacity expansion and contraction are combined, so that service request closed-loop processing is realized, distributed transaction coordination is eliminated through local transaction and memory sharing, and transaction time consumption is reduced.
Owner:ANHUI YACHUANG ELECTRONICS TECH CO LTD

Dynamic shared memory control method and device, medium and program product

The embodiment of the invention provides a dynamic shared memory control method and device, a medium and a program product, and relates to the technical field of databases. The method comprises the following steps: setting an initial reference count and an initial fixed mark of a dynamic shared memory block; updating the reference count and / or the fixed mark according to the association operation or the disassociation operation of the process on the dynamic shared memory block; under the condition that the reference count is a first preset value and the fixed mark is in a first state, the operation of releasing the dynamic shared memory block is triggered, the first preset value is a preset reference count threshold value used for triggering the memory release operation, and the first state is a preset fixed mark state used for triggering the memory release operation. Based on the method, high flexibility and safety can be taken into account, so that the dynamic shared memory block is flexibly and safely released, and the requirements in an actual application scene are met.
Owner:CETC JINCANG (BEIJING) TECH CO LTD

Incremental reference counting validation based on a copy-on-write indexing structure

A system can maintain a copy-on-write b+ tree for a data storage system that is configured to store data as chunks, and a mapping between tree page identifiers of the b+ tree and first in-use chunk identifiers. The system can perform reference count validation, comprising iterating over the mapping, wherein the iterating over the mapping comprises, based on a first tree page identifier of the tree page identifiers being present in the mapping, loading second in-use chunk identifiers that correspond to the first tree page identifier directly from the mapping; and based on the first tree page identifier of the tree page identifiers being absent, loading corresponding content, extracting third in-use chunk identifiers from object metadata of the content, and updating the mapping based on the third in-use chunk identifiers, to produce an updated mapping. The system can perform garbage collection of the data storage system based on the validation.
Owner:DELL PROD LP

Data storage system with automated correction of reference counts in addressing structure to avoid capacity leakage

A method in a data storage system includes foreground processes that produce incorrect-high reference counts greater than corresponding numbers of actual references to associated data instances (e.g., virtual large blocks), representing loss of storage capacity. A background process corrects a reference count for a data instance by (1) copying the data instance to a new data instance with an initial reference count of zero and making the new data instance unavailable for reclaiming, (2) scanning a set of referencing structures for all references to the data instance, and for each reference (i) replacing the reference with a new reference to the new data instance, and (ii) incrementing the reference count of the new data instance, and (3) upon completing the scanning, making the new data instance available for eventual reclaiming based on its reference count reaching zero. The process restores reference count accuracy and reduces loss / leakage of storage capacity.
Owner:DELL PROD LP

Efficient deduplication in a metadata delta log architecture

In at least one embodiment, destaging a write of a user data (UD) log may include: performing deduplication that determines content written by the write to a logical address LA is a duplicate of existing content; and establishing mapping information of a chain of metadata (MD) pages mapping LA to a physical address PA storing the existing content, wherein the chain includes a MD leaf page and a VLB (virtual layer block) page. An update of a MD log can update an indirect pointer (IDP) field of a MD leaf entry of the MD leaf page to reference a VLB entry of the VLB page where the VLB entry further includes PA. A second update of the MD log can increment a reference count of the VLB entry. The second update can be recorded blindly in the MD log without reading or accessing reference count during destaging the write.
Owner:DELL PROD LP

Memory management method and device, medium and product

The invention provides a memory management method and device, a medium and a product, and relates to the technical field of memory management. The memory management method comprises the following steps: acquiring memory management information corresponding to an application, wherein the memory management information comprises at least one of reference quantity information, object creation and destruction information, graphic element feature information and memory block information; and calling a corresponding memory management mode to optimize the memory according to the memory management information, wherein the memory management mode comprises at least one of a reference counting method, memory pool management, image compression and memory fragmentation. According to the embodiment of the invention, a new memory management mode is added in addition to the memory management mode of Avalia, and the corresponding memory management mode can be selected according to the acquired memory management information for memory optimization, so that the memory occupation of data is effectively reduced, the memory resource waste is reduced, the possibility that the application runs slowly and lags is reduced, and the application performance is improved. And the user experience is improved.
Owner:GUANGZHOU BAOLUN ELECTRONICS CO LTD

Elastic, ephemeral in-line deduplication service

A deduplication service can be provided to a storage domain from a services framework that expands and contracts to both meet service demand and to conform to resource management of a compute domain. The deduplication service maintains a fingerprint database and reference count data in compute domain resources, but persists these into the storage domain for use in the case of a failure or interruption of the deduplication service in the compute domain. The deduplication service responds to service requests from the storage domain with indications of paths in a user namespace and whether or not a piece of data had a fingerprint match in the fingerprint database. The indication of a match guides the storage domain to either store the piece of data into the storage backend or to reference another piece of data. The deduplication service uses the fingerprints to define paths for corresponding pieces of data.
Owner:NETAPP INC

Memory management method and device for vehicle-mounted communication service, equipment and medium

The invention discloses a memory management method and device for a vehicle-mounted communication service, equipment and a medium. According to the scheme, a state management module is deployed in a microprocessor unit; performing registration processing on the server and the client through the state management module to obtain a theme identification number; wherein the server and the client communicate through the shared memory; performing reference counting processing on the shared memory according to the theme identification number to obtain a counting result; and recycling the shared memory according to a counting result, and cancelling the shared memory. According to the scheme, the shared memory is recycled and managed by counting the global shared memory object references of the platform, so that the waste of platform memory resources can be reduced and the reliability of platform communication can be improved in the communication operation process of the vehicle-mounted platform, and the method can be widely applied to the technical field of software development.
Owner:CHINA FAW CO LTD

Aggregate file dynamic recovery method and device, electronic equipment and storage medium

The invention discloses an aggregation file dynamic recovery method and device, electronic equipment and a storage medium, and relates to a distributed storage system.According to the technical scheme, a reference counter is set for each aggregation file, the numerical value of the counter is obtained in real time, and the change frequency of the numerical value is calculated; therefore, the problems of high I / O pressure and response delay caused by disk persistence when a counter is frequently updated and resource waste caused by memory residence of a cold data counter can be solved, and the technical effects of reducing performance overhead and optimizing the resource utilization rate are achieved. When the target file is migrated from the source aggregation file to the target aggregation file, the metadata direction of the target file is updated through the atomization operation, and the source aggregation file counter is adjusted to decrease progressively and the target aggregation file counter is adjusted to increase progressively. The problems that the verification overhead is large and the consistency is difficult to guarantee due to the fact that a verification mechanism is complex during small file migration can be solved.
Owner:JINAN INSPUR DATA TECH CO LTD