79 results about "File caching" patented technology

The invention relates to a method and a device for realizing synchronization of an oversized transaction to an openGauss database based on Debezium. The method comprises the following steps: configuring the size of a file cache slice; starting an incremental synchronization service, and obtaining a data change record (EVENT) by analyzing an Oracle source library log; dividing the EVENT into an independent file cache queue according to the transaction ID, and triggering dynamic segmentation (the storage capacity exceeds the standard or is overtime); when the service is abnormal, positioning a site through an index to realize breakpoint resume; after the transactions are submitted, the file cache queue is read in sequence and pushed to the downstream, and incomplete transactions are rolled back when the transactions are abnormal; events containing LOB / XML columns are combined into DML statements, and the DML statements are issued according to a primary key strategy; and converting the downstream system into a standard format for the consumer to subscribe. According to the scheme, memory caching is replaced by file caching, OOM is avoided, dynamic segmentation and breakpoint resume improve the large transaction processing capacity, the LOB / XML merging strategy guarantees the data consistency, general transaction optimization is supported, and the oversized transaction synchronization efficiency and stability are remarkably improved.

The invention provides a 5G NR broadcast terminal cross-process multi-service scheduling system, and belongs to the technical field of 5G communication, and the system comprises an application interface module which is responsible for interacting with other applications, providing a service list and processing a subscription request; the task scheduling module focuses on solving broadcast session opening conflicts and reasonably arranging a task execution sequence; the data transmission module is responsible for data transmission in different modes; the data storage module realizes task list and file cache management; and the chip adaptation module carries out packaging adaptation on SDKs of different chips. According to the invention, the terminal performance can be optimized, the user experience can be improved, and the advantages of the 5G NR broadcast technology can be fully exerted.

The embodiment of the invention discloses a file reading method and a related device, the method is applied to a computing node, the computing node is a computing node in a cluster system, the cluster system further comprises a storage system and a cache system, the cache system is used for caching files in the storage system, and the method comprises the following steps: obtaining a reading request, the read request is used for reading a first file in the storage system; obtaining a storage path of a first file in the read request in a storage system; determining whether the first file is stored in a cache system according to a cache mapping table, wherein the cache mapping table records a corresponding relationship between a storage path of the file cached in the cache system in the storage system and the storage path cached in the cache system; and when the first file is in the cache system, obtaining a storage path of the first file in the cache system from the cache mapping table, and reading the first file from the cache system. According to the method, the operation burden of the user can be relieved, and the file storage management efficiency and accuracy are improved.

Managing access to digital files in a computing environment, including facilitating protection of digital files; including facilitating the user experience for authentication, file encryption, file decryption, dynamic read access of protected files, dynamic write access of protected files, file cache management, and logging of access and updates to the protected files.

The invention relates to an edge data caching method and device, a medium and equipment, belongs to the technical field of edge storage, and solves the problems that a caching scheme is difficult to determine and the scheme result effect is poor in the existing edge data caching process. According to the technical scheme, the method mainly comprises the steps of obtaining file data, server data and user data; constructing a to-be-selected cache scheme space, wherein the to-be-selected cache scheme space comprises a plurality of to-be-selected cache schemes; maximizing the delay reduction amount of the caching scheme as a target function; according to a preset condition required to be met by the caching scheme, constructing a constraint through a relationship among the first binary variable, the file data, the server data, the user data and the request delay reduction amount; forming a penalty term according to the constraint, and adding the penalty term into the target function to form a quadratic unconstrained binary optimization model; and solving the secondary unconstrained binary model to determine the value of each first binary variable, and determining the file cache position according to the values of the first binary variables.

The application provides a method and system for realizing storage-computing separation through a distributed file cache system, comprising the following steps: S1, deploying a service end of a network file system in a bottom-layer storage system, and sharing a source data directory to an active area where a computing cluster is located; S2, deploying a distributed file cache system in the active area where the computing cluster is located, and mounting the shared source data directory in the distributed file cache system; and S3, selecting a submitted computing task node in the active area where the computing cluster is located, reading data, and writing the data into a working node. According to the application, the working node of the cluster can pull data from a remote storage and then locally cache the data to realize one-time reading and multiple-time use, thereby reducing the number of cross-city network access times.

The application relates to a same-layer resource processing method and device, computer equipment and a storage medium. The method comprises the following steps: when a same-layer component loading request is detected in the running process of an application program basic package, determining a same-layer component identifier corresponding to the same-layer component loading request, and installing a same-layer component extension package matched with the same-layer component identifier. A same-layer native library file matched with the same-layer component extension package is acquired, a file soft connection is constructed based on a basic library cache path of the application program basic package and a native file cache path of the same-layer native library file, the file soft connection is accessed, the same-layer native library file is acquired from the native file cache path by linking from the basic library cache path to the native file cache path, and the same-layer component loading logic is executed to load the same-layer native library file. By using the method, the same-layer native library file can be directly accessed and loaded by accessing the file soft connection in the running process of the application program basic package, and the same-layer extension function is provided to a use object.

The invention provides a data storage method, device and equipment, and relates to the technical field of cloud, the method is applied to a cloud platform, the cloud platform is deployed with a file system, the file system is used for caching training files, and the training files are used for training an artificial intelligence model. The method comprises the steps of determining a cold data time elimination condition according to latest access time corresponding to each training file cached in a file system and the data volume of the training files cached in the file system under the condition that the data volume of the training files cached in the file system reaches the upper limit of a data volume elimination threshold; querying a target training file of which the corresponding latest access time meets the cold data time elimination condition in the file system; and deleting the target training file in the file system. According to the invention, the storage space of the file system can be saved.

The application provides a video file caching method, an electronic device, a storage medium and a chip, relates to the technical field of data caching, and tags a video file according to the content of the video file, determines a label content score of each video file according to the number of video files with various labels, determines a play frequency score of the video file according to the play times of the video file, and finally obtains a probability score of video file play according to the label content score, the play frequency score and the corresponding weight of each video file, pre-caches the video file with the probability score greater than a score threshold. After the video file is pre-cached, the video frame of the video file can be quickly obtained from the cache space when the video file is played in frames in the future, so that the picture flow of frame play is smooth.

Disclosed in embodiments of the present application are a file reading method and a related device. The method is applied to a computing node, the computing node is a computing node in a cluster system, the cluster system further comprises a storage system and a cache system, and the cache system is used for caching files in the storage system. The method comprises: acquiring a read request, the read request being used for reading a first file in the storage system; acquiring a storage path of the first file in the storage system from the read request; on the basis of a cache mapping table, determining whether the first file is stored in the cache system, the cache mapping table recording the correspondence between storage paths of files cached in the cache system in the storage system and storage paths of said files in the cache system; and when the first file is in the cache system, acquiring the storage path of the first file in the cache system from the cache mapping table, and reading the first file from the cache system. The method can reduce the operation burden of users, and improve the efficiency and accuracy of file storage management.

The invention discloses a data multi-level cache collaborative acceleration method and system, and mainly relates to the technical field of data cache acceleration. Comprising the steps of obtaining data access features and inputting a model fusing sliding window statistics and exponential decay to calculate a dynamic heat value; dynamically scheduling the data to a high-speed memory, a distributed sharing or a local file cache level according to the heat value; optimizing a cross-level query path based on the intelligent routing model; the multi-level cache consistency is maintained by adopting an asynchronous version control mechanism; executing hierarchical collaborative cleaning and elastic capacity expansion according to the real-time popularity and the capacity utilization rate; and realizing hot data preloading by utilizing the prediction model. The method has the beneficial effects that through intelligent scheduling and collaborative optimization, the cache hit rate and the system throughput are remarkably improved, the access delay and the storage cost are reduced, and the stability and the data consistency under high concurrency are enhanced.

The embodiment of the invention provides a hotspot diffusion method and device and a readable storage medium, a cache node updates loads of multiple dimensions after receiving an access request every time, and determines whether the service capability of the cache node is sufficient or not according to the updated loads of the multiple dimensions. And when the service capability of the cache node is insufficient, the cache node determines a hot file from a plurality of cache files cached locally and diffuses the hot file. By adopting the scheme, whether hot spot diffusion is triggered or not is determined according to the service capability of the cache node, so that the aim of improving the accuracy and real-time performance of hot spot diffusion is fulfilled. Moreover, the hot files are diffused to more cache nodes, so that the diffusion nodes share pressure, the load of the cache nodes is reduced, differentiated configuration for different cache nodes and different cache files is not needed, manual configuration and intervention are reduced, the coupling degree between the cache client and the cache nodes is reduced, and maintenance is facilitated.

An aircraft recording system provides gapless asynchronous data recording. A data acquisition system is connected to a data ingestion interface, such that data acquisition system captures avionics data. An endpoint is associated with the aircraft data acquisition system and located remotely from the aircraft, which persists the plurality of events. An asynchronous messaging platform is provided that is adapted to digest and publish a plurality of packet payload words. A data file caching system is provided that is adapted to cache aircraft data files by ingesting data from the asynchronous messaging platform, writing the ingested data to a cache buffer, and based on a determination that a file descriptor is available, flushing the cache buffer to the file descriptor. A data file persistence system persists the aircraft data files by claiming the file descriptor, flushing the cache buffer to physical storage, and releasing the file descriptor.

The embodiment of the invention provides a file caching method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring a request for caching a file, wherein the request for acquiring the cache file comprises a target name of a first cache file and an offset of the first cache file; obtaining a target base address from the index table according to the target name of the first cache file; determining a target index tree and an address of a second cache file in the target index tree in a memory pool according to the offset of the first cache file and the target base address; and performing read-write operation on the second cache file according to the address of the second cache file in the memory pool in the target index tree. The method is used for achieving the effects of dynamically expanding the capacity of the memory pool and caching data for multi-node sharing.

The invention discloses a file caching performance improvement method and system based on distributed storage, and the method comprises the steps: firstly obtaining first file caching information of a to-be-cached file in a local caching node, the information comprises at least one of IO mode information generated through an improved virtual file system read-write function, cache page IO type marks and memory state information collected through a memory monitoring thread; inputting the first file cache information into a pre-trained first cache strategy model to obtain a file cache feature vector; the feature vector is sent to a remote storage node, the remote node inputs the feature vector and second file cache information of the to-be-cached file into a pre-trained second cache strategy model together, and a final cache instruction is obtained; and the local node executes cache strategy adjustment according to the instruction. According to the method, global collaborative intelligent optimization of the cache strategy is realized, and the cache hit rate and the system response performance are remarkably improved.

The present application relates to the field of cloud technology, and discloses a metadata access method and apparatus, a device, and a storage medium. According to the method, a global metadata cache scheme is used, and relates to a three-layer architecture, i.e., a global cache file, a cache description array, and a memory block; storage areas can be divided according to the data length of metadata in a shared memory; and when target metadata needs to be acquired for a target process, metadata can be queried layer by layer according to the three-layer architecture until target address information of target metadata that has been cached is obtained, and the target address information is returned to the target process for realizing the access of the target metadata. In this way, the metadata required by all the working processes can be accessed by means of the method, so that metadata caches of the working processes do not need to be respectively maintained by the working processes, thereby reducing the repeated metadata cache between the working processes, reducing the occupied memory resources, reducing the transmission resource occupation of the repeated metadata, and improving the resource utilization rate.

The embodiment of the invention provides a kernel integrity protection method and device, electronic equipment and a medium, and relates to the technical field of computers.The method comprises the steps that when a to-be-executed file is executed, an abstract list file of a target file package to which the to-be-executed file belongs is searched from a specified cache space, the abstract list file comprises an expected hash value of each file in the target file package; if the to-be-executed file is found, searching an expected hash value of the to-be-executed file from the abstract list file, and performing integrity verification on the to-be-executed file based on the expected hash value; if the abstract list file is not found, obtaining a storage path of the abstract list file of the target file package from the extension attribute of the to-be-executed file; caching the abstract list file to a specified cache space; and searching an expected hash value of the to-be-executed file from the abstract list file of the specified cache space, and performing integrity verification on the to-be-executed file based on the expected hash value. According to the scheme, the file verification efficiency can be improved, and the system performance is prevented from being influenced.

The invention provides a video file caching method, electronic equipment, a storage medium and a chip, and relates to the technical field of data caching, and the method comprises the steps: labeling video files according to the contents of the video files, and determining the label content score of each video file according to the number of the video files with each label; determining a playing frequency score of the video file according to the playing frequency of the video file; and finally, according to the label content score, the playing frequency score and the corresponding weight of each video file, obtaining a probability score of playing the video file, and pre-caching the video file of which the probability score of playing is greater than a score threshold. After the video file is pre-cached, the video frame of the video file can be quickly acquired from the cache space when the video file is subsequently subjected to frame playing, so that the frame playing picture is smooth.

The application discloses a kind of unmanned vehicle dynamic distribution-oriented auxiliary V2V cache network optimization methods, comprising: step one designs multi-unmanned auxiliary layered cache strategy, the file in the library is arranged in descending order from high to low according to popularity, and is successively divided into core layer, supplementary layer and low-frequency layer, the core layer file cache mode is determined cache for each vehicle, the supplementary layer file cache mode is vehicle probability cache, and the low-frequency layer file is not cached by vehicle;When vehicle sends file request, establish own cache, V2V cache, obtain request file from cloud server through unmanned vehicle three-level response;Step two analyzes cache performance average service delay;Step three jointly optimizes cache strategy and unmanned vehicle deployment;The application forms local priority, cooperation second, and efficient file distribution system of cloud supplement, and optimizes unmanned vehicle quantity, and balance is obtained between cost and performance.

The embodiment of the application relates to the technical field of communication, and discloses a response method, electronic equipment and a storage medium. The response method is applied to a CDN node, and the method comprises the following steps: receiving a user request, wherein the user request is used for requesting data of a target interval in a target file; in the case that the target file is locally cached and file holes exist in the target file cached locally, requesting target data corresponding to the file holes in the target interval from a source station, so as to respond to the user request according to the target data; and storing the target data in the file holes corresponding to the target file. At least the utilization of the local storage space and the performance of the cache system are improved, and the throughput is improved.

Systems and methods of persistent file protection integrate cloud-based backups with EDR file protection. Use of local persistent file cache (PFC) data with cloud-stored file segments, facilitates reconstruction of file states from multiple sources. File storage and retrieval efficiency are optimized, even when local session caches are invalidated or when extensive file modifications exceed storage quotas.

This application relates to a file caching method and device based on a content delivery network cache file system. The method is based on the content delivery network cache file system, which includes a logic layer and a storage layer. The storage layer includes multiple key-value pair storage systems. The file caching method includes: obtaining a source file to be cached; splitting the source file to be cached into multiple shard storage data based on the logic layer in the content delivery network cache file system, and storing each shard storage data in multiple key-value pair storage systems in the storage layer. The present invention enables the cache to be written and read quickly, speeds up the timeliness in the data transmission process, reduces the disk load, improves the network transmission speed, reduces the disk overhead cost, and alleviates the pressure on system maintenance by operation and maintenance.

The invention relates to an aircraft component test report analysis system based on an anomaly inference model, and relates to the technical field of component testing, and the aircraft component test report analysis system comprises a data analysis module which is configured with a file format analysis strategy to perform file format feature extraction on a test file, carrying out aircraft part test information analysis on the test file by matching a corresponding analysis step in the format analysis database so as to generate a test report; the file caching module is configured with a multi-stage data caching strategy, reads, writes and pre-caches the test file through a pre-constructed memory, file and database caching architecture, and keeps the data read-write efficiency and the system response speed within a preset good threshold range; and the exception analysis module is configured with an exception report identification model and is used for identifying and analyzing the test report through similar fault identification logic and a fault analysis engine so as to determine exception test information in the test report. The method has the effect of improving the analysis efficiency and accuracy of the test report of the aircraft component.

The invention discloses an operator execution method and device, equipment and a medium. The method comprises the steps that when it is detected that a new operator flow is created in a deep learning processor, an operator queue and a file cache matched with the new operator flow are determined, all operators are stored in the operator queue, and operator binary files are stored in the file cache; and when an operator flow calling request is obtained, detecting whether a target operator flow of the operator flow calling request meets a packaging execution condition or not, after the condition is determined to be met, packaging operators in the operator queue matched with the target operator flow to obtain an operator group, and sending the operator group to the driving module. According to the embodiment of the invention, after the operator flow is automatically determined to meet the packaging execution condition, the operators in the operator flow are packaged into one operator group, and the operators in the operator flow are sent to the driving module at one time, so that the interaction frequency of the driving module is effectively reduced, and the time consumption of the operator execution process is reduced.

The present application provides a long-term resource allocation method, device, and equipment for a caching system. The method includes: obtaining a file caching request of a user; determining whether there are unused resources in the current caching system; if not, determining whether the access times of the file with the lowest priority in the current caching system are greater than a set threshold; and if not, determining that the current caching system belongs to a first state and adopting an efficiency-based resource allocation method; otherwise, determining that the current caching system belongs to a second state and adopting a fairness-based resource allocation method. The long-term resource allocation method and device for the caching system of the present application support multi-user sharing of the cache to improve the caching efficiency, adopt different resource allocation methods according to different states of the current caching system, ensure fairness among users, and improve the allocation efficiency at the same time.

The embodiment of the invention discloses a file processing method and device, electronic equipment and a medium. The file processing method comprises the following steps: in response to a file acquisition request for a first file, querying file index information of the first file; the file index information comprises disk information of a plurality of first disks; the first disk is used for storing a first sub-file obtained after the first file is cut; responding to at least one damaged first disk in the plurality of first disks, and generating first information; the first information is used for representing that the first file fails to be acquired; and acquiring the first file from the source end of the first file based on the first information, and sending the first file to the terminal equipment corresponding to the file acquisition request. According to the method and the device, when the disk in the file cache system is damaged, the complete file can still be provided for the terminal equipment, and the influence of the disk damage condition on the file acquisition service of the terminal equipment is avoided.

The present application relates to a kind of on-chain off-chain cache optimization method of memory-computing separation block chain architecture, it is related to on-chain off-chain cache optimization technical field.The present application constructs the overall architecture of multistage cache including data cache and file cache, data cache area is divided into four sub-regions according to preset proportion, ledger data cache, state data cache, contract code cache and off-chain index cache;Special cache control strategy is designed for four types of on-chain data, ledger data, state data, contract code and off-chain file index: ledger data cache uses time locality replacement cache strategy control, state data cache uses lease-aware dual-hotness cache strategy, contract code cache uses size-aware execution frequency cache strategy, and off-chain index cache uses semantic metadata and hash check cache strategy;File cache uses dynamic hierarchical data block and full-amount file collaborative cache strategy to realize dynamic hierarchical cache mechanism;On the premise of guaranteeing data consistency, system access performance is improved.

The application provides a file downloading method, a file caching method and a server, and is applied to a server cluster. When the server cluster executes the file downloading method, if it is determined that no instance in the server cluster caches a file requested to be downloaded by a client, the downloading request is redirected to an instance that has cached the file, and the instance that has cached the file sends the file to the client. According to the application, only one copy of a file is cached in the server cluster, the disk space utilization of the server cluster can be improved, and storage resource waste can be avoided.

Disclosed in embodiments of the present application are a file reading method and a related device. The method is applied to a computing node, the computing node is a computing node in a cluster system, the cluster system further comprises a storage system and a cache system, and the cache system is used for caching files in the storage system. The method comprises: acquiring a read request, the read request being used for reading a first file in the storage system; acquiring a storage path of the first file in the storage system from the read request; on the basis of a cache mapping table, determining whether the first file is stored in the cache system, the cache mapping table recording the correspondence between storage paths of files cached in the cache system in the storage system and storage paths of said files in the cache system; and when the first file is in the cache system, acquiring the storage path of the first file in the cache system from the cache mapping table, and reading the first file from the cache system. The method can reduce the operation burden of users, and improve the efficiency and accuracy of file storage management.

Embodiments of the present application provide a high-concurrency network file processing system, method, computer device and storage medium. The system comprises a DPDK data plane module configured to receive network data packets from a network card transceiver queue and store the network data packets in an MBUF memory pool; a coroutine scheduling module configured to preprocess the network data packets based on an event loop and generate corresponding files; a task dispatching module configured to add a file writing task to a task queue and assign the task to a target file cache shard according to system load conditions; a file cache module configured to receive and cache data segments of the writing task and reorder the data segments according to offset information; and an asynchronous I / O module configured to submit the reordered data to a storage device interface in an asynchronous manner. The introduction of the DPDK enables user mode direct access to the network card transceiver queue, avoids the overhead of system calls and kernel protocol stacks, and significantly improves network I / O performance.