92 results about "File caching" patented technology

The present application relates to the field of cloud technology, and discloses a metadata access method and apparatus, a device, and a storage medium. According to the method, a global metadata cache scheme is used, and relates to a three-layer architecture, i.e., a global cache file, a cache description array, and a memory block; storage areas can be divided according to the data length of metadata in a shared memory; and when target metadata needs to be acquired for a target process, metadata can be queried layer by layer according to the three-layer architecture until target address information of target metadata that has been cached is obtained, and the target address information is returned to the target process for realizing the access of the target metadata. In this way, the metadata required by all the working processes can be accessed by means of the method, so that metadata caches of the working processes do not need to be respectively maintained by the working processes, thereby reducing the repeated metadata cache between the working processes, reducing the occupied memory resources, reducing the transmission resource occupation of the repeated metadata, and improving the resource utilization rate.

The invention relates to a method and a device for realizing synchronization of an oversized transaction to an openGauss database based on Debezium. The method comprises the following steps: configuring the size of a file cache slice; starting an incremental synchronization service, and obtaining a data change record (EVENT) by analyzing an Oracle source library log; dividing the EVENT into an independent file cache queue according to the transaction ID, and triggering dynamic segmentation (the storage capacity exceeds the standard or is overtime); when the service is abnormal, positioning a site through an index to realize breakpoint resume; after the transactions are submitted, the file cache queue is read in sequence and pushed to the downstream, and incomplete transactions are rolled back when the transactions are abnormal; events containing LOB / XML columns are combined into DML statements, and the DML statements are issued according to a primary key strategy; and converting the downstream system into a standard format for the consumer to subscribe. According to the scheme, memory caching is replaced by file caching, OOM is avoided, dynamic segmentation and breakpoint resume improve the large transaction processing capacity, the LOB / XML merging strategy guarantees the data consistency, general transaction optimization is supported, and the oversized transaction synchronization efficiency and stability are remarkably improved.

The invention provides a 5G NR broadcast terminal cross-process multi-service scheduling system, and belongs to the technical field of 5G communication, and the system comprises an application interface module which is responsible for interacting with other applications, providing a service list and processing a subscription request; the task scheduling module focuses on solving broadcast session opening conflicts and reasonably arranging a task execution sequence; the data transmission module is responsible for data transmission in different modes; the data storage module realizes task list and file cache management; and the chip adaptation module carries out packaging adaptation on SDKs of different chips. According to the invention, the terminal performance can be optimized, the user experience can be improved, and the advantages of the 5G NR broadcast technology can be fully exerted.

The embodiment of the invention discloses a file reading method and a related device, the method is applied to a computing node, the computing node is a computing node in a cluster system, the cluster system further comprises a storage system and a cache system, the cache system is used for caching files in the storage system, and the method comprises the following steps: obtaining a reading request, the read request is used for reading a first file in the storage system; obtaining a storage path of a first file in the read request in a storage system; determining whether the first file is stored in a cache system according to a cache mapping table, wherein the cache mapping table records a corresponding relationship between a storage path of the file cached in the cache system in the storage system and the storage path cached in the cache system; and when the first file is in the cache system, obtaining a storage path of the first file in the cache system from the cache mapping table, and reading the first file from the cache system. According to the method, the operation burden of the user can be relieved, and the file storage management efficiency and accuracy are improved.

Managing access to digital files in a computing environment, including facilitating protection of digital files; including facilitating the user experience for authentication, file encryption, file decryption, dynamic read access of protected files, dynamic write access of protected files, file cache management, and logging of access and updates to the protected files.

The invention discloses a batch log storage method. The method mainly comprises the following steps: writing normalized logs into a data bucket; refreshing the log data in the data bucket to a cache file; outputting the cache file to a database import file queue; and taking out the data file from the queue, and executing data import. According to the method, the execution process of log data from receiving to storage is modularized, different types of log data can be subjected to different resource configurations, and the storage efficiency is improved; a data bucket and a cache file are provided with a double-threshold mechanism, so that the real-time performance of log data storage is improved; tmpfs is adopted, a traditional log cache file caching mode is changed, and a database import file is rapidly generated from batch log data; therefore, the problems of low log data caching efficiency, difficulty in cache file management, low database data importing efficiency and the like are effectively solved.

The invention relates to an edge data caching method and device, a medium and equipment, belongs to the technical field of edge storage, and solves the problems that a caching scheme is difficult to determine and the scheme result effect is poor in the existing edge data caching process. According to the technical scheme, the method mainly comprises the steps of obtaining file data, server data and user data; constructing a to-be-selected cache scheme space, wherein the to-be-selected cache scheme space comprises a plurality of to-be-selected cache schemes; maximizing the delay reduction amount of the caching scheme as a target function; according to a preset condition required to be met by the caching scheme, constructing a constraint through a relationship among the first binary variable, the file data, the server data, the user data and the request delay reduction amount; forming a penalty term according to the constraint, and adding the penalty term into the target function to form a quadratic unconstrained binary optimization model; and solving the secondary unconstrained binary model to determine the value of each first binary variable, and determining the file cache position according to the values of the first binary variables.

The application provides a method and system for realizing storage-computing separation through a distributed file cache system, comprising the following steps: S1, deploying a service end of a network file system in a bottom-layer storage system, and sharing a source data directory to an active area where a computing cluster is located; S2, deploying a distributed file cache system in the active area where the computing cluster is located, and mounting the shared source data directory in the distributed file cache system; and S3, selecting a submitted computing task node in the active area where the computing cluster is located, reading data, and writing the data into a working node. According to the application, the working node of the cluster can pull data from a remote storage and then locally cache the data to realize one-time reading and multiple-time use, thereby reducing the number of cross-city network access times.

The application relates to a same-layer resource processing method and device, computer equipment and a storage medium. The method comprises the following steps: when a same-layer component loading request is detected in the running process of an application program basic package, determining a same-layer component identifier corresponding to the same-layer component loading request, and installing a same-layer component extension package matched with the same-layer component identifier. A same-layer native library file matched with the same-layer component extension package is acquired, a file soft connection is constructed based on a basic library cache path of the application program basic package and a native file cache path of the same-layer native library file, the file soft connection is accessed, the same-layer native library file is acquired from the native file cache path by linking from the basic library cache path to the native file cache path, and the same-layer component loading logic is executed to load the same-layer native library file. By using the method, the same-layer native library file can be directly accessed and loaded by accessing the file soft connection in the running process of the application program basic package, and the same-layer extension function is provided to a use object.

The invention provides a data storage method, device and equipment, and relates to the technical field of cloud, the method is applied to a cloud platform, the cloud platform is deployed with a file system, the file system is used for caching training files, and the training files are used for training an artificial intelligence model. The method comprises the steps of determining a cold data time elimination condition according to latest access time corresponding to each training file cached in a file system and the data volume of the training files cached in the file system under the condition that the data volume of the training files cached in the file system reaches the upper limit of a data volume elimination threshold; querying a target training file of which the corresponding latest access time meets the cold data time elimination condition in the file system; and deleting the target training file in the file system. According to the invention, the storage space of the file system can be saved.

The application provides a video file caching method, an electronic device, a storage medium and a chip, relates to the technical field of data caching, and tags a video file according to the content of the video file, determines a label content score of each video file according to the number of video files with various labels, determines a play frequency score of the video file according to the play times of the video file, and finally obtains a probability score of video file play according to the label content score, the play frequency score and the corresponding weight of each video file, pre-caches the video file with the probability score greater than a score threshold. After the video file is pre-cached, the video frame of the video file can be quickly obtained from the cache space when the video file is played in frames in the future, so that the picture flow of frame play is smooth.

Disclosed in embodiments of the present application are a file reading method and a related device. The method is applied to a computing node, the computing node is a computing node in a cluster system, the cluster system further comprises a storage system and a cache system, and the cache system is used for caching files in the storage system. The method comprises: acquiring a read request, the read request being used for reading a first file in the storage system; acquiring a storage path of the first file in the storage system from the read request; on the basis of a cache mapping table, determining whether the first file is stored in the cache system, the cache mapping table recording the correspondence between storage paths of files cached in the cache system in the storage system and storage paths of said files in the cache system; and when the first file is in the cache system, acquiring the storage path of the first file in the cache system from the cache mapping table, and reading the first file from the cache system. The method can reduce the operation burden of users, and improve the efficiency and accuracy of file storage management.

The invention discloses a data multi-level cache collaborative acceleration method and system, and mainly relates to the technical field of data cache acceleration. Comprising the steps of obtaining data access features and inputting a model fusing sliding window statistics and exponential decay to calculate a dynamic heat value; dynamically scheduling the data to a high-speed memory, a distributed sharing or a local file cache level according to the heat value; optimizing a cross-level query path based on the intelligent routing model; the multi-level cache consistency is maintained by adopting an asynchronous version control mechanism; executing hierarchical collaborative cleaning and elastic capacity expansion according to the real-time popularity and the capacity utilization rate; and realizing hot data preloading by utilizing the prediction model. The method has the beneficial effects that through intelligent scheduling and collaborative optimization, the cache hit rate and the system throughput are remarkably improved, the access delay and the storage cost are reduced, and the stability and the data consistency under high concurrency are enhanced.

The embodiment of the invention provides a hotspot diffusion method and device and a readable storage medium, a cache node updates loads of multiple dimensions after receiving an access request every time, and determines whether the service capability of the cache node is sufficient or not according to the updated loads of the multiple dimensions. And when the service capability of the cache node is insufficient, the cache node determines a hot file from a plurality of cache files cached locally and diffuses the hot file. By adopting the scheme, whether hot spot diffusion is triggered or not is determined according to the service capability of the cache node, so that the aim of improving the accuracy and real-time performance of hot spot diffusion is fulfilled. Moreover, the hot files are diffused to more cache nodes, so that the diffusion nodes share pressure, the load of the cache nodes is reduced, differentiated configuration for different cache nodes and different cache files is not needed, manual configuration and intervention are reduced, the coupling degree between the cache client and the cache nodes is reduced, and maintenance is facilitated.

The present invention relates to a method for improving the batch processing execution efficiency of spring-batch framework, which is based on file thread database cache interface, creates thread sharding file, then loads and writes the number of paging entries, splits a single large file or a large result set into small files or small result sets according to rules for subsequent processing; developers use business reader interface to write query / read logic according to business needs, loads pending data, and transfers pending data to processor layer; uses file cache entity interface to configure single cache number, reads corresponding row data from thread sharding file at one time, converts it into row data entity, and writes it into redis cache; uses file reading interface to configure basic information of reading file; creates spring-batch framework public class through Reader interface of springframework. Compared with the prior art, the present invention has the advantages of greatly improving batch execution efficiency, reducing business risks of large data volume and high data timeliness, etc.

An aircraft recording system provides gapless asynchronous data recording. A data acquisition system is connected to a data ingestion interface, such that data acquisition system captures avionics data. An endpoint is associated with the aircraft data acquisition system and located remotely from the aircraft, which persists the plurality of events. An asynchronous messaging platform is provided that is adapted to digest and publish a plurality of packet payload words. A data file caching system is provided that is adapted to cache aircraft data files by ingesting data from the asynchronous messaging platform, writing the ingested data to a cache buffer, and based on a determination that a file descriptor is available, flushing the cache buffer to the file descriptor. A data file persistence system persists the aircraft data files by claiming the file descriptor, flushing the cache buffer to physical storage, and releasing the file descriptor.

The embodiment of the invention provides a file caching method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring a request for caching a file, wherein the request for acquiring the cache file comprises a target name of a first cache file and an offset of the first cache file; obtaining a target base address from the index table according to the target name of the first cache file; determining a target index tree and an address of a second cache file in the target index tree in a memory pool according to the offset of the first cache file and the target base address; and performing read-write operation on the second cache file according to the address of the second cache file in the memory pool in the target index tree. The method is used for achieving the effects of dynamically expanding the capacity of the memory pool and caching data for multi-node sharing.

A content distribution server, including a central server and a plurality of edge servers. The central server may be configured to obtain popularity ranking information of the video files according to video file requesting history of user terminals, and obtain sections of multiple video files based on the popularity ranking information from a content server and transmit the sections to the edge server. The plurality of edge servers may include a file caching server and a mobile edge computing server of a user terminal, and each of the edge server(s) may be configured to cache sections of the multiple video files which are received from the central server, and send a video file requested by the user terminal to the user terminal.

The invention discloses a file caching performance improvement method and system based on distributed storage, and the method comprises the steps: firstly obtaining first file caching information of a to-be-cached file in a local caching node, the information comprises at least one of IO mode information generated through an improved virtual file system read-write function, cache page IO type marks and memory state information collected through a memory monitoring thread; inputting the first file cache information into a pre-trained first cache strategy model to obtain a file cache feature vector; the feature vector is sent to a remote storage node, the remote node inputs the feature vector and second file cache information of the to-be-cached file into a pre-trained second cache strategy model together, and a final cache instruction is obtained; and the local node executes cache strategy adjustment according to the instruction. According to the method, global collaborative intelligent optimization of the cache strategy is realized, and the cache hit rate and the system response performance are remarkably improved.

The embodiment of the invention discloses a cache updating method, a server, terminal equipment and a chip system, which are suitable for the technical field of computers, and the method comprises the following steps: the server receives a file request sent by the terminal equipment; when the file request comprises path information and a cache file identifier, obtaining a target file from a file path indicated by the path information, the cache file identifier being a file identifier of a target cache file cached in the terminal device; when the target file is not matched with the cache file identifier, a historical cache file of the target file is obtained from a server cache, and the historical cache file conforms to the target cache file; a difference file used for describing the difference between the target file and the historical cache file is determined, the difference file is sent to the terminal device, and the difference file is used for updating the target cache file in the terminal device. According to the embodiment of the invention, the file cached by the terminal equipment can be updated in time.

The present application relates to the field of cloud technology, and discloses a metadata access method and apparatus, a device, and a storage medium. According to the method, a global metadata cache scheme is used, and relates to a three-layer architecture, i.e., a global cache file, a cache description array, and a memory block; storage areas can be divided according to the data length of metadata in a shared memory; and when target metadata needs to be acquired for a target process, metadata can be queried layer by layer according to the three-layer architecture until target address information of target metadata that has been cached is obtained, and the target address information is returned to the target process for realizing the access of the target metadata. In this way, the metadata required by all the working processes can be accessed by means of the method, so that metadata caches of the working processes do not need to be respectively maintained by the working processes, thereby reducing the repeated metadata cache between the working processes, reducing the occupied memory resources, reducing the transmission resource occupation of the repeated metadata, and improving the resource utilization rate.

The embodiment of the invention provides a kernel integrity protection method and device, electronic equipment and a medium, and relates to the technical field of computers.The method comprises the steps that when a to-be-executed file is executed, an abstract list file of a target file package to which the to-be-executed file belongs is searched from a specified cache space, the abstract list file comprises an expected hash value of each file in the target file package; if the to-be-executed file is found, searching an expected hash value of the to-be-executed file from the abstract list file, and performing integrity verification on the to-be-executed file based on the expected hash value; if the abstract list file is not found, obtaining a storage path of the abstract list file of the target file package from the extension attribute of the to-be-executed file; caching the abstract list file to a specified cache space; and searching an expected hash value of the to-be-executed file from the abstract list file of the specified cache space, and performing integrity verification on the to-be-executed file based on the expected hash value. According to the scheme, the file verification efficiency can be improved, and the system performance is prevented from being influenced.

The invention provides a video file caching method, electronic equipment, a storage medium and a chip, and relates to the technical field of data caching, and the method comprises the steps: labeling video files according to the contents of the video files, and determining the label content score of each video file according to the number of the video files with each label; determining a playing frequency score of the video file according to the playing frequency of the video file; and finally, according to the label content score, the playing frequency score and the corresponding weight of each video file, obtaining a probability score of playing the video file, and pre-caching the video file of which the probability score of playing is greater than a score threshold. After the video file is pre-cached, the video frame of the video file can be quickly acquired from the cache space when the video file is subsequently subjected to frame playing, so that the frame playing picture is smooth.

The application discloses a kind of unmanned vehicle dynamic distribution-oriented auxiliary V2V cache network optimization methods, comprising: step one designs multi-unmanned auxiliary layered cache strategy, the file in the library is arranged in descending order from high to low according to popularity, and is successively divided into core layer, supplementary layer and low-frequency layer, the core layer file cache mode is determined cache for each vehicle, the supplementary layer file cache mode is vehicle probability cache, and the low-frequency layer file is not cached by vehicle;When vehicle sends file request, establish own cache, V2V cache, obtain request file from cloud server through unmanned vehicle three-level response;Step two analyzes cache performance average service delay;Step three jointly optimizes cache strategy and unmanned vehicle deployment;The application forms local priority, cooperation second, and efficient file distribution system of cloud supplement, and optimizes unmanned vehicle quantity, and balance is obtained between cost and performance.

The present invention discloses a method and system for improving the ejection speed of a USB flash drive under a Linux system. The method for improving the ejection speed of a USB flash drive under a Linux system includes counting the amount of written data and the usage rate of the USB flash drive per unit time, and predicting the maximum write speed of the USB flash drive according to the statistical data, and then determining a threshold value according to the predicted maximum write speed of the USB flash drive and the expected ejection time of the USB flash drive, and finally using the threshold value to limit the number of cache pages to be written back to the USB flash drive, and using it in a controllable state, thereby improving the ejection speed of the USB flash drive. The present invention aims to solve the problems of long ejection time of the USB flash drive and poor user experience that may exist under the current Linux file cache management strategy.

The embodiment of the application relates to the technical field of communication, and discloses a response method, electronic equipment and a storage medium. The response method is applied to a CDN node, and the method comprises the following steps: receiving a user request, wherein the user request is used for requesting data of a target interval in a target file; in the case that the target file is locally cached and file holes exist in the target file cached locally, requesting target data corresponding to the file holes in the target interval from a source station, so as to respond to the user request according to the target data; and storing the target data in the file holes corresponding to the target file. At least the utilization of the local storage space and the performance of the cache system are improved, and the throughput is improved.

Systems and methods of persistent file protection integrate cloud-based backups with EDR file protection. Use of local persistent file cache (PFC) data with cloud-stored file segments, facilitates reconstruction of file states from multiple sources. File storage and retrieval efficiency are optimized, even when local session caches are invalidated or when extensive file modifications exceed storage quotas.

This application relates to a file caching method and device based on a content delivery network cache file system. The method is based on the content delivery network cache file system, which includes a logic layer and a storage layer. The storage layer includes multiple key-value pair storage systems. The file caching method includes: obtaining a source file to be cached; splitting the source file to be cached into multiple shard storage data based on the logic layer in the content delivery network cache file system, and storing each shard storage data in multiple key-value pair storage systems in the storage layer. The present invention enables the cache to be written and read quickly, speeds up the timeliness in the data transmission process, reduces the disk load, improves the network transmission speed, reduces the disk overhead cost, and alleviates the pressure on system maintenance by operation and maintenance.

The invention relates to an aircraft component test report analysis system based on an anomaly inference model, and relates to the technical field of component testing, and the aircraft component test report analysis system comprises a data analysis module which is configured with a file format analysis strategy to perform file format feature extraction on a test file, carrying out aircraft part test information analysis on the test file by matching a corresponding analysis step in the format analysis database so as to generate a test report; the file caching module is configured with a multi-stage data caching strategy, reads, writes and pre-caches the test file through a pre-constructed memory, file and database caching architecture, and keeps the data read-write efficiency and the system response speed within a preset good threshold range; and the exception analysis module is configured with an exception report identification model and is used for identifying and analyzing the test report through similar fault identification logic and a fault analysis engine so as to determine exception test information in the test report. The method has the effect of improving the analysis efficiency and accuracy of the test report of the aircraft component.

The invention discloses an operator execution method and device, equipment and a medium. The method comprises the steps that when it is detected that a new operator flow is created in a deep learning processor, an operator queue and a file cache matched with the new operator flow are determined, all operators are stored in the operator queue, and operator binary files are stored in the file cache; and when an operator flow calling request is obtained, detecting whether a target operator flow of the operator flow calling request meets a packaging execution condition or not, after the condition is determined to be met, packaging operators in the operator queue matched with the target operator flow to obtain an operator group, and sending the operator group to the driving module. According to the embodiment of the invention, after the operator flow is automatically determined to meet the packaging execution condition, the operators in the operator flow are packaged into one operator group, and the operators in the operator flow are sent to the driving module at one time, so that the interaction frequency of the driving module is effectively reduced, and the time consumption of the operator execution process is reduced.