2302 results about "File system" patented technology

A video streaming system including a video cache system having a cache database and a video storage device, and including a video server configured to communicate with a gateway system configured to store multiple video streams, communicate with a cloud backup system configured to store video in a file system, and communicate with a cloud server to receive a stream request, the stream request indicating requested video that comprises multiple video segments. The video server is further configured to receive the stream request and determine a playlist that indicates where on the video cache system, the cloud backup system, and the gateway system the video segments that are needed to fulfill the stream request are stored, and to communicate needed video segments to the video cache system in preparation for providing streaming video to fulfill the stream request.

The invention discloses a permanent incremental backup method based on an S3 bucket-level virtual snapshot, and relates to the technical field of data storage and backup, the method comprises the following steps: initializing a backup system and starting a backup agent program, an index management service and an S3 object storage service; complete backup is executed, and a VID and S3 bucket-level virtual snapshoot are generated; efficiently managing a mapping relation between a file path and a snapshot identifier through a B + tree and an inverted index by utilizing a MongoDB index service; incremental backup is executed, and file changes are accurately recognized through Hash comparison and modification time comparison; a storage position is optimized by adopting an incremental storage and deduplication storage mechanism and an IncDSI optimization algorithm; executing backup chain integration and cleaning; and performing data recovery based on the VID of the file and the directory. According to the method, the backup efficiency is remarkably improved, the storage overhead is reduced, the data recovery process is optimized, and the method is suitable for data backup and management of a large-scale file system and has important application value and prospect.

A video gateway system at a worksite is coupled to multiple cameras on a network, and backs-up video streams generated by the cameras to a backend cloud backup video storage system and a frontend (cache) video storage system. The video gateway system generates an aggregated video asset from a plurality of streams of video from the multiple cameras, and generates metadata and a backup report associated with the video asset. The video asset, metadata, and backup report are stored on the backend cloud backup video storage system in an file system, and are also stored on the frontend (cache) video storage system such that for at least a period of time, the video asset and the associated the metadata and backup report are stored on both the cloud backup video storage system, facilitating quick access and retrieval to the stored video for retrieval for streaming, activity detection, and other uses.

Techniques are described for enabling concurrent cross-region replications and garbage collection while maintaining consistency and data integrity among file systems. In some embodiments, techniques for garbage collection fencing utilize a system-level garbage fencing key (GC fencing key) and one or more job-level GC fencing keys in a source file system that perform one or more cross-region replications with one or more target file systems, one replication and one job-level GC fencing key per target file system. In some embodiments, one job-level GC fencing key in a source file system and one job-level GC fencing key in a source file system together provide garbage fencing for a cross-region replication. In certain embodiments, the metadata information in a GC fencing key can inform, instruct, or be used to configure garbage collectors to skip garbage collection for a range of snapshots in a file system.

Techniques are described for efficient replication and maintaining snapshot data consistency during file storage replication between file systems in different cloud infrastructure regions. In certain embodiments, provenance IDs are used to efficiently identify a starting point (e.g., a base snapshot) for a cross-region replication process, conserve cloud resources while reducing network and IO traffic.

Techniques are described for enabling concurrent and non-blocking replication object deletion during cross-region replications. In some embodiments, in a target file system, a target replication pipeline as part of a cross-region replication, and a deletion pipeline operate in parallel. The deletion pipeline deletes processed objects reaching the last pipeline stage of the target replication pipeline after each checkpoint in the target replication pipeline. In some embodiments, after a non-recoverable failure during the cross-region replication, the cross-region replication can be restarted from the beginning (i.e., fresh restart) without waiting for its unused objects in the Object Store to be deleted by utilizing a generation number associated with each object to delete the unused objects in a background process while allowing deleting processed objects as normal for the freshly restarted cross-region replication.

The invention provides a double-stream video conference privacy tracing method based on a block chain, and the method comprises the steps: building a tracing evidence obtaining model which comprises information uplink and tracing evidence obtaining; the information uploading comprises the following steps: in a video conference process, audio and video data and related conference information are uploaded to an interstellar file system for storage and verification after being subjected to encryption and signature processing through a threshold ring signature mechanism, and key evidence is uploaded to a block chain; the intelligent contract is used for automatically managing data storage; the traceability evidence obtaining comprises the following steps: after leaked picture information is obtained, detecting a traceability identifier in a video or a picture, and comparing the traceability identifier with information stored on a chain; a secret divulging block point is automatically tracked and positioned through an intelligent contract, and the real identity of a secret divulging party is decrypted. According to the method, tampering resistance, transparency, decentration and anonymity of chain information in the traceability evidence obtaining process are achieved, and automatic evidence obtaining and traceability operation is supported.

A method for implementing data loss prevention (DLP) includes: generating an asset lineage map from file system metadata; identifying, based on the asset lineage map, an input feature linked to the asset, a type of the asset, and a plurality of activities linked to the asset; obtaining a sensitivity score for the asset based on the input feature and the type of the asset; obtaining, based on the plurality of activities, a malicious score and a data loss score for the asset; determining a user level of a user; and initiating implementation of a first DLP policy for the user based on the user level, the malicious score, the data loss score, and the sensitivity score.

A snapshot of metadata of a file system on a source storage system is stored in one or more of storage devices of a target storage system. A request to access data stored in the file system is received. A portion of the metadata including identification information of the data is identified. The data stored in the file system is retrieved by providing the source storage system with the identification information of the data.

The invention discloses an intelligent resource scheduling method and system based on task characteristics and system state perception, and relates to an operating system kernel optimization technology. According to the method, resource demand types, importance labels and time sensitivity of tasks are captured in real time through an eBPF technology, system bottlenecks are recognized by combining / proc and sysfs file system data, a dynamic scheduling strategy is generated by adopting an improved Q-Learning model, and resource allocation is executed through a schemed * function and a cgroup v2 interface. The problems that a traditional scheduler is poor in self-adaptive capacity and resources are mismatched are solved, the actually-measured resource utilization rate is increased to 85%-92%, and the key task delay standard reaching rate gt is achieved; the method is suitable for edge computing, cloud servers and other high-concurrency scenes.

Embodiments are directed to a file system that include object stores. An object store for write requests may be provided. Write ahead log (WAL) entries that include data blocks may be generated. A WAL object may be generated based on the WAL entries and stored in the object store. An in-memory overlay may be updated to associate the data blocks with the WAL object. A checkpoint operation may be executed to: generate an index object that includes index entries that associate other data blocks with data objects stored in the object store; update the index object to include index entries that associate the data blocks with the WAL object; store the updated index object in the object store; update the in-memory overlay to remove the association of the data blocks and the WAL object and update the in memory WAL to remove records of successfully checkpointed WAL objects.

The invention relates to a notebook software vulnerability scanning method based on a security policy. According to the method, firstly, a security policy library of a target notebook is obtained, and the security policy library comprises access control rules, data operation limiting conditions and vulnerability detection reference requirements for different software types; collecting software running data of the target notebook computer, wherein the software running data comprises current running process information, file system operation records, network connection states and software configuration parameters; performing matching analysis on the software operation data and the security policy library to generate a preliminary scanning result; evaluating the risk level of the software vulnerability according to the preliminary scanning result; and finally, based on the risk level and the repair guide terms in the security policy library, generating software vulnerability repair guide information. Therefore, notebook software vulnerabilities can be scanned comprehensively and accurately, and effective repair guidance is provided.

Ransomware detection accuracy is improved by analyzing patterns of changes in filename extension counts, relative to each other, that occur in a file system over time. The disclosed approach is malware-agnostic and does not rely on recognizing malware extensions or on real-time monitoring of the target file system. Instead, during each successive backup job of the target file system, the disclosed technology counts different types of filename extensions and compares the counts to each other and to corresponding counts taken in earlier backup jobs. Preferably, the anomaly detection analysis uses machine learning to discern a behavior pattern of the file system, which indicates how filename extensions are distributed and how much they change between backup jobs over time.

The invention belongs to the field of data processing, particularly relates to a standardized processing method, system and equipment for original engineering parameters of a flight simulator, and aims to solve the problems of serious field limitation, low processing efficiency and poor adaptive capacity in the prior art. The method comprises the following steps: analyzing a predefined configuration file, and dynamically generating a database temporary table matched with an original data file structure according to field structure information and a parameter mapping rule defined in the configuration file; directly loading original engineering parameter files into the temporary table by adopting a batch loading mechanism based on a file system path, and executing multi-file parallel import based on a task queue; and converting calculation logic in the parameter mapping rule into an executable query expression, and directly exporting a standardized parameter file after calculation is executed through the temporary table. According to the method, field limitation is broken through, efficient processing of mass data is achieved, instant updating of service logic is achieved through configuration rules, and the maintenance cost is greatly reduced.

Systems and methods for generating user-specific textual and image-based outputs, in response to a user query, for provision of a matching item are disclosed herein. For example, the system may receive a query that includes a textual description. The system may retrieve a user profile for a user associated with the query. Based on the query and the user profile, the system may generate outputs and images using a machine learning model. Based on the outputs, the system may generate graphical representations. The system may receive a selection of a graphical representation. Based on the selection of the graphical representation, the system may enable access to an item that corresponds to the graphical representation.

In various examples, data storage is managed using a distributed storage management system that is resilient. Data blocks of a logical block device may be distributed across multiple nodes in a cluster. The logical block device may correspond to a file system volume associated with a file system instance deployed on a selected node within a distributed block layer of a distributed file system. Each data block may have a location in the cluster identified by a block identifier associated with each data block. Each data block may be replicated on at least one other node in the cluster. A metadata object corresponding to a logical block device that maps to the file system volume may be replicated on at least another node in the cluster. Each data block and the metadata object may be hosted on virtualized storage that is protected using redundant array independent disks (RAID).

Embodiments manage data in a file system over a network. A plurality of file system operations in the file system may be executed based on a file system client action or a file system administrative action such that the file system may be integrated with a network. Characteristics of the plurality network components in the network infrastructure that may be associated with the file system may be determined. Tests may be generated based on the characteristics of the network components such that the tests may be executed to evaluate the network components. Results of the tests may be employed to perform further actions, including determining non-compliant network components based on the results; modifying the network infrastructure based on the non-compliant network components such that one or more of file system operations are modified based on the non-compliant network devices; executing the modified file system operations on the modified network infrastructure.

Techniques are described for performing different types of restart operations for a file storage replication between a source file system and a target file system in different cloud infrastructure regions. In certain embodiments, the disclosed techniques perform a restart operation to terminate a current cross-region replication by synchronizing resource cleanup operations in the source file system and the target file system, respectively. In other embodiments, disclosed techniques perform a restart operation to allow a customer to reuse the source file system by identifying a restartable base snapshot in the source file system without dependency on the target file system.

The embodiment of the invention discloses an access method in a distributed file system, computing equipment and file access equipment. The method is applied to a first metadata node, the first metadata node belongs to the multiple metadata nodes, the same directory information is stored in each metadata node in the multiple metadata nodes, and the directory information is used for storing directory metadata of the distributed file system. The method comprises the following steps: receiving a first access request from a client, wherein the first access request comprises an object path to be accessed; determining first file fragment information corresponding to first metadata according to the directory information and the to-be-accessed object path; determining an operation result corresponding to the first metadata according to the first file fragment information and the fragment routing information of the first metadata node; and sending the operation result to the client.

The invention discloses a network data sharing method and system based on dual identity authentication, and the method comprises the steps: a user A logs in a client A through the dual authentication of a password and a digital certificate, and transmits an encrypted file data packet formed after an original file is encrypted to a server; the user A selects target data at the client A and appoints a shared user B, and the server updates a shared file list of the user A; the user B submits a data downloading request on the client B, and the server verifies the authority of the user B and sends the encrypted file data packet to the client B; the client B decrypts the encrypted file data packet to obtain an original file; the system comprises a client module, a storage module and a server module. According to the method, a dynamic key generation mechanism is adopted, the requirement for real-time change of the authority in a multi-user cooperation scene is met, the leakage risk of a preset key is avoided, and the conflict between user privacy and sharing convenience is effectively solved.

An illustrative storage system may detect a command to create a directory within the file system, determine that the command satisfies one or more criteria, execute the command by creating the directory within the file system, and automatically generate, based on the determining that the command satisfies the one or more criteria, a managed entity associated with the directory. In some situations, the managed entity may be usable by a storage management system to provide storage management functionality to a dataset of the directory.

The invention relates to a mirror image file processing method and device, and relates to the technical field of container mirror image generation. The method comprises the steps of obtaining target file system directory information of a target container in response to a mirror image file export request for the target container; the target file system directory information of the target container is used for representing file system directory information of a writable layer of the target container; comparing the target file system directory information of the basic mirror image with the target file system directory information of the target container to obtain a difference file of the target container relative to the basic mirror image; constructing a layer file of a new incremental layer based on the difference file; writing a layer file of the incremental layer into a mirror image layer file directory of the basic mirror image, and updating a list file and an index file of the basic mirror image to obtain a mirror image file of the target container; the manifest file comprises metadata of each mirror image layer in the basic mirror image, and the index file comprises metadata of the manifest file. By adopting the method, the export efficiency of the container mirror image file can be improved.

The invention provides a method, a system and equipment for improving the computing performance stability of a hardware platform and a medium, and belongs to the technical field of computer system performance optimization. The method comprises the following steps: acquiring architecture characteristic data of a bottom hardware platform through a hardware detection tool, and generating and loading a first-stage system kernel parameter configuration based on the architecture characteristic data; reading the architecture characteristic data, carrying out behavior analysis and type identification on the running process based on the architecture characteristic data, and generating a second-level process scheduling strategy according to the behavior analysis and type identification; reading cache structure information in the architecture characteristic data, and driving a file system to intelligently prefetch and reconstruct a storage layout of memory data based on the cache structure information; and continuously collecting performance index flow data when the system runs, performing real-time analysis on the performance index flow data by using a performance degradation model, generating a feedback control instruction according to an analysis result, and dynamically correcting the kernel parameter configuration of the first-stage system and the scheduling strategy of the second-stage process.

An illustrative method includes detecting a log-in request associated with a managed directory within the file system, determining that the log-in request satisfies one or more criteria, and automatically generating, based on the determining that the log-in request satisfies the one or more criteria, a managed home directory within the managed directory. In some embodiments, the managed home directory may be usable by a storage management system to provide user-specific storage management functionality to a dataset of the directory.

Embodiments manage data in a file system. A plurality of file system operations may be executed in the file system based on a file system client action or a file system administrative action such that the file system may be integrated with an infrastructure of the network that includes network devices. Messages associated with some of network devices may be employed to perform one or more actions, including: determining one or more file system operations of the plurality of file system operations that may be associated with the network devices based on the messages; modifying the infrastructure associated with the network devices in response to the messages or the one or more file system operations; modifying the one or more file system operations in response to the modified infrastructure; executing the one or more modified file system operations; or the like.

The invention relates to the technical field of chip development, and discloses a transplanting method and device of a swan gap system, a storage medium and computer equipment, and the method comprises the steps: constructing a swan gap system to be transplanted based on a target chip, obtaining a corresponding source code, and then configuring a compiling configuration file in the source code, the compiling subsystem is used for establishing a compiling environment based on configuration information in the compiling configuration file, performing transplantation adaptation on a kernel layer in the compiling environment to generate a kernel mirror image, configuring the starting recovery subsystem, generating a root file system mirror image based on the file system configuration file, and starting the starting recovery subsystem. And finally burning the kernel mirror image and the root file system mirror image to a target chip, and performing function verification and test on the target chip. According to the method disclosed by the present invention, the swan-gap system transplantation including kernel layer transplantation, subsystem adaptation and root file system customization is designed and realized, so that the swan-gap system is successfully adapted to the target chip, and the accelerator equipment carrying the target chip can stably and reliably operate.

Embodiments are directed to a file system that include object stores. An object store for write requests may be provided. Write ahead log (WAL) entries that include data blocks may be generated. A WAL object may be generated based on the WAL entries and stored in the object store. An in-memory overlay may be updated to associate the data blocks with the WAL object. A checkpoint operation may be executed to: generate an index object that includes index entries that associate other data blocks with data objects stored in the object store; update the index object to include index entries that associate the data blocks with the WAL object; store the updated index object in the object store; update the in-memory overlay to remove the association of the data blocks and the WAL object and update the in memory WAL to remove records of successfully checkpointed WAL objects.

A system can include a memory device comprising a plurality of dynamic capacity devices and a processing device, operatively coupled with the memory device. The processing device is configured to perform operations including sending, to a memory device, an allocation request to allocate, for a file system, a first memory section of a plurality of memory sections of a plurality of dynamic capacity devices associated with the memory device; reserving, for the file system, a first portion of the first memory section for storing file metadata of file system data; reserving, for the file system, a second portion of the first memory section for storing file data of file system data; sending, by a first process running on the host system, to the memory device, the file metadata for storing the file metadata in the first portion of the first memory section; sending, by a second process running on the host system, to the memory device, a second request to modify the file metadata stored in the first portion of the first memory section; and receiving, from the memory device, a second response indicating an error regarding the second request.

Examples of file analytics systems are described that may obtain metadata data from a virtualized file server using a snapshot of the file virtualized file. The analytics system may mount a snapshot to retrieve metadata used in determining an overall size, structure, owner, permissions, and / or storage locations of parts of a file system managed by the virtualized file server. The analytics tool may communicate directly with each of file server virtual machines of the virtualized file server during the metadata collection process to retrieve the respective portions of the metadata. During the metadata collection process, the analytics system may add a checkpoint or marker (e.g., index) after every completed metadata transaction to indicate where it left off accessing the metadata snapshots. The checkpoint may allow the analytics system to return to the checkpoint to resume the process should the process be interrupted for some reason.

The invention discloses a power distribution terminal upgrading method, system and device based on an SD card and a medium. The method specifically comprises the steps of generating an incremental package of a Linux kernel and / or a root file system, calculating a hash value file of the incremental package, and generating a block chain certificate with a timestamp; encrypting and packaging the incremental package, the hash value file and the block chain certificate into a container mirror image, performing digital signature to form an upgrade package, and writing the upgrade package into an SD card; after it is detected that the SD card is inserted, mounting a boot partition, analyzing metadata of an upgrade package, and setting an upgrade flag bit and an upgrade operation type of Boot; and performing digital signature verification, hash value file verification and block chain certificate verification on the upgrade package in the SD card, if the verification is passed, backing up user parameters to the encrypted storage area, and restarting to enter the Recovery OS to complete replacement upgrade of the Linux kernel and / or the root file system. According to the method, safe, efficient and flexible upgrading of the operating system of the power distribution automation terminal is achieved, the rollback function is achieved, and the upgrading risk and cost are effectively reduced.