56 results about "File allocation" patented technology

The invention discloses a hybrid expert model deployment method and device based on a heterogeneous hardware equipment cluster, equipment and a medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: carrying out the data abstraction processing of all equipment in a target heterogeneous hardware equipment cluster, so as to obtain an initial equipment description vector, compiling the target expert model file into a binary intermediate representation; generating a hardware probe for the target equipment based on the hardware type of the target equipment in the target heterogeneous hardware equipment cluster and the binary intermediate representation, and obtaining a real-time equipment description vector based on hardware; determining a hardware sensing routing weight of each device in the target heterogeneous hardware device cluster based on the real-time device description vector, and determining a to-be-activated expert model file from the target expert model file based on the sensing routing weight; and distributing the to-be-activated expert model file to the target heterogeneous hardware equipment cluster for deployment and installation. Therefore, heterogeneous deployment of the hybrid expert model can be realized through unified arrangement.

A storage system includes a storage device and a host device. The storage device includes a non-volatile memory device configured to store data and a storage controller configured to control the non-volatile memory device to isolate and store the data by grouping storage spaces of the non-volatile memory device into a plurality of isolated storage spaces. The host device includes a file system configured to generate and manage a plurality of file allocation instances corresponding to a plurality of file allocation system calls from an application program by allocating, with respect to each of the plurality of file allocation system calls, a logical address range and an isolated identifier of a plurality of isolated identifiers corresponding to the plurality of isolated storage spaces.

Systems, apparatuses, methods, and computer program products are disclosed for linking remote file storage. An example method includes identifying, by discovery agent circuitry, a set of files for storage from a set of candidate files, and classifying, by file metadata circuitry, each file from the set of files for storage into file usage categories. The example method also includes assigning, by file metadata circuitry, permission categories to each file from the set of files for storage based on the file usage categories, and providing, by communications hardware, the set of files for storage and associated metadata to a file storage host, wherein the associated metadata comprises the file usage categories and the permission categories The example method also includes updating, by the discovery agent circuitry, content stored on the file storage host, the content comprising the set of files for storage.

The invention discloses a file processing method and device, equipment, a storage medium and a computer program product, and the method comprises the steps: obtaining the attribute information of a file in response to a file processing request, and determining the to-be-read data volume of the file; performing priority ranking on the files based on the attribute information and the to-be-read data volume to obtain a ranking result; distributing the file to a first queue or a second queue according to the sorting result, the attribute information and the current processing capability; wherein the first queue is configured to instantly process the file, and the second queue is configured to delay processing of the file. Through the design of a flow pool system and double-queue parallel processing, the system load can be effectively balanced, resource waste is avoided, and quick response of file processing is ensured. Besides, the automatic length adjustment of the processing queue enables the system to better adapt to the fluctuation of the business volume, and the system can keep stable operation, reduce the failure rate and improve the user experience no matter whether the peak or the valley is processed.

In one embodiment, an illustrative method herein comprises: obtaining, by a process, a file having a given size; assigning, by the process, the file to a particular size-range bucket of a plurality of size-range buckets of a data ingestion pipeline, the plurality of size-range buckets having a corresponding size-based configuration; and forwarding, by the process, the file into a particular size-based service mesh ingress gateway of the data ingestion pipeline according to the particular size-range bucket to cause processing of the file within the data ingestion pipeline according to the corresponding size-based configuration of the particular size-range bucket.

A dynamic data extraction, distillation, and enhancement system is disclosed that includes a dynamic extraction, distillation, and enhancement framework. The framework includes an allocator, extractor, and deconstructor stored in a non-transitory memory that, when executed by a processor, receive files in different formats from data sources, determine a native format of each file, identify and extract an embedded object from a file, deconstruct the file into components, assign each file to one of a plurality of streams based on the native format of the file, assign the embedded object to a stream based on a format of the embedded object, and assign a deconstructed component to a stream based on a format of the deconstructed component. The native format includes one of text, video, image, or audio. Each stream corresponds to one native format. The streams include a text stream, an audio stream, a video stream, and an image stream.

This invention discloses a file upload method, apparatus, device, and medium. The method includes: obtaining network bandwidth rate, storage space of a target file, and the number of available data nodes; calculating the storage capacity of each available data node based on the storage space of the target file and the number of available data nodes; splitting the target file according to the network bandwidth rate to obtain at least one target sub-file; the target file includes at least one target sub-file; allocating each target sub-file to each available data node according to the storage capacity of each available data node, obtaining a mapping relationship between each target sub-file and each available data node; and uploading each target sub-file to each available data node according to the mapping relationship. Embodiments of this invention can improve the efficiency of file transfer.

Various embodiments relate generally to data science and data analysis, and computer software and systems to provide an interface between repositories of disparate datasets and computing machine-based entities that seek access to the datasets, and, more specifically, to a computing and data storage platform that facilitates consolidation of one or more datasets, whereby data ingestion is performed to form data representing layered data files and data arrangements to facilitate, for example, interrelations among a system of networked collaborative datasets. In some examples, a method may include forming a first layer data file and a second layer data file, assigning addressable identifiers to uniquely identify units of data and data units to facilitate the linking of data, and implementing selectively one or more of a unit of data and a data unit as a function of a context of a data access request for a collaborative dataset.

To enable a user to more easily set a rule relating to folder allocation and file naming at the time of filing data, such as a document scanned image and a received facsimile, and storing the file. On a user interface screen for setting a rule relating to property information that is used at the time of allocating a file to a folder for storage, at least a first area for editing the rule and a second area displaying an item a user can select are provided. The item is a unit element for a user to set a character string configuring the property information and corresponds to a character string of an arbitrary attribute a user can set. Then, the rule is created by the item selected by a user from the second area being arranged at a position designated by the user in the first area.

Embodiments include translation of an integrated circuit design data into a production file. Aspects of the invention include identifying a top cell in the integrated circuit design data and a first cell-file associated with the top cell, assigning the first cell-file to a first translation module of a plurality of translation modules, and receiving, from the first translation module, an identification of child cells referenced by the first cell-file. Aspects also include assigning cell-files of the integrated circuit design data that correspond to each of the child cells to different translation modules of the plurality of translation modules, receiving, by a merger module, a single-cell production file from each of the plurality of translation modules, wherein each single-cell production file corresponds to a cell in the integrated circuit design data, and combining, by the merger module, the single-cell production files to create the production file.

The application discloses a storage method applied to an access device, and the method comprises the following steps: in the case that a storage request for a first file is received, an identification code is allocated to the first file; an index number of the first file is generated by using an existing parent directory of the first file and the identification code; an association between a name of the first file and the index number of the first file is stored; and the first file and the index number of the first file are sent to an index device. The application also discloses a device, a system and a computer storage medium, and the reliability of file storage is improved, and the storage efficiency of file storage is improved.

The present invention discloses a server and a method, device, and medium for controlling edit permissions for stored files. The method comprises: receiving an edit request from a client, parsing the edit request to obtain ownership information of the file, as well as a logged-in user ID and a logged-in client ID; the ownership information includes a current client ID and a current user ID; and assigning ownership to the file and granting or denying edit permissions to the logged-in user based on a comparison between the logged-in user ID and the current user ID, and between the logged-in client ID and the current client ID. The present invention avoids file "deadlock" issues and frees up server resources.

The invention provides a data analysis method. The data analysis method comprises the following steps that a list of files to be analyzed is obtained according to a file path, and a file queue is formed; distributing the files in the file queue to N threads determined by the number of processor cores to realize load balancing; searching packet headers and packet tails of the data packets from the distributed file data in parallel through N threads so as to identify and extract complete or partial data packets; and carrying out analysis and integrity verification on the extracted data packet, and writing the analyzed data into a database system. According to the data analysis method provided by the invention, through multi-thread parallel processing and load balanced distribution, the data analysis efficiency is remarkably improved, and the overall processing time is shortened.

To enable a user to more easily set a rule relating to folder allocation and file naming at the time of filing data, such as a document scanned image and a received facsimile, and storing the file. On a user interface screen for setting a rule relating to property information that is used at the time of allocating a file to a folder for storage, at least a first area for editing the rule and a second area displaying an item a user can select are provided. The item is a unit element for a user to set a character string configuring the property information and corresponds to a character string of an arbitrary attribute a user can set. Then, the rule is created by the item selected by a user from the second area being arranged at a position designated by the user in the first area.

The present application relates to a kind of on-chip system file data management method and system for prolonging the service life of memory.The storage space allocated by the present application for file is divided into file space and standby space, when file data needs to be modified, the relevant control information and data of each modification are written into standby space, the original file space is unchanged, until the margin of standby space is insufficient, then the corresponding storage block of file space is erased, the file data after multiple modifications is written, and the standby space is erased for next modification.The data stored in the corresponding storage block in file space is read first when reading file data, then the data in the specified reading range in standby space is used to construct the final reading data, or the specified record stored in standby space is directly read.The technical scheme of the present application can effectively reduce the erasing times of storage block, prolong the service life of storage block, and has power failure protection function in the case of large amount of local modification of file data.

The invention discloses a file uploading method and device, equipment and a medium. The method comprises the steps of obtaining a network bandwidth rate, a storage space of a target file and the number of available data nodes; calculating the storage capacity in each available data node according to the storage space of the target file and the number of the available data nodes; splitting the target file according to the network bandwidth rate to obtain at least one target sub-file; the target file comprises at least one target sub-file; according to the storage capacity in each available data node, distributing each target sub-file to each available data node to obtain a mapping relationship between each target sub-file and each available data node; and according to the mapping relationship, uploading each target sub-file to each available data node. According to the embodiment of the invention, the file transmission efficiency can be improved.

This invention discloses a storage system, read / write method, and controller for reducing file fragmentation, belonging to the field of data storage technology. The system includes a host and a solid-state drive (SSD). The host has a file system and a space pre-allocator, while the SSD has a segment allocator. The file system allocates logical address space to files according to segments. The segment allocator maintains a global translation table, which records the mapping relationship between segment IDs and mapped page addresses. The mapped pages are flash pages used to record the logical page addresses within a segment to the physical page addresses. The segment allocator also calculates free segment IDs in a monotonically increasing manner and publishes them to the circular buffer when there is remaining space. The space pre-allocator retrieves the free segment ID from the head of the circular buffer and allocates it to the file when space needs to be allocated. This invention can reduce the generation of file fragmentation and free space fragmentation in the file system, improving the overall performance and stability of the system.

The invention provides a file analysis system which comprises a file reading module used for obtaining the name and the size of a to-be-analyzed file according to a file path and putting the file into a file queue; the file distribution module is used for sorting the files in the file queue according to the sizes of the files, creating a plurality of data buffer areas and a plurality of threads according to the core number of the processor, and distributing the files to the threads by adopting a load balancing method; each analysis execution module runs in an independent thread, and comprises a file search unit used for identifying and extracting a complete or partial data packet from file data distributed to the thread by searching a packet header and a packet tail of the data packet; and the file analysis unit is used for carrying out analysis and integrity verification on the data packet extracted by the file search unit and writing the analyzed data into a database system. According to the system provided by the invention, multi-file parallel analysis can be realized, and the analysis efficiency and the system throughput are remarkably improved.

File system full and incremental replication to object storage systems (e.g., using a computerized tool), is enabled. For example, a system can comprise at least one processor, and at least one memory that stores executable instructions that, when executed by the at least one processor, facilitate performance of operations. The operations can comprise identifying a file system, wherein the file system comprises a group of files, based on an attribute applicable to the file system, determining a dataset identifier representative of the file system, for respective files represented in the group of files, allocating the dataset identifier and allocating respective file identifiers, and copying the group of files to an object store with respective object names corresponding to the respective files, wherein the copying comprises, for the respective files, assigning the dataset identifier to respective object names and assigning the respective file identifiers to the respective object names.

Detecting malicious / spoofed Universal Resource Locators (URLs) and / or files in received electronic mail (i.e., email) and, in response to detection, automatically taking responsive measures to prevent interaction / access to the malicious / spoofed URLs and / or files and / or limit, network-wide, the exposure to the malicious / spoofed URLs and / or files. Valid URLs and / or files are assigned a token, such as an alpha-numeric passcode or the like and the assigned token is verified upon receipt of the email by the intended recipient. In the event that the token cannot be verified or a token is not received for a URL or file, the automated responsive actions ensue.

The invention provides a file system processing method and system, a server, a storage medium and a program product. The method is suitable for a container migration scene, in a container storage stage, a source end reads an original device number of a mounting point of original tmpfs in a source container and an original index node number of a file, and transmits the original device number and the original index node number to a target end through a container management and control end; in the container recovery stage, the target end explicitly allocates the corresponding original device number to the created mounting point when creating the mounting point, so that the consistency of the device numbers before and after the tmpfs is stored and recovered can be ensured. When the file is created in the mounting point, the corresponding original index node number is explicitly allocated to the created file, so that the consistency of the file index node numbers before and after the tmpfs is stored and recovered can be ensured, the abnormality / fault caused by the inconsistency of the device number and the index node number before and after the tmpfs is migrated can be avoided, and the security and stability of the system are improved.

The invention belongs to the technical field of computer storage, and particularly relates to a ZNS SSD partition allocation optimization method based on machine learning. Comprising the following steps: constructing a file life prediction system based on a machine learning model, and establishing a file life prediction model according to an overlapping rate between a hierarchy where an SST file is created and a next hierarchy file, a total file size of the hierarchy, a global compression operation count, an interval between the hierarchy and a previous compression operation, and a latest global compression operation count of the hierarchy. The file life range is accurately predicted as features; implementing a partition allocation algorithm based on life labels, and preferentially allocating files with the same life labels to the same partition according to the predicted file life labels; and designing a time-aware dynamic partition allocation strategy, measuring the partition life according to the predicted life label, and dynamically adjusting the partition life label according to time lapse. According to the method, the data migration amount during garbage collection is remarkably reduced, write amplification is reduced, the throughput of the ZNS SSD is improved, and the service life of the ZNS SSD is prolonged.

The invention discloses a file processing method, a file system and a readable storage medium. The method is applied to a system comprising a private client, a master file server and fragmented file servers, the master file server segments a target file and distributes obtained N fragmented files to M file servers for management, and the method comprises the following steps: the private client performs data distribution on the fragmented files according to a first data layout table, when it is determined that a first fragment identifier of to-be-written first data is larger than a first maximum fragment identifier, first fragment creation information is sent to the master file server, so that the master file server creates a first fragment, the maximum fragment identifier included in the second data layout table is updated to be the first fragment identifier, and the first fragment is written into the second data layout table; and sending the updated second data layout table to the first private client. According to the fragment creation process, the accuracy of the data layout table can be ensured by updating the data layout table, so that the accuracy of file processing operations such as attribute acquisition executed by a subsequent private client is ensured.

A system and method for managing unstructured data stored across multiple data repositories is disclosed. The method includes accessing unstructured data items from distinct data repositories and generating classification metadata for each unstructured data item using a machine learning-based classifier. A residency pointer is generated for each unstructured data item based on residency metadata that identifies its storage location within the data repositories. An unstructured data metafile is automatically constructed for each unstructured data item. Each unstructured data metafile is assigned to one or more virtual queues based on its classification metadata, enabling organized data management. The method further integrates the virtual queues with a web-based data handling interface that facilitates batch operations on unstructured data items stored across the distinct repositories. The disclosed approach improves data classification, retrieval efficiency, and governance of unstructured data without requiring physical relocation of stored data items.

The application provides a method and system for stream storage management and persistence of files, and relates to the technical field of computer storage, which comprises the following steps: dividing a hard disk storage space into a plurality of file blocks with the same size to obtain a plurality of initial state unused file blocks; creating and managing a plurality of virtual domains based on the obtained plurality of initial state unused file blocks; allocating the plurality of initial state unused file blocks to different virtual domains for management; when a to-be-written file is received according to the established virtual domain, matching to the corresponding virtual domain based on the attribute of the to-be-written file, and allocating a target file block and a unique file serial number for the to-be-written file in the corresponding virtual domain; and generating a corresponding file ID based on the allocated target file block and unique file serial number. The application realizes low-latency, high-throughput and sequential read-write storage management of sequentially, massively and quickly continuously arriving stream files in application scenarios.

The invention provides an SATA (Serial Advanced Technology Attachment) disk array access method, which comprises the following steps of: acquiring write-in data of FIFO (First In First Out), carrying out circular detection on the data volume written in the FIFO based on a timestamp, initiating a file write-in application, and carrying out file distribution on the written-in data volume so as to correspondingly write the write-in data into an SATA write queue, the file distribution is based on disk information of each SATA disk, and the disk information at least comprises disk capacity and a relative wear factor; arbitrating each path of SATA write queue through an arbitration strategy, performing data caching on the arbitrated SATA write queue, and sending a write completion message; and reading the cached data according to the received write completion message, forming a cache queue, and transmitting the data in the cache queue to a rear-end SATA disk. According to the method, a multi-level storage strategy is adopted, so that the high-speed reliability and anti-risk capability of the multi-level cache can be improved, and the power consumption cost of the system is greatly saved.

A management and updating system for automation devices connected to an OT network of an automation plant, includes a device management and updating user unit programmed to provide a service for providing a communications interface for producing a communication link with connected ones of the automation devices, a read-out service for reading-out device-type information stored in each automation device, and for storing read-out device-type information, a service for planning update requests for each automation device having saved read-out device-type information, a service for communicating with at least one memory unit and retrieving update files stored therein, a service for checking each retrieved update file for its release relative to the automation devices in functional dependency on the stored device-type information, and, if the checking of one of the retrieved update files leads to a release for an automation device, for allocating this update file to the device-type information saved for this automation device, and a service for transmitting according to saved device-type information to each automation device of the update file allocated thereto, along with an update command, and functionally dependent on the update requests planned for each automation device.

This application provides a distributed data reading method, apparatus, and server, relating to the field of data processing. The method includes: a server retrieving multiple file sets from a disk. Each file set may include at least one file. Each file includes metadata for at least one document. The server determines the number of files allocated in each partition based on the quotient of the total number of files in each file set and the number of parallel processes. The server may allocate the files in the file sets to several partitions of parallel processes. Each partition includes at least several allocated files. After determining the number of partitions for the parallel processes, the server may allocate one partition to each parallel process. The parallel process may then read the document corresponding to the metadata in the partition after retrieving it. This method improves data reading efficiency.

Provided are a device, system, method, and computer program for inferring an attacker group by analyzing malicious code. The system includes a sandbox pool manager configured to allocate analysis target files for inferring an attacker group to one or more nodes and separately execute the analysis target files in separate malicious code analysis environments by controlling each node, an event manager configured to determine in real time whether all events related to the analysis target files have been collected on the basis of running state information of each node and collect events which are recorded in the malicious code analysis environments of each of the nodes and related to the analysis target files, an attacker group inference part configured to infer an attacker group by analyzing the collected events, and an analysis result provider configured to provide information on the inferred attacker group.