74 results about "File allocation" patented technology

The embodiment of the invention discloses a distributed file processing method and system.The method comprises the steps that connection with child nodes is established, and load information sent by the child nodes is received regularly; when a to-be-processed file is received, the to-be-processed file is fragmented, and a serial number is allocated to each sub-file; calculating the number of distributable sub-files of each sub-node according to the load information, and distributing the sub-files to the corresponding sub-nodes according to the number of distributable sub-files; adjusting the number of sub-nodes and the distribution of sub-files regularly according to the load information; and receiving execution results of the sub-nodes on the sub-files, sorting the execution results of all the sub-files according to the serial numbers, and splicing the sorted execution results to obtain a final execution result of the to-be-processed file.

The invention discloses a hybrid expert model deployment method and device based on a heterogeneous hardware equipment cluster, equipment and a medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: carrying out the data abstraction processing of all equipment in a target heterogeneous hardware equipment cluster, so as to obtain an initial equipment description vector, compiling the target expert model file into a binary intermediate representation; generating a hardware probe for the target equipment based on the hardware type of the target equipment in the target heterogeneous hardware equipment cluster and the binary intermediate representation, and obtaining a real-time equipment description vector based on hardware; determining a hardware sensing routing weight of each device in the target heterogeneous hardware device cluster based on the real-time device description vector, and determining a to-be-activated expert model file from the target expert model file based on the sensing routing weight; and distributing the to-be-activated expert model file to the target heterogeneous hardware equipment cluster for deployment and installation. Therefore, heterogeneous deployment of the hybrid expert model can be realized through unified arrangement.

Example networks and methods are provided for defining a common data model for redundant data in a data specification. One example method includes accessing a message standard, the message standard having multiple message types, each message type including a plurality of classes, each of the classes defined by an .xsd file; splitting the .xsd files into named complexTypes, where each .xsd file includes one complexType which is named for the one complexType; updating references to duplicate named complexTypes to reference an unnumbered superset complexType; and absorbing each of the unique data elements of the duplicate named complexTypes into the unnumbered superset complexType.

The invention discloses a fast file system scheme applied to embedded hardware. Aiming at the characteristics of fixed storage size and few file directories of embedded hardware files, file management of a file system is simplified by adopting a file distribution area and a data area. The fast file system is divided into a file distribution area and a file data area. The file distribution area records a file name, the maximum file capacity and a file data area address. The file data area stores actual file content. In order to solve the problem that the file system is damaged due to sudden power failure during reading and writing of a common file system, the file distribution area is solidified according to the file number and the maximum capacity in the created file system in a pre-distribution mode, and the situation that the whole file system is damaged due to accidental interruption of file writing operation is avoided. According to the fast file system, file retrieval and file data area positioning can be accelerated through a method for hiding file names and file data area addresses. The method is simple in structure and suitable for embedded hardware.

A file distribution system provided with a file distribution apparatus and a file reception apparatus, wherein: the file distribution apparatus has one or more first processors; and the one or more first processors execute a process including transmitting, to the file reception apparatus, multiple layers included in a first file to be distributed to the file reception apparatus, and among multiple layers included in a second file to be distributed to the file reception apparatus, a layer other than a common layer that is included in common in the first file and the second file.

A storage system includes a storage device and a host device. The storage device includes a non-volatile memory device configured to store data and a storage controller configured to control the non-volatile memory device to isolate and store the data by grouping storage spaces of the non-volatile memory device into a plurality of isolated storage spaces. The host device includes a file system configured to generate and manage a plurality of file allocation instances corresponding to a plurality of file allocation system calls from an application program by allocating, with respect to each of the plurality of file allocation system calls, a logical address range and an isolated identifier of a plurality of isolated identifiers corresponding to the plurality of isolated storage spaces.

Systems, apparatuses, methods, and computer program products are disclosed for linking remote file storage. An example method includes identifying, by discovery agent circuitry, a set of files for storage from a set of candidate files, and classifying, by file metadata circuitry, each file from the set of files for storage into file usage categories. The example method also includes assigning, by file metadata circuitry, permission categories to each file from the set of files for storage based on the file usage categories, and providing, by communications hardware, the set of files for storage and associated metadata to a file storage host, wherein the associated metadata comprises the file usage categories and the permission categories The example method also includes updating, by the discovery agent circuitry, content stored on the file storage host, the content comprising the set of files for storage.

The invention discloses a file processing method and device, equipment, a storage medium and a computer program product, and the method comprises the steps: obtaining the attribute information of a file in response to a file processing request, and determining the to-be-read data volume of the file; performing priority ranking on the files based on the attribute information and the to-be-read data volume to obtain a ranking result; distributing the file to a first queue or a second queue according to the sorting result, the attribute information and the current processing capability; wherein the first queue is configured to instantly process the file, and the second queue is configured to delay processing of the file. Through the design of a flow pool system and double-queue parallel processing, the system load can be effectively balanced, resource waste is avoided, and quick response of file processing is ensured. Besides, the automatic length adjustment of the processing queue enables the system to better adapt to the fluctuation of the business volume, and the system can keep stable operation, reduce the failure rate and improve the user experience no matter whether the peak or the valley is processed.

In one embodiment, an illustrative method herein comprises: obtaining, by a process, a file having a given size; assigning, by the process, the file to a particular size-range bucket of a plurality of size-range buckets of a data ingestion pipeline, the plurality of size-range buckets having a corresponding size-based configuration; and forwarding, by the process, the file into a particular size-based service mesh ingress gateway of the data ingestion pipeline according to the particular size-range bucket to cause processing of the file within the data ingestion pipeline according to the corresponding size-based configuration of the particular size-range bucket.

A dynamic data extraction, distillation, and enhancement system is disclosed that includes a dynamic extraction, distillation, and enhancement framework. The framework includes an allocator, extractor, and deconstructor stored in a non-transitory memory that, when executed by a processor, receive files in different formats from data sources, determine a native format of each file, identify and extract an embedded object from a file, deconstruct the file into components, assign each file to one of a plurality of streams based on the native format of the file, assign the embedded object to a stream based on a format of the embedded object, and assign a deconstructed component to a stream based on a format of the deconstructed component. The native format includes one of text, video, image, or audio. Each stream corresponds to one native format. The streams include a text stream, an audio stream, a video stream, and an image stream.

This invention discloses a file upload method, apparatus, device, and medium. The method includes: obtaining network bandwidth rate, storage space of a target file, and the number of available data nodes; calculating the storage capacity of each available data node based on the storage space of the target file and the number of available data nodes; splitting the target file according to the network bandwidth rate to obtain at least one target sub-file; the target file includes at least one target sub-file; allocating each target sub-file to each available data node according to the storage capacity of each available data node, obtaining a mapping relationship between each target sub-file and each available data node; and uploading each target sub-file to each available data node according to the mapping relationship. Embodiments of this invention can improve the efficiency of file transfer.

Various embodiments relate generally to data science and data analysis, and computer software and systems to provide an interface between repositories of disparate datasets and computing machine-based entities that seek access to the datasets, and, more specifically, to a computing and data storage platform that facilitates consolidation of one or more datasets, whereby data ingestion is performed to form data representing layered data files and data arrangements to facilitate, for example, interrelations among a system of networked collaborative datasets. In some examples, a method may include forming a first layer data file and a second layer data file, assigning addressable identifiers to uniquely identify units of data and data units to facilitate the linking of data, and implementing selectively one or more of a unit of data and a data unit as a function of a context of a data access request for a collaborative dataset.

To enable a user to more easily set a rule relating to folder allocation and file naming at the time of filing data, such as a document scanned image and a received facsimile, and storing the file. On a user interface screen for setting a rule relating to property information that is used at the time of allocating a file to a folder for storage, at least a first area for editing the rule and a second area displaying an item a user can select are provided. The item is a unit element for a user to set a character string configuring the property information and corresponds to a character string of an arbitrary attribute a user can set. Then, the rule is created by the item selected by a user from the second area being arranged at a position designated by the user in the first area.

The invention provides an improved RSU (Road side unit) popular file distribution method based on deep reinforcement learning, belongs to the technical field of edge computing of the Internet of Vehicles, and solves the technical problems of relatively low file response rate and relatively large file transmission delay caused by wrong cache files of RSU. According to the technical scheme, the method comprises the following steps that S1, in each round, based on file requests of all vehicles in a communication range, two RSUs determine Fc to-be-cached popular files; s2, the RSU initializes a main Q network, a target Q network and an experience playback buffer area; s3, the RSU trains the target Q network and updates the main Q network; and S4, based on a deep reinforcement learning result, the local RSU and the neighbor RSU distribute popular files so as to respond to a vehicle file request. The method has the beneficial effects that the RSU popular file allocation algorithm based on deep reinforcement learning is provided, and the response performance of cache files can be improved.

Embodiments include translation of an integrated circuit design data into a production file. Aspects of the invention include identifying a top cell in the integrated circuit design data and a first cell-file associated with the top cell, assigning the first cell-file to a first translation module of a plurality of translation modules, and receiving, from the first translation module, an identification of child cells referenced by the first cell-file. Aspects also include assigning cell-files of the integrated circuit design data that correspond to each of the child cells to different translation modules of the plurality of translation modules, receiving, by a merger module, a single-cell production file from each of the plurality of translation modules, wherein each single-cell production file corresponds to a cell in the integrated circuit design data, and combining, by the merger module, the single-cell production files to create the production file.

The application discloses a storage method applied to an access device, and the method comprises the following steps: in the case that a storage request for a first file is received, an identification code is allocated to the first file; an index number of the first file is generated by using an existing parent directory of the first file and the identification code; an association between a name of the first file and the index number of the first file is stored; and the first file and the index number of the first file are sent to an index device. The application also discloses a device, a system and a computer storage medium, and the reliability of file storage is improved, and the storage efficiency of file storage is improved.

The invention discloses a dynamic mapping strategy for improving the space utilization rate of a ZNS solid state disk. The dynamic mapping strategy comprises the following steps that 1, Strip strips are distributed according to needs; the method comprises the following steps: dynamically allocating different numbers of Strip strips for a logic Zone according to different sizes of SST / WAL files during load operation; 2, only one file is stored in one Zone; a new null logic Zone is allocated to each SST / WAL file, and it is guaranteed that only one SST / WAL file is stored in one Zone; and 3, providing an infinite logic Zone number. The equipment provides an infinite logic Zone number for an upper-layer application so as to fully utilize resources in the equipment; and 4, resetting the Zone if the file fails. And when one SST / WAL file is invalid, the upper application directly sends a reset command to the Zone, and the Zone space is quickly recovered. According to the method, the problems of space amplification and data migration of LSM tree storage on the ZNS SSD can be well solved, the space utilization rate can be remarkably increased, the data migration overhead is eliminated, and then the overall I / O performance is improved.

The present invention discloses a server and a method, device, and medium for controlling edit permissions for stored files. The method comprises: receiving an edit request from a client, parsing the edit request to obtain ownership information of the file, as well as a logged-in user ID and a logged-in client ID; the ownership information includes a current client ID and a current user ID; and assigning ownership to the file and granting or denying edit permissions to the logged-in user based on a comparison between the logged-in user ID and the current user ID, and between the logged-in client ID and the current client ID. The present invention avoids file "deadlock" issues and frees up server resources.

The invention provides a data analysis method. The data analysis method comprises the following steps that a list of files to be analyzed is obtained according to a file path, and a file queue is formed; distributing the files in the file queue to N threads determined by the number of processor cores to realize load balancing; searching packet headers and packet tails of the data packets from the distributed file data in parallel through N threads so as to identify and extract complete or partial data packets; and carrying out analysis and integrity verification on the extracted data packet, and writing the analyzed data into a database system. According to the data analysis method provided by the invention, through multi-thread parallel processing and load balanced distribution, the data analysis efficiency is remarkably improved, and the overall processing time is shortened.

To enable a user to more easily set a rule relating to folder allocation and file naming at the time of filing data, such as a document scanned image and a received facsimile, and storing the file. On a user interface screen for setting a rule relating to property information that is used at the time of allocating a file to a folder for storage, at least a first area for editing the rule and a second area displaying an item a user can select are provided. The item is a unit element for a user to set a character string configuring the property information and corresponds to a character string of an arbitrary attribute a user can set. Then, the rule is created by the item selected by a user from the second area being arranged at a position designated by the user in the first area.

The invention discloses a batch printing method, system and device based on file type classification and a medium. The method comprises the following steps that a to-be-printed file set uploaded by a user is received; identifying the type of each file; distributing a proper printing format for each file according to an identification result; all printing tasks are integrated; executing the printing task; providing preview before printing, and executing content verification after printing; the method can automatically identify the file types, summarize the printing files in batches, select a proper printing format for classified printing according to the file types, and preview and check the printing result, so that the efficiency and accuracy of batch printing are remarkably improved, and the method is particularly suitable for office scenes needing to process a large number of different types of files.

The invention provides a block chain-based trusted adaptive file distribution method, which comprises the following steps of: S1, reading application information of a node to generate an application information form; s2, comparing the number of application nodes in the application information form with a preset initial demand number to obtain a size relationship; s3, selecting a decentralized distribution model or a centralized distribution model as a file distribution strategy according to different size relationships; s4, distributing the files through the determined file distribution strategy to generate a file distribution form; and S5, uploading the file distribution form to a block chain network.

The present application relates to a kind of on-chip system file data management method and system for prolonging the service life of memory.The storage space allocated by the present application for file is divided into file space and standby space, when file data needs to be modified, the relevant control information and data of each modification are written into standby space, the original file space is unchanged, until the margin of standby space is insufficient, then the corresponding storage block of file space is erased, the file data after multiple modifications is written, and the standby space is erased for next modification.The data stored in the corresponding storage block in file space is read first when reading file data, then the data in the specified reading range in standby space is used to construct the final reading data, or the specified record stored in standby space is directly read.The technical scheme of the present application can effectively reduce the erasing times of storage block, prolong the service life of storage block, and has power failure protection function in the case of large amount of local modification of file data.

The invention discloses a file uploading method and device, equipment and a medium. The method comprises the steps of obtaining a network bandwidth rate, a storage space of a target file and the number of available data nodes; calculating the storage capacity in each available data node according to the storage space of the target file and the number of the available data nodes; splitting the target file according to the network bandwidth rate to obtain at least one target sub-file; the target file comprises at least one target sub-file; according to the storage capacity in each available data node, distributing each target sub-file to each available data node to obtain a mapping relationship between each target sub-file and each available data node; and according to the mapping relationship, uploading each target sub-file to each available data node. According to the embodiment of the invention, the file transmission efficiency can be improved.

This invention discloses a storage system, read / write method, and controller for reducing file fragmentation, belonging to the field of data storage technology. The system includes a host and a solid-state drive (SSD). The host has a file system and a space pre-allocator, while the SSD has a segment allocator. The file system allocates logical address space to files according to segments. The segment allocator maintains a global translation table, which records the mapping relationship between segment IDs and mapped page addresses. The mapped pages are flash pages used to record the logical page addresses within a segment to the physical page addresses. The segment allocator also calculates free segment IDs in a monotonically increasing manner and publishes them to the circular buffer when there is remaining space. The space pre-allocator retrieves the free segment ID from the head of the circular buffer and allocates it to the file when space needs to be allocated. This invention can reduce the generation of file fragmentation and free space fragmentation in the file system, improving the overall performance and stability of the system.

The invention provides a file analysis system which comprises a file reading module used for obtaining the name and the size of a to-be-analyzed file according to a file path and putting the file into a file queue; the file distribution module is used for sorting the files in the file queue according to the sizes of the files, creating a plurality of data buffer areas and a plurality of threads according to the core number of the processor, and distributing the files to the threads by adopting a load balancing method; each analysis execution module runs in an independent thread, and comprises a file search unit used for identifying and extracting a complete or partial data packet from file data distributed to the thread by searching a packet header and a packet tail of the data packet; and the file analysis unit is used for carrying out analysis and integrity verification on the data packet extracted by the file search unit and writing the analyzed data into a database system. According to the system provided by the invention, multi-file parallel analysis can be realized, and the analysis efficiency and the system throughput are remarkably improved.

File system full and incremental replication to object storage systems (e.g., using a computerized tool), is enabled. For example, a system can comprise at least one processor, and at least one memory that stores executable instructions that, when executed by the at least one processor, facilitate performance of operations. The operations can comprise identifying a file system, wherein the file system comprises a group of files, based on an attribute applicable to the file system, determining a dataset identifier representative of the file system, for respective files represented in the group of files, allocating the dataset identifier and allocating respective file identifiers, and copying the group of files to an object store with respective object names corresponding to the respective files, wherein the copying comprises, for the respective files, assigning the dataset identifier to respective object names and assigning the respective file identifiers to the respective object names.

Detecting malicious / spoofed Universal Resource Locators (URLs) and / or files in received electronic mail (i.e., email) and, in response to detection, automatically taking responsive measures to prevent interaction / access to the malicious / spoofed URLs and / or files and / or limit, network-wide, the exposure to the malicious / spoofed URLs and / or files. Valid URLs and / or files are assigned a token, such as an alpha-numeric passcode or the like and the assigned token is verified upon receipt of the email by the intended recipient. In the event that the token cannot be verified or a token is not received for a URL or file, the automated responsive actions ensue.

The invention provides a file system processing method and system, a server, a storage medium and a program product. The method is suitable for a container migration scene, in a container storage stage, a source end reads an original device number of a mounting point of original tmpfs in a source container and an original index node number of a file, and transmits the original device number and the original index node number to a target end through a container management and control end; in the container recovery stage, the target end explicitly allocates the corresponding original device number to the created mounting point when creating the mounting point, so that the consistency of the device numbers before and after the tmpfs is stored and recovered can be ensured. When the file is created in the mounting point, the corresponding original index node number is explicitly allocated to the created file, so that the consistency of the file index node numbers before and after the tmpfs is stored and recovered can be ensured, the abnormality / fault caused by the inconsistency of the device number and the index node number before and after the tmpfs is migrated can be avoided, and the security and stability of the system are improved.