888 results about "File storage" patented technology

The invention provides a private deployment-oriented multi-modal AI knowledge base construction system. The private deployment-oriented multi-modal AI knowledge base construction system comprises a knowledge storage module, an intelligent document loading module, a document partitioning engine module, a data enhancement engine module, a multi-language semantic vector alignment module and a private deployment module, the knowledge storage module comprises a knowledge authority management sub-module and a knowledge source management sub-module, the knowledge authority management sub-module is used for managing and storing knowledge from different sources, and the knowledge source management sub-module is used for managing electronic documents and multimedia documents; according to the method, intelligent identification, partitioning, vectorization and source file storage can be carried out on different types of electronic files, knowledge graph construction is carried out for specific fields, semantic relevance between texts and topics and key entities is fully considered, the method has wider applicability, higher robustness and controllability, the data leakage risk is effectively reduced, and the method is suitable for popularization and application. The method is suitable for enterprise sensitive data protection and personal user elastic computing power requirements.

A system for data collection related to a fluid conveyance environment includes a data acquisition circuit comprising inputs and outputs; input sensors to provide sensor data values, coupled to a component in the fluid conveyance environment; and a processor comprising the data acquisition circuit. The processor is configured to determine a data storage profile; responsive to the data storage profile, configure the data acquisition circuit to selectively couple at least one of the inputs to at least one of the outputs; interpret the at least one of the sensor data values; store at least a portion of the at least one of the sensor data values in response to the data storage profile; analyze a set of the sensor data values and determine a data quality parameter; and adjust at least one of the data storage profile and a data collection routine in response to the data quality parameter.

The invention relates to the field of financial science and technology, and particularly discloses a cross-platform financial data integration and real-time analysis system and method. The system is characterized in that a distributed heterogeneous data acquisition module accesses multi-source data in parallel through a pluggable protocol adapter; the streaming data cleaning engine realizes field-level normalization based on a dynamic template and semantic mapping; the hybrid storage architecture forms a three-level system by a memory database, a time sequence database and distributed file storage; the stream processing core engine adopts an elastic sliding window to execute CEP complex event processing; and the dynamic analysis decision module integrates online machine learning to generate a real-time transaction signal. The method comprises the steps of multi-source concurrent acquisition and microsecond-level timestamp injection, context sensing data standardization, dual-channel data storage, event time window aggregation, dynamic risk calculation and block chain audit evidence storage. The defects of low data fusion efficiency, high real-time analysis delay and the like in the prior art are overcome, and cross-platform data processing timeliness and decision accuracy are remarkably improved.

The invention relates to the technical field of computational fluid mechanics data post-processing, and particularly discloses an OpenFOAM unsteady state simulation result-oriented boundary surface physical quantity calculation method, which comprises the following steps of: setting a working root directory, reading a grid file and establishing an example grid; accessing a time control function, and reading simulation time information of each time folder; establishing a global blank file to store boundary surface physical quantity results, first line writing time and physical quantity variable names at all moments; sequentially reading field quantity data required by calculation in the time folder; appointing and searching the name of a concerned boundary surface, creating a scalar field magSf of the boundary surface, and calculating and storing the area of a grid surface element; defining and initializing physical quantities and intermediate quantities of boundary surfaces, calculating physical quantities and intermediate variables of surface elements of each grid, and traversing the grids; and writing a result from the second line of the global file according to a time sequence, and traversing all times. By the adoption of the method, key data at all moments can be accurately extracted, errors and complexity caused by manual processing are avoided, and the calculation efficiency is greatly improved.

The invention discloses a load balancing optimization method and device, electronic equipment and a storage medium, and relates to the technical field of load balancing optimization, and the method comprises the steps: carrying out the differentiation processing of a data migration task according to a file storage threshold value, and combining a migration performance index and server resource occupation information collected in real time, the task allocation strategy is adjusted through the dynamic task allocation algorithm, so that the reasonable allocation of the files with different sizes and the dynamic load optimization in the migration process are realized. The technical problems that in the prior art, server overload and resource idling coexist due to a single migration server or a static task allocation strategy, high delay and data inconsistency risks are generated due to network congestion and uneven task allocation in the migration process, and the overall performance and reliability of a system are affected can be solved. The technical effects of optimizing the load balance degree, reducing overload and idle conditions of the server, reducing migration delay, ensuring data consistency and improving the overall performance and reliability of the system are achieved.

Techniques are described for maintaining data consistency when failure events occur during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, two generation numbers (or different identifications) are assigned to two groups of processed B-tree key-value pairs, one before and one after a failure event, within a key range. In some embodiments, the two generation numbers are assigned to a group of B-tree key-value pairs processed by a failed thread and another group of B-tree key-value pairs processed by a substitute thread taking over the failed thread to avoid potential data corruption.

The embodiment of the invention provides a chip verification method, device, system and equipment, a medium and a program.The verification method comprises the steps that a chip is simulated, the chip comprises a program running module and a memory storage module, the program running module is used for running a software program, and a program file of the software program is stored in the memory storage module; executing the simulation case to enable the program running module to call the program file for processing; in the process that the program running module calls the program file, a read-write event of a memory storage module is monitored so as to track the variable change condition of the software program, and the memory storage module is provided with a memory area distributed for variables of the software program; the variable change condition is synchronized and displayed to reflect the running state of the software program, and the running state is used for judging whether the processing of the program running module on the software program reaches the design expectation or not. According to the verification method provided by the embodiment of the invention, software and hardware joint simulation verification of the chip is realized.

The invention provides a lightweight large model operation method based on end-side deployment. The lightweight large model operation method comprises the following steps: acquiring equipment operation data and storing a model file in a mixed precision format according to quantization precision supported by terminal equipment; determining a mixing precision quantification strategy and carrying out strategy analysis on the equipment operation data; a Key-Value cache file obtained through model reasoning in the historical dialogue is reserved; calculating a semantic embedding vector of the second user request, calculating a cosine similarity between the semantic embedding vector and the first V vector, and judging whether the second user request has a similar intention or not according to a calculation result; and if yes, performing incremental reasoning on the second user request by multiplexing the Key-Value cache file. By monitoring the state of the terminal equipment in real time, dynamically adjusting the quantization level, multiplexing the cache data of the semantic related requests and merging batch request reasoning, the effects of balancing the energy consumption and performance of the terminal equipment, reducing repeated calculation and improving the service time of the equipment and the user experience are achieved.

The embodiment of the invention discloses a virtual file management method, system and device and a storage medium. According to the technical scheme provided by the embodiment of the invention, under the condition that the file request is received, the virtual path contained in the file request is extracted, the virtual path is created when the target file is generated, path classification is performed according to the module identifier of the associated target service module, and the target file is stored in the system path mapped by the virtual path. The system path is generated by mapping based on the virtual path and a pre-constructed mapping rule; mapping the virtual path into a corresponding system path based on a mapping rule; and accessing the target file based on the system path to execute a file operation corresponding to the file request. By means of the technical means, the situation that file directory structures of different service modules are disordered can be avoided, unified file management of the different service modules in the application is achieved, the file management efficiency of the different service modules is improved, the application service effect is improved, and therefore the user experience is improved.

The invention discloses a file hiding method and system based on an ext4 file system, and the method comprises the steps: obtaining a target file and metadata information of a directory where the target file is located according to a target file path needing to be hidden, and distributing a new mapping item in an internal mapping table of the ext4 file system for recording; creating a new hard link in a directory where the target file is located, and associating the new hard link with an inode node for storing metadata information of the target file; reopening the ext4 file system, and deleting the directory entry of the target file, so that the target file cannot be found in the ext4 file system; and re-mounting the ext4 file system, refreshing the Linux kernel cache, and recording the new hard link, the original name of the target file, the inode node number of the target file and the file type into a new mapping item. According to the invention, data security can be ensured, and efficient hidden storage of files is realized.

The invention relates to the technical field of file storage management, and discloses a Lustre architecture-based high-performance file storage and processing method and system, the method comprises the following steps: obtaining spatio-temporal characteristics of file access; predicting a popularity value of the file based on the spatial-temporal characteristics; determining a storage strategy of the file based on the popularity value of the file; carrying out distributed storage on the file based on the storage strategy, and continuously detecting a node state; if it is detected that the node load is unstable, adaptive data migration is carried out on the node with the unstable load; if the node is detected to have a fault, generating a data recovery path based on the three-dimensional topological coordinates of the node, and migrating a file stored in the node having the fault based on the data recovery path; according to the method, the storage strategy is determined through the file popularity value, and the abnormal condition of the node is efficiently processed, so that the file storage and processing performance and efficiency are comprehensively improved.

The invention discloses a resource usage amount statistical method and system oriented to an application mirror image, and belongs to the technical field of cloud computing and containerization. The method comprises the steps that duplicate removal and calculation are conducted on files of all mirror image layers according to the identification result of the files, and the file storage amount is obtained; collecting resource use state data of the application mirror image and inputting the data into the resource competition game model, so that the resource competition game model outputs the calling weight of each process in each mirror image layer and determines a resource occupation ratio by solving a Nash equilibrium strategy; dividing the current system resource usage data to each current process based on the corresponding relationship between each current process and each current mirror image layer, and determining the resource usage amount of each current mirror image layer according to each resource occupation proportion; and summarizing the resource usage amount and the file storage amount of each layer to obtain the total resource usage amount of the current application mirror image, so that accurate statistics of the resource usage amount can be realized.

The invention relates to the field of data management, and provides an equipment label information hierarchical storage method and device based on a prefix tree structure, the method comprises the following steps: caching equipment label information into a memory in the prefix tree structure, the prefix tree structure being a data structure combining a prefix tree and a linked list, each cached node holds a pointer pointing to a child node storage space on a disk; when the amount of data cached in the memory reaches a preset threshold value, newly adding or modifying the equipment label information cached in the prefix tree into a physical log file; after the newly-added or modified operation is completely written into the physical log file, the equipment label information in the memory is stored in the prefix tree file, and the prefix tree file is stored on a disk by adopting a structure of fusing a prefix tree with a B + tree. According to the invention, the problem of overlarge memory consumption of a full-memory storage scheme in the prior art is solved, and the memory occupation is obviously reduced.

The invention discloses a distributed medical image retrieval and hierarchical storage management system and method. The system comprises an interface processing module, a distributed image storage cluster, a routing forwarding module, a metadata management module and a storage management module. The interface processing module receives the image retrieval request, generates a routing identifier for positioning the storage position of a target medical image file and sends the routing identifier to the routing forwarding module; and the routing forwarding module queries a mapping relationship between the medical image file and the storage node and / or the storage hierarchy based on the routing identifier, determines a first storage node and forwards the request, so that the first storage node returns a target medical image file. And the storage management module obtains the access statistical information, determines a target storage level and / or a target storage node, migrates the medical image file, and triggers updating of the mapping relation after migration is completed. Therefore, cross-node accurate positioning and unified retrieval are realized, positioning consistency is kept after file migration, and hierarchical storage management based on access conditions is supported.

A method for optimizing file storage includes receiving columnar data to store at a columnar data store with columns ordered with an initial ordering. The method includes determining, based on historical access patterns for the columnar data store, an updated ordering for the columns. The method includes storing the columnar data at a first location of the columnar data store using the updated ordering. The method includes determining that the stored columnar data is to be compacted and compressing at least a portion of the columnar data using each of a plurality of compression techniques. The method includes, based on compressing the at least a portion of the columnar data, selecting one of the plurality of compression techniques. The method includes storing the columnar data at a second location of the columnar data store using the selected one of the plurality of compression techniques.

The present disclosure provides a data storage method, device, electronic equipment and storage medium. The data storage method comprises: in response to a file storage request for a first target data file, obtaining the first target data file to be stored in a target storage system, the target storage system being configured with table metadata, the table metadata indicating storage branch information corresponding to the target storage system, the storage branch information comprising a main branch and at least one experimental branch, the main branch being used to mount data files that have been used for model training and the training results of which meet preset requirements, and the experimental branch being used to mount data files that have not been used for model training; based on the table metadata, determining a first target experimental branch corresponding to the file storage request, and newly creating a first target snapshot corresponding to the first target experimental branch; and based on the first target snapshot, storing the first target data file to a corresponding storage location in the target storage system. The embodiments of the present application can improve the data storage efficiency.

The invention relates to a distributed high-availability code warehouse management method and system and a medium, and relates to the technical field of version control and code management.The method comprises the steps that a pre-configured external load balancer, a plurality of GitLab Web nodes, an internal load balancer, a PostgreSQL database cluster, a Redis cluster, a Gitaly cluster and object storage or file storage are obtained; carrying out load balancing on the plurality of GitLab Web nodes through the external load balancer; performing load balancing on the internal connection of the GitLab application program through the internal load balancer; the data of the GitLab Web and the data of the GitLab Pracect are stored through the PostgreSQL database cluster, and the data of the GitLab Web and the data of the GitLab Pracect are And providing access to a Git warehouse by using the Gitaly cluster. According to the invention, high availability, high performance and expandability of code warehouse management are realized through a distributed architecture and redundancy configuration.

A file security processing method based on selective encryption and adaptive transmission comprises the following steps: analyzing file metadata in an upload request of a client to obtain a file feature and a user identity; based on the file features and the user identity, calling a configured encryption seed model to generate an initial encryption seed, and inputting the initial encryption seed into the optimized pseudo-random sequence model to generate an encryption position index so as to obtain an encryption position index; on the basis of the encryption position index and the initial encryption seed, performing encryption processing on a data block in a file data stream uploaded by the client in a streaming manner to obtain an encrypted data block; meanwhile, the transmission rate is adjusted based on the real-time transmission quantity and the network quality, and a transmission rate parameter of the adaptive network is obtained so as to control the time sequence scheduling and the data sending frequency of encrypted data block streaming transmission; and performing fragmentation aggregation and data splicing processing on the transmitted encrypted data blocks according to a transmission sequence to generate a complete encrypted file, and updating a file storage record in a database according to the complete encrypted file.

The invention discloses a data interaction management and control system and method for judicial and administrative business resource integration. The system comprises a presentation layer, an application layer, a service layer, a data access layer and an infrastructure layer, wherein the presentation layer is used for providing a visual interface of a judicial administration data collaborative sharing portal and an interactive management and control portal, and supporting data collaborative sharing and management and control operation of a plurality of business scenes; the application layer is used for combining functions of the service layer, realizing a business scenarized application module and recording an operation behavior log in real time; the service layer is used for packaging preset business logic and providing an atomization service interface; the infrastructure layer comprises network resource load balancing, an API gateway, a hardware security device, a computing resource server cluster, a message queue, a log storage, a relational database, a distributed file storage and a block chain platform node. The block chain is utilized to realize on-chain data access, behavior data is prevented from being tampered, traceability and truth restoration are facilitated, and a person in charge is positioned.

The invention provides a multi-type storage cluster dynamic nano-management method based on a cloud platform, and belongs to the technical field of cloud platform storage resource management, a storage pool is used as a minimum management unit, a configuration center provides a unified nano-management entrance and is associated with a platform initialization process, and initialization of standardized parameters such as a storage cluster IP, an account and the like is automatically completed; after management, block storage, file storage or object storage types and exclusive parameters thereof are configured for the storage pool according to business requirements, and available storage pools are screened according to binding relations when storage services are created, so that resource mismatching is avoided. Bidirectional synchronization is carried out between the operation and maintenance resource management module and the storage module through a full life cycle by using an equipment SN number, so that state consistency and traceability are realized; expansion of newly added storage pools, unbinding of the storage pools and resource recovery are supported, and the resource utilization rate and the operation and maintenance efficiency are improved.

The invention discloses a drive program installation method and electronic equipment, and relates to the technical field of software. According to the scheme, the driver files of all the devices of the server are stored in the preset storage medium in advance, so that the completeness of driver programs of all the devices is ensured, which is the premise of executing device driver updating and installation; meanwhile, when the operation system mirror image is installed, the target drive file of the undriven device in the storage medium can be read, loaded and installed based on the target installation environment of the operation system mirror image, the drive file does not need to be integrated into the operation system mirror image in advance, and the operation system mirror image version release is not depended on; therefore, the problems that the period of releasing the new ISO mirror image by the system issuer is long, and the new parts cannot be used due to the update lag of the drive program are solved, and the rapid popularization and application of the new parts of the server are facilitated.

The invention provides a file data migration method and device, equipment, a storage medium and a program. The method comprises the following steps: receiving object storage access information, wherein the object storage access information is used for requesting migration of target file data; according to the object storage access information, constructing mapping information between a file local directory of the file storage system and an object storage bucket of an object storage cluster; according to the file path under the local directory of the file, generating a target object identifier in the object storage cluster, the target object identifier being generated by converting a hierarchical structure of the file path; and migrating the target file data from the file storage system to an object storage cluster based on the mapping information and the target object identifier. According to the method, the file storage flexibility can be improved.

Systems, computer program products, and methods are described herein for automatic requirements capture and generation of system builds using artificial intelligence. The present invention is configured to receive a plurality of architectural requirements for a system build; render an interactive graphical user interface (GUI) for display on user devices, wherein the GUI includes information for a plurality of system builds; receive an input from a user device displaying the GUI, wherein the input includes information for the system build; validate the input for the system build; transmit, upon invalidation of the input, a notification of an invalid input to the user device; transmit, upon validation of the input, the input to networked devices associated with the system build; receive a build output file for the system build; and store the build output file in at least one memory device.

One or more implementations of this specification provide model training and checkpoint file storage systems and methods. In an implementation, a method includes executing, by a model training module of a storage system, a training task of an artificial intelligence model, during execution of the training task, suspending, by the model training module, the training task if a first checkpoint file is generated and sending a request to a checkpoint file processing module of the storage system to cache the first checkpoint file, locally caching, by the checkpoint file processing module, the first checkpoint file based on the request, and concurrently performing, by the check file processing module, a notification operation and a storage operation.

This specification provides methods, systems, and computer-readable media for training an artificial intelligence model and storing a checkpoint file. An example system includes a computing node and a first cache node. The computing node is configured to: read a dataset for training from the first cache node to execute a training task of an artificial intelligence model; and in a process of executing the training task, generate a checkpoint file and send it to the first cache node. The first cache node is configured to: identify a type of obtained to-be-stored data; if the type of the to-be-stored data is a dataset, first write the to-be-stored data into a local buffer, and then store the to-be-stored data in a local hard disk from the local buffer; or if the type of the to-be-stored data is a checkpoint file, directly store the to-be-stored data in the local hard disk.

The present specification provides an article flow information tracing method, device, medium and equipment. The method comprises: receiving a tracing request for a specified article sent by a tracing end, the tracing request comprising at least a tracing party identifier and a specified article identifier; extracting a permission file storage address of a target node party associated with the specified article identifier; the permission file storage address is configured in a tracing platform; accessing a flow information permission file of the target node party according to the permission file storage address; the flow information permission file is used at least to configure access permission of a flow information storage unit; the flow information storage unit refers to a storage unit corresponding to a node party and used to store article flow information of the node party; in a case where it is confirmed based on the flow information permission file that the tracing party identifier has access permission to the flow information storage unit of the target node party, feeding back position information of the flow information storage unit of the target node party to the tracing end, so that the tracing end obtains flow information of the specified article.

The invention discloses a user working environment persistent storage method based on a double-layer container. The method comprises the steps that mirror images of all public containers in mirror image private server are pulled to a shared storage space to form a hot loading area; dividing a corresponding working space from the shared storage space, querying a target storage block corresponding to a target public container mirror image from a hot loading area based on a public container mirror image identifier in a creation request from a client, copying the target storage block to the working space to serve as a system disk, and further establishing a double-layer container and starting the double-layer container; in response to a modification request of a client, performing corresponding modification operation on the initial state inner layer container to obtain a target state inner layer container, so as to run a user working environment based on the target state inner layer container to execute a computing power task; and storing a file corresponding to the modification operation in the remaining space except the system disk in the workspace to form a user mirror image layer. According to the invention, the persistent storage efficiency of the container state and the user working environment can be improved, and the rapid restarting of the container can be realized.

The invention discloses an IO merging mechanism based on a block index, relates to the technical field of big data and information retrieval, and optimizes an original multi-file stream output mechanism of open source Lucene into single-file stream output by expanding a Directory interface of the Lucene, so that IO operation merging is realized in an index writing process, and the operation efficiency is improved. The method specifically comprises three parts of a file storage structure, a writing process and a reading process. The file storage structure comprises a single file structure of a file header, a data area and a file tail; during writing, data is cached in the memory block firstly, and after the memory block is full, the data is added to a file by taking a data block as a unit; during reading, quick positioning is realized by analyzing the block index at the end of the file; according to the method, the problems of performance bottleneck, small file flooding, difficulty in atomicity guarantee and the like caused by multi-file IO in the prior art are solved, and the index construction efficiency and the system stability are remarkably improved.

The invention discloses a file storage processing method and system based on artificial intelligence, and the method comprises the steps: obtaining access description data of a file entity in a file storage data set, and the access description data comprises an access activeness parameter and access result data; mapping the data partition to a storage unit of a corresponding activeness interval based on the activeness parameter; controlling each storage unit to construct access sequence structure information containing sequence and non-sequence arrangement description; generating active and inactive file corresponding relation result data based on the information; evaluating the access change of the target scene file in combination with the multi-interval result to obtain an access change measurement result; therefore, the storage structure is adjusted, the storage resource configuration is matched with the scene file access change characteristics, and the storage resource utilization rate and the access performance are improved.

The application relates to the technical field of file storage equipment, in particular to a file safe storage device and an operating method, which comprises a mobile hard disk, the mobile hard disk is slidingly installed in a hard disk box, one side of the hard disk box is provided with a plug-in plate, the other side of the hard disk box is abutted with an anti-falling cover, slots are formed in the two sides of the anti-falling cover, the slots are slidably connected with one end of plug-in blocks, the other end of the plug-in blocks is provided with a transmission assembly, the transmission assembly comprises a first connecting plate, a second connecting plate and a third connecting plate, one side of the third connecting plate is rotationally connected with a connecting block, the connecting block is fixedly sleeved at the top of a sliding rod, and the bottom of the sliding rod is slidingly sleeved in a fixed block; one end of the hard disk box is plugged with the plug-in plate, so that the falling of the mobile hard disk is preliminarily prevented; the anti-falling cover is buckled on one side of the hard disk box, so that the falling of the mobile hard disk is further prevented; the plug-in blocks are plugged into the slots, so that the anti-falling cover is limited and fixed, and the anti-falling cover is further prevented from loosening, so that the mobile hard disk is also prevented from falling due to vibration after being plugged.