1395results about "File access structures" patented technology

The invention discloses an unstructured storage hierarchical strategy optimization method based on machine learning, and relates to the technical field of data storage management, and the method comprises the steps: monitoring a file access event in real time, generating an access log, and extracting a multi-dimensional feature data set; utilizing the trained multi-dimensional decision tree model to distribute a corresponding storage hierarchy for the storage object to obtain a storage hierarchy decision; according to a storage level decision, distributing the storage objects to different storage layers, and carrying out resource configuration and storage operation; monitoring the access condition of the storage object in the new storage hierarchy in real time, and collecting file access performance, storage cost and response time to obtain feedback data; by monitoring the file access event in real time and extracting the multi-dimensional feature data set containing the basic attribute, the access behavior and the context information, efficient response of hotspot data and reasonable utilization of resources are ensured, and the overall performance utilization rate and the cost effectiveness of storage are remarkably improved.

The invention provides a data weaving platform-oriented self-service data preparation method, which comprises the following steps of: accessing original data and associated structured metadata of a target data source into a data weaving platform to generate standardized accessed original data and associated structured metadata; based on a semantic model of the data weaving platform, performing classified semantic annotation on the associated structured metadata, and positioning a transfer node of the associated structured metadata to generate a data blood relationship record; performing bidirectional quality verification on the data consanguinity record according to semantics and path continuity, and matching with a federated governance rule of a data weaving platform to generate a metadata governance list; and generating a unified metadata directory based on the metadata governance list, so that a user of the data knitting platform can obtain key segments containing complete data consanguinity records. According to the method, the problem of insufficient accuracy and efficiency in self-service data preparation can be effectively relieved, and the efficient preparation requirement of a data weaving platform for multi-source data is met.

A metadata access method, apparatus, and computer-readable storage medium for efficient metadata retrieval through cache management. The method receives metadata query requests including target index information from processes and performs matching operations on a global cache file containing records with index information and slot identifiers. Each cache description array corresponds to memory blocks caching metadata. Upon successful matching, the target cache description array is accessed using the target slot identifier. The data state of target metadata is determined from the cache description array, and target address information indicating the location of the target memory block in shared memory is obtained and returned to the requesting process, enabling efficient shared memory-based metadata access.

The invention relates to the technical field of data processing, and provides a BMC fault recording method and device, computer equipment and a storage medium, and the method comprises the steps: responding to a kernel fault of a target operating system of a target BMC, and printing fault log information; obtaining a pre-configured memory technology device partition; transmitting the fault log information to an annular fault cache region to form cache log data; storing the cached log data in a memory technical equipment partition, and generating a fault log file; and in response to a user reading instruction, reading and outputting the fault log file of the memory technical equipment partition. According to the technical scheme provided by one or more embodiments of the invention, the fault log can be effectively recorded, maintenance personnel can timely and accurately know the fault position and the fault reason, and BMC fault diagnosis and repair can be efficiently completed.

The invention provides a ship-based unified storage and management design data method, which comprises the following steps that: an operation end generates and sends an editing request with a node identifier, an editing type, a timestamp and a digital signature, and a WEB front end receives and verifies the legality of the request through an encryption channel; the storage service generates an intention lock based on the permission table and version number matching, and concurrent access control is achieved. Then, the WEB front end carries out content definition partitioning (CDC) and Hash fingerprint calculation on a design file stream, the storage service realizes partitioning duplicate removal and incremental storage according to a fingerprint matching result, and a Merkle tree is utilized to construct a file version integrity Hash root; on this basis, the system analyzes model differences, constructs a semantic difference chart, and calls a rule engine to perform automatic pre-check. According to the method, efficient storage, collaborative editing, security and credibility and hierarchical scheduling management of ship design data are realized, and the method has the technical advantages of high data consistency, high resource utilization rate, traceability auditing and the like.

A system can include a memory device comprising a plurality of dynamic capacity devices and a processing device, operatively coupled with the memory device. The processing device is configured to perform operations including sending, to a memory device, an allocation request to allocate, for a file system, a first memory section of a plurality of memory sections of a plurality of dynamic capacity devices associated with the memory device; reserving, for the file system, a first portion of the first memory section for storing file metadata of file system data; reserving, for the file system, a second portion of the first memory section for storing file data of file system data; sending, by a first process running on the host system, to the memory device, the file metadata for storing the file metadata in the first portion of the first memory section; sending, by a second process running on the host system, to the memory device, a second request to modify the file metadata stored in the first portion of the first memory section; and receiving, from the memory device, a second response indicating an error regarding the second request.

The invention belongs to the technical field of data processing, and particularly relates to a fingerprint generation method of a virtual evaluation model based on Hash operation, which comprises the following steps of: firstly, constructing a dependency tree by taking a main model file specified by a user as a root node, and initializing an access record table; establishing a static keyword protection area containing a core parameter rule base, scanning a dynamic file operated by a user, and comparing parameter definitions to generate an early warning; reading the binary data of the current node file, and generating a character string through SHA-256 after standardization; recursively processing the sub-nodes to obtain a node tree hash value, sorting the sub-nodes, splicing the sub-nodes according to a fixed format, and generating a character string through SHA-256 operation; then, a current node standardized path is spliced and generated through SHA-256 operation; and finally, by taking the root node as a starting point and the DFS as a sequential serialization tree structure, executing SHA-256 operation after adding a global collision salt value, and outputting a global unique fingerprint of the model. The problems of poor stability, low efficiency and high generation risk in a model fingerprint generation technology in the prior art can be solved.

The invention relates to a simulation result compression method and system based on semantic segmentation, equipment and a medium. The method comprises the following steps: performing feature extraction on a simulation result file to obtain semantic features; generating a segmentation point sequence according to the features by using a semantic segmentation model, and segmenting the file into a plurality of semantic blocks; extracting a compressibility feature vector for each block, and dynamically distributing an optimal compression strategy for the block through a prediction model; compressing each block by using the distributed strategy to generate a compressed data block; and finally, constructing a block sensing index based on the block type and the used strategy, and integrating the index and the data block to generate a compressed file. By adopting the method, the defects of limited compression ratio and incapability of supporting rapid random access caused by lack of semantic understanding of a general compression algorithm can be overcome, and by understanding the internal logic structure of the file and applying differential compression, the rapid retrieval and analysis capability of specific data is kept while the high compression ratio is realized.

The invention provides a file positioning management method and system based on artificial intelligence, and relates to the technical field of artificial intelligence. According to the method, files are collected from multiple sources and subjected to standardization processing, an element set is generated in combination with multi-modal analysis of texts, images, audios, videos and tables, cross-modal alignment is achieved through a semantic representation model, hierarchical indexes of semantics, keywords and relations are constructed, and a unique traceability identifier is generated; in the query stage, intention recognition and joint retrieval are carried out, a result subjected to permission verification and traceability information labeling is output, online optimization and incremental reconstruction are executed based on user feedback, and comprehensiveness, accuracy, traceability and self-adaptive optimization of file positioning are achieved.

The invention provides a log data reading method and device, equipment, a medium and a program product, and relates to the field of big data. Obtaining a mapping relationship between a plurality of timeline identifiers in a database and corresponding start log serial numbers; the timeline identifier is an identifier generated when the main database and the standby database are switched, whether the log data cross different timeline identifiers or not is judged according to the mapping relation and the starting log serial number and the ending log serial number of the log data, and under the condition that the log data cross different timeline identifiers, the log data cross different timeline identifiers. Reconstructing a log reading process according to a start log serial number of a new timeline identifier, wherein the new timeline identifier is different from the plurality of timeline identifiers; according to the method, automatic reconstruction of the log reading process after main and standby switching is achieved, the problems of low efficiency and poor real-time performance caused by manual operation are avoided, and continuity and consistency of data synchronization of the source end and the target end are guaranteed.

The invention discloses a batch file intelligent processing method and system in office development. The method comprises the steps of receiving and verifying parameters including a source directory, a target directory, a processing range and the like; traversing the directory based on the related path and the processing range, generating a structure, and collecting information to establish a file index; positioning the changed file by means of a file index, carrying out multi-dimensional difference comparison and recording a result; detecting codes and row separators in parallel for files in the file index and a target file provided by a user, generating a repair strategy and executing format repair; analyzing the natural language instruction based on the replacement rule to generate an executable rule set, and executing batch processing operation on the processed files; and summarizing various results to generate and store an output report. According to the method, intelligent batch processing of the files is realized, and the file processing efficiency and accuracy in office development are improved.

The embodiment of the invention provides an Internet of Things control system and method, electronic equipment, a storage medium and a program product, and relates to the technical field of industrial Internet of Things. The system comprises a connection management module used for establishing and maintaining connections between a plurality of Internet of Things devices and a plurality of MQTT servers corresponding to the Internet of Things devices; the dynamic equipment mapping module is used for mapping the data of the Internet of Things equipment to the MQTT server through the configuration file; and the message queue and network disconnection continuous transmission module is used for putting the message into the tail of the message queue to continuously transmit the message if the MQTT client instance corresponding to the Internet of Things equipment is detected to be in a network disconnection state when the message is published to the MQTT server. According to the scheme, dynamic expansion of heterogeneous equipment in the industrial Internet of Things, parallel butt joint of multiple cloud platforms, network disconnection continuous transmission and two-way communication capability are realized.

The embodiment of the invention discloses a virtual file management method, system and device and a storage medium. According to the technical scheme provided by the embodiment of the invention, under the condition that the file request is received, the virtual path contained in the file request is extracted, the virtual path is created when the target file is generated, path classification is performed according to the module identifier of the associated target service module, and the target file is stored in the system path mapped by the virtual path. The system path is generated by mapping based on the virtual path and a pre-constructed mapping rule; mapping the virtual path into a corresponding system path based on a mapping rule; and accessing the target file based on the system path to execute a file operation corresponding to the file request. By means of the technical means, the situation that file directory structures of different service modules are disordered can be avoided, unified file management of the different service modules in the application is achieved, the file management efficiency of the different service modules is improved, the application service effect is improved, and therefore the user experience is improved.

A system can include a memory device comprising a plurality of dynamic capacity devices and a processing device, operatively coupled with the memory device. The processing device is configured to perform operations including receiving, from a host system, a request to perform a computational operation on first file system data, wherein the request comprises an identifier of the first file system data and specifies the computational operation, wherein the first file system data is stored in a first memory section of a plurality of memory sections of a plurality of dynamic capacity devices of a memory device, wherein the first memory section is associated with a first tag; performing the computational operation on the first file system data to obtain second data; determining a second memory section of the plurality of dynamic capacity devices and associating the second memory section with a second tag; and storing the second data in the second memory section associated with the second tag.

The invention discloses a distributed file backup method for a sonar database system, which belongs to the technical field of sonar data processing and comprises the following steps: acquiring file data containing underwater acoustic signal data from the sonar database system; fragmenting the file data to generate data blocks and dynamically adjusting the sizes of the data blocks; generating a unique identifier for the data block based on the content hash value and the metadata; selecting distributed storage nodes based on availability, storage capacity, network delay and geographical distribution factors; the transmission data block uses an encryption protocol; updating the storage position and state of the backup metadata record; periodically verifying the integrity and availability of the backup data; executing a data recovery process to recover the original file data; the dynamically adjusted backup policy is based on real-time operation requirements and data importance levels. According to the method, through distributed storage and dynamic optimization, the backup efficiency, the data security and the system reliability are improved, and the method is suitable for sonar data management in an underwater detection system.

The invention provides a storage device, a storage method and a storage access method based on lake and warehouse integration. The storage device comprises a storage management unit, a unified metadata management unit and a federal query engine unit, the storage management unit is used for realizing hybrid storage and dynamic hierarchical storage; the unified metadata management unit is used for metadata atlas construction, concurrency control and GDPR compliance support; the federal query engine is used for intelligent routing decision, SQL dialect unification and intelligent push-down optimization; based on a lake and warehouse integrated storage device and an access technology thereof, unified management of structured and unstructured data is supported, the problem of data islands is avoided, and cross-engine data visibility is realized through unified metadata management; according to the lake and warehouse integrated storage device and the access method, technologies such as a dynamic hierarchical storage strategy, intelligent routing decision, intelligent push-down optimization and the like are fused, and the storage and access efficiency and the robustness of the system are greatly improved.

The invention discloses an edge AIGC environment-oriented verifiable ownership storage method and framework. The method comprises the following steps: constructing a perceptual hash and verifiable storage ownership framework comprising a user, an edge AIGC generation server, an ownership evidence calculation node and a distributed evidence warehouse; a user submits an AIGC generation request to the edge AIGC generation server, and the edge AIGC generation server generates a target AIGC image; performing perceptual feature extraction on the target AIGC image to generate a unique hash sequence; binding the unique hash sequence with a user ID and sending the unique hash sequence and the user ID to the distributed evidence warehouse, and after the user confirms that the target AIGC image is received, storing binding information of the unique hash sequence and the user ID by the distributed evidence warehouse; and when a suspected infringement image appears, calculating a hash sequence of the suspected infringement image and querying an originally stored unique hash sequence to judge and determine whether infringement occurs, so as to realize verifiable storage of the ownership of the AIGC image.

The present disclosure describes a computer-implemented method for generating certification data for an electronic document, which comprises carrying out, in a computing unit, the steps of: a) authenticating a user by means of an identity verification process, the identity verification process including carrying out at least a first authentication and a second authentication; b) receiving, from a user terminal, a status certification request associated with a file, wherein the file is stored in a database, and wherein the file includes at least the electronic document; c) transmitting, to a validating server, a verification request that includes the file and a piece of identification data associated with the user; and d) receiving, from the validating server, a piece of certification data associated with the verification request.

The invention relates to the technical field of file storage management, and discloses a Lustre architecture-based high-performance file storage and processing method and system, the method comprises the following steps: obtaining spatio-temporal characteristics of file access; predicting a popularity value of the file based on the spatial-temporal characteristics; determining a storage strategy of the file based on the popularity value of the file; carrying out distributed storage on the file based on the storage strategy, and continuously detecting a node state; if it is detected that the node load is unstable, adaptive data migration is carried out on the node with the unstable load; if the node is detected to have a fault, generating a data recovery path based on the three-dimensional topological coordinates of the node, and migrating a file stored in the node having the fault based on the data recovery path; according to the method, the storage strategy is determined through the file popularity value, and the abnormal condition of the node is efficiently processed, so that the file storage and processing performance and efficiency are comprehensively improved.

In general, techniques for efficient data retrieval from a backup system are described. An example computing system includes one or more storage devices and processing circuitry having access to the one or more storage devices and configured to: process an input to generate a filter, wherein the input indicates a context for one or more queries; apply the filter to backup data to obtain filtered data from the backup data; generate an index of embeddings from the filtered data; process, based on the index of embeddings, a query to generate a response for the query; and output the response.

The invention provides a PDF analysis method and system based on image-text semantic alignment, and belongs to the technical field of document processing. The method comprises the following steps: reading PDF or picture file byte data, and converting pictures into PDF byte data in a unified format; creating an image and a Markdown directory according to the output directory, the PDF name and an analysis method; extracting byte data of a specified page of the PDF according to starting and ending page numbers; selecting a back end to analyze PDF byte data to obtain a reasoning result, an image list and the like; generating an intermediate JSON containing PDF detailed information according to an analysis result; analyzing the positions of the text and the picture, and associating information to realize semantic alignment of the picture and the text; and generating multiple types of machine readable files according to the intermediate JSON and storing the files in a specified directory. According to the method, the PDF multi-mode content can be accurately extracted and subjected to semantic alignment, the PDF multi-mode content is efficiently converted into a machine readable format, the analysis accuracy and usability are improved, and the method is suitable for PDF analysis of multiple image-text tables such as scientific and technical literatures.

The invention discloses a multi-algorithm fusion big data analysis method based on integrated collaboration. The method specifically comprises the steps that a multi-algorithm fusion engine layer receives a feature flow, a strategy flow and a resource quota from an integrated collaboration control layer; calling a plurality of basic models in an algorithm storage, wherein the basic models comprise a machine learning model, a deep learning model, a graph model or a federal learning model; generating a sparse gating weight through the sparse gating network; performing weighted fusion on the activated basic model output based on the sparse gating weight to obtain multi-model fusion output; in the training process, differential privacy protection is executed through the security aggregation gateway; the collaborative feedback channel continuously monitors a performance drift index, and if it is detected that performance reduction exceeds a preset threshold value and continuously exceeds a tolerance round, the system automatically rolls back to an optimal model state or triggers incremental training; and the training result and the performance index are pushed to the A / B index board in real time.

The invention discloses a rapid identification and grading early warning method for sensitive files of customs field mobile equipment, and relates to the technical field of electronic data forensics, and the method comprises the following specific steps: constructing a sensitive key feature library, collecting and analyzing mirror image data, fusing and judging sensitive information, and automatically generating a sensitive grading early warning and risk report. According to the method, sensitive information samples of customs law enforcement cases, supervision notification cases and compliance public data sources are collected, and accurate extraction of four modal features of images, texts, audios and videos is ensured by using a multi-modal sensitive feature extraction formula; in the preliminary screening stage, file feature vectors are compared in batches through a preliminary screening similarity calculation formula, and suspicious files are quickly marked; in the recognition stage, multi-dimensional matching is adopted for files of different formats, and recognition confidence coefficients of all modes are generated; finally, whether the to-be-detected file is a sensitive file or not is judged through a comprehensive confidence coefficient calculation formula, and the omission ratio is reduced.

The invention relates to the technical field of archive management, in particular to an electronic archive full life cycle management method and system, and the method comprises the steps: obtaining a to-be-managed electronic archive and a life cycle stage identifier, matching a corresponding stage management rule set, and recognizing a core and an auxiliary management action; checking a core action completion condition, counting an auxiliary action matching degree and judging whether an execution threshold value is reached or not; and after reaching the standard, generating a stage conversion instruction and an operation list, updating a stage identifier and distributing a next stage task. According to the method and the device, the core defect of lack of staged management and control in existing electronic archive management can be overcome, and the accuracy and the efficiency of electronic archive management are improved.

The invention relates to an electric power big data efficient collection and storage system and method based on brain-computer interaction, and the system comprises an electroencephalogram signal collection module which is used for obtaining an electroencephalogram signal of a user through a brain-computer interface, and transmitting the electroencephalogram signal to an edge node in real time; the intention recognition module is used for recognizing the current intention of the user by adopting a pre-training intention recognition model based on the brain wave signal of the user and generating a corresponding electric power data acquisition instruction; the data acquisition control module is used for dynamically scheduling the power sensor nodes in different areas to carry out power data acquisition according to the current intention of the user in combination with the state of the power sensor nodes fed back by the edge nodes in real time; the edge calculation processing module is used for carrying out edge processing and feature compression on the collected power data by adopting edge nodes; and the distributed storage module is used for carrying out distributed storage on the power data subjected to feature compression. Compared with the prior art, the method has the advantages of non-contact rapid instruction input, improvement of response efficiency and the like.

The invention discloses a method and a system for realizing compatible file hard link and S3 shallow copy based on a distributed key-value pair, a file metadata key-value pair is used as a metadata storage main body of a file or a directory, a key consists of a file system ID (Identity), a parent directory ID and a file or directory name, and a value contains metadata information; a reverse index file handle key value pair is introduced to serve as a unique invariant identifier, a key is composed of a file system ID, an initial parent directory ID, an initial file or first N characters of a directory name and a file ID, and a value contains a current file metadata key and a reference count. When files or directories are created and deleted, the files or the directories are located at the same one-key value pair node in a large probability, and one-stage submission is supported to improve performance; hard linking and S3 shallow copying are achieved at the same time, independent file metadata key-value pairs and reverse index file handle key-value pairs are created for the hard linking and the S3 shallow copying, reverse index file handle keys of the target file are recorded by file metadata values of the file metadata key-value pairs, reference counts of the reverse index file handle values of the target file are updated, and performance and compatibility are both considered.

The invention provides a deep processing method and system for a digital delivery result, and relates to the technical field of data processing. A delivery result file set is disassembled and recombined into a unified object main data set and a three-dimensional entity set; each object attribute record and each object relation record have a unified object identifier, a source identifier and evidence positioning information at the same time, and multiple sets of coding calibers of the same object can be gathered to the same unified object identifier; refined model replacement is executed by taking a unified object identifier as an anchor point in three-dimensional treatment, continuity of a composition relationship and a connection relationship is kept, and after project handover and commissioning, online and field inconsistency found by operation and maintenance personnel is converted into a revision chain capable of being rechecked, judged and audited, so that the operation and maintenance efficiency is improved. Therefore, the problems that the unstructured data is difficult to reuse, the coding calibers are different, the object relationship is easy to lose and the data in the operation and maintenance period is not credible are solved.

A method for protecting a computer which includes a processor configured to access a set of files, the method including the processor detecting a first request to access a given file. in response to the first request, a notification of the first request is conveyed by the processor to a server prior to executing the first request. In response to receiving the notification, a second request to retrieve metadata from the given file is conveyed by the server to the computer. In response to receiving the second request, the requested metadata is retrieved from the given file by the processor. The retrieved metadata is conveyed by the computer to the server, and in response to receiving the conveyed metadata, a decision as to whether to authorize the first request is conveyed from the server to the computer. Finally, the processor responds to the first request in accordance with the decision.

The embodiment of the invention provides a data processing method and device based on a graphics processor, equipment and a storage medium, and the method comprises the steps: responding to a data processing request of a large model application, accessing a target file system in a user mode, and obtaining an access position of target data corresponding to the data processing request in the target file system, the target file system is used for managing the storage unit, and the target data is persistent data stored in the storage unit; creating a data transmission path between the storage unit and the graphics processor based on the access position; and based on the data transmission path, calling a graphics processor to process the target data, and generating a request result corresponding to the data processing request. By accessing the target file system for uniformly managing the storage unit, the graphics processor can directly read and write data for the storage unit, and the efficiency of processing the target data corresponding to the data processing request by the graphics processor is improved.

The application discloses a block data archiving method and device for a blockchain and a medium. The method comprises the following steps: determining a block start number and a block end number corresponding to to-be-archived block data, constructing a to-be-executed archiving application through a private key of a to-be-archived blockchain node corresponding to the to-be-archived block data; publishing the to-be-executed archiving application on a chain, auditing the to-be-executed archiving application by an archiving smart contract in the blockchain, and publishing an archiving audit result; in the case that the audit is passed, packaging the to-be-archived block data by the to-be-archived blockchain node to generate a to-be-archived block data file and uploading the to-be-archived block data file to a preset storage area; after receiving a message notification of successful uploading, constructing a successful archiving notification based on an archiving application transaction Hash corresponding to the to-be-executed archiving application and a storage address of the to-be-archived block data file contained in the message notification, and publishing the successful archiving notification. The application avoids the risk of data loss or leakage and ensures that the archiving behavior is traceable.