1528results about "File access structures" patented technology

The invention relates to the technical field of archive management, and provides an archive resource intelligent classification and association indexing method based on multi-modal ai analysis, which comprises the following steps: preprocessing collected archive data to form a multi-modal data set comprising text data, image data and audio data; extracting feature vectors of the text data, the image data and the audio data, and fusing the feature vectors into a unified archive representation vector through a cross-modal attention fusion mechanism; constructing a hierarchical classification model, and performing intelligent classification on the archives in combination with a domain knowledge graph; the method comprises the following steps: constructing an associated index of an archive from four dimensions of entities, semantics, time and space and events, establishing a feedback mechanism, dynamically updating and optimizing the associated index, and providing multi-modal retrieval and visual display based on a classification system and the associated index. According to the intelligent classification and association indexing method for the archive resources, the problems that in an existing archive classification and indexing method, the multi-modal processing capacity is insufficient, the association mining depth is insufficient, and the efficiency is low are solved.

The invention discloses an unstructured storage hierarchical strategy optimization method based on machine learning, and relates to the technical field of data storage management, and the method comprises the steps: monitoring a file access event in real time, generating an access log, and extracting a multi-dimensional feature data set; utilizing the trained multi-dimensional decision tree model to distribute a corresponding storage hierarchy for the storage object to obtain a storage hierarchy decision; according to a storage level decision, distributing the storage objects to different storage layers, and carrying out resource configuration and storage operation; monitoring the access condition of the storage object in the new storage hierarchy in real time, and collecting file access performance, storage cost and response time to obtain feedback data; by monitoring the file access event in real time and extracting the multi-dimensional feature data set containing the basic attribute, the access behavior and the context information, efficient response of hotspot data and reasonable utilization of resources are ensured, and the overall performance utilization rate and the cost effectiveness of storage are remarkably improved.

The invention provides a layout file geometric data processing method, a computer device and a storage medium. The method comprises the steps that a layout file of a three-dimensional chip is obtained, polygonal entities and layer information are extracted, the polygonal entities are distributed to corresponding layer containers, and an independent R tree index is created for each layer; performing connectivity analysis on the polygonal entity, and returning a connected component set and performance analysis statistical information; when an independent R-tree index is created for each layer, data of all polygonal entities of a single layer is obtained, a minimum boundary rectangle is generated for each polygonal entity, R-tree indexes are constructed for different layers in parallel by adopting a plurality of threads, whether the number of node entries of the R-tree indexes exceeds a splitting threshold value or not is judged, and if yes, splitting processing is carried out on nodes; and verifying the consistency of the created R tree index and the original data. The computer device and the storage medium can realize the method. The geometric data processing efficiency of the layout file can be improved.

The invention provides a data weaving platform-oriented self-service data preparation method, which comprises the following steps of: accessing original data and associated structured metadata of a target data source into a data weaving platform to generate standardized accessed original data and associated structured metadata; based on a semantic model of the data weaving platform, performing classified semantic annotation on the associated structured metadata, and positioning a transfer node of the associated structured metadata to generate a data blood relationship record; performing bidirectional quality verification on the data consanguinity record according to semantics and path continuity, and matching with a federated governance rule of a data weaving platform to generate a metadata governance list; and generating a unified metadata directory based on the metadata governance list, so that a user of the data knitting platform can obtain key segments containing complete data consanguinity records. According to the method, the problem of insufficient accuracy and efficiency in self-service data preparation can be effectively relieved, and the efficient preparation requirement of a data weaving platform for multi-source data is met.

A metadata access method, apparatus, and computer-readable storage medium for efficient metadata retrieval through cache management. The method receives metadata query requests including target index information from processes and performs matching operations on a global cache file containing records with index information and slot identifiers. Each cache description array corresponds to memory blocks caching metadata. Upon successful matching, the target cache description array is accessed using the target slot identifier. The data state of target metadata is determined from the cache description array, and target address information indicating the location of the target memory block in shared memory is obtained and returned to the requesting process, enabling efficient shared memory-based metadata access.

The invention relates to the technical field of data processing, and provides a BMC fault recording method and device, computer equipment and a storage medium, and the method comprises the steps: responding to a kernel fault of a target operating system of a target BMC, and printing fault log information; obtaining a pre-configured memory technology device partition; transmitting the fault log information to an annular fault cache region to form cache log data; storing the cached log data in a memory technical equipment partition, and generating a fault log file; and in response to a user reading instruction, reading and outputting the fault log file of the memory technical equipment partition. According to the technical scheme provided by one or more embodiments of the invention, the fault log can be effectively recorded, maintenance personnel can timely and accurately know the fault position and the fault reason, and BMC fault diagnosis and repair can be efficiently completed.

The invention relates to a mirror image file processing method and device, and relates to the technical field of container mirror image generation. The method comprises the steps of obtaining target file system directory information of a target container in response to a mirror image file export request for the target container; the target file system directory information of the target container is used for representing file system directory information of a writable layer of the target container; comparing the target file system directory information of the basic mirror image with the target file system directory information of the target container to obtain a difference file of the target container relative to the basic mirror image; constructing a layer file of a new incremental layer based on the difference file; writing a layer file of the incremental layer into a mirror image layer file directory of the basic mirror image, and updating a list file and an index file of the basic mirror image to obtain a mirror image file of the target container; the manifest file comprises metadata of each mirror image layer in the basic mirror image, and the index file comprises metadata of the manifest file. By adopting the method, the export efficiency of the container mirror image file can be improved.

The invention provides a ship-based unified storage and management design data method, which comprises the following steps that: an operation end generates and sends an editing request with a node identifier, an editing type, a timestamp and a digital signature, and a WEB front end receives and verifies the legality of the request through an encryption channel; the storage service generates an intention lock based on the permission table and version number matching, and concurrent access control is achieved. Then, the WEB front end carries out content definition partitioning (CDC) and Hash fingerprint calculation on a design file stream, the storage service realizes partitioning duplicate removal and incremental storage according to a fingerprint matching result, and a Merkle tree is utilized to construct a file version integrity Hash root; on this basis, the system analyzes model differences, constructs a semantic difference chart, and calls a rule engine to perform automatic pre-check. According to the method, efficient storage, collaborative editing, security and credibility and hierarchical scheduling management of ship design data are realized, and the method has the technical advantages of high data consistency, high resource utilization rate, traceability auditing and the like.

A system can include a memory device comprising a plurality of dynamic capacity devices and a processing device, operatively coupled with the memory device. The processing device is configured to perform operations including sending, to a memory device, an allocation request to allocate, for a file system, a first memory section of a plurality of memory sections of a plurality of dynamic capacity devices associated with the memory device; reserving, for the file system, a first portion of the first memory section for storing file metadata of file system data; reserving, for the file system, a second portion of the first memory section for storing file data of file system data; sending, by a first process running on the host system, to the memory device, the file metadata for storing the file metadata in the first portion of the first memory section; sending, by a second process running on the host system, to the memory device, a second request to modify the file metadata stored in the first portion of the first memory section; and receiving, from the memory device, a second response indicating an error regarding the second request.

The invention belongs to the technical field of data processing, and particularly relates to a fingerprint generation method of a virtual evaluation model based on Hash operation, which comprises the following steps of: firstly, constructing a dependency tree by taking a main model file specified by a user as a root node, and initializing an access record table; establishing a static keyword protection area containing a core parameter rule base, scanning a dynamic file operated by a user, and comparing parameter definitions to generate an early warning; reading the binary data of the current node file, and generating a character string through SHA-256 after standardization; recursively processing the sub-nodes to obtain a node tree hash value, sorting the sub-nodes, splicing the sub-nodes according to a fixed format, and generating a character string through SHA-256 operation; then, a current node standardized path is spliced and generated through SHA-256 operation; and finally, by taking the root node as a starting point and the DFS as a sequential serialization tree structure, executing SHA-256 operation after adding a global collision salt value, and outputting a global unique fingerprint of the model. The problems of poor stability, low efficiency and high generation risk in a model fingerprint generation technology in the prior art can be solved.

The invention relates to a simulation result compression method and system based on semantic segmentation, equipment and a medium. The method comprises the following steps: performing feature extraction on a simulation result file to obtain semantic features; generating a segmentation point sequence according to the features by using a semantic segmentation model, and segmenting the file into a plurality of semantic blocks; extracting a compressibility feature vector for each block, and dynamically distributing an optimal compression strategy for the block through a prediction model; compressing each block by using the distributed strategy to generate a compressed data block; and finally, constructing a block sensing index based on the block type and the used strategy, and integrating the index and the data block to generate a compressed file. By adopting the method, the defects of limited compression ratio and incapability of supporting rapid random access caused by lack of semantic understanding of a general compression algorithm can be overcome, and by understanding the internal logic structure of the file and applying differential compression, the rapid retrieval and analysis capability of specific data is kept while the high compression ratio is realized.

The invention provides a file positioning management method and system based on artificial intelligence, and relates to the technical field of artificial intelligence. According to the method, files are collected from multiple sources and subjected to standardization processing, an element set is generated in combination with multi-modal analysis of texts, images, audios, videos and tables, cross-modal alignment is achieved through a semantic representation model, hierarchical indexes of semantics, keywords and relations are constructed, and a unique traceability identifier is generated; in the query stage, intention recognition and joint retrieval are carried out, a result subjected to permission verification and traceability information labeling is output, online optimization and incremental reconstruction are executed based on user feedback, and comprehensiveness, accuracy, traceability and self-adaptive optimization of file positioning are achieved.

The invention provides a log data reading method and device, equipment, a medium and a program product, and relates to the field of big data. Obtaining a mapping relationship between a plurality of timeline identifiers in a database and corresponding start log serial numbers; the timeline identifier is an identifier generated when the main database and the standby database are switched, whether the log data cross different timeline identifiers or not is judged according to the mapping relation and the starting log serial number and the ending log serial number of the log data, and under the condition that the log data cross different timeline identifiers, the log data cross different timeline identifiers. Reconstructing a log reading process according to a start log serial number of a new timeline identifier, wherein the new timeline identifier is different from the plurality of timeline identifiers; according to the method, automatic reconstruction of the log reading process after main and standby switching is achieved, the problems of low efficiency and poor real-time performance caused by manual operation are avoided, and continuity and consistency of data synchronization of the source end and the target end are guaranteed.

The invention discloses a batch file intelligent processing method and system in office development. The method comprises the steps of receiving and verifying parameters including a source directory, a target directory, a processing range and the like; traversing the directory based on the related path and the processing range, generating a structure, and collecting information to establish a file index; positioning the changed file by means of a file index, carrying out multi-dimensional difference comparison and recording a result; detecting codes and row separators in parallel for files in the file index and a target file provided by a user, generating a repair strategy and executing format repair; analyzing the natural language instruction based on the replacement rule to generate an executable rule set, and executing batch processing operation on the processed files; and summarizing various results to generate and store an output report. According to the method, intelligent batch processing of the files is realized, and the file processing efficiency and accuracy in office development are improved.

The embodiment of the invention provides an Internet of Things control system and method, electronic equipment, a storage medium and a program product, and relates to the technical field of industrial Internet of Things. The system comprises a connection management module used for establishing and maintaining connections between a plurality of Internet of Things devices and a plurality of MQTT servers corresponding to the Internet of Things devices; the dynamic equipment mapping module is used for mapping the data of the Internet of Things equipment to the MQTT server through the configuration file; and the message queue and network disconnection continuous transmission module is used for putting the message into the tail of the message queue to continuously transmit the message if the MQTT client instance corresponding to the Internet of Things equipment is detected to be in a network disconnection state when the message is published to the MQTT server. According to the scheme, dynamic expansion of heterogeneous equipment in the industrial Internet of Things, parallel butt joint of multiple cloud platforms, network disconnection continuous transmission and two-way communication capability are realized.

The embodiment of the invention discloses a virtual file management method, system and device and a storage medium. According to the technical scheme provided by the embodiment of the invention, under the condition that the file request is received, the virtual path contained in the file request is extracted, the virtual path is created when the target file is generated, path classification is performed according to the module identifier of the associated target service module, and the target file is stored in the system path mapped by the virtual path. The system path is generated by mapping based on the virtual path and a pre-constructed mapping rule; mapping the virtual path into a corresponding system path based on a mapping rule; and accessing the target file based on the system path to execute a file operation corresponding to the file request. By means of the technical means, the situation that file directory structures of different service modules are disordered can be avoided, unified file management of the different service modules in the application is achieved, the file management efficiency of the different service modules is improved, the application service effect is improved, and therefore the user experience is improved.

A system can include a memory device comprising a plurality of dynamic capacity devices and a processing device, operatively coupled with the memory device. The processing device is configured to perform operations including receiving, from a host system, a request to perform a computational operation on first file system data, wherein the request comprises an identifier of the first file system data and specifies the computational operation, wherein the first file system data is stored in a first memory section of a plurality of memory sections of a plurality of dynamic capacity devices of a memory device, wherein the first memory section is associated with a first tag; performing the computational operation on the first file system data to obtain second data; determining a second memory section of the plurality of dynamic capacity devices and associating the second memory section with a second tag; and storing the second data in the second memory section associated with the second tag.

The invention discloses a distributed file backup method for a sonar database system, which belongs to the technical field of sonar data processing and comprises the following steps: acquiring file data containing underwater acoustic signal data from the sonar database system; fragmenting the file data to generate data blocks and dynamically adjusting the sizes of the data blocks; generating a unique identifier for the data block based on the content hash value and the metadata; selecting distributed storage nodes based on availability, storage capacity, network delay and geographical distribution factors; the transmission data block uses an encryption protocol; updating the storage position and state of the backup metadata record; periodically verifying the integrity and availability of the backup data; executing a data recovery process to recover the original file data; the dynamically adjusted backup policy is based on real-time operation requirements and data importance levels. According to the method, through distributed storage and dynamic optimization, the backup efficiency, the data security and the system reliability are improved, and the method is suitable for sonar data management in an underwater detection system.

The invention discloses an MCP-based log data interactive retrieval visualization and report generation system, and relates to the technical field of log visualization. The system comprises a semantic analysis engine, an intelligent visualization engine, a structured report generation engine, an intelligent filing engine and an adaptive optimization system. Each module constructs a closed loop through a data stream driven by context semantics and a feedback path, and supports the whole process from original log analysis, visual analysis and semantic report generation to archiving management and strategy iteration. The system has a context-enhanced semantic modeling capability, an interactive multi-view linkage mechanism, an automatic report arrangement structure and a feedback-driven optimization capability, improves the structural property, interpretability and adaptability of log analysis compared with a traditional scheme, and is particularly suitable for complex scenes such as troubleshooting, security audit and system situation awareness.

The invention provides a storage device, a storage method and a storage access method based on lake and warehouse integration. The storage device comprises a storage management unit, a unified metadata management unit and a federal query engine unit, the storage management unit is used for realizing hybrid storage and dynamic hierarchical storage; the unified metadata management unit is used for metadata atlas construction, concurrency control and GDPR compliance support; the federal query engine is used for intelligent routing decision, SQL dialect unification and intelligent push-down optimization; based on a lake and warehouse integrated storage device and an access technology thereof, unified management of structured and unstructured data is supported, the problem of data islands is avoided, and cross-engine data visibility is realized through unified metadata management; according to the lake and warehouse integrated storage device and the access method, technologies such as a dynamic hierarchical storage strategy, intelligent routing decision, intelligent push-down optimization and the like are fused, and the storage and access efficiency and the robustness of the system are greatly improved.

The invention discloses an edge AIGC environment-oriented verifiable ownership storage method and framework. The method comprises the following steps: constructing a perceptual hash and verifiable storage ownership framework comprising a user, an edge AIGC generation server, an ownership evidence calculation node and a distributed evidence warehouse; a user submits an AIGC generation request to the edge AIGC generation server, and the edge AIGC generation server generates a target AIGC image; performing perceptual feature extraction on the target AIGC image to generate a unique hash sequence; binding the unique hash sequence with a user ID and sending the unique hash sequence and the user ID to the distributed evidence warehouse, and after the user confirms that the target AIGC image is received, storing binding information of the unique hash sequence and the user ID by the distributed evidence warehouse; and when a suspected infringement image appears, calculating a hash sequence of the suspected infringement image and querying an originally stored unique hash sequence to judge and determine whether infringement occurs, so as to realize verifiable storage of the ownership of the AIGC image.

The present disclosure describes a computer-implemented method for generating certification data for an electronic document, which comprises carrying out, in a computing unit, the steps of: a) authenticating a user by means of an identity verification process, the identity verification process including carrying out at least a first authentication and a second authentication; b) receiving, from a user terminal, a status certification request associated with a file, wherein the file is stored in a database, and wherein the file includes at least the electronic document; c) transmitting, to a validating server, a verification request that includes the file and a piece of identification data associated with the user; and d) receiving, from the validating server, a piece of certification data associated with the verification request.

The invention relates to the technical field of file storage management, and discloses a Lustre architecture-based high-performance file storage and processing method and system, the method comprises the following steps: obtaining spatio-temporal characteristics of file access; predicting a popularity value of the file based on the spatial-temporal characteristics; determining a storage strategy of the file based on the popularity value of the file; carrying out distributed storage on the file based on the storage strategy, and continuously detecting a node state; if it is detected that the node load is unstable, adaptive data migration is carried out on the node with the unstable load; if the node is detected to have a fault, generating a data recovery path based on the three-dimensional topological coordinates of the node, and migrating a file stored in the node having the fault based on the data recovery path; according to the method, the storage strategy is determined through the file popularity value, and the abnormal condition of the node is efficiently processed, so that the file storage and processing performance and efficiency are comprehensively improved.

In general, techniques for efficient data retrieval from a backup system are described. An example computing system includes one or more storage devices and processing circuitry having access to the one or more storage devices and configured to: process an input to generate a filter, wherein the input indicates a context for one or more queries; apply the filter to backup data to obtain filtered data from the backup data; generate an index of embeddings from the filtered data; process, based on the index of embeddings, a query to generate a response for the query; and output the response.

The application provides a data processing method and a data storage system. The method comprises training, by the data storage system, a first streaming model according to a plurality of first access data features corresponding to the first file at an i-1th access request and a first access attribute parameter of the first file, then inputting a plurality of second access data features corresponding to the first file at an ith access request into the first streaming model to obtain a second access attribute parameter possessed by the first file at an i+1th access request, and then pre-fetching or migrating the first file according to the second access attribute parameter. In this way, the method trains the first streaming model using the plurality of first access data features, thereby consuming relatively few storage resources and computing resources, and enabling the first streaming model to adapt to changing file access patterns, thereby effectively improving the business performance of the data storage system.

The invention provides a PDF analysis method and system based on image-text semantic alignment, and belongs to the technical field of document processing. The method comprises the following steps: reading PDF or picture file byte data, and converting pictures into PDF byte data in a unified format; creating an image and a Markdown directory according to the output directory, the PDF name and an analysis method; extracting byte data of a specified page of the PDF according to starting and ending page numbers; selecting a back end to analyze PDF byte data to obtain a reasoning result, an image list and the like; generating an intermediate JSON containing PDF detailed information according to an analysis result; analyzing the positions of the text and the picture, and associating information to realize semantic alignment of the picture and the text; and generating multiple types of machine readable files according to the intermediate JSON and storing the files in a specified directory. According to the method, the PDF multi-mode content can be accurately extracted and subjected to semantic alignment, the PDF multi-mode content is efficiently converted into a machine readable format, the analysis accuracy and usability are improved, and the method is suitable for PDF analysis of multiple image-text tables such as scientific and technical literatures.

The invention discloses a multi-algorithm fusion big data analysis method based on integrated collaboration. The method specifically comprises the steps that a multi-algorithm fusion engine layer receives a feature flow, a strategy flow and a resource quota from an integrated collaboration control layer; calling a plurality of basic models in an algorithm storage, wherein the basic models comprise a machine learning model, a deep learning model, a graph model or a federal learning model; generating a sparse gating weight through the sparse gating network; performing weighted fusion on the activated basic model output based on the sparse gating weight to obtain multi-model fusion output; in the training process, differential privacy protection is executed through the security aggregation gateway; the collaborative feedback channel continuously monitors a performance drift index, and if it is detected that performance reduction exceeds a preset threshold value and continuously exceeds a tolerance round, the system automatically rolls back to an optimal model state or triggers incremental training; and the training result and the performance index are pushed to the A / B index board in real time.

The invention relates to the technical field of distributed storage system monitoring, and particularly discloses a metadata real-time monitoring method, device and equipment of a distributed storage system. The method comprises the following steps of: dynamically loading a file system mirror image to construct an initial memory metadata model by deploying an auditing service which is physically isolated from a main node and a standby node; subscribing metadata change event streams published by log nodes in real time, and incrementally updating the model after time sequence sorting and conflict detection; key dimension fields are extracted and converted into a structured format, and after batch compression, the key dimension fields are synchronized to an index engine to construct a dual-path index; and generating cross-level directory topology analysis, a user behavior portrait and storage block distribution early warning in response to query. According to the method, the problems of high time delay, metadata splitting and main node performance interference caused by offline analysis in a traditional scheme are solved, second-level updated full-quantity metadata acquisition and multi-dimensional real-time analysis are realized, and the monitoring efficiency and storage optimization capability of a massive file system are remarkably improved.

The invention discloses a rapid identification and grading early warning method for sensitive files of customs field mobile equipment, and relates to the technical field of electronic data forensics, and the method comprises the following specific steps: constructing a sensitive key feature library, collecting and analyzing mirror image data, fusing and judging sensitive information, and automatically generating a sensitive grading early warning and risk report. According to the method, sensitive information samples of customs law enforcement cases, supervision notification cases and compliance public data sources are collected, and accurate extraction of four modal features of images, texts, audios and videos is ensured by using a multi-modal sensitive feature extraction formula; in the preliminary screening stage, file feature vectors are compared in batches through a preliminary screening similarity calculation formula, and suspicious files are quickly marked; in the recognition stage, multi-dimensional matching is adopted for files of different formats, and recognition confidence coefficients of all modes are generated; finally, whether the to-be-detected file is a sensitive file or not is judged through a comprehensive confidence coefficient calculation formula, and the omission ratio is reduced.

The invention relates to the technical field of archive management, in particular to an electronic archive full life cycle management method and system, and the method comprises the steps: obtaining a to-be-managed electronic archive and a life cycle stage identifier, matching a corresponding stage management rule set, and recognizing a core and an auxiliary management action; checking a core action completion condition, counting an auxiliary action matching degree and judging whether an execution threshold value is reached or not; and after reaching the standard, generating a stage conversion instruction and an operation list, updating a stage identifier and distributing a next stage task. According to the method and the device, the core defect of lack of staged management and control in existing electronic archive management can be overcome, and the accuracy and the efficiency of electronic archive management are improved.