2344results about "File access structures" patented technology

The invention provides a meteorological metadata storage method and system based on machine learning, and the method comprises the steps: obtaining a meteorological metadata set of a target region, inputting the meteorological metadata set into a pre-trained time sequence feature extraction model, generating the time sequence distribution features of all meteorological file files, and inputting a spatial topology mapping model, generating spatial correlation topological features of the meteorological elements; based on a cross fusion result of the time sequence distribution features and the space correlation topological features, a hierarchical index structure of meteorological metadata is constructed, a meteorological storage optimization model is called to generate a storage node distribution strategy of a meteorological file, and the meteorological file is stored according to the storage node distribution strategy. And writing each meteorological file file in the meteorological metadata set into the corresponding distributed storage node, and dynamically adjusting the mapping relationship between the physical position and the index of the stored file based on a real-time updating mechanism of the hierarchical index. The meteorological metadata storage efficiency and retrieval accuracy can be improved, and storage resource waste and operation and maintenance cost are reduced.

The invention discloses an optical fiber data storage management system and method based on big data, and relates to the technical field of storage management. According to the optical fiber data storage management system and method based on big data, the spatial-temporal characteristics of the historical access records are extracted through the LSTM network, the data heat value is calculated in combination with the time decay function, and the thermal distribution diagram of the data blocks is generated. And based on the thermal distribution map, constructing a storage resource allocation model by using a deep reinforcement learning algorithm, and dynamically adjusting the optical path priority and copy distribution. A data prefetching strategy is optimized through the spatio-temporal joint index and the vector similarity index, and a high-frequency data cache index distribution diagram is generated. And comprehensively considering the data access delay, the index hit rate and the copy migration frequency, obtaining an optical fiber storage efficiency index, and adjusting a storage strategy according to the index. According to the method, the dynamic scheduling and utilization efficiency of data storage resources is effectively improved, the data access delay is reduced, the copy management is optimized, and the overall performance of a storage system is remarkably improved.

The invention provides an AI agent memory management method and system based on cold and hot stratification, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining and preprocessing the historical information of an AI agent, and carrying out the vectorization storage of the historical information; calculating a score based on the access frequency and the semantic association; constructing a dual-threshold triggering mechanism to realize data cold and hot layered migration; establishing index mapping and dynamically monitoring data popularity for returning; and a storage water level dynamic adjustment migration strategy is monitored in real time. According to the invention, the problem of low memory management efficiency of the AI agent is solved, efficient access of memory content is realized, and the storage cost is reduced.

The invention belongs to the technical field of data storage, and discloses a multi-source marine environment data storage method and system based on HDF5. According to the method, a hierarchical storage framework is constructed, a domain-type-source three-level directory architecture is constructed, a space-time dimension dynamic extension mechanism is executed, and space-time composite coding is performed; designing an HDF5 nested packet storage architecture, wherein grid topological structure definition, dynamic resolution self-adaptive storage, multi-source data processing and metadata management are carried out; constructing distributed data association and indexes, including data standardization preprocessing and cross-source data association; the storage performance is optimized and expanded, wherein density sensitive blocking and layered compression are carried out. According to the method, redundant storage of a high-latitude region is greatly reduced, and dynamic adaptation of a local encryption grid is supported. According to the invention, open circulation and collaborative utilization of interdisciplinary data resources are facilitated.

The invention discloses a human resource archive filing management system, and relates to the technical field of data management, the system comprises an acquisition module, a processing module and a management module, the acquisition module acquires original archive data of employees, transmits the data to the processing module, classifies the original archive data according to a preset first rule, and stores the classified data to the management module; the first archive data is obtained, controlled and managed, a calling permission and a distribution strategy are generated, the first archive data is recognized through a preset second rule, second archive data is generated, corresponding processing is conducted according to the processing priority, a processing log is reserved, third archive data are obtained and transmitted to the management module, and the management module is used for managing the first archive data. A distributed cloud storage cluster and a local storage node are constructed, an overall storage architecture and a retrieval path of third archive data are optimized, archive management is refined through three-dimensional classification, data integrity is guaranteed through field-level verification and automatic completion, scene-based dynamic authority control safety is achieved, compliance risks are intelligently cleared and avoided, and efficiency is improved through mixed storage. And active management of archive management is promoted.

The invention discloses a smart city data sharing system based on a block chain technology, and relates to the crossing field of medical data processing and information technologies. The system comprises the following modules: 1, a data storage and block chain mapping module, which generates an abstract by using a salting SHA-256 algorithm, stores the abstract in a block chain, stores original data in an IPFS distributed manner, sets redundancy and cache, and regulates abstract storage according to data popularity; 2, a data sharing module driven by the intelligent contract, which is used for subdividing authorization levels, setting pre-request verification, dynamically updating rules and automatically executing functions to guarantee compliance and controllability; the privacy protection technology fusion module expands zero-knowledge proof application and optimizes multi-party security calculation by adopting hybrid encryption; and 4, a data tracing and compliance auditing module which records logs to form a historical chain, traces and outputs an auditing report. According to the invention, the security, efficiency and compliance of medical data sharing can be comprehensively improved, and the application prospect is wide.

The invention relates to the technical field of archive management, and provides an archive resource intelligent classification and association indexing method based on multi-modal ai analysis, which comprises the following steps: preprocessing collected archive data to form a multi-modal data set comprising text data, image data and audio data; extracting feature vectors of the text data, the image data and the audio data, and fusing the feature vectors into a unified archive representation vector through a cross-modal attention fusion mechanism; constructing a hierarchical classification model, and performing intelligent classification on the archives in combination with a domain knowledge graph; the method comprises the following steps: constructing an associated index of an archive from four dimensions of entities, semantics, time and space and events, establishing a feedback mechanism, dynamically updating and optimizing the associated index, and providing multi-modal retrieval and visual display based on a classification system and the associated index. According to the intelligent classification and association indexing method for the archive resources, the problems that in an existing archive classification and indexing method, the multi-modal processing capacity is insufficient, the association mining depth is insufficient, and the efficiency is low are solved.

The invention discloses an archive retrieval method and system based on cloud computing, and the method comprises the steps: receiving retrieval request information inputted by a user, and analyzing a retrieval intention according to the retrieval content in the retrieval request information, the historical retrieval behavior of the user and a retrieval field database by using a context perception semantic analysis model, generating a user retrieval semantic vector in combination with the retrieval intention; the method comprises the following steps: constructing an archive data index map for archive data stored in a distributed manner in a cloud storage environment; according to the user retrieval semantic vector, a search domain related to retrieval semantics is dynamically positioned in the archive data index map in a multi-layer projection calculation mode; and in the search domain, screening a candidate archive set according with the query intention by using a matching algorithm of multi-modal semantic fusion, and sorting and displaying the candidate archive set. By utilizing the embodiment of the invention, the accuracy and efficiency of file retrieval and the user experience can be improved.

The invention discloses a channel model dynamic association method based on a multi-source data index, and relates to channel engineering digital management. The channel model dynamic association method comprises the steps that component codes of channel engineering are constructed and used for identifying the same physical entity; establishing a three-layer index table structure according to the component codes; constructing a dynamic partition management mechanism of a three-level cache architecture, wherein the three-level cache architecture comprises a first-level cache stored in a memory, a second-level cache stored in a local disk and a third-level cache stored in a distributed manner; according to data access information in the three-layer index table structure, data migration is carried out among all levels of caches; according to the dynamic partition management mechanism, a data version management mechanism based on a timestamp concurrency control protocol is established, the data version management mechanism generates incremental data by comparing original data with updated data, and incremental updating or full updating is selected according to a threshold value. According to the method, the problem that data evolution tracking is difficult due to fragmentation of multi-source heterogeneous data and unbalanced access efficiency in navigation channel engineering is solved.

The invention relates to the technical field of data processing, and discloses a real-time data stream processing method and device, electronic equipment and a storage medium. The method comprises the following steps: distributing a received to-be-processed data stream to a first distributed node through a preset load balancer; fragmenting the received distribution data stream at the first distributed node according to a preset fragmentation rule, then writing the fragmented data stream into a message queue, and performing parallel processing on fragmented data in the message queue through multiple threads; storing the intermediate state data in the fragmented data processing process, generating snapshot data according to the fragmented data, and synchronizing the snapshot data to the distributed storage system; acquiring load data of the first distributed node, and dynamically updating a data flow distribution strategy of the load balancer according to the load data through the resource use prediction model; and writing a processing result of the to-be-processed data stream into persistent storage through an asynchronous batch mechanism, and returning the processing result through a preset real-time communication protocol. The method improves the data stream processing efficiency and stability.

The invention discloses an intelligent archive content evaluation system based on deep learning, which relates to the technical field of deep learning, and comprises the steps of inputting a standardized feature vector into a deep learning model, calculating a time sequence waveform factor and a semantic kurtosis factor in combination with a rule matching reference, and generating an evaluation confidence coefficient matrix; the cosine similarity and the Spearman correlation coefficient are included, and the comparison index threshold triggers abnormal marking and grading early warning. According to the method, a multi-model fusion deep learning evaluation model is constructed and comprises a semantic understanding model, a risk evaluation model and a value classification model, split attributes are dynamically optimized in combination with an evaluation rule decision tree, file contents can be comprehensively evaluated from multiple dimensions of semantic understanding, risk evaluation, value classification and the like, and the evaluation efficiency is improved. And an accurate comprehensive evaluation result and a rule matching reference are generated, so that the comprehensiveness and scientificity of file content evaluation are improved.

The invention discloses an intelligent document duplicate checking system and method based on a vector database and a large language model, and the method comprises the following steps: S1, collecting document data in various formats, and carrying out the preprocessing of the document data; s2, semantic coding is carried out through a large language model, and a document semantic vector is generated; s3, storing the document semantic vector into a vector database, constructing a vector index and recording historical query data; s4, carrying out preliminary candidate document retrieval, and carrying out approximate nearest neighbor retrieval based on outlier identification; s5, calculating the similarity between the candidate document and the document to be subjected to duplicate checking, and screening a final high-similarity document; s6, generating a duplicate checking report, and recording user operation behaviors; and S7, receiving user feedback, and dynamically updating the document semantic vector and the vector index. According to the method, efficient and accurate intelligent document duplicate checking is realized by utilizing the large language model and the vector database, the semantic matching capability is improved, the duplicate checking efficiency is optimized, and the intelligence and adaptability of a duplicate checking system are improved.

The invention provides a data classification and grading method and device based on an artificial intelligence large model technology, and relates to the field of data security. The method comprises the following steps: constructing a data classification and grading catalog according to a preset classification rule, and storing each row of the classification and grading catalog as a document record in a target vector database; respectively carrying out first retrieval operation, second retrieval operation and third retrieval operation on the data classification and grading catalog by taking the business system name, the data table name and the field name as first query conditions so as to obtain a first matching document, a second matching document and a third matching document in the target vector database; sorting the first matching document, the second matching document and the third matching document according to a preset sorting mode, and obtaining a sorting result; and inputting a sorting result into the large language model, and performing data security early warning through the large language model. According to the method and the device, the security problems that corresponding security measures cannot be taken, sensitive data is leaked and the like when a data governance engineer identifies the metadata according to the data classification and classification catalog and judges the classification and security level of the metadata are solved.

The invention discloses an unstructured storage hierarchical strategy optimization method based on machine learning, and relates to the technical field of data storage management, and the method comprises the steps: monitoring a file access event in real time, generating an access log, and extracting a multi-dimensional feature data set; utilizing the trained multi-dimensional decision tree model to distribute a corresponding storage hierarchy for the storage object to obtain a storage hierarchy decision; according to a storage level decision, distributing the storage objects to different storage layers, and carrying out resource configuration and storage operation; monitoring the access condition of the storage object in the new storage hierarchy in real time, and collecting file access performance, storage cost and response time to obtain feedback data; by monitoring the file access event in real time and extracting the multi-dimensional feature data set containing the basic attribute, the access behavior and the context information, efficient response of hotspot data and reasonable utilization of resources are ensured, and the overall performance utilization rate and the cost effectiveness of storage are remarkably improved.

The invention provides a layout file geometric data processing method, a computer device and a storage medium. The method comprises the steps that a layout file of a three-dimensional chip is obtained, polygonal entities and layer information are extracted, the polygonal entities are distributed to corresponding layer containers, and an independent R tree index is created for each layer; performing connectivity analysis on the polygonal entity, and returning a connected component set and performance analysis statistical information; when an independent R-tree index is created for each layer, data of all polygonal entities of a single layer is obtained, a minimum boundary rectangle is generated for each polygonal entity, R-tree indexes are constructed for different layers in parallel by adopting a plurality of threads, whether the number of node entries of the R-tree indexes exceeds a splitting threshold value or not is judged, and if yes, splitting processing is carried out on nodes; and verifying the consistency of the created R tree index and the original data. The computer device and the storage medium can realize the method. The geometric data processing efficiency of the layout file can be improved.

The invention provides a data weaving platform-oriented self-service data preparation method, which comprises the following steps of: accessing original data and associated structured metadata of a target data source into a data weaving platform to generate standardized accessed original data and associated structured metadata; based on a semantic model of the data weaving platform, performing classified semantic annotation on the associated structured metadata, and positioning a transfer node of the associated structured metadata to generate a data blood relationship record; performing bidirectional quality verification on the data consanguinity record according to semantics and path continuity, and matching with a federated governance rule of a data weaving platform to generate a metadata governance list; and generating a unified metadata directory based on the metadata governance list, so that a user of the data knitting platform can obtain key segments containing complete data consanguinity records. According to the method, the problem of insufficient accuracy and efficiency in self-service data preparation can be effectively relieved, and the efficient preparation requirement of a data weaving platform for multi-source data is met.

A metadata access method, apparatus, and computer-readable storage medium for efficient metadata retrieval through cache management. The method receives metadata query requests including target index information from processes and performs matching operations on a global cache file containing records with index information and slot identifiers. Each cache description array corresponds to memory blocks caching metadata. Upon successful matching, the target cache description array is accessed using the target slot identifier. The data state of target metadata is determined from the cache description array, and target address information indicating the location of the target memory block in shared memory is obtained and returned to the requesting process, enabling efficient shared memory-based metadata access.

The invention discloses an information processing method, and the method comprises the steps: inputting query information for a target object into an intention recognition model for intention recognition, and obtaining an information retrieval type corresponding to the query information; wherein the query information is used for querying all related log information or part of related log information of the target object; retrieving the query information through a retrieval strategy corresponding to the information retrieval type by utilizing a pre-constructed index database to obtain a retrieval result meeting an information length condition corresponding to an input requirement of a second large language model; wherein different information retrieval types correspond to different retrieval strategies; performing fusion processing on the query information and the retrieval result to generate context reference information meeting the input requirement of the second large language model; and reasoning the context reference information by using the second large language model to generate reply content corresponding to the query information.

The invention provides a star catalogue file generation method and device, a celestial body data query method and device, computer equipment and a storage medium. The star catalogue file generation method comprises the steps of obtaining star catalogue data; dividing the celestial sphere surface area corresponding to the plurality of celestial bodies into a plurality of local areas, and determining an identifier of each local area; determining an identifier matched with each celestial body data according to the right ascension and declination included in the celestial body data and the identifier of each local area in the celestial sphere surface area; sorting the celestial body data of the plurality of celestial bodies based on the identifiers corresponding to the celestial body data; and writing the sequenced celestial body data of the plurality of celestial bodies into an initial column storage file matched with the identifier to obtain a target file stored with the celestial body data, and generating file index information corresponding to the target file. The celestial body data is stored by utilizing the column type storage file, so that the problems of high maintenance cost and high use threshold caused by utilizing a database to store data are relieved while the data query efficiency is guaranteed.

The invention relates to the technical field of data processing, and provides a BMC fault recording method and device, computer equipment and a storage medium, and the method comprises the steps: responding to a kernel fault of a target operating system of a target BMC, and printing fault log information; obtaining a pre-configured memory technology device partition; transmitting the fault log information to an annular fault cache region to form cache log data; storing the cached log data in a memory technical equipment partition, and generating a fault log file; and in response to a user reading instruction, reading and outputting the fault log file of the memory technical equipment partition. According to the technical scheme provided by one or more embodiments of the invention, the fault log can be effectively recorded, maintenance personnel can timely and accurately know the fault position and the fault reason, and BMC fault diagnosis and repair can be efficiently completed.

The invention discloses a cloud storage service implementation method and system for unified management of multiple types of storage, and belongs to the technical field of cloud storage, and the method specifically comprises the steps: building a unified abstract model oriented to heterogeneous storage media, mapping distributed files, object storage and block storage devices into standardized storage units, deploying a cloud block chain, and carrying out uplink, a storage index table under a chain is constructed, an index state is dynamically updated through a loose coupling consensus mechanism, the loose coupling consensus mechanism comprises a weighted voting algorithm based on a storage node reputation value and an asynchronous verification mechanism based on space-time certification, and data is divided into data pieces and verification pieces; fragmentation information is written into a storage metadata registration chain and a service quality proof chain at the same time, cross-chain storage is carried out on data, and cloud storage resources are dynamically distributed; according to the method, the storage management function is decoupled through the block chain, so that the retrieval accuracy is improved, the retrieval response time is effectively shortened, and the data transmission cost is reduced.

A method is described. The method includes processing first meta data for first units of stored content in a first storage system. The processing includes constructing a first tree of first nodes where first lower nodes represent fewer of the first units of stored content and first higher nodes represent more of the first units of stored content. The first nodes include respective first hash values calculated from the first meta data to represent the respective content of the respective first units of stored content that the first nodes respectively represent. The method includes processing second meta data for second units of stored content in a second storage system. The processing includes constructing a second tree of second nodes where second lower nodes represent fewer of the second units of stored content and second higher nodes represent more of the second units of stored content. The second nodes include respective second hash values calculated from the second meta data to represent the respective content of the respective second units of stored content that the second nodes respectively represent. The method includes comparing at least a subset of the first hash values with a corresponding at least a subset of the second hash values to ascertain sameness between the first and second units of stored content.

The invention discloses a mirror image management method and device, electronic equipment and a computer program product. Relates to the field of cloud and edge computing, and the method comprises the following steps: obtaining network load data of a storage node cluster, fragmenting a to-be-deployed mirror image according to the network load data to obtain N mirror image fragments, and uploading the N mirror image fragments to the storage node cluster, the to-be-deployed mirror image refers to a file required for deploying the preset model, and N is a positive integer; under the condition that a server deployment request is received, N mirror image fragments are extracted from the storage node cluster according to the server deployment request, and a mirror image file is formed by the N mirror image fragments; and loading the mirror image file, and deploying the preset model to the target area. Through the method and the device, the technical problems of low uploading efficiency and low storage resource utilization rate when the mirror image is uploaded and stored in related technologies are solved.

The invention belongs to the technical field of archive management, and discloses an intelligent archive supervision method, device and equipment and a storage medium, and the method comprises the steps: carrying out the standardization processing of an archive feature data set which is established based on entity feature information and electronic feature information, and generating the digitalized archive information which can be analyzed; processing the parsable digital archive information through a preset rule to obtain archive metadata; constructing a dynamic supervision model based on the file metadata, and performing real-time association analysis by using the dynamic supervision model; when the correlation analysis result does not meet the preset condition, a corresponding grading response mechanism is matched according to the label attribute, and a disposal instruction containing an exception type identifier is generated; and outputting a visual supervision report based on the execution result of the disposal instruction and the historical supervision data, thereby breaking through the limitation of physical feature and electronic data splitting analysis, establishing entity-electronic feature space-time correlation mapping, and realizing accurate matching of hierarchical response strategies based on file metadata tags.

The invention discloses an automobile production data tracing method, system and device based on a block chain and a medium, and the method comprises the steps: obtaining a product identifier of a production process, original production data and key metadata, storing the original production data to a distributed file system, and generating a unique hash value; packaging the unique hash value and the key metadata to generate a data block, generating block index information according to the unique hash value and the product identifier, and uploading the data block and the block index information to a block chain; obtaining a target hash value and a target data block through indexing on the block chain according to the target product identifier, and determining target metadata according to the target data block; and searching in the distributed file system according to the target hash value to obtain target production data, and generating a target tracing record according to the target production data and the target metadata. The automobile production data tracing efficiency and the accuracy, real-time performance and safety of the automobile production data are improved, and the automobile production data tracing method can be applied to the technical field of automobile production.

The embodiment of the invention discloses an access method in a distributed file system, computing equipment and file access equipment. The method is applied to a first metadata node, the first metadata node belongs to the multiple metadata nodes, the same directory information is stored in each metadata node in the multiple metadata nodes, and the directory information is used for storing directory metadata of the distributed file system. The method comprises the following steps: receiving a first access request from a client, wherein the first access request comprises an object path to be accessed; determining first file fragment information corresponding to first metadata according to the directory information and the to-be-accessed object path; determining an operation result corresponding to the first metadata according to the first file fragment information and the fragment routing information of the first metadata node; and sending the operation result to the client.

The invention provides an intelligent document management method and system, and the method comprises the steps: obtaining an uploaded file set, processing each file through a text analysis algorithm, an image analysis algorithm and a structured data analysis algorithm, generating a first feature set, and storing the first feature set in a temporary database; obtaining a distribution record from the first permission configuration file, writing the distribution record into an audit database through a log audit module, and generating a first operation traceability chain; according to the first operation tracing chain, a storage management module is called to perform fragmentation processing on the file and upload the file to cloud storage, a first storage index is generated, and the metadata table and the archive position are associated; and according to the second storage index, calling a disaster recovery backup algorithm to perform multi-region copying on the file, generating a first backup confirmation signal, and recording the first backup confirmation signal into an operation and maintenance log. According to the system and the method, intelligent classification, sensitive information protection and whole-course traceable management of the files are realized, the automation level and the safety of document management are improved, and a comprehensive solution is provided for document management.

The invention discloses a data archiving processing method and system based on a shutdown system, and the method comprises the steps: carrying out the classification of data through employing a multi-dimensional classification model based on deep learning according to the data type, access frequency and business importance of the shutdown system, and obtaining a classified data set and a priority label; inputting the classified data set into a compression and encryption joint processing model based on a neural network to obtain a compressed and encrypted data packet; distributing the compressed and encrypted data packet to a distributed storage system, and generating a data storage path index by adopting an index construction algorithm based on a graph database to obtain a distributed storage index table; and according to the distributed storage index table, verifying the data integrity by using a block chain-based archiving verification technology, and generating an archiving verification report. By utilizing the embodiment of the invention, intelligent classification, effective compression and safe storage of the data in the shutdown system can be realized, and the integrity and traceability of the data are finally ensured.

The invention relates to a mirror image file processing method and device, and relates to the technical field of container mirror image generation. The method comprises the steps of obtaining target file system directory information of a target container in response to a mirror image file export request for the target container; the target file system directory information of the target container is used for representing file system directory information of a writable layer of the target container; comparing the target file system directory information of the basic mirror image with the target file system directory information of the target container to obtain a difference file of the target container relative to the basic mirror image; constructing a layer file of a new incremental layer based on the difference file; writing a layer file of the incremental layer into a mirror image layer file directory of the basic mirror image, and updating a list file and an index file of the basic mirror image to obtain a mirror image file of the target container; the manifest file comprises metadata of each mirror image layer in the basic mirror image, and the index file comprises metadata of the manifest file. By adopting the method, the export efficiency of the container mirror image file can be improved.

An illustrative method includes detecting a log-in request associated with a managed directory within the file system, determining that the log-in request satisfies one or more criteria, and automatically generating, based on the determining that the log-in request satisfies the one or more criteria, a managed home directory within the managed directory. In some embodiments, the managed home directory may be usable by a storage management system to provide user-specific storage management functionality to a dataset of the directory.