1916results about "File system types" patented technology

The invention discloses a geographic information big data-oriented multi-source data processing system, and relates to the technical field of data acquisition and sensors, and the system comprises a data acquisition module which accesses a remote sensing satellite, an unmanned aerial vehicle, an Internet of Things sensor and social media in real time through a multi-source heterogeneous interface, and carries out the adaptive analysis of a data format and metadata marking; the distributed storage module is used for performing partition storage on the geographic information data based on a space-time database and an object storage architecture, and establishing dynamic space-time index and version control; the data fusion module is used for realizing coordinate system conversion, time sequence calibration and semantic knowledge graph matching based on a multi-source data alignment method of dynamic weight distribution; and an intelligent analysis module and a security control module. According to the method, core pain points such as data splitting, low storage efficiency, extensive analysis and compliance risks in the geographic information field are systematically solved, and a full-stack type technical base is provided for scenes such as smart cities, emergency disaster relief and environment monitoring.

The invention provides an intelligent agent tool calling knowledge optimization method based on empirical path graph evolution, which comprises the following steps: when an intelligent agent successfully completes a task for the first time, recording an intelligent agent tool calling sequence, input and output parameters and an execution result, generating a structured calling log, the calling log is converted into a standardized calling path knowledge unit; performing structured representation and semantic representation on the calling path knowledge unit, storing the structured representation in a graph database, and storing the semantic representation in a vector database; task intentions, tool entities and calling paths are used as heterogeneous nodes, an experience path knowledge graph is constructed, the experience path knowledge graph is used for recording the multi-dimensional relation among tasks, paths and tools, execution performance attributes and feedback attributes are added to path nodes in the graph, and agent tool calling knowledge optimization is completed. And the purpose of improving the tool calling efficiency and robustness of the intelligent agent in the multi-task environment is achieved.

The invention discloses a retrieval enhancement generated document screening system and method fusing a verification mechanism, and relates to the technical field of document screening, the system comprises a user input and query analysis module for extracting key information through natural language processing, and converting the key information into a high-dimensional semantic vector, a meta-tag and a keyword set; the multi-source document retrieval module is used for obtaining documents from multiple data sources through mixed retrieval and generating a candidate set through preliminary screening and sorting; the credibility evaluation and security verification module is used for generating scores and labels after multi-dimensional evaluation and screening qualified documents; the document consistency detection module is used for detecting document conflicts, processing and sequencing, and ensuring logic consistency; the document acquisition and generation module is used for inputting qualified documents into a generation model and generating answers with references; and the result output and tracing module is used for outputting answers and recording whole-process data to ensure traceability. The invention aims to ensure the accuracy and credibility of the generated content through a multi-dimensional verification mechanism.

The invention discloses a log quality detection method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring a multi-modal input source; performing reverse anchoring operation in a dynamic analysis stage of the running log to establish direct mapping between the running log and the source code; compiling the abstract syntax tree based on the source code, establishing a grammar dependence graph according to the abstract syntax tree, and traversing the abstract syntax tree to construct a structured data set; based on the running log, obtaining a quality detection result through structured analysis, and generating a semantic detection result by using a large language model; performing cooperative operation on the structured data set and the grammar dependency graph to generate a code log coverage score, and performing arrangement according to a quality detection result and a semantic detection result to obtain an abnormal mode feature; and performing feedback adjustment based on the code log coverage score and the abnormal mode feature in combination with a preset correction rule. The method can efficiently and accurately realize log quality detection, and can be widely applied to the technical field of data processing.

The invention discloses an intelligent log retrieval and analysis system based on a model context protocol MCP, and relates to the technical field of log analysis. The system comprises an MCP log semantic conversion and retrieval engine, a log format irrelevant feature extraction and indexer, a dynamic log format recognition and context enhancement system, a real-time retrieval analysis and aggregation controller and a multi-level semantic retrieval and visualization engine. According to the context enhancement system, unknown formats can be identified, semantic tags can be complemented, and semantic consistency and behavior tracking capability can be improved. The retrieval analysis controller supports semantic expression analysis, strategy generation and feedback closed loop, and strategy scheduling and multi-dimensional aggregation analysis are achieved. And finally, outputting a structured semantic result by the system, and visually displaying the structured semantic result through components such as a semantic composition device and a context expander. All the modules are managed in a unified mode through a capability registration mechanism, dynamic arrangement and upstream and downstream closed-loop linkage are supported, and a log intelligent analysis framework with high semantic driving and a clear structure is formed.

Techniques are described for efficient replication and maintaining snapshot data consistency during file storage replication between file systems in different cloud infrastructure regions. In certain embodiments, provenance IDs are used to efficiently identify a starting point (e.g., a base snapshot) for a cross-region replication process, conserve cloud resources while reducing network and IO traffic.

The invention relates to the technical field of distributed system transaction management, and discloses an inventory consistency management method and device based on distributed transactions, and the method comprises the following steps: pre-occupying an inventory, and recording the state information of the inventory to a Redis hash structure; after the inventory is pre-occupied successfully, generating a pre-deducted snapshot, generating a compensation identifier, and writing the pre-deducted snapshot and the compensation identifier into a MySQL transaction log together; verifying the uniqueness of the compensation identifier, and submitting the current database transaction after the uniqueness of the compensation identifier is confirmed; when the submission of the current database transaction fails, executing a compensation operation on the current database transaction; and after the compensation operation is executed, performing data consistency verification on the MySQL transaction log and the Redis hash structure. According to the method and the system, the consistency of inventory management during distributed transaction processing is highly guaranteed.

Embodiments are described herein for document authentication certification using information stored on a distributed ledger such as a blockchain. A distributed ledger may securely store document data describing the document. Use of a distributed ledger may provide an immutable, readily auditable record of the history of the document. Each user participating in the system may be assigned a unique identifier to be used for conducting transactions on the distributed ledger network. A user may also be provided with a digital security token such as a cryptographic key that is useable to authenticate the user and enable access to the document data stored on the distributed ledger(s).

The invention relates to the technical field of digital twinning, in particular to a steel structure building construction whole process mechanical property evaluation method based on digital twinning, which comprises the following steps of: acquiring multi-source heterogeneous steel structure state data from a stress sensor, BIM model component information and a displacement monitoring point, comparing timestamps from all data sources, and determining the state of the multi-source heterogeneous steel structure according to the comparison result; and calculating a transmission delay value of each data source relative to the unified reference time, correcting the original data timestamps one by one, and establishing a steel structure state data stream with aligned time sequences. According to the method, the timestamps of the multi-source heterogeneous data are subjected to transmission delay correction through unified reference time, the timestamps of the original data are corrected one by one, the steel structure state data streams with the aligned time sequences are generated, the time deviation among the sensors, the BIM model and the monitoring points is eliminated, and the synchronism and coherence of dynamic changes of mechanical parameters are guaranteed.

A snapshot of metadata of a file system on a source storage system is stored in one or more of storage devices of a target storage system. A request to access data stored in the file system is received. A portion of the metadata including identification information of the data is identified. The data stored in the file system is retrieved by providing the source storage system with the identification information of the data.

The invention relates to the technical field of computer data processing, and discloses an automatic format conversion and incremental synchronization method for cross-platform heterogeneous document data. The method comprises the following steps: capturing an atomic data change event from a source system transaction log; aggregating into logic transaction units according to the transaction identifiers; converting into a unified intermediate normal form; generating a target system operation instruction set based on the metadata mapping rule; the synchronization sites are executed and persisted in a transactional manner. According to the method, through log-level capture and transactional synchronization, data integrity, sequence and high consistency are ensured, and efficient and reliable data synchronization between heterogeneous systems is supported.

The invention discloses an information processing method, and the method comprises the steps: inputting query information for a target object into an intention recognition model for intention recognition, and obtaining an information retrieval type corresponding to the query information; wherein the query information is used for querying all related log information or part of related log information of the target object; retrieving the query information through a retrieval strategy corresponding to the information retrieval type by utilizing a pre-constructed index database to obtain a retrieval result meeting an information length condition corresponding to an input requirement of a second large language model; wherein different information retrieval types correspond to different retrieval strategies; performing fusion processing on the query information and the retrieval result to generate context reference information meeting the input requirement of the second large language model; and reasoning the context reference information by using the second large language model to generate reply content corresponding to the query information.

In various examples, data storage is managed using a distributed storage management system that is resilient. Data blocks of a logical block device may be distributed across multiple nodes in a cluster. The logical block device may correspond to a file system volume associated with a file system instance deployed on a selected node within a distributed block layer of a distributed file system. Each data block may have a location in the cluster identified by a block identifier associated with each data block. Each data block may be replicated on at least one other node in the cluster. A metadata object corresponding to a logical block device that maps to the file system volume may be replicated on at least another node in the cluster. Each data block and the metadata object may be hosted on virtualized storage that is protected using redundant array independent disks (RAID).

The invention relates to the technical field of data processing, and provides a BMC fault recording method and device, computer equipment and a storage medium, and the method comprises the steps: responding to a kernel fault of a target operating system of a target BMC, and printing fault log information; obtaining a pre-configured memory technology device partition; transmitting the fault log information to an annular fault cache region to form cache log data; storing the cached log data in a memory technical equipment partition, and generating a fault log file; and in response to a user reading instruction, reading and outputting the fault log file of the memory technical equipment partition. According to the technical scheme provided by one or more embodiments of the invention, the fault log can be effectively recorded, maintenance personnel can timely and accurately know the fault position and the fault reason, and BMC fault diagnosis and repair can be efficiently completed.

Techniques are described for performing different types of restart operations for a file storage replication between a source file system and a target file system in different cloud infrastructure regions. In certain embodiments, the disclosed techniques perform a restart operation to terminate a current cross-region replication by synchronizing resource cleanup operations in the source file system and the target file system, respectively. In other embodiments, disclosed techniques perform a restart operation to allow a customer to reuse the source file system by identifying a restartable base snapshot in the source file system without dependency on the target file system.

The invention relates to the field of cold and hot data storage, in particular to a cold and hot data storage optimization method based on a large model, which comprises a business integration module, a data dependence module, a mode updating module, a cold and hot partitioning module and a file merging module, and is characterized in that the business integration module is used for collecting business data to form a training data set; the data dependency module is used for fitting business data to obtain a data consanguinity model, the mode updating module is used for predicting an access mode, the cold and hot partitioning module is used for calculating the heat of the data and performing dynamic scheduling, and the file merging module is used for automatically merging small files. A data storage structure is standardized, storage space is reduced, more efficient and more accurate cold and hot data classification can be achieved, the resource utilization rate of data storage hardware is improved, hot data response delay is reduced, the system storage utilization rate is remarkably improved, and the storage efficiency is improved.

The invention provides a system abnormity intelligent diagnosis and recovery method and system fusing time sequence logs, and relates to the field of distributed system operation and maintaining.The method comprises the steps that operation logs are collected from a plurality of service nodes of a distributed system, time sequence alignment is conducted, and calling relation and performance measurement data are extracted; time sequence change features are calculated in the sliding time window, dynamic feature vectors are generated through fusion coding, and a cross-service time sequence association graph is constructed; identifying an abnormal service node and matching the abnormal service node with historical fault data to obtain a historical propagation path and a feature vector; calculating a propagation convergence coefficient and a characteristic deviation degree based on a historical propagation path to obtain a causal intensity score; and identifying a fault influence degree according to the score, and adaptively adjusting a resource isolation and flow scheduling strategy. According to the invention, accurate diagnosis and efficient recovery of distributed system faults are realized.

The invention belongs to the technical field of system operation and maintenance monitoring, and particularly discloses a system health degree assessment method and system, a terminal and a medium. Comprising the following steps: collecting log stream data of a target system and a heterogeneous subsystem to obtain log feature data; inputting the log feature data into an exception detection engine of a preset training model, and performing exception judgment on the log feature data based on the exception detection engine to obtain an exception detection result; according to the anomaly detection result and the operation index of the target system, performing multi-dimensional comprehensive calculation on the operation index to obtain a system health degree score; and performing processing calculation according to the log feature data, the anomaly detection result and the system health degree score, and outputting a system operation state analysis result. By introducing the anomaly detection model based on the support vector data description and the improved grey wolf optimization algorithm, efficient analysis and anomaly recognition of complex heterogeneous system logs are realized, and the system stability and the operation and maintenance efficiency can be improved.

The embodiment of the invention provides a log processing method, computing equipment, a storage medium and a program product. The method comprises the following steps: under the condition that a target transaction of a database is detected to meet a target condition, storing a temporary file generated by the target transaction to a storage node; the temporary file stores a binary log for the database; and in response to a transaction submission operation of the target transaction, processing the temporary file in the storage node to generate a binary log file. According to the technical scheme provided by the embodiment of the invention, the transaction submission efficiency is improved.

The invention discloses a one-time plot rule mining method and device for a process event log, and belongs to the field of plot mining. The distance between two events is limited by adopting a time interval constraint, and repeated use of the events is avoided by adopting a one-time condition so as to avoid mining excessive meaningless plots. Moreover, when a strong one-time plot rule (i.e., a one-time plot rule with confidence greater than or equal to a minimum confidence threshold) is mined, a triple mechanism of candidate plot generation, support degree calculation and rule generation is provided. In a candidate plot generation stage, eliminating redundant candidate plot extension by adopting a plot connection strategy; in the support degree calculation stage, the support degree is calculated based on the position index; in a rule generation stage, firstly, all frequent one-time plots are mined, and then strong one-time plot rules are screened according to a minimum support degree threshold value and a minimum confidence coefficient threshold value. The technical effect of efficiently and accurately mining the strong one-time plot rule is achieved.

The invention discloses a file metadata multi-version storage management method and device, computer equipment and a storage medium. The method comprises the steps of obtaining file metadata; in response to the received generation instruction, generating a snapshot version for the file metadata to obtain a snapshot version set; extracting a first snapshot version and a second snapshot version from the snapshot version set, updating the snapshot version set, and storing the snapshot version set in a memory area; packaging the file metadata, the first snapshot version and the second snapshot version to obtain a coded data structure, and storing the coded data structure in a memory area; storing the updated snapshot version set and the coded data structure in a storage device; querying the memory area according to the query instruction, querying the storage device if the query fails, and returning a query result; by means of the method, decoupling storage of multiple versions of file metadata can be achieved, and the system memory utilization rate and the snapshot version management efficiency can be improved.

The invention discloses a cross-platform file transfer method in the mine field, which comprises the following steps: S1, mine file source identification and fluidization: docking an industrial internet platform, reading an original file generated by a specific business scene, and converting the original file into a file stream; s2, intelligent unified transcoding for the mining terminal: performing unified transcoding on the file stream through an intelligent transcoding engine; s3, encryption based on the mine data security level: carrying out encryption processing on the transcoded file; and S4, safe unloading driven by the mine service: receiving the encrypted transcoding file, and decrypting and decoding the encrypted transcoding file at a target mine terminal or a mine appointed storage node. According to the method, safe and efficient cross-platform transmission among mine multi-service systems is realized, the problem of mine terminal file compatibility is thoroughly solved, the file transmission and access efficiency in a mine weak network environment is remarkably improved, and a mine-level data security protection system is constructed.

The invention provides an offline transaction control method and a state prediction model training method and device.The method is applied to the technical field of computers and comprises the steps that if an offline transaction in an online operation environment is in an operation state, an online operation log corresponding to the online operation environment within a preset historical duration before the current moment is obtained; extracting transaction state data and equipment resource data from the online log data, and performing data integration on the transaction state data and the equipment resource data to obtain a formatted running log; inputting the formatted operation log into a pre-trained state prediction model to obtain a prediction result generated by the offline transaction on the environment state of the online operation environment; and obtaining a state intervention strategy corresponding to the prediction result, and adjusting the operation state of the offline transaction based on the state intervention strategy, thereby improving the environmental stability of the online operation environment.

The embodiment of the invention discloses a big data preprocessing method and system based on machine learning, and the method comprises the steps: obtaining a running log data set of an RPA system, the running log data set comprising a plurality of session interaction records; performing data cleaning processing on the running log data set to obtain a preprocessed running log data set; performing feature extraction processing on the preprocessed running log data set based on a preset feature extraction algorithm to obtain a text semantic feature and an operation sequence feature of each session interaction record; and calling a machine learning model to perform joint modeling processing on the text semantic features and the operation sequence features, generating an anomaly detection result of the session interaction record, and adjusting a task execution strategy of the RPA system based on the anomaly detection result. Therefore, the stability and efficiency of the RPA system can be improved, and smooth operation of the business process is guaranteed.

Embodiments of the present disclosure provide a log storage method, a database system, a server, and a storage medium. In the log storage method, any database node in a distributed database can use a storage engine component to acquire a transaction log of a target transaction, and write the transaction log into a target online log file. The database node can use the storage engine to persistently store the transaction log in the target online log file into a corresponding target block device, and perform a transaction-level persistent storage operation on metadata information of the target online log file. In the implementation, on the basis of the online log file supporting a write operation at any location, the database node can directly store the transaction log in a block storage mode on the basis of the storage engine assembly of the database node without depending on a file system, thereby reducing resource overhead caused by incorporating the file system and a file storage mode.

The invention discloses a kernel probe generation method and device, computer equipment and a storage medium, and relates to the technical field of operating system kernel monitoring, and the kernel probe generation method comprises the steps of obtaining a target metadata file corresponding to a current operating system kernel in response to a kernel probe generation request; determining a target declarative strategy file; and determining a target program logic file corresponding to the target declarative strategy file, and generating a probe code according to the target program logic file and the target metadata file to obtain a kernel probe program. The probe program source code adaptive to the current kernel version can be dynamically generated in combination with the target metadata file and the target declarative strategy file, kernel probe development logic and underlying kernel knowledge can be decoupled through the source code generation function, and developers do not need to deeply understand kernel internal data structures and version differences; the development threshold is lower, a specific operating system or probe type is supported, and a universal Linux kernel and multiple probe types are also supported.

The invention provides a log data reading method and device, equipment, a medium and a program product, and relates to the field of big data. Obtaining a mapping relationship between a plurality of timeline identifiers in a database and corresponding start log serial numbers; the timeline identifier is an identifier generated when the main database and the standby database are switched, whether the log data cross different timeline identifiers or not is judged according to the mapping relation and the starting log serial number and the ending log serial number of the log data, and under the condition that the log data cross different timeline identifiers, the log data cross different timeline identifiers. Reconstructing a log reading process according to a start log serial number of a new timeline identifier, wherein the new timeline identifier is different from the plurality of timeline identifiers; according to the method, automatic reconstruction of the log reading process after main and standby switching is achieved, the problems of low efficiency and poor real-time performance caused by manual operation are avoided, and continuity and consistency of data synchronization of the source end and the target end are guaranteed.

The invention discloses a data change capturing method, system and device, a medium and a program product, and relates to the technical field of data processing.The data change capturing method comprises the steps that a snapshot metadata table is constructed, and creation operation, deletion operation and submission timestamps corresponding to the operation of data files are recorded through the snapshot metadata table, the data file comprises a table data file and a tombstone data file, actual service data of the table is stored in the table data file, and deleted data of the table is stored in the tombstone data file; by querying a submission timestamp in the snapshot metadata table, an effective file list in a target time period of capturing data change is determined, and the effective file list comprises an effective table data file and / or an effective tombstone data file; and filtering and merging the effective file list based on the submission timestamp in the target time period to obtain final change data. The processing efficiency of data change capture is improved.

The invention discloses a WebGL-based Unity asset cross-platform preview method and system, and a storage medium. The preview method comprises the following steps: 1) preprocessing an editor plug-in: scanning a target asset through the Unity editor plug-in, extracting a resource GUID,. Meta metadata and a dependency relationship, generating a lightweight AB packet and adding a dynamic thumbnail; step 2) cloud dependency analysis: the server side recursively constructs a dependency tree based on GUID, filters redundant resources, and generates a resource loading list which can be identified by the browser side; and step 3) a WebGL rendering engine: loading an AB packet in a browser, realizing particle effect regeneration, animation control and scene interaction by simulating a Unity Runtime environment, and adapting to a WebGL 1.0 / 2.0 rendering pipeline. The invention aims to provide a WebGL-based Unity asset cross-platform preview method and system and a storage medium, aims to solve the problem that a browser cannot directly preview Unity resource files, and provides a set of resource management workflow based on Unity-cloud storage-browser preview for the Unity resource files.

The invention relates to the technical field of cloud storage, can be applied to business system platforms of financial science and technology, medical treatment and health and the like, and discloses a hierarchical storage method, device and equipment for data files and a medium, and the method comprises the following steps: acquiring the data files in a storage unit, and extracting data characteristic parameters in the data files; performing multi-dimensional quantitative analysis on the data feature parameters to obtain a dimensional feature analysis value; identifying a dynamic weight of the data feature parameter, and performing weighted calculation on the data feature parameter according to the dimension feature analysis value and the dynamic weight to obtain a dynamic storage analysis value; identifying a data storage type label according to the dynamic storage analysis value; matching the data storage type label with strategies in a storage strategy library to obtain a target storage strategy; determining a storage hierarchy according to the target storage strategy; a storage address of the data file in the storage hierarchy is identified and the data file is stored in the storage address. The accuracy of hierarchical storage of the data files can be improved.