2498results about "File system types" patented technology

The invention discloses a geographic information big data-oriented multi-source data processing system, and relates to the technical field of data acquisition and sensors, and the system comprises a data acquisition module which accesses a remote sensing satellite, an unmanned aerial vehicle, an Internet of Things sensor and social media in real time through a multi-source heterogeneous interface, and carries out the adaptive analysis of a data format and metadata marking; the distributed storage module is used for performing partition storage on the geographic information data based on a space-time database and an object storage architecture, and establishing dynamic space-time index and version control; the data fusion module is used for realizing coordinate system conversion, time sequence calibration and semantic knowledge graph matching based on a multi-source data alignment method of dynamic weight distribution; and an intelligent analysis module and a security control module. According to the method, core pain points such as data splitting, low storage efficiency, extensive analysis and compliance risks in the geographic information field are systematically solved, and a full-stack type technical base is provided for scenes such as smart cities, emergency disaster relief and environment monitoring.

The invention relates to the technical field of data analysis, and discloses a business data analysis method, device and equipment based on big data and a storage medium, the method comprises the following steps: collecting and preprocessing original log data to obtain standardized log data, and performing adaptive multi-modal feature extraction to obtain comprehensive feature representation; inputting a heterogeneous graph neural network to carry out anomaly detection and multi-level root cause analysis to obtain a hierarchical anomaly event; executing task scheduling and resource allocation of reinforcement learning to obtain a task allocation scheme; performing deep learning enhanced root cause reasoning and knowledge graph construction on the task allocation scheme to obtain a system state diagram and a causal relationship; and generating a multi-dimensional task data analysis report. According to the method, comprehensive analysis, anomaly detection, root cause reasoning and intelligent scheduling of log data are realized. The reliability and performance of the system can be improved, and more intuitive and accurate decision support can be provided for operation and maintenance personnel.

The invention relates to the technical field of information management, and comprises a distribution automation operation and maintenance work log intelligent recording platform based on a CS architecture, and the platform comprises a log field analysis module, a task chain construction module, a task logic verification module, and a log storage tracing module. According to the method, missing and abnormal fields are effectively marked and corrected through meticulous analysis and comparison of uploaded log records, the accuracy and integrity of data are improved, task scheduling is optimized and the operation and maintenance management efficiency is improved through association of device numbers and task types and adjustment of execution time in sorting and grouping of task sequences, and the task scheduling efficiency is improved. Task chains are recombined to solve conflicts and adjust priorities, smooth execution of operation and maintenance tasks is guaranteed, delay is reduced, logic verification enhances task reliability through rule verification and error marking, unassociated tasks are deduced, analyzed and corrected, the exception processing capacity is enhanced, the data tracing and storage efficiency is improved, and the tracing capacity of data management is enhanced.

The invention provides an intelligent agent tool calling knowledge optimization method based on empirical path graph evolution, which comprises the following steps: when an intelligent agent successfully completes a task for the first time, recording an intelligent agent tool calling sequence, input and output parameters and an execution result, generating a structured calling log, the calling log is converted into a standardized calling path knowledge unit; performing structured representation and semantic representation on the calling path knowledge unit, storing the structured representation in a graph database, and storing the semantic representation in a vector database; task intentions, tool entities and calling paths are used as heterogeneous nodes, an experience path knowledge graph is constructed, the experience path knowledge graph is used for recording the multi-dimensional relation among tasks, paths and tools, execution performance attributes and feedback attributes are added to path nodes in the graph, and agent tool calling knowledge optimization is completed. And the purpose of improving the tool calling efficiency and robustness of the intelligent agent in the multi-task environment is achieved.

The invention discloses a retrieval enhancement generated document screening system and method fusing a verification mechanism, and relates to the technical field of document screening, the system comprises a user input and query analysis module for extracting key information through natural language processing, and converting the key information into a high-dimensional semantic vector, a meta-tag and a keyword set; the multi-source document retrieval module is used for obtaining documents from multiple data sources through mixed retrieval and generating a candidate set through preliminary screening and sorting; the credibility evaluation and security verification module is used for generating scores and labels after multi-dimensional evaluation and screening qualified documents; the document consistency detection module is used for detecting document conflicts, processing and sequencing, and ensuring logic consistency; the document acquisition and generation module is used for inputting qualified documents into a generation model and generating answers with references; and the result output and tracing module is used for outputting answers and recording whole-process data to ensure traceability. The invention aims to ensure the accuracy and credibility of the generated content through a multi-dimensional verification mechanism.

The invention discloses an intelligent nuclear radiation dose real-time monitoring system based on deep learning, and the system comprises the steps: S1, a multi-mode sensing module which is used for outputting radiation intensity data with timestamps, environment image data and geographic space position information; s2, a preprocessing module used for outputting a structured feature sequence; s3, a fusion modeling module which is used for generating a unified fusion feature vector; s4, an end-edge reasoning module which is used for processing the fusion feature vector in stages and generating a nuclear radiation dose prediction result; s5, the task scheduling module is used for dynamically determining whether the reasoning task is executed on the end side or the edge node; s6, an anomaly detection module which is used for determining whether there is an abnormal change of the nuclear radiation dose; and S7, the system control module is used for coordinating the modules. The method has the advantages of being high in environmental adaptability, fast in reasoning response and high in anomaly recognition precision.

The invention discloses a log quality detection method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring a multi-modal input source; performing reverse anchoring operation in a dynamic analysis stage of the running log to establish direct mapping between the running log and the source code; compiling the abstract syntax tree based on the source code, establishing a grammar dependence graph according to the abstract syntax tree, and traversing the abstract syntax tree to construct a structured data set; based on the running log, obtaining a quality detection result through structured analysis, and generating a semantic detection result by using a large language model; performing cooperative operation on the structured data set and the grammar dependency graph to generate a code log coverage score, and performing arrangement according to a quality detection result and a semantic detection result to obtain an abnormal mode feature; and performing feedback adjustment based on the code log coverage score and the abnormal mode feature in combination with a preset correction rule. The method can efficiently and accurately realize log quality detection, and can be widely applied to the technical field of data processing.

The invention discloses an intelligent log retrieval and analysis system based on a model context protocol MCP, and relates to the technical field of log analysis. The system comprises an MCP log semantic conversion and retrieval engine, a log format irrelevant feature extraction and indexer, a dynamic log format recognition and context enhancement system, a real-time retrieval analysis and aggregation controller and a multi-level semantic retrieval and visualization engine. According to the context enhancement system, unknown formats can be identified, semantic tags can be complemented, and semantic consistency and behavior tracking capability can be improved. The retrieval analysis controller supports semantic expression analysis, strategy generation and feedback closed loop, and strategy scheduling and multi-dimensional aggregation analysis are achieved. And finally, outputting a structured semantic result by the system, and visually displaying the structured semantic result through components such as a semantic composition device and a context expander. All the modules are managed in a unified mode through a capability registration mechanism, dynamic arrangement and upstream and downstream closed-loop linkage are supported, and a log intelligent analysis framework with high semantic driving and a clear structure is formed.

The invention belongs to the technical field of data synchronization, and particularly relates to a real-time data synchronization method for cloud computing heterogeneous resources. In order to solve the problem of data synchronization among heterogeneous resources, the method comprises the following steps: aggregating a plurality of files in batches to generate a packaged file; performing metadata compression processing on the generated packaged file, and eliminating repeated data through a deduplication algorithm; dynamically selecting a lightweight data transmission protocol; establishing a multi-dimensional index structure; only retaining the file of the latest version through a version control mechanism; dynamically adjusting the allocation of synchronization tasks based on the load condition of available resources in the cloud computing environment; and rescheduling the uncompleted synchronization task. Through a multi-dimensional index structure, version control, self-adaptive transmission scheduling and a dynamic resource allocation mechanism, efficient and stable file synchronization among heterogeneous resources is realized, repeated transmission and storage waste are reduced, and real-time performance, reliability and resource utilization efficiency of synchronization are improved.

Techniques are described for efficient replication and maintaining snapshot data consistency during file storage replication between file systems in different cloud infrastructure regions. In certain embodiments, provenance IDs are used to efficiently identify a starting point (e.g., a base snapshot) for a cross-region replication process, conserve cloud resources while reducing network and IO traffic.

The invention provides a business arrangement scheduling method based on event driving, which comprises the following steps of: constructing a unified business arrangement framework, describing a business process by using a unified process definition language, and defining a process metadata standard; adding a general process engine with a plug-in mechanism in the business arrangement framework; establishing a real-time event processing architecture by taking an event as a core, and adding an event-driven business process engine; constructing a monitoring platform on the basis of the real-time event processing architecture, performing real-time monitoring on an event stream and event processing, and performing visual display; a unified retry mechanism is introduced, and a retry strategy and retry management are defined; according to the method, a unified, efficient and easy-to-monitor and easy-to-maintain business arrangement framework is established, so that the scheduling process is conveniently arranged through event-driven business, the endpoint retry and business flow monitoring capability is provided, the realization difficulty of monitorable and retry in the business is simplified, and the robustness of the business is improved.

The invention relates to the technical field of big data and the field of financial science and technology, and discloses a database table synchronization method based on log data, which comprises the following steps: recording operation information of a business database table to log data, writing the log data into a message queue, obtaining the log data through a streaming data processing framework, and deserializing the log data into structured data; and extracting operation records, classifying the operation records according to operation type fields, distributing the operation records to processing channels corresponding to the table names, and updating or adjusting the table structure of the target table according to classification, or executing data change operation. Through the log data-based synchronization method, quasi-real-time synchronization from the business database table to the operation data storage module is realized, and the table structure is adaptively adjusted when the business table structure is changed, so that the data consistency problem and the system interruption risk are reduced, the data synchronization efficiency and stability are effectively improved, and the user experience is improved. And the efficient and stable data synchronization requirements of the service are met.

The invention relates to the technical field of distributed system transaction management, and discloses an inventory consistency management method and device based on distributed transactions, and the method comprises the following steps: pre-occupying an inventory, and recording the state information of the inventory to a Redis hash structure; after the inventory is pre-occupied successfully, generating a pre-deducted snapshot, generating a compensation identifier, and writing the pre-deducted snapshot and the compensation identifier into a MySQL transaction log together; verifying the uniqueness of the compensation identifier, and submitting the current database transaction after the uniqueness of the compensation identifier is confirmed; when the submission of the current database transaction fails, executing a compensation operation on the current database transaction; and after the compensation operation is executed, performing data consistency verification on the MySQL transaction log and the Redis hash structure. According to the method and the system, the consistency of inventory management during distributed transaction processing is highly guaranteed.

Embodiments are described herein for document authentication certification using information stored on a distributed ledger such as a blockchain. A distributed ledger may securely store document data describing the document. Use of a distributed ledger may provide an immutable, readily auditable record of the history of the document. Each user participating in the system may be assigned a unique identifier to be used for conducting transactions on the distributed ledger network. A user may also be provided with a digital security token such as a cryptographic key that is useable to authenticate the user and enable access to the document data stored on the distributed ledger(s).

The invention relates to an ADS-B (Automatic Dependent Surveillance-Broadcast)-based civil aviation flight pre-arrival data calculation method, which integrates ADS-B data, meteorological satellite data, airport runway state monitoring data and air traffic control real-time instruction data through multi-source data fusion, and dynamically adjusts the weight of each data source according to the flight stage of a flight. A dynamic adjustment time synchronization interval mechanism is adopted, a short synchronization interval is set in the take-off stage and the landing stage so as to capture rapidly-changed data, and the synchronization interval is prolonged in the cruise stage so as to optimize computing resources. Meanwhile, the deep learning model based on the Transform architecture utilizes a multi-head self-attention mechanism and position coding to enhance the feature capture capability, and reduces the calculation burden through a sparse activation strategy. The system is provided with a real-time monitoring layer, log records and visual analysis tools are integrated, and interpretability and traceability of the model are ensured. An efficient aviation management decision is supported, and the efficiency and safety of overall aviation traffic management are improved.

The invention relates to the technical field of digital twinning, in particular to a steel structure building construction whole process mechanical property evaluation method based on digital twinning, which comprises the following steps of: acquiring multi-source heterogeneous steel structure state data from a stress sensor, BIM model component information and a displacement monitoring point, comparing timestamps from all data sources, and determining the state of the multi-source heterogeneous steel structure according to the comparison result; and calculating a transmission delay value of each data source relative to the unified reference time, correcting the original data timestamps one by one, and establishing a steel structure state data stream with aligned time sequences. According to the method, the timestamps of the multi-source heterogeneous data are subjected to transmission delay correction through unified reference time, the timestamps of the original data are corrected one by one, the steel structure state data streams with the aligned time sequences are generated, the time deviation among the sensors, the BIM model and the monitoring points is eliminated, and the synchronism and coherence of dynamic changes of mechanical parameters are guaranteed.

The invention provides a professional online collaborative editing method and system for hydropower project cost schemes. The professional online collaborative editing method comprises a standard management process, an intelligent collaborative editing process and a release state synchronous updating process. The standard management process comprises the steps of performing automatic analysis and data standardization conversion on an industrial standard file based on a deep learning model, generating a system rule set and storing the system rule set in a database; and based on the system rule set, automatically generating a standardized filling template of each specialty. According to the micro-service architecture-based intelligent online collaborative editing method and system for the hydropower engineering, a natural language processing NLP technology, a deep learning algorithm and a transaction management mechanism are combined, and the system has the functions of real-time grammar and spelling inspection of filled content, intelligent recommendation, semantic understanding, real-time difference detection, multi-user conflict processing and version control; the problems that communication cost is high, version control is difficult, data verification is not strict and multi-user cooperation is not smooth in water-power engineering cost scheme compiling are solved.

A snapshot of metadata of a file system on a source storage system is stored in one or more of storage devices of a target storage system. A request to access data stored in the file system is received. A portion of the metadata including identification information of the data is identified. The data stored in the file system is retrieved by providing the source storage system with the identification information of the data.

The invention relates to the technical field of computer data processing, and discloses an automatic format conversion and incremental synchronization method for cross-platform heterogeneous document data. The method comprises the following steps: capturing an atomic data change event from a source system transaction log; aggregating into logic transaction units according to the transaction identifiers; converting into a unified intermediate normal form; generating a target system operation instruction set based on the metadata mapping rule; the synchronization sites are executed and persisted in a transactional manner. According to the method, through log-level capture and transactional synchronization, data integrity, sequence and high consistency are ensured, and efficient and reliable data synchronization between heterogeneous systems is supported.

Systems and methods are described for performing an instant recovery of data associated with a locked snapshot. In various examples, the amount of time for performing a recovery of data associated with a locked snapshot is significantly reduced by making use of enhanced volume cloning functionality instead of making an actual copy of the data to be recovered. In one embodiment, the resulting volume clone representing the recovery volume is cleared of all data protection information (e.g., WORM flags and / or lock metafiles) that was previously used to protect the content from being changed when stored on the data protection volume so as allow the recovery volume to be used in read-write mode.

The invention discloses a data recovery method and a related device, the method can be applied to a server running a database system, and the method comprises the steps that in response to modification of a first data page, a first log is generated, modification of the first data page is recorded in the first log, and a first identifier indicates the version of the first data page; under the condition that the database system fails, the version of a second data page is determined, the second data page is the data page of the data to be recovered, and the page identifier of the second data page is the same as the page identifier of the first data page; and under the condition that the version of the first log is greater than that of the second data page and the version indicated by the first identifier is the same as that of the second data page, performing data recovery on the second data page by using the first log. Through the first identifier, data recovery on the second data page by using a log with a wrong version is avoided, and the accuracy of data recovery is ensured.

The invention discloses an information processing method, and the method comprises the steps: inputting query information for a target object into an intention recognition model for intention recognition, and obtaining an information retrieval type corresponding to the query information; wherein the query information is used for querying all related log information or part of related log information of the target object; retrieving the query information through a retrieval strategy corresponding to the information retrieval type by utilizing a pre-constructed index database to obtain a retrieval result meeting an information length condition corresponding to an input requirement of a second large language model; wherein different information retrieval types correspond to different retrieval strategies; performing fusion processing on the query information and the retrieval result to generate context reference information meeting the input requirement of the second large language model; and reasoning the context reference information by using the second large language model to generate reply content corresponding to the query information.

The invention discloses a dynamic log collection method and system and a server. According to the method, a flexible strategy configuration management system is introduced, user-defined configuration is supported, a plurality of target servers, different log files and different collection frequencies can be configured in one log collection strategy, and flexible collection strategies can be configured for different collection objects; in addition, in the method, strategies are automatically distributed, and the complexity of deployment and management is simplified; after the change of the file is monitored, the log file in the appointed path with the change is automatically collected, the automatic log collection and storage process reduces the requirement of manual operation, reduces the possibility of error occurrence, and improves the integrity and reliability of log data; in addition, the method supports the simultaneous storage of the logs in a plurality of target storage systems, such as a local file system, a cloud storage, a database and the like, so that the flexible configuration of log storage is realized.

In various examples, data storage is managed using a distributed storage management system that is resilient. Data blocks of a logical block device may be distributed across multiple nodes in a cluster. The logical block device may correspond to a file system volume associated with a file system instance deployed on a selected node within a distributed block layer of a distributed file system. Each data block may have a location in the cluster identified by a block identifier associated with each data block. Each data block may be replicated on at least one other node in the cluster. A metadata object corresponding to a logical block device that maps to the file system volume may be replicated on at least another node in the cluster. Each data block and the metadata object may be hosted on virtualized storage that is protected using redundant array independent disks (RAID).

The invention discloses an operation and maintenance method and device of a query system, a computer readable storage medium and a processor. The method comprises the steps that a query request of a query object is received, and the query request at least comprises target identification information; based on target identification information in the query request, target full-link log information matched with the target identification information is retrieved from a retrieval server; based on the target full-link log information, determining a service scene targeted by the query task; detecting a query message corresponding to the query task based on the execution standard information in the service scene to obtain a first detection result, and detecting the target full-link log information based on the execution standard information to obtain a second detection result; and adjusting the query message based on the first detection result, and adjusting the service execution process based on the second detection result. The technical problem that the operation and maintenance difficulty of a query system for data interaction based on the extensible markup language is large is solved.

The invention relates to the technical field of data processing, and provides a BMC fault recording method and device, computer equipment and a storage medium, and the method comprises the steps: responding to a kernel fault of a target operating system of a target BMC, and printing fault log information; obtaining a pre-configured memory technology device partition; transmitting the fault log information to an annular fault cache region to form cache log data; storing the cached log data in a memory technical equipment partition, and generating a fault log file; and in response to a user reading instruction, reading and outputting the fault log file of the memory technical equipment partition. According to the technical scheme provided by one or more embodiments of the invention, the fault log can be effectively recorded, maintenance personnel can timely and accurately know the fault position and the fault reason, and BMC fault diagnosis and repair can be efficiently completed.

The embodiment of the invention discloses a data processing method and device, equipment, a medium and a product, and the method comprises the steps: obtaining the operation data of a power system, carrying out the analysis, obtaining a clustering analysis data set, inputting the clustering analysis data set into an abnormal value detection model, determining and correcting an abnormal value, obtaining a data set corrected by the abnormal value, and carrying out the detection of the abnormal value. And deleting redundant data in the data set subjected to abnormal value correction through importance analysis, and inputting the data set without the redundant data into a data quality optimization model to obtain a target operation data processing result. The abnormal value detection model and the data quality optimization model are obtained by training accurate data meeting a preset data quality requirement in the historical operation data. According to the technical scheme of the embodiment of the invention, the problem that the currently obtained operation data is inaccurate is solved, the data quality can be improved through an intelligent algorithm and a machine learning model, the reliability and availability of the data are guaranteed, and reliable data support is provided for data analysis and decision making of a power system.

Techniques are described for performing different types of restart operations for a file storage replication between a source file system and a target file system in different cloud infrastructure regions. In certain embodiments, the disclosed techniques perform a restart operation to terminate a current cross-region replication by synchronizing resource cleanup operations in the source file system and the target file system, respectively. In other embodiments, disclosed techniques perform a restart operation to allow a customer to reuse the source file system by identifying a restartable base snapshot in the source file system without dependency on the target file system.

The invention relates to the field of cold and hot data storage, in particular to a cold and hot data storage optimization method based on a large model, which comprises a business integration module, a data dependence module, a mode updating module, a cold and hot partitioning module and a file merging module, and is characterized in that the business integration module is used for collecting business data to form a training data set; the data dependency module is used for fitting business data to obtain a data consanguinity model, the mode updating module is used for predicting an access mode, the cold and hot partitioning module is used for calculating the heat of the data and performing dynamic scheduling, and the file merging module is used for automatically merging small files. A data storage structure is standardized, storage space is reduced, more efficient and more accurate cold and hot data classification can be achieved, the resource utilization rate of data storage hardware is improved, hot data response delay is reduced, the system storage utilization rate is remarkably improved, and the storage efficiency is improved.

The invention discloses a voice platform full-link log management analysis method and system, and relates to the technical field of information, and the method comprises the steps: obtaining log data; storing the log data by using a log storage engine; analyzing the log data through a machine learning algorithm; visually displaying the log data; and log security control. According to the method, a distributed architecture is adopted, and a collector and a storage engine are deployed at a log generation source, so that comprehensive collection and storage of full-link log data are realized, real-time monitoring and analysis are ensured, and data collection efficiency and data comprehensiveness are improved; by utilizing the efficient storage engine, the system can process a large amount of log data, and the query efficiency is improved; by introducing a machine learning algorithm, intelligent analysis and mining of log data are realized, and potential problems and optimization points in the log data are found.