9803results about "Database updating" patented technology

The invention provides a time sequence knowledge graph federation collaborative optimization method, system and device based on causal inference and a storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: generating an enhanced knowledge unit with a causal mark through the real-time access of a multi-field heterogeneous data stream and the execution of a space-time alignment operation; by calculating new and old knowledge conflict scores, conflict resolution and version management are realized through a decision tree mechanism, and a time sequence knowledge graph with history tracing is output; node weights are dynamically distributed among distributed nodes based on knowledge entropy, a hierarchical aggregation strategy is adopted to update an entity embedding layer and a relation prediction layer, and a global optimization model is output; the method comprises the following steps: analyzing a natural language query containing an anti-fact condition, extracting a factor sub-graph from a time sequence knowledge graph, executing intervention calculation, and generating an anti-fact influence report, thereby solving the technical problems of a traditional time sequence knowledge graph in the aspects of multi-source heterogeneous data fusion, knowledge conflict resolution and privacy protection; and the accuracy and the interpretability of the knowledge graph are improved.

The present invention relates to a digital twin processing method and system, and a cloud platform. The method comprises: acquiring production system elements, carrying out abstraction definition and parameterization description on the production system elements by means of digital-to-analog knowledge fusion of a mathematical model and a mechanistic model, so as to construct a digital twin ontology model; analyzing and reconstructing model data to obtain a mapping model of which object variables can be directly accessed and operated by a collective motion control method, so that the model is visualized at the cloud; and using an external data source to drive parameter update and operation matching of the model by means of a motion control method, so as to complete cooperative deployment and synchronous evolution of an actual physical device and the model in the production process on a cloud server. According to the present invention, a model is constructed by means of digital-to-analog knowledge fusion of a mathematical model and a mechanistic model, the model is mapped to achieve motion visualization, model parameter update and operation matching on the cloud are achieved, and then cooperative deployment and synchronous evolution of a physical device and the model are completed.

A system and method for adaptive real-time multi-modal compression with dynamic resource allocation provides intelligent compression optimization based on continuously monitored device conditions. The system monitors battery level, CPU utilization, and memory availability while classifying incoming multi-modal data streams comprising image, audio, text, and sensor data to determine processing priorities. Multi-objective optimization balances compression efficiency, reconstruction quality, and energy consumption using evolutionary algorithms that generate optimal parameters for an adaptive variational autoencoder. The autoencoder features dynamically selectable processing complexity, adjustable latent space dimensionality, and modality-specific processing layers. The system automatically switches between operational modes including emergency mode triggered by resource constraints, which applies maximum compression settings and intelligent data triage. Continuous learning adapts compression parameters based on observed performance outcomes, improving future optimization decisions. The system enables homomorphic operations on compressed data and provides enhanced compression performance under varying resource constraints across diverse edge computing applications.

The invention discloses a multi-modal bill processing method based on dynamic knowledge enhancement. The multi-modal bill processing method comprises the following steps: S1, constructing a dynamic knowledge base containing an aging weight; s2, synchronously processing text, image and format features of the bill by adopting a multi-modal feature fusion network to generate a composite feature vector; s3, semantic-level, format-level and timeliness three-level fusion retrieval is carried out based on the composite feature vector, and a three-level fusion retrieval engine comprises dynamic weighted sorting with timeliness attenuation, a difference degree triggered artificial review mechanism and a policy sensitive slope adjustment algorithm; s4, setting a multi-expert cooperative verification system, wherein the multi-expert cooperative verification system comprises cooperative work of a rule engine, a large language model and a logical reasoning module; s5, implementing a dynamic knowledge updating mechanism, and automatically triggering incremental learning of the knowledge base when policy change or format update is detected; and S6, outputting structured data, and synchronously generating an auditing traceability chain containing a decision path. According to the method, the key field identification accuracy can be improved, and auditing traceability and non-perceptual increment updating in the whole process are realized.

The invention relates to the technical field of information retrieval, and discloses a distributed storage and indexing method, which comprises the following steps of: dynamically fragmenting an original file through an improved consistent Hash algorithm to generate a plurality of data blocks with timestamps; constructing a three-dimensional Bloom filter index matrix containing timestamps, data types and content features for the data blocks with the timestamps, associating a B + tree local index with an inverted global index through a hierarchical index structure, and establishing an index update priority queue by adopting a two-channel synchronization mechanism, performing real-time increment synchronization on the hotspot index through a heartbeat mechanism, performing batch synchronization on the cold data layer index according to a cold data synchronization period, predicting a data distribution probability through a distributed query statistics probability table during query, and initiating multi-path query in parallel based on probability weight, and the query path is dynamically optimized according to the node group storage medium type and the inter-node network transmission delay, so that rapid distribution adjustment and access of distributed storage are realized.

The invention provides an intelligent station multi-source heterogeneous data acquisition and remote collaborative analysis method and system, and relates to the technical field of heterogeneous data, and the method comprises the steps: collecting multi-source heterogeneous data, carrying out the standardization processing, constructing a station real-time adaptive three-dimensional model through a self-calibration laser scanning technology, and carrying out the semantic mapping of the model and a BIM model; constructing a self-evolution feature library based on a digital twinborn model, and carrying out space-time correlation analysis by adopting a graph convolutional neural network; a remote cooperation platform is established to realize real-time cooperation of multiple experts, and a diagnosis process is recorded through a block chain technology. According to the invention, efficient acquisition, analysis and remote collaborative diagnosis of the data of the smart station are realized, and the accuracy and efficiency of fault diagnosis are improved.

The embodiment of the invention provides a BIM-based hydraulic engineering digital management method and system. The method comprises the following steps: performing multi-dimensional monitoring system deployment on a to-be-monitored area to obtain multi-dimensional hydrological data; constructing a first BIM based on the topographic data of the to-be-monitored area, the real-time work area image, the hydraulic engineering construction information of the design stage and the equipment deployment information; fusing the multi-dimensional hydrological data, the water area change data, the construction progress data and the operation and maintenance monitoring data into the first BIM, and constructing a second BIM including a full life cycle; and through the equipment characteristic curve and the water conservancy project physical model, in combination with the multi-dimensional hydrological data and the water area change data, predicting a water conservancy project structure change trend and potential risk factors in the second BIM, fusing the water conservancy project structure change trend and the potential risk factors into the second BIM, and displaying an implementation effect and improvement suggestions in real time in the second BIM. A user is assisted to realize hydraulic engineering digital management, and the engineering management efficiency is improved.

The invention relates to the technical field of transaction processing systems, and particularly discloses a trusted monitoring auditing system fusing TEE and a block chain, which comprises the following steps: generating a transaction instantaneous state abstract containing a key information field at a key node of a business process through a trusted execution environment, signing, anchoring a hash value of the transaction instantaneous state abstract to a block chain network, and storing the signature abstract under the chain; and during auditing, by comparing integrity verification of on-chain hash and under-chain abstract and TEE signature verification, dual authentication of transaction authenticity and environment reliability is realized. Through the collaborative architecture of the TEE and the block chain, the transaction core data state and the credible environment proof are deeply fused, so that the audit verification can be independently completed without depending on a centralized database, the problem that post-event tracing depends on the credibility of the log in the traditional audit is fundamentally avoided, and the audit verification efficiency is improved. And a lightweight credible verification basis is provided for a cross-mechanism cooperation scene.

The invention discloses a data quality intelligent auditing system and method based on a dynamic rule base, and belongs to the technical field of data auditing, and the system comprises a rule base construction module which is used for analyzing business scene parameters through a scene analysis unit according to business scene demands and data type features to generate a rule configuration instruction; the multi-source monitoring engine module is connected to the rule base construction module and is used for collecting multi-source data in real time and loading corresponding checking rules; the automatic verification execution module is used for executing normalized quality verification on the multi-source data based on the verification rule base; and the feedback optimization module analyzes a rule hit rate and a false alarm rate in a verification result through a reinforcement learning algorithm, and dynamically iteratively updates a rule threshold value and a logic combination in the rule base. By constructing a full-automatic process of rule generation, execution, feedback and updating, the problems that a traditional system depends on manual intervention, response is slow, the industry average rule updating period is 3-7 days, and real-time updating is achieved through the scheme are solved.

The embodiment of the invention discloses a database multi-source heterogeneous data synchronization method and device.The method comprises the steps that a data change event is monitored through an RPA robot deployed in a server where a source database is located, and when change operation of structured data or unstructured data is detected, an initial data set is generated; performing fragmentation processing on the initial data set to generate a plurality of data fragmentation tasks, and allocating the data fragmentation tasks to distributed computing nodes to execute parallel format conversion operation so as to generate a target data set; detecting a task execution state of the distributed computing node, and when a generation completion event of a target data set is detected, activating a data synchronization channel and transmitting the target data set to a target database; and performing conflict detection and identification based on an RPA conflict processing strategy configured in the target database, when a primary key conflict or a uniqueness constraint conflict is identified, updating the target database based on preset conflict merging indication information, and generating a data synchronization log.

The invention discloses a logistics traceability system and method based on a block chain technology, and relates to the technical field of logistics traceability, and the method comprises the steps: constructing an event graph structure containing a plurality of behavior event nodes, and calculating a structure closure signature value; judging whether the original data of the behavior event node is written into the block chain or not according to the responsibility structure weight value; constructing a verification path mapping graph based on the structure closure signature value and the responsibility structure weight value; when a traceability request is received, calling the verification path mapping graph, performing consistency verification on the original data of the target behavior event node and the structure mapping hash value, and generating a traceability verification result; according to the method, the credibility of the behavior event data structure is improved, meanwhile, the on-chain storage cost is effectively reduced, and the technical contradiction between full-uplink data and non-uplink data is solved.

The invention discloses an elderly disease health question and answer method based on multiple agents and a knowledge graph, and belongs to the crossing field of artificial intelligence and medical information technologies. Structured representation of medical knowledge is realized by constructing a multi-modal medical knowledge graph and through multiple entities and association relationships thereof. In combination with a domain customized large language model and a multi-agent dynamic routing mechanism, a two-stage training strategy is adopted to optimize a base model, including all-parameter pre-training and parameter efficient fine tuning based on a low-rank decomposition technology, so that the professionality and reliability of the model are effectively improved. In a multi-agent collaborative architecture, the system dynamically selects an optimal processing path according to input characteristics, including knowledge graph query, professional answer generation or external knowledge retrieval, and maintains context continuity of multiple rounds of conversations through a dynamic abstract compression algorithm. According to the method, the multi-agent collaborative architecture is combined with the knowledge graph reasoning ability, and the advantages of relation query and semantic matching are integrated through the mixed retrieval strategy.

The invention belongs to the technical field of computers, and particularly relates to an information integration management method and system supporting real-time updating, and the method comprises the steps: accessing a heterogeneous data source through a distributed gateway, carrying out the adaptive analysis of a protocol, and standardizing a structure; adding a millisecond timestamp and a unique identifier, and generating a semantic data cluster through streaming engine sliding window association analysis; after multi-dimensional consistency verification is executed, writing and storing are carried out by adopting a double-buffering and snapshot isolation mechanism; and dynamically assembling the customized view according to the user role and the context and safely returning the customized view. The system correspondingly comprises a protocol analysis module, a time sequence marking module, a streaming association module, a semantic verification module, a double-buffer control module, an index updating module, a view assembling module and a secure transmission module. According to the invention, millisecond-level real-time convergence, conflict-free fusion and on-demand view output are realized, and the data access flexibility, the quality guarantee capability and the service adaptation flexibility are comprehensively improved.

The invention relates to the technical field of data analysis, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a data consanguinity map processing method, device, equipment and medium, and the method comprises the steps: collecting metadata change information and data operation event information, and building a mixed consanguinity map model; monitoring a metadata change event and a data operation event, and triggering incremental updating based on an event type; and performing data change influence analysis based on the updated blood relationship map model, generating an analysis result, and outputting the mixed blood relationship map model and the data change influence analysis result. According to the invention, by collecting metadata, incrementally updating the blood relationship map and carrying out data change influence analysis, full-link data tracing and influence analysis are realized, the problems of blood relationship breakage and insufficient real-time data tracing in traditional data blood relationship analysis are solved, and the accuracy of data tracing and the timeliness of business decision are improved.

The invention discloses a database interaction method, device and equipment based on multi-agent cooperation and a medium, and relates to the technical field of database interaction. The method comprises the steps of obtaining a query request, performing semantic partitioning and vectorization, and obtaining a retrieval vector set; and selecting a search source and searching to obtain an information block. And obtaining candidate knowledge through knowledge base retrieval. And integrating the information blocks and the candidate knowledge to obtain a candidate information set. And constructing a dynamic demand understanding model according to the query request and the candidate information set, and obtaining a task state matrix and a tool calling task. And calling a corresponding auxiliary tool and operating, updating the task state matrix according to an operation result and returning the task state matrix to the task planning agent to update the tool calling task until the tool calling task is no longer output, and obtaining a task result. And when the database interaction module is called, generating a chart according to an output result of the database interaction module, and outputting the chart and a task result together. Otherwise, directly outputting a task result.

The invention relates to the technical field of storage optimization, in particular to a data storage management system and method based on big data, and the system comprises an efficiency monitoring module, a medium evaluation module, a decision generation module, a strategy execution module and an effect feedback module. According to the method, input and output delay, a medium health attenuation curve and a bandwidth fluctuation map are collected, a dynamic feature space is constructed, the stability of the system is judged in combination with a Lyapunov index, early warning of performance abnormity is achieved, health indexes such as an erasing cycle, a seeking error rate and a charge retention rate are fused, and the access frequency and the load pressure are combined. Calculating a compensation coefficient, improving heterogeneous medium cooperation efficiency, pre-migration evaluation cost and topology adjustment income, improving resource configuration accuracy, dynamically adjusting hot and cold data distribution and node load, enhancing system scheduling flexibility, comparing throughput, medium wear rate and synchronous delay, and quantifying stability gain brought by configuration change; and a closed-loop process from identification to feedback is constructed.

The invention relates to the technical field of data processing and storage management, and discloses a multi-source index data intelligent storage management method and system, and the method comprises the steps: building a unified data integration and processing channel, and forming a standardized index data set; meanwhile, an enterprise-level index unified management system is established, and a credible index data set is generated; the credible index data set is stored through an optimized storage strategy and a data organization model, a data consanguinity association map is constructed based on data full-link processing, and visual and traceable index data assets are obtained; according to the method, a real-time quality evaluation engine is used for carrying out continuous monitoring and intelligent warning on all links of synchronization, processing, storage and service of an index data set so as to construct a data anomaly detection and repair mechanism, and quality controllable supervision is carried out on the whole processing process of the index data set through a data quality billboard and an anomaly processing workflow; therefore, the intelligent management level and the data quality guarantee capability of multi-source index data processing and storage can be improved.

The invention relates to the technical field of data processing, in particular to a data fusion intelligent equipment linkage management and control system based on a large model, and the system comprises a data collection module which outputs standardized data carrying privacy grading identifiers; the data processing module outputs an optimization algorithm identifier and a collaborative strategy identifier which are synchronously bound; the privacy algorithm fusion scheduling module generates atomization calculation graph data and strategy audit logs fusing privacy operation; the multiple optimization resource module executes calculation graph data and drives a parallel simulation environment rehearsal resource scheme, and the double-closed-loop feedback module is activated when an arbiter continuously detects that deviation exceeds the standard for three times; the safety instruction generation module fuses the output result and the audit log to generate a digital signature equipment control instruction; and the double-closed-loop feedback module updates knowledge base weight parameters through a data feedback loop, and controls the feedback loop to adjust a resource allocation threshold in a grading manner according to simulation confidence, so that dynamic balance between resource efficiency and privacy intensity is realized.

The invention discloses a territorial space planning dynamic monitoring method based on multi-source data fusion, and belongs to the technical field of territorial space planning intelligent monitoring, and the method comprises the steps: obtaining data, and generating a multi-source heterogeneous data set; performing space-time alignment processing by using a preset regional association rule of a planning knowledge base to generate a space-time unified data set; constructing a dynamic knowledge graph taking planning elements as a core based on the data set, and updating node relation weights in real time; guiding a multi-source data fusion direction through the map relation weight to generate a fusion feature vector; incremental learning monitoring processing is carried out on the feature vectors, parameters are optimized, and a planning implementation state monitoring result is output; and updating the knowledge graph node relation weight in a closed loop manner according to a monitoring result, and synchronously optimizing incremental learning monitoring processing. According to the method, a dynamic knowledge graph is adopted to guide data fusion and an incremental learning closed-loop optimization mechanism in real time, and accurate perception and adaptive decision support of a planning implementation state can be realized.

The invention relates to a prefabricated building quality defect tracing system and method applied to the field of building informatization, and the system comprises a user interaction layer, a data processing and storage layer, a data collection layer and a function layer, and the data collection layer is used for collecting key quality data of prefabricated building components in different links; the user interaction layers interact with each other through a network and the function module layer; the data processing and storage layer performs encryption, hash operation, scattered storage and index verification on the quality data transmitted by the data acquisition layer by using an intelligent contract, an IPFS network and a block chain; the function module layer provides support for each unit of the user interaction layer to trace quality defects, judge responsibility and take prevention measures, and the system can collect the quality data of the whole process of the prefabricated building, comprehensively manage the quality tracing process, strengthen multi-party collaboration and sharing, and improve the quality of the prefabricated building. And it is ensured that tracing of the quality defects of the fabricated building has comprehensiveness, transparency and non-tampering performance.

The invention provides a medical wholesale and retail comprehensive service method and system based on the Internet, and relates to the technical field of medicines.The medical wholesale and retail comprehensive service method comprises the steps that a hospital HIS system is docked in real time through an API, and electronic prescription data are collected; the collected electronic prescription data is connected with a medicine wholesale enterprise ERP system, the inventory state, batch number and validity period information of each medicine are obtained, and inventory information is obtained. According to the invention, through real-time connection of the hospital HIS system, the drug wholesale enterprise ERP system and the retail terminal POS system, inventory information can be obtained and dynamic management can be carried out. The real-time data updating not only improves the inventory turnover rate, but also can automatically warn the medicines close to the expiration date, and reduces the loss of the expired medicines. The LSTM neural network is adopted to predict the drug demand, and the replenishment suggestion is generated in combination with the linear programming model, so that the replenishment process can be optimized while the operation cost is minimized, and sufficient and efficient inventory is ensured.

The invention provides an online map tile fast loading and caching optimization method, medium and system, and belongs to the technical field of map processing.The method includes the steps that a tile index database and an exchange priority matrix are established, a viewport moving trend vector is calculated according to user sliding operation, a future viewport range is predicted, and a preloaded tile list is screened; the method comprises the following steps: performing cache validity check on tiles, establishing a small change detection vector to judge whether updating is needed, establishing a tile hit rate statistical matrix, fusing a time decay factor and a spatial correlation factor to calculate a popularity score, and gathering high-popularity tiles in a quick access area by adopting a thermodynamics-imitating entropy reduction algorithm to realize cache ordering. And the next tile demand of the user is predicted, so that the technical problem that the overall loading response speed is reduced due to an overlong high-value tile access path caused by disordered distribution of cache resources in the online map tile loading process is solved.

A system may be configured for cooperation between language model agents. An agent may be, for example, a computer system, or a software component executing on a computer system, that can accept text and / or natural language inputs, draw upon an LM to process the inputs and perform a function, and respond via text and / or natural language outputs. An agent may act as a mediator to interact with a user, identify a task requested by the user, and delegate one or more subtasks to another agent or other resource. An agent may act as a delegate to handle tasks or subtasks delegated by a mediator. Agents may communicate with each other using a combination of structured and unstructured language; for example, one or more parameters and a natural language message.

The invention relates to the technical field of medical decision, and discloses a medical decision support system based on a knowledge graph, and the system comprises a knowledge graph construction module which constructs an initial knowledge graph based on a medical ontology library, and the knowledge graph comprises entities and association relationships of diseases, symptoms and drugs; the data acquisition module is used for acquiring data from an electronic medical record, wearable equipment, a medical literature library and a hospital information system and normalizing the data through a standardized protocol; the dynamic knowledge updating module is used for processing normalized data through an incremental graph neural network; a multi-source knowledge fusion module; a context awareness module; a dynamic deduction module; and a decision optimization closed loop module. And triggering a preset clinical rule in real time based on the pathological state of the patient, dynamically adjusting the intensity value of the related edge in the factor graph, and persistently storing the intensity value back to the knowledge graph, so that logic adaptation and individualized experience precipitation of general medical knowledge in a special pathological state are realized, and the individualized treatment accuracy is ensured.

The invention relates to the technical field of smart factory management platforms, and discloses a data real-time monitoring method based on a smart factory management platform system, and the method comprises the steps: carrying out the standard format conversion of supplier Excel data, carrier GPS track data flow and warehouse RFID goods location data in a smart factory ERP system, and obtaining standardized business data; calculating comprehensive scores of suppliers based on the standardized business data and generating a dynamic sorting list; receiving purchase order information and creating a logistics scheduling plan, and pre-allocating an optimal goods allocation for the warehousing commodities to obtain a warehousing pre-allocation scheme; monitoring the execution progress of the storage pre-distribution scheme in real time, and synchronously updating service state information; the auditing state of the purchase order information is monitored based on the service state information, and an intelligent collaborative decision-making scheme is generated, real-time synchronous updating of the service states of the supplier management module, the logistics scheduling module and the warehouse management module is ensured, and the supply chain collaborative management efficiency is improved.

The invention discloses a multi-mode fusion product document and source code association retrieval method based on a knowledge graph, and relates to the technical field of software engineering and artificial intelligence. The method comprises the steps that source codes are preprocessed, and code structure information and business semantics are mapped in combination with a predefined business term dictionary; performing controlled induction on a code file and a product document by utilizing a large model, extracting business terms, logic intentions and a subject relationship, fusing with original codes, and establishing a vector retrieval index system; further analyzing a code structure by using an abstract syntax tree, and extracting an entity and a calling relationship; semantic enhancement and relation normalization are performed in combination with the large model, entities and relations are stored in a graph database, and a knowledge graph is formed; and performing parallel processing on user query based on a full-text retrieval index, a vector retrieval index system and a knowledge graph, and finally generating a product concept. According to the method, the retrieval speed, the semantic depth and the logical reasoning ability can be considered at the same time, and the retrieval accuracy is improved.

The invention discloses a verifiable medical data query method based on a DAG block chain architecture, and belongs to the technical field of block chains and medical information management. According to the method, medical events such as diagnosis and treatment records and image reports are modeled into DAG nodes, an on-chain topology with time sequence and causal association is constructed by encrypting a hash pointer, dynamic indexing and multi-keyword verifiable query are achieved in combination with an off-chain authentication data structure, a verification object containing a matching node set and a verification path is received, and the verification object is subjected to dynamic indexing and multi-keyword verifiable query. The authenticity, integrity and timeliness verification of the result is independently completed based on the authentication root stored on the chain without depending on a third party. According to the method, through an on-chain and off-chain collaborative storage and incremental authentication updating mechanism, on the premise of ensuring privacy and tamper-proofing of medical data, query efficiency, verification performance and expandability under high concurrency are remarkably improved, communication and resource expenses are lower, and the method is suitable for large-scale popularization and application. The method is suitable for application scenes such as cross-institution medical data sharing, remote diagnosis and treatment and compliance auditing.

The invention relates to the technical field of data processing, and discloses a knowledge graph incremental updating and consistency checking method and system. The method and the device are used for solving the problem of low incremental updating and consistency checking efficiency of a large-scale knowledge graph. The method comprises the following steps: collecting a to-be-updated data source, sorting and separately storing data, and marking priority labels; performing change detection on the marked data source, identifying change items by comparing entities and relationships, and generating a log; incremental updating is executed based on the log, and nodes and edges are processed in a hierarchical fusion mode; preliminary consistency verification is carried out, and attribute uniqueness and relation directivity are checked; expanding a verification range, traversing an association path through cascade check, and recording problems; and optimizing storage according to the record, merging the update area, updating the index and cleaning the log. The method solves the problem of low efficiency of incremental updating and consistency verification of the large-scale knowledge graph, improves the response speed and the data accuracy of the system, and is suitable for a high-frequency dynamic data environment.

The invention discloses a high-performance AIOT real-time data processing method and device based on Flink, Kafka and Dores, and the method comprises the steps: constructing an end-to-end hierarchical architecture, capturing multi-source database change data in real time through Flink CDC, and analyzing semi-structured data in combination with Apache NiFi, and generating a standardized data stream; the method comprises the following steps: realizing message routing based on KafkaTopic division of a service domain, ensuring the consistency of a data structure by using Schema Registry, and optimizing the transmission efficiency through a dynamic compression algorithm; an Flink real-time computing engine is adopted to implement window aggregation, asynchronous dimension table association and incremental state persistence strategies, and the resource fluctuation and state management problems in streaming computing are solved; time pre-partitioning and intelligent cold and hot layering are achieved through a Dores storage layer, the query performance is improved in combination with materialized views and multi-level indexes, and finally a full-link closed loop of collection-transmission-calculation-storage-management is formed. According to the invention, the problems that the real-time performance and the consistency are difficult to balance and the system complexity is high in the prior art are solved.

The invention discloses a method for carrying out data cleaning decision making and automatic tuning by using AI workflow, and relates to the technical field of data processing, the method comprises the following steps: obtaining original data from various heterogeneous data sources, and generating a correlation table and a result through field extraction, structure analysis and language attribute recognition; based on this, storage and task generation are performed, and standardized data and task queue information are output; identifying a quality problem through feature analysis, defining a target and constructing a cleaning task atlas, and generating cleaning metadata through preliminary screening and marking; and an AI cleaning workflow containing multiple strategy paths is constructed and optimized, after execution, a downstream feedback evaluation effect is combined and adjustment is carried out, and finally a multi-version cleaning result is integrated and output. According to the method, the adaptation capability to different types of data sources is effectively improved, the dependence on a static preset strategy is reduced, the manual intervention requirement is reduced, the tuning efficiency is improved, and the systematicness and normalization of data cleaning are enhanced.