183 results about "Data warehouse" patented technology

The invention discloses a fire-fighting spatial-temporal trajectory data processing and off-line data warehouse construction method based on a stream batch fusion architecture, and relates to the field of digital fire fighting, and the core steps of the method are as follows: after multi-source data are collected by an edge gateway, high throughput distribution and storage are realized by Kafka; real-time optimization such as track cross repair and offset correction is completed through Flink streaming calculation; performing offline batch processing on the historical data by the Spark to generate an analysis result; generating an optimal rescue path through a distributed Dijkstra algorithm in combination with road topology, real-time traffic and fire fighting truck parameters; the method comprises the following steps: realizing mixed storage by adopting ElasticSearch secondary index and HBase total storage; based on hot and cold data hierarchical management, intelligent applications such as fire-fighting commanding and dispatching, battle comment and review and the like are supported. Practice verifies that the method can shorten the fire control time of the main urban area by 60%, significantly improves the fire emergency response capability, and is suitable for large-scale fire track data scenes.

The invention provides a control method for layer-by-layer carrying in a segmented data warehouse, which belongs to the technical field of layer-by-layer carrying, and comprises the following steps: step 1, carrying out segmented division on the segmented data warehouse; step 2, performing independent manufacturing on non-adjustment segments in the data warehouse for segment manufacturing; step 3, carrying out non-adjustment segmented welding and precision measurement on the data bins manufactured in a segmented manner; step 4, performing adjustment segmentation allowance compensation on the data bins manufactured in a segmentation manner; step 5, carrying out layer-by-layer carrying operation on the data bins which are manufactured in a segmented manner; 6, front equipment and piping system installation is carried out on the data bins manufactured in a segmented mode; and 7, carrying the segmented data bins in a segmented manner, and integrally folding the segmented data bins. Through scientific segmentation division, accurate precision compensation, a reasonable carrying sequence and preposed equipment installation procedures, effective control over the overall size precision of the data bin is achieved, and the construction quality and efficiency are improved.

The invention relates to the technical field of electric power intelligent interaction, and discloses an AI large language model intelligent interaction system oriented to the electric power industry. The system comprises a multi-source data acquisition module for acquiring power industry multi-dimensional data from a plurality of heterogeneous data sources and integrating the power industry multi-dimensional data into a data warehouse; the semantic feature extraction module performs semantic analysis and feature extraction on the integrated data warehouse to obtain a semantic feature matrix; the industry knowledge graph construction module is used for constructing a power industry knowledge graph based on the semantic feature matrix; the user query analysis module is used for analyzing user query and extracting query feature vectors; the core semantic screening module is used for screening a core semantic sequence based on the semantic feature matrix and the query feature vector; and the intelligent response generation module generates an intelligent response based on the core semantic sequence and the power industry knowledge graph. The system can deeply understand user demands, generates professional and accurate responses, and is suitable for diversified intelligent interaction scenes in the power industry.

Embodiments described herein comprise an advanced Software as a Service (SaaS) platform addressing challenges including trust, governance, quality, data accuracy, and transparency, ensuring full visibility of enterprise data warehouse data pipelines. These embodiments seamlessly integrate with cloud-based and legacy data platforms, enhancing visibility through monitoring data quality, tracking metadata changes, and generating exceptions for review. Data quality management allows for custom script creation to validate pipelines and business rules. Cloud-native connectors facilitate integration with one or more cloud-based services streamlining data delivery processes. Personalized notifications aid administrators in responsive action, while data profiling capabilities identify quality issues and provide distribution insights. Actionable data intelligence for privacy, security, and governance is enabled through data classification. Automation features encompass data quality checks, metadata monitoring, profiling, and classification, orchestrated by a flexible scheduler. The architecture leverages a scalable serverless design with independent microservices, dynamically scaling based on demand.

The invention discloses a data asset management system and method, and relates to the field of data management, and the system comprises a data collection layer which is used for collecting multi-source heterogeneous data; the data integration layer is used for integrating the multi-source heterogeneous data and constructing a data warehouse; the data storage layer is used for carrying out structured storage on the data in the data warehouse; the data governance layer is used for performing standardization processing and quality management on the data stored in the data storage layer to obtain target data; and the data application layer is used for performing dynamic tracking inventory and classification level-to-level management on the data assets based on the target data. Through cooperative work of five layers of architectures, full-process closed-loop management of the multi-source heterogeneous data from collection to application is realized, and the problems of difficulty in integration of the multi-source data, low management efficiency, uncontrollable quality and the like are effectively solved by the overall scheme.

The application discloses a data bloodline collection method and device based on a Gbase stored procedure, and the method comprises the following steps: removing fixed keywords of a stored procedure definition process body; capturing a stored procedure variable through a regular expression and converting the stored procedure variable into a Key-Value form; replacing a variable name with a variable value according to the Key; processing a loop structure and a branch structure, and processing a self-defined label; and performing syntax compatibility and replacing or removing SQL statements according to some keywords. The application processes some general keywords of the stored procedure, removes redundant information irrelevant to the data bloodline, processes some syntaxes specific to the Gbase in a compatible manner, so that the syntaxes can be parsed by open-source SQL tools, and the application is adapted to the stored procedure of the Gbase, so that the application can help some manufacturers using the Gbase stored procedure to process data in an automatic form to the data bloodline, and the application is convenient for data warehouse construction.

The application discloses an intelligent accounting method for industrial water consumption, and relates to the technical field of industrial water consumption monitoring and intelligent accounting, and comprises the following steps: step one: receiving and analyzing water consumption accounting requirements, simultaneously acquiring historical data and real-time data from a historical data warehouse and a real-time data acquisition system, respectively, pre-processing, and generating a pre-processed data set. The intelligent accounting method for industrial water consumption realizes real-time self-adaptive adjustment of time-varying factors in industrial water consumption accounting by adopting multi-level data fusion, an improved deep learning network, data sliding window online learning, feedback closed-loop control and a hierarchical incremental updating strategy, significantly improves the prediction accuracy and system response speed of nonlinear water consumption behavior, and thus effectively overcomes the deficiency of static offline training of an accounting model in the prior art.

The invention belongs to the technical field of nuclear fuel assembly detection, and particularly relates to a fuel assembly operation robot positioning method and system based on adaptive contour feature matching. Comprising the steps that a global surface optimization tool converts acquired contour data into optimized ideal contour surface information, a fitting contour surface evaluation optimization tool feeds back a fitting contour surface overall deviation evaluation result, and a task data integration tool is an algorithm tool for relevant data configuration in the operation execution process of an execution assembly. And the task data integration tool constructs a data warehouse, and obtains trajectory data corresponding to the contour from the contour surface optimization module. The method has the advantages that one-key self-adaptive execution of assembly operation work can be achieved, accurate operation positioning is achieved through high-precision contour modeling, the method adapts to various motion control cards through a configuration type mode, dependence on the operation technology of operators is reduced, and the operation safety of equipment is improved.

The present application relates to a data storage method and apparatus, a computer device, and a computer readable storage medium. The method comprises: a data middle platform first acquiring current application environment information, user requirement information, and data information to be stored, and generating corresponding configuration information on the basis of the current application environment information and the user requirement information; constructing a corresponding storage directory tree on the basis of the configuration information; and finally storing said data information into a corresponding data storage node on the basis of data warehouse elements and data indicators of said data information.

This application discloses a resource scheduling method, apparatus, device, and medium based on a real-time data warehouse, relating to the fields of financial technology and data processing technology. It can ensure real-time service while significantly improving the release efficiency of idle resources during periods of low traffic, thereby greatly improving the overall resource utilization of the cluster while ensuring system stability. The method includes: predicting the data traffic of computing nodes in the real-time data warehouse using a time-sharing traffic prediction model; monitoring the load status of computing nodes in the real-time data warehouse using a load-aware model; determining the initial resource configuration of computing nodes based on the data traffic predicted by the time-sharing traffic prediction model before the start of the real-time data warehouse scheduling cycle; dynamically adjusting the initial resource configuration of computing nodes based on the load status monitored by the load-aware model during the real-time data warehouse scheduling cycle, generating resource scheduling instructions; and allocating resources to computing nodes in the real-time data warehouse according to the resource scheduling instructions.

The present application belongs to the technical field of data management, and particularly relates to a data management system for a twisted wire automatic detection platform, to solve the technical problem that the prior art cannot identify periodic defects and the traceability analysis process cannot correlate defects. The management system comprises: a data acquisition and synchronization unit for forming a longitudinal data profile; a main data modeling unit for constructing a pedigree main data model containing twisted wire product, single wire raw material and production process data; a data warehouse hierarchical processing unit for establishing a hierarchical data warehouse; an identification unit for constructing a data theme layer above the data detail layer; and an application service unit for constructing a data application layer above the data theme layer. The present application realizes traceability from quality problems to production, and provides data decision basis for process improvement and quality control.

The invention discloses an e-commerce big data analysis method and system based on artificial intelligence, and belongs to the field of data analysis, and the analysis method comprises the following specific steps: I, obtaining various e-commerce data from various channels, preprocessing the e-commerce data of different sources and different formats, and constructing an integrated data warehouse; iI, based on the collected e-commerce data, mining behavior characteristics of the user, extracting a fine-grained behavior pattern, and constructing a personalized label of the user; iII, analyzing each piece of text data uploaded by the user, and extracting the psychological demand, risk perception and consumption intention of the user; according to the method, the workload of manual cleaning, labeling and merging is greatly reduced, the user portrait does not stay on a static label any more, an interpretable dynamic behavior mode is formed, the causal inference and real-time key factor recognition capability is improved, and platform treatment, compliance requirements and manual checking are facilitated.

In an example embodiment, a data warehousing environment, or a similar architecture, is extended to permit data storage in either an in-memory database or a Lakehouse architecture, which leverages one or more hyperscalers for the underlying storage. More specifically, a single artifact is defined in Datasphere that stores data in either the in-memory database or the Lakehouse architecture, and does so in a storage-agnostic manner.

The invention relates to the technical field of power grid project cost management and business intelligence, and discloses a project cost multi-stage intelligent analysis and examination method, system, equipment and medium, and the method comprises the steps: building a power grid project cost data warehouse, and achieving the integration and standardization of multi-source cost data; establishing an OLAP multi-dimensional analysis model, and supporting general cost comparison and key index analysis in a budget calculation stage, feasibility research estimation, preliminary design budget calculation, completion settlement three-calculation comparison, first supply material check and review list consistency check in a settlement stage; designing an intelligent decision engine based on data mining, and generating review opinions through association rules, a classification algorithm and an anomaly detection technology; and a BI visual instrument panel is developed, and multi-dimensional display, real-time monitoring and interactive analysis of the power grid cost data are realized. Through systematic application of the BI technology, deep mining, intelligent decision and visual display of engineering cost analysis are realized, and the efficiency and precision of cost review are remarkably improved.

The invention provides an electro-hydrogen shared energy data multi-microgrid energy management system based on neurodynamics, and relates to the technical field of electro-hydrogen energy data processing and optimization. According to the system, a multi-source heterogeneous data source of historical electro-hydrogen energy from multiple microgrids is collected through a data integration module, after data extraction, an energy scheduling optimization problem is solved through a neurodynamics optimization engine, and then optimized data is integrated and stored. The data integration process comprises the steps of data standardization, cleaning, loading, conversion and the like, so that centralized integration of data from different sources is realized, and the data are uniformly output to the energy data storage module; the data storage module stores the electric hydrogen energy data in a fixed format, an HDFS is used as a file storage engine to support data reading and writing, multi-level parallel processing is provided, and high efficiency and stability during large data volume processing are ensured. And the data quality management module monitors the processed energy data according to a preset rule, ensures the accuracy and integrity of the data, and generates a data quality report. By means of the system, efficient processing and optimization control over the electricity-hydrogen shared energy multi-microgrid data can be achieved, high reliability, expandability and fault tolerance are achieved, and the requirements for low cost and real-time intelligent optimization are met.

This invention discloses a knowledge graph-based intelligent question-and-answer method for audit guidance, relating to the field of digital auditing. Through knowledge graph framework design and ETL data warehouse technology, this invention extracts, cleans, and transforms audit business data relevant to business environment optimization audits, achieving the fusion of various business data to provide data support for intelligent question-and-answer. By constructing an intelligent question-and-answer system, and designing a multi-task-oriented approach based on real-world audit scenarios, it utilizes word segmentation, matching, and query modules to achieve knowledge graph-based intelligent question-and-answer for business environment optimization audit guidance. This reduces the workload of the audit process, freeing auditors from tedious and repetitive tasks, allowing them to focus more on risk identification, problem analysis, and root cause analysis, effectively improving audit efficiency and quality.

The invention discloses an enterprise internal acquisition cost optimization decision support system based on big data analysis. The system comprises a data aggregation and integration module, a purchase behavior feature extraction module, a cost driving factor deconstruction module, a supplier multi-dimensional evaluation module, a purchase strategy intelligent customization module, an enterprise purchase data warehouse and an enterprise purchase supervision terminal. The purchase strategy suitable for the enterprise is intelligently generated by integrating the purchase behavior characteristics, the cost driving factors and the supplier evaluation result, the purchase cost composition and the dynamic change can be accurately informed, scientific and reasonable purchase decision suggestions are provided for the enterprise, the purchase cost is effectively reduced, and the purchase efficiency is improved. And the execution effect of the purchasing strategy is monitored, evaluated and fed back in real time, so that each link of the purchasing process can be monitored in real time, potential problems can be found and solved in time, the purchasing efficiency and quality are improved, the optimization of the purchasing cost and the maximization of the overall benefit are realized, the purchasing supervision difficulty of an enterprise is remarkably reduced, and the intelligent level is high.

The application provides a method and system for ensuring consistency of data synchronization of a distributed file system. The method comprises the following steps: responding to an update period of acquired business data; performing secondary processing on the data according to an actual update period of the acquired business data; and responding to periodic full data processing to ensure that all data is up-to-date. The method is used in an offline data warehouse based on HDFS storage+HIVE analysis, is applicable to consistency of business data in various industries, and is particularly suitable for a scenario in which the amount of synchronized data is large and the actual update period of data is short. The method ensures data consistency by performing secondary processing on large amounts of business data according to the actual update period of the business data and combining periodic full data processing. Since the amount of data read from a source library is reduced, the read pressure of the source library is reduced.

The application relates to the technical field of digital security management of assets, and discloses a digital security management system and method, which extracts to-be-scanned asset identification and scanning strategies by analyzing a digital security scanning task, considers network bandwidth and scanning result data warehousing queues of a scanning engine performing scanning, combines asset risk attributes, scanning features and system resource constraints, generates a digital security management strategy, and further controls scanning and data warehousing actions performed by the scanning engine. Thus, the application realizes accurate matching of scanning frequency and asset risk through a dynamic risk assessment mechanism, avoids the complexity and subjective bias of traditional manual strategy setting, comprehensively considers global system resource restrictions such as scanning engine load, network bandwidth and data warehousing queues, optimizes scanning task scheduling, completely solves the resource competition problem that is prone to occurring in traditional scanning, avoids system performance bottlenecks, and significantly improves scanning efficiency and the timeliness of security assessment.

The present disclosure provides an automatic driving data link generation method and device and electronic equipment, relates to the technical field of big data, in particular to the technical fields of cloud computing, information flow and automatic driving, and the automatic driving data link generation method comprises the following steps: obtaining target data, the target data is data for automatic driving, and the target data comprises a scheduling job group; identifying a plurality of jobs included in the scheduling job group, wherein the plurality of jobs have a calling relationship; generating a first data link according to the calling relationship; obtaining a second data link generated in advance, and generating a first target data link according to the first data link and the second data link, wherein the first target data link is a data link for data tracking, and the second data link is a data link corresponding to data cleaning in a data warehouse.

The invention provides an intelligent database table structure self-adaption and data import system and method based on Excel, and the system comprises a file uploading and analyzing module which is used for receiving and analyzing an Excel file uploaded by a user, and extracting header information and a data sample; the database configuration management module is used for storing and managing connection configuration information of a plurality of databases; the intelligent table structure inference engine is used for carrying out multi-dimensional statistical analysis based on Excel data samples and automatically inferring an optimal database table structure design; the multi-database dialect adapter is used for converting the general table structure definition into SQL grammar of a specific database; and the transactional execution engine executes table establishment and insertion operations in an atomic transaction mode to ensure data consistency. According to the system and the method, the Excel spreadsheet data can be automatically imported into the database, the database table structure is intelligently inferred and created according to the data characteristics, and the system and the method are particularly suitable for enterprise data migration, data warehouse construction and data analysis preprocessing scenes.

The application discloses a data quality management method and device, and belongs to the technical field of data processing. The data quality management method comprises the following steps: receiving a first input of a user, wherein the first input is used for inputting a target configuration; in response to the first input, a data model is constructed based on the target configuration; a data source corresponding to a target layer in a data warehouse is associated to a corresponding position of the data model and field mapping is performed; at least part of data in the data source corresponding to the target layer is checked based on a target data checking rule, and a quality checking report is obtained. The data quality management method can unify the data quality standard, ensure the consistency of data processing in the subsequent application process, unify the data storage structure, directly associate the data when the data of multiple systems is called, and realize the data sharing between different systems.

This invention discloses a human resource intelligent planning method and system based on big data analysis, relating to the field of human resource management. It includes constructing a two-layer storage architecture of a human resource data lake and a data warehouse; extracting basic features from the data in the data warehouse using multi-dimensional features; selecting features strongly correlated with human resource needs from these basic features to construct a knowledge graph and then mining hidden correlation features; building a multi-model fusion prediction model to output prediction results; based on the prediction results, using an improved non-dominated sorting genetic algorithm to optimize human resource allocation and obtain a human resource allocation strategy pool; introducing digital twin technology to construct a digital twin model of enterprise human resources and business scenarios; simulating the implementation effect of the human resource allocation strategy pool for comprehensive evaluation; and determining the target human resource allocation strategy based on the comprehensive evaluation. This invention improves the accuracy of human resource demand prediction and the adaptability of planning by employing the above method, while reducing human resource costs and manual maintenance costs.

The invention discloses a data migration method, device, equipment and product, and relates to the technical field of storage, the method comprises the following steps: receiving a data migration request, and building a data migration environment according to the data migration request; and migrating to-be-migrated data in a to-be-migrated data bin to a target data bin based on the data migration environment to obtain a data migration result, and performing performance optimization on the target data bin based on the data migration result to obtain an optimization result. Therefore, the corresponding data migration environment is established according to the received data migration request, then the data in the to-be-migrated data bin is migrated to the target data bin based on the data migration environment, and the performance of the target data bin is optimized, so that the problem of low data migration efficiency caused by insufficient automation and low architecture adaptation during data bin migration is solved, and the data migration efficiency is improved. And the data migration efficiency is improved.

The invention relates to the technical field of batteries, and discloses a battery test data processing method and device, equipment, a medium and a program product. The method comprises the following steps: based on battery test data corresponding to a target test item, generating a plurality of data tables with an association relationship; based on each feature dimension in the battery test data, a data label corresponding to each feature dimension is determined, and the data labels are used for representing the continuity of the corresponding feature dimensions; establishing a data bin model corresponding to the target test item based on the plurality of data tables; and on the basis of each feature dimension and the data label corresponding to each feature dimension, scene analysis data corresponding to the target test item is generated, and the scene analysis data represents chart types correspondingly generated by different data analysis scenes. According to the method, redundant data can be reduced, an efficient and convenient data query service is provided, the battery test data analysis and chart generation efficiency is improved, the battery research and development cost is reduced, and the analysis difficulty of massive battery test data is reduced.

The application discloses an intelligent mine management and control platform and method based on a beneficiation whole-process optimization large model, relates to the technical field of industrial control, and comprises the following steps: constructing a unified data warehouse, collecting multi-source heterogeneous data, performing quality verification and space-time alignment, and generating beneficiation data; using the beneficiation data to train a twin prediction sub-model and an optimization decision sub-model respectively, forming a linkage model set; running the linkage model set in the intelligent mine management and control hub node to generate future state trends and candidate set values, and distributing the candidate set values to each business item; each business item forms an optimization instruction according to the candidate set value and a preset optimization target, and issues the optimization instruction to the bottom control layer for execution; and the new beneficiation data generated by the execution is returned to the unified data warehouse. The application constructs a callback closed loop in an application programming interface, prepositions the candidate set value as a twin prediction input, and returns the candidate set value in the same data closed loop, so that the cross-business item execution stability is improved.

The present application relates to the technical field of load identification, in particular to a non-intrusive load identification method based on high-frequency power data, which comprises the following steps: collecting high-frequency power signal data of user end voltage and current, and constructing a hierarchical data warehouse according to high-frequency power signal data processing requirements; performing signal analysis on the collected high-frequency power signal, extracting transient and steady-state characteristics, and constructing a multi-view feature matrix based on the transient and steady-state characteristics; using a multi-branch structured learning model based on the multi-view feature matrix to perform load prediction, at the same time, constructing a deep correlation analysis model to process the uncertainty problem in the high-frequency power signal data in the prediction process, and constructing a time domain correction function using the inherent periodic characteristics of the load. Through the fusion of high-frequency transient characteristics and time domain dynamics decoupling mechanism, the present application realizes high-precision and structured identification of complex composite loads and dynamic similar equipment.

A system and method for providing a unified, evidentiary artificial intelligence architecture integrates a local Large Language Model (LLM) executed within a hardware-secured enclave with a versioned data repository to perform synchronic, point-in-time correlation of data against historical operational rules. The LLM includes a novel Temporal Block Sparse Attention (TBSA) mechanism for computationally efficient analysis of long-context time-series data. The system captures a persistent, immutable ‘Evidentiary Analyze State’ using cryptographic hashing, creating a verifiable audit trail of the AI's reasoning process. A Retrieval-Augmented Generation (RAG) framework enables this correlation and drives a closed-loop proactive feedback mechanism, generating recommendations to update operational procedures based on real-time risk analysis. This unified architecture provides a specific technological improvement, yielding quantifiable gains in prediction accuracy and latency while ensuring privacy and auditability for high-stakes applications in domains such as finance, healthcare, and cybersecurity.

The application provides a reconciliation method for JSON data and data warehouse base table data, which comprises the following steps: receiving reconciliation scheduling information for controlling task execution, reconciliation mapping relationship for associating JSON data and data warehouse base table data, and alarm configuration; constructing reconciliation execution instructions for data comparison according to a preset template and the reconciliation mapping relationship; triggering distributed computing resources to load JSON data and data warehouse base table data and performing data comparison based on the reconciliation scheduling information and the reconciliation execution instructions, and returning reconciliation result data; and analyzing the reconciliation result data according to the alarm configuration to generate corresponding alarm results. Thus, the application realizes data consistency checking between heterogeneous data and fixed table data without relying on manual customization, eliminates the structural bottleneck formed by the fact that the existing method is difficult to efficiently and accurately complete comparison in a large-scale reconciliation scenario, and improves reconciliation efficiency and accuracy.

This application discloses a drilling risk early warning method, apparatus, equipment, and medium, relating to the field of shale gas drilling. The method includes: establishing a drilling parameter prediction model based on historical drilling parameters, and determining corresponding drilling risk impact parameters based on the drilling parameter prediction model; constructing a downhole risk identification model based on historical downhole risk data in a preset drilling risk data warehouse and the drilling risk impact parameters; acquiring current drilling parameters and current downhole risk parameters, and determining the current drilling risk based on the downhole risk identification model, the current drilling parameters, and the current downhole risk parameters, so as to provide early warning for the current drilling risk based on a preset drilling risk early warning method. In this application, the drilling parameter prediction model and the downhole risk identification model are constructed using machine learning and big data technologies, thereby achieving comprehensive early warning for the current drilling risk.