49 results about "Data mart" patented technology

The invention discloses an enterprise information visual management system and method based on data sharing, and belongs to the technical field of enterprise information management.The method comprises the steps that by connecting business systems in an enterprise, user behavior data and business data are collected and preprocessed, and an internal data set is constructed; acquiring data from an external data source, and constructing an external data set through semantic analysis, screening and classification; internal and external data are integrated, a dynamic cleaning rule is formulated, multi-source fusion is realized through data association, data are stored in a classified manner according to business requirements, and a data warehouse with a layered architecture is constructed; meanwhile, a personalized data mart is generated based on user roles and permissions, user demands are automatically predicted, and analysis indexes and dimensions are pre-generated; and finally, constructing a visual analysis platform, generating a preliminary recommendation scheme in combination with the pre-generated content and the user operation track, fusing user-defined setting, and outputting a final visual result in real time, thereby realizing efficient management and visual presentation of enterprise information.

Adaptable data warehousing computing systems and methods are disclosed for improved and flexible declarative control for programmatic modeling and database building, including in compute intense environments. A declarative scripting engine executes computing commands defined in a domain specific language (DSL), which is configured to programmatically configure a conceptual data model defining business object(s). A logical data model may be generated based on a declarative command defining the conceptual data model. A physical data model may be generated based on the logical data model, where the physical data model defines stage, core layer, and / or a data mart layers that may be deployed to a target database or data warehouse, and where the stage, core layer, and / or a data mart layers are the same or similar to those of the logical data model.

The invention discloses a power distribution network line loss intelligent sensing and tracing decision-making method and system, and belongs to the technical field of power distribution network operation and management.The power distribution network line loss intelligent sensing and tracing decision-making method comprises the steps that multi-source heterogeneous data are connected, timestamp interpolation is carried out, and data are obtained; associating the position and GIS coordinates of the electrical equipment in the electrical network through the unique identifier of the electrical equipment, and outputting a standardized data mart; a dynamic line loss separation engine calculates total loss according to the standardized data mart and a photovoltaic back-feeding compensation item, dynamically corrects technical line loss through a photovoltaic output coefficient, subdivides a line loss component by using a K-means algorithm, and outputs a line loss component matrix; and calculating a feature vector and inputting the feature vector into the traceability decision tree to trace the line loss abnormal root cause. Time sequence and space dislocation of multi-source heterogeneous data is eliminated through timestamp intelligent interpolation and equipment ID two-dimensional mapping fusion data, accurate decoupling of management line loss and technical line loss is achieved through a corrected dynamic line loss separation engine, and the misjudgment rate of a new energy scene is reduced through a traceability decision tree.

The invention discloses a fusion method and device based on multi-source water conservancy data, equipment and a medium, and relates to the technical field of water conservancy data processing, and the technical scheme is characterized in that the method comprises the steps: collecting water conservancy original data, carrying out the classified storage of the water conservancy original data according to a water conservancy data specification, and obtaining a water conservancy classification data set; preprocessing the water conservancy classification data set to obtain a preprocessed water conservancy classification data set, combining similar data of multiple sources, and carrying out missing compensation processing on the similar data to obtain a fused similar data set; and carrying out hierarchical modeling on the fused data sets of the same kind by utilizing distributed processing and a multi-bin hierarchical architecture design to obtain a standardized unified water conservancy data mart. The system has the advantages of high-efficiency water conservancy data acquisition, high-quality similar data fusion, high-standard heterogeneous data fusion, high-availability water conservancy data service and the like.

The invention provides a data processing method and system, and the method comprises the steps: obtaining multi-modal data according to business demand data for each business, converting the multi-modal data into structured data through a specific rule, solving a management problem of unstructured data, further constructing a label system comprising static and dynamic labels, and improving the management efficiency. Multi-dimensional high-precision and high-efficiency processing of data is realized by establishing a data mart and combining generative artificial intelligence with a data bin covering granularity, dimensionality, fact tables and themes, and finally, accurate user group customized recommendation is realized by adopting customized data recommendation rules for different target crowds by constructing the data mart and combining with the generative artificial intelligence.

The invention discloses a hydropower station operation data intelligent management system and method based on a data mart, and relates to the technical field of operation abnormity identification of a hydroelectric generating set, and the method comprises the steps: dividing a working condition interval of the hydroelectric generating set; obtaining a reference feature group of all the working condition intervals; actual electric power sampling is carried out on a certain working condition interval, a curve is fitted according to sampling data points, and a first fitted curve of the working condition interval is determined through a correlation calculation method; obtaining a first fitting curve of all the working condition intervals; storing the reference feature group and the first fitting curve into a data mart; carrying out real-time monitoring on a certain hydroelectric generating set, and carrying out sampling inspection to construct a first monitoring sequence; and based on the first monitoring sequence, retrieving the data mart, obtaining a reference feature group and a first fitting curve under the same working condition, and performing early warning according to different conditions. The method has good expandability and adaptability, and is suitable for intelligent monitoring and health diagnosis in a multi-unit cooperative operation scene.

The invention provides a bank customer portrait determination method and device, and the method comprises the steps: carrying out the processing of obtained bank internal system data and bank external system data, and generating a customer data mart; constructing a customer value index system according to the customer data marts; the customer value index system comprises customer multi-dimensional value index classes, each value index class comprises at least one value index, and the customer value index system further comprises a definition of each value index; carrying out weight analysis on the customer value index system by adopting an analytic hierarchy process, and constructing a customer evaluation model; based on customer data corresponding to a target customer group in the customer data mart and a customer evaluation model, generating a customer portrait of each target customer in the target customer group, and outputting the customer portrait in an interactive interface of the terminal equipment; the customer portrait at least indicates the index value and the overall grade of each target customer under each value index class. Therefore, a more comprehensive and accurate customer portrait of the bank customer can be generated.

The present invention discloses a method for storing and detecting multi-version metadata of a power grid data warehouse, which includes the following steps: constructing a power grid Data Vault (DV) data warehouse; constructing a metamodel for a power grid DV data warehouse environment, including a DV mode and a multidimensional (MD) mode part; performing metadata consistency detection of the DV mode; performing metadata consistency detection of the MD mode; performing consistency detection of the integrity constraints of the attribute table Attributes; performing metadata consistency detection of the DV mode and the MD mode; the present invention utilizes the metamodel of the power grid metadata repository to respectively check the multi-version metadata consistency relationship between the data warehouse area, the data mart area, and the data warehouse and the data mart. The present invention can automatically discover metadata missing, duplication and conflict at the power grid metadata level, thereby achieving the purpose of improving the quality of power grid data.

The invention relates to a financial data index testing method and device, electronic equipment and a storage medium. The financial data index test method comprises the steps of generating a first data index set according to a bank supervision meeting off-site supervision 1104 report, data indexes and a mapping relation; compiling a cross-system index cross verification scheme; performing data mart table and field correspondence, establishing an index cross-check mapping relation, and developing an index cross-check SQL script set; performing blood relationship analysis to obtain a front dependency table based on a data warehouse, and desensitizing batch data of the front dependency table from production recovery to a test environment; running a data mart script scheduling job, and processing and loading the data of the data warehouse into a data mart target table; and carrying out index cross-check rule entry, generating an index cross-check rule, and generating a data index cross-check test case. According to the financial data index test method, the supervision submitted data index can be subjected to cross-system automatic verification, and the test efficiency is improved.

This invention discloses an intelligent management system and method for hydropower station operation data based on a data mart, relating to the field of hydropower unit operation anomaly identification technology. The method includes: dividing the hydropower unit into operating condition intervals; obtaining a baseline feature set for all operating condition intervals; sampling actual power for a certain operating condition interval, fitting a curve based on the sampled data points, and determining a first fitted curve for that operating condition interval using a correlation calculation method; obtaining the first fitted curves for all operating condition intervals; storing the baseline feature set and the first fitted curve in a data mart; performing real-time monitoring of a certain hydropower unit and constructing a first monitoring sequence through random sampling; and based on the first monitoring sequence, retrieving the data mart to obtain the baseline feature set and the first fitted curve under the same operating condition, and issuing early warnings according to different situations. This invention has good scalability and adaptability, and is suitable for intelligent monitoring and health diagnosis in multi-unit collaborative operation scenarios.

The invention discloses an architecture design method of an operation management system, and relates to the technical field of enterprise operation management, and the method comprises the following specific steps: firstly, through multi-source data integration and cross-system verification, data integrity and accuracy are ensured; then ETL processing and layered modeling are carried out, DIM and DW layers are constructed, and asset information is reversely written; secondly, performing secondary data processing on a Data Mart layer, screening high-credibility data, and configuring a security and scaling mechanism; and finally, realizing visual display and linkage management, presenting data in multiple dimensions, early warning abnormity, and ensuring close linkage of an operation management platform, a data model and a mart. Comprehensive and accurate data are ensured through multi-source data integration and intelligent verification, ETL intelligent processing, bidirectional blood relationship tracing and fine-grained safety control are adopted, the intuition, accuracy and safety of operation management are improved, and efficient application of data and tight linkage of a platform are achieved.

Adaptable data warehousing computing systems and methods are disclosed for improved and flexible declarative control for programmatic modeling and database building, including in compute intense environments. A declarative scripting engine executes computing commands defined in a domain specific language (DSL), which is configured to programmatically configure a conceptual data model defining business object(s). A logical data model may be generated based on a declarative command defining the conceptual data model. A physical data model may be generated based on the logical data model, where the physical data model defines stage, core layer, and / or a data mart layers that may be deployed to a target database or data warehouse, and where the stage, core layer, and / or a data mart layers are the same or similar to those of the logical data model.

The invention discloses an industrial data exception attribution method, device and equipment, a storage medium and a product, is applied to a smart power plant data platform, and relates to the technical field of exception detection. The method comprises the following steps: acquiring multi-source industrial data in real time; performing standardization processing on the multi-source industrial data to generate a standardized data stream in a unified format; performing anomaly detection on the standardized data through a pre-trained integrated learning model, and identifying an abnormal mode in the data; and in response to the detected abnormal mode, performing multi-dimensional attribution analysis on the abnormal mode based on an association rule constructed by a data topic and a data mart, and determining a root cause of the abnormality. Through progressive processing of data integration, intelligent detection and business attribution, a complete closed loop from data perception to intelligent decision is realized, and mass data is converted into high-value decision information for directly driving business optimization.

In some implementations, an analytics platform may process a data lineage configuration to identify one or more data sources storing one or more data sets associated with a data analytics use case and identify one or more data relations associated with the one or more data sets. The analytics platform may parse a source code template defining one or more functions to build a data mart and a data pipeline that enables the data analytics use case. The source code template may include one or more tokens to specify elements that define the one or more data sources, data sets, and data relations identified in the data lineage configuration. The analytics platform may automatically generate source code that is executable to build a data mart and a data pipeline to enable the data analytics use case based on the data lineage configuration and the source code template.

The invention relates to the technical field of financial systems, and particularly discloses a centralized processing method for a certificate inquiry letter, which comprises the following steps of: acquiring a paper certificate inquiry letter, and scanning the paper certificate inquiry letter into an electronic image; extracting certification inquiry home page information and certification inquiry seal page information in the electronic image; checking the formats of the information of the certifying home page and the information of the certifying seal page, the information accuracy of the information of the certifying home page and the seal integrity of the certifying seal page; printing verification and charging are carried out based on the information of the certification inquiry home page; inputting the information of the certification inquiry home page and the information of the certification inquiry seal page into a preset certification inquiry letter template to generate an electronic certification inquiry letter; the electronic confirmation request is stored in a data mart, the data mart comprises a query interface, and the query interface is used for checking the information consistency of the electronic confirmation request and the corresponding paper confirmation request; and returning the qualified electronic confirmation request to the user equipment. According to the method, the processing efficiency of the paper inquiry certificate is improved, centralized processing of the inquiry certificate is realized, and the labor cost is reduced.

This application discloses an intelligent generation method and apparatus for decision analysis reports in the insurance field. The method includes: constructing a thematic data mart containing insurance business data, and vectorizing and semantically slicing insurance field rules and indicator texts to form a knowledge unit index; responding to a report generation request, constructing a computational agent based on a matched report template; using this agent, generating analytical text by using retrieved data and knowledge units as a context-driven generative model, and synthesizing a decision analysis report; collecting interactive operation data related to the report, and adjusting the configuration parameters of the report template accordingly. This application achieves full-process automation from multi-source data integration and professional knowledge structuring to intelligent analysis, report generation, and continuous optimization, solving the problems of low efficiency, poor accuracy, and difficulty in adapting to large-scale data and complex business logic in traditional manual analysis in the insurance industry, significantly improving decision support efficiency and intelligence.

This application discloses a data mart management method, apparatus, electronic device, and storage medium. The method includes: receiving data to be processed from a source system; storing the data to be processed in a predetermined table structure to obtain a standard-format data table; processing the data to be processed sequentially at multiple levels according to a hierarchical order based on the standard-format data table to obtain a processing result table of the data to be processed; and sending the processing result table of the data to be processed to a downstream system. The method of this application can standardize the creation and storage format of data tables, directly process the standardized data, reduce the application and subsequent maintenance difficulty of data tables in data marts, and improve the management efficiency of data marts.

The invention discloses a data warehouse layering and domain dividing design method, and belongs to the technical field of data warehouses, and the method comprises the steps: constructing a data warehouse, and carrying out the layering of the data warehouse; extracting traffic flow related data from a source end service system in a data operation layer, and establishing a data model in a zoning manner; in the data detail layer, standardizing and cleaning the data of the data operation layer, and establishing a data model based on the processed data; preprocessing the data of the detail layer in the data intermediate layer, establishing a fact table and performing light summarization; exchanging the data to a Dores database for processing; in the data summarization layer, summarization processing is provided for the theme library based on the data of the middle layer, and business indexes are developed according to different business dimensions; in the data mart layer, the fact table and the index data are subjected to domain processing according to business requirements and are docked to an application thematic library. According to the method, the problems of high data processing delay, long period and analysis singleness are effectively solved, and data redundancy and delay are remarkably reduced.

A data management system includes a computing device. The computing device includes one or more processors to execute instructions stored in memory of the computing device. The instructions, when executed by the computer device, cause the computing device to receive an indication that multiple data elements are ready for evaluation, as well as determine whether metadata registration for the multiple data elements is valid. The instructions also cause the computing device to determine a data quality of the multiple data elements by evaluating multiple data elements based on multiple data quality dimensions including at least two of: validity, completeness, uniqueness, accuracy, consistency, and timeliness. The instructions also cause the computing device to provide an indication of the data quality of the multiple data elements via a graphical user interface (GUI) generated via a centralized data mart.

The invention provides an event prompting method and device, and the method is applied to a customer management platform, and the method comprises the steps: building a customer data mart based on the obtained bank internal system data and bank external system data; configuring a business event rule set for the customer data mart from at least one business participation object dimension; monitoring the customer data mart by using the business event rule set corresponding to each business participation object dimension, and identifying a target business event under each business participation object dimension; in response to the generation of the target business event, determining event prompt information, an information push mode and target business personnel for the target business event according to the target business event rule; and automatically distributing the event prompt information to the target business personnel through an information push channel included in the information push mode. In this way, timely and unified monitoring and targeted personalized information pushing of the business events can be achieved.

A device and method are described for a universal analytical data mart and data structure for same. The analytical data mart (ADM) associated data structure is designed to allow data from disparate sources to be integrated, enabling streamlined business intelligence, reporting and ad hoc analysis. Conceptually, the ADM enables analytics and business intelligence from multiple frames of reference including people, such as parties and actors including individuals and organizations, places, such as addresses with geographic information at various levels of view, objects, such as insured properties, automobiles and machinery, and events, milestones which happen at points in time and provide analytical / business value.

The application discloses a business index acquisition method and device based on distributed processing and a readable medium. The method comprises the following steps: collecting background transaction data from a source database to a distributed processing system as source layer data; partitioning and storing the source layer data to obtain partition storage data corresponding to different business requirements and / or functions as model layer data; processing the partition storage data corresponding to different business requirements and / or functions in the model layer data based on a distributed processing mode to obtain corresponding marketing indexes as data mart layer data; and finally outputting the data mart layer data to a downstream business system. The application can make more accurate and customer demand-conforming marketing recommendations by introducing background transaction data into a marketing scenario. By introducing a distributed processing system and designing a hierarchical structure of a database table for data storage and processing, the application solves the problems of slow calculation, poor accuracy and stability of indexes and the like in the traditional mode for large data volume.

In some implementations, an analytics platform may process a data lineage configuration to identify one or more data sources storing one or more data sets associated with a data analytics use case and identify one or more data relations associated with the one or more data sets. The analytics platform may parse a source code template defining one or more functions to build a data mart and a data pipeline that enables the data analytics use case. The source code template may include one or more tokens to specify elements that define the one or more data sources, data sets, and data relations identified in the data lineage configuration. The analytics platform may automatically generate source code that is executable to build a data mart and a data pipeline to enable the data analytics use case based on the data lineage configuration and the source code template.

The present invention discloses a scheduling task configuration method and apparatus, relating to the field of computer technology. A specific implementation manner of the method includes collecting relevant information of a task from a data mart historical execution log according to the id of a spark scheduling task, inputting same into the Actor network of a DDPG model to obtain an action space value; inputting the action space value and the information into the Critic network in the DDPG model to train and obtain a reward value, calculating the network loss function to obtain a DDPG model corresponding to the maximum reward value as a configuration model; obtaining the target information of the current spark scheduling task, obtaining spark configuration parameters through the configuration model, and executing the current spark scheduling task. Thereby, the problem that the processing performance of the scheduling task is affected due to the low precision and poor efficiency of setting Spark configuration parameters in the prior art is solved.

The invention discloses a financial risk rule automatic test method and device, equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: responding to a rule adjustment instruction, and determining a to-be-tested rule according to the rule adjustment instruction; determining external connection decision data required by testing from a data market according to the to-be-tested rule, wherein financial data of a user is stored in the data market; and testing the to-be-tested rule based on the external connection decision data to obtain a rule test result. According to the method, the external connection decision data required by the test is determined from the data mart according to the to-be-tested rule, and the to-be-tested rule is tested based on the external connection decision data, so that compared with the existing method that demand review and decision data generation need to be performed every time the rule is changed, the method can improve the rule test efficiency.

The present application provides a data processing method, comprising: constructing an operator set for each data layer in a data warehouse to obtain an operator set corresponding to each data layer; configuring an operator link based on the operator set corresponding to each data layer to obtain a data processing sub-link corresponding to each data layer; constructing a data processing link corresponding to the data warehouse based on the data processing sub-link corresponding to each data layer using a particle swarm algorithm; and processing source data based on the data processing link to obtain a data mart for the data warehouse. This method shortens data processing time, improves data processing efficiency, and thereby enhances the overall performance of the data processing link.

The invention relates to the technical field of machine learning, and particularly provides a method and a system for identifying abnormal mobile retailers of real cigarettes. Firstly, data resources in all aspects are managed in a centralized manner and stored in a unified manner, a data mart is established, and the abnormal mobile retailers of the real cigarettes are identified according to the characteristics of outflow of the tobacco cigarettes; quantifiable outflow features are combed in the whole process of cigarette orders, distribution, sales, inventory, historical outflow and cigarette-related case terminal sales, and data support is provided for illegal circulation early warning of real cigarettes. Compared with the prior art, the method has the advantages that the real smoke outflow prediction decision tree model can be established, and the accuracy of the prediction model is improved through training of machine learning and continuous optimization and improvement of the decision tree. Technical support is provided for business personnel, and illegal circulation of real cigarettes is effectively prevented.

The invention relates to a data processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps that a target product data set is acquired from a data mart, multiple pieces of product data sent by a data management system are stored in the data mart, and the target product data set comprises the multiple pieces of product data; obtaining the data volume of the target product data set, and determining whether an alarm condition is satisfied based on the data volume and the historical data volume of the historical product data set obtained in the historical time period; if the alarm condition is met, executing an alarm operation; and if the alarm condition is not met, performing product production based on the target product data set. By adopting the method, the accuracy of the product data can be guaranteed, so that the safety problem caused by poor accuracy of the product data during product production based on the product data is avoided.

A system is provided.SOLUTION: A system comprising: means for analyzing input requirements using a production AI; means for selecting appropriate data marts based on the analyzed requirements; means for automatically generating SQL queries based on the selected data marts; means for presenting the generated SQL queries to a user and accepting user confirmations and modifications; and means for executing the confirmed or modified SQL queries and presenting the results to the user.SELECTED DRAWING: Figure 1

The invention discloses a BI enterprise-level report analysis system construction method, and belongs to the technical field of business intelligence and data analysis. According to the method, a five-layer architecture system is constructed and comprises a data source layer, a data processing layer, a data storage layer, an analysis layer and a display layer, so that comprehensive data management of bank retail business and public business is realized. According to the system, data cleaning, conversion and loading are carried out on a data processing layer by adopting an ETL / PLUS technology, and the data accuracy is ensured through metadata management and data quality management; a data warehouse and a data mart are established in the data storage layer, and large-scale data storage and rapid query are supported; and functions of multi-dimensional analysis, decision support, flexible query and the like are provided in the analysis layer. According to the method, the problems of data islands, low processing efficiency, insufficient safety and the like of a traditional report system are solved, centralized management, automatic processing and real-time analysis of the data are realized, and the data processing efficiency and the decision support capability are remarkably improved.