38 results about "Data mart" patented technology

Adaptable data warehousing computing systems and methods are disclosed for improved and flexible declarative control for programmatic modeling and database building, including in compute intense environments. A declarative scripting engine executes computing commands defined in a domain specific language (DSL), which is configured to programmatically configure a conceptual data model defining business object(s). A logical data model may be generated based on a declarative command defining the conceptual data model. A physical data model may be generated based on the logical data model, where the physical data model defines stage, core layer, and / or a data mart layers that may be deployed to a target database or data warehouse, and where the stage, core layer, and / or a data mart layers are the same or similar to those of the logical data model.

The invention discloses a power distribution network line loss intelligent sensing and tracing decision-making method and system, and belongs to the technical field of power distribution network operation and management.The power distribution network line loss intelligent sensing and tracing decision-making method comprises the steps that multi-source heterogeneous data are connected, timestamp interpolation is carried out, and data are obtained; associating the position and GIS coordinates of the electrical equipment in the electrical network through the unique identifier of the electrical equipment, and outputting a standardized data mart; a dynamic line loss separation engine calculates total loss according to the standardized data mart and a photovoltaic back-feeding compensation item, dynamically corrects technical line loss through a photovoltaic output coefficient, subdivides a line loss component by using a K-means algorithm, and outputs a line loss component matrix; and calculating a feature vector and inputting the feature vector into the traceability decision tree to trace the line loss abnormal root cause. Time sequence and space dislocation of multi-source heterogeneous data is eliminated through timestamp intelligent interpolation and equipment ID two-dimensional mapping fusion data, accurate decoupling of management line loss and technical line loss is achieved through a corrected dynamic line loss separation engine, and the misjudgment rate of a new energy scene is reduced through a traceability decision tree.

The invention discloses a fusion method and device based on multi-source water conservancy data, equipment and a medium, and relates to the technical field of water conservancy data processing, and the technical scheme is characterized in that the method comprises the steps: collecting water conservancy original data, carrying out the classified storage of the water conservancy original data according to a water conservancy data specification, and obtaining a water conservancy classification data set; preprocessing the water conservancy classification data set to obtain a preprocessed water conservancy classification data set, combining similar data of multiple sources, and carrying out missing compensation processing on the similar data to obtain a fused similar data set; and carrying out hierarchical modeling on the fused data sets of the same kind by utilizing distributed processing and a multi-bin hierarchical architecture design to obtain a standardized unified water conservancy data mart. The system has the advantages of high-efficiency water conservancy data acquisition, high-quality similar data fusion, high-standard heterogeneous data fusion, high-availability water conservancy data service and the like.

The invention discloses a hydropower station operation data intelligent management system and method based on a data mart, and relates to the technical field of operation abnormity identification of a hydroelectric generating set, and the method comprises the steps: dividing a working condition interval of the hydroelectric generating set; obtaining a reference feature group of all the working condition intervals; actual electric power sampling is carried out on a certain working condition interval, a curve is fitted according to sampling data points, and a first fitted curve of the working condition interval is determined through a correlation calculation method; obtaining a first fitting curve of all the working condition intervals; storing the reference feature group and the first fitting curve into a data mart; carrying out real-time monitoring on a certain hydroelectric generating set, and carrying out sampling inspection to construct a first monitoring sequence; and based on the first monitoring sequence, retrieving the data mart, obtaining a reference feature group and a first fitting curve under the same working condition, and performing early warning according to different conditions. The method has good expandability and adaptability, and is suitable for intelligent monitoring and health diagnosis in a multi-unit cooperative operation scene.

The invention provides a bank customer portrait determination method and device, and the method comprises the steps: carrying out the processing of obtained bank internal system data and bank external system data, and generating a customer data mart; constructing a customer value index system according to the customer data marts; the customer value index system comprises customer multi-dimensional value index classes, each value index class comprises at least one value index, and the customer value index system further comprises a definition of each value index; carrying out weight analysis on the customer value index system by adopting an analytic hierarchy process, and constructing a customer evaluation model; based on customer data corresponding to a target customer group in the customer data mart and a customer evaluation model, generating a customer portrait of each target customer in the target customer group, and outputting the customer portrait in an interactive interface of the terminal equipment; the customer portrait at least indicates the index value and the overall grade of each target customer under each value index class. Therefore, a more comprehensive and accurate customer portrait of the bank customer can be generated.

This invention discloses an intelligent management system and method for hydropower station operation data based on a data mart, relating to the field of hydropower unit operation anomaly identification technology. The method includes: dividing the hydropower unit into operating condition intervals; obtaining a baseline feature set for all operating condition intervals; sampling actual power for a certain operating condition interval, fitting a curve based on the sampled data points, and determining a first fitted curve for that operating condition interval using a correlation calculation method; obtaining the first fitted curves for all operating condition intervals; storing the baseline feature set and the first fitted curve in a data mart; performing real-time monitoring of a certain hydropower unit and constructing a first monitoring sequence through random sampling; and based on the first monitoring sequence, retrieving the data mart to obtain the baseline feature set and the first fitted curve under the same operating condition, and issuing early warnings according to different situations. This invention has good scalability and adaptability, and is suitable for intelligent monitoring and health diagnosis in multi-unit collaborative operation scenarios.

The invention discloses an architecture design method of an operation management system, and relates to the technical field of enterprise operation management, and the method comprises the following specific steps: firstly, through multi-source data integration and cross-system verification, data integrity and accuracy are ensured; then ETL processing and layered modeling are carried out, DIM and DW layers are constructed, and asset information is reversely written; secondly, performing secondary data processing on a Data Mart layer, screening high-credibility data, and configuring a security and scaling mechanism; and finally, realizing visual display and linkage management, presenting data in multiple dimensions, early warning abnormity, and ensuring close linkage of an operation management platform, a data model and a mart. Comprehensive and accurate data are ensured through multi-source data integration and intelligent verification, ETL intelligent processing, bidirectional blood relationship tracing and fine-grained safety control are adopted, the intuition, accuracy and safety of operation management are improved, and efficient application of data and tight linkage of a platform are achieved.

Adaptable data warehousing computing systems and methods are disclosed for improved and flexible declarative control for programmatic modeling and database building, including in compute intense environments. A declarative scripting engine executes computing commands defined in a domain specific language (DSL), which is configured to programmatically configure a conceptual data model defining business object(s). A logical data model may be generated based on a declarative command defining the conceptual data model. A physical data model may be generated based on the logical data model, where the physical data model defines stage, core layer, and / or a data mart layers that may be deployed to a target database or data warehouse, and where the stage, core layer, and / or a data mart layers are the same or similar to those of the logical data model.

The invention discloses an industrial data exception attribution method, device and equipment, a storage medium and a product, is applied to a smart power plant data platform, and relates to the technical field of exception detection. The method comprises the following steps: acquiring multi-source industrial data in real time; performing standardization processing on the multi-source industrial data to generate a standardized data stream in a unified format; performing anomaly detection on the standardized data through a pre-trained integrated learning model, and identifying an abnormal mode in the data; and in response to the detected abnormal mode, performing multi-dimensional attribution analysis on the abnormal mode based on an association rule constructed by a data topic and a data mart, and determining a root cause of the abnormality. Through progressive processing of data integration, intelligent detection and business attribution, a complete closed loop from data perception to intelligent decision is realized, and mass data is converted into high-value decision information for directly driving business optimization.

The invention relates to the technical field of financial systems, and particularly discloses a centralized processing method for a certificate inquiry letter, which comprises the following steps of: acquiring a paper certificate inquiry letter, and scanning the paper certificate inquiry letter into an electronic image; extracting certification inquiry home page information and certification inquiry seal page information in the electronic image; checking the formats of the information of the certifying home page and the information of the certifying seal page, the information accuracy of the information of the certifying home page and the seal integrity of the certifying seal page; printing verification and charging are carried out based on the information of the certification inquiry home page; inputting the information of the certification inquiry home page and the information of the certification inquiry seal page into a preset certification inquiry letter template to generate an electronic certification inquiry letter; the electronic confirmation request is stored in a data mart, the data mart comprises a query interface, and the query interface is used for checking the information consistency of the electronic confirmation request and the corresponding paper confirmation request; and returning the qualified electronic confirmation request to the user equipment. According to the method, the processing efficiency of the paper inquiry certificate is improved, centralized processing of the inquiry certificate is realized, and the labor cost is reduced.

This application discloses an intelligent generation method and apparatus for decision analysis reports in the insurance field. The method includes: constructing a thematic data mart containing insurance business data, and vectorizing and semantically slicing insurance field rules and indicator texts to form a knowledge unit index; responding to a report generation request, constructing a computational agent based on a matched report template; using this agent, generating analytical text by using retrieved data and knowledge units as a context-driven generative model, and synthesizing a decision analysis report; collecting interactive operation data related to the report, and adjusting the configuration parameters of the report template accordingly. This application achieves full-process automation from multi-source data integration and professional knowledge structuring to intelligent analysis, report generation, and continuous optimization, solving the problems of low efficiency, poor accuracy, and difficulty in adapting to large-scale data and complex business logic in traditional manual analysis in the insurance industry, significantly improving decision support efficiency and intelligence.

This application discloses a data mart management method, apparatus, electronic device, and storage medium. The method includes: receiving data to be processed from a source system; storing the data to be processed in a predetermined table structure to obtain a standard-format data table; processing the data to be processed sequentially at multiple levels according to a hierarchical order based on the standard-format data table to obtain a processing result table of the data to be processed; and sending the processing result table of the data to be processed to a downstream system. The method of this application can standardize the creation and storage format of data tables, directly process the standardized data, reduce the application and subsequent maintenance difficulty of data tables in data marts, and improve the management efficiency of data marts.

The invention discloses a data warehouse layering and domain dividing design method, and belongs to the technical field of data warehouses, and the method comprises the steps: constructing a data warehouse, and carrying out the layering of the data warehouse; extracting traffic flow related data from a source end service system in a data operation layer, and establishing a data model in a zoning manner; in the data detail layer, standardizing and cleaning the data of the data operation layer, and establishing a data model based on the processed data; preprocessing the data of the detail layer in the data intermediate layer, establishing a fact table and performing light summarization; exchanging the data to a Dores database for processing; in the data summarization layer, summarization processing is provided for the theme library based on the data of the middle layer, and business indexes are developed according to different business dimensions; in the data mart layer, the fact table and the index data are subjected to domain processing according to business requirements and are docked to an application thematic library. According to the method, the problems of high data processing delay, long period and analysis singleness are effectively solved, and data redundancy and delay are remarkably reduced.

A data management system includes a computing device. The computing device includes one or more processors to execute instructions stored in memory of the computing device. The instructions, when executed by the computer device, cause the computing device to receive an indication that multiple data elements are ready for evaluation, as well as determine whether metadata registration for the multiple data elements is valid. The instructions also cause the computing device to determine a data quality of the multiple data elements by evaluating multiple data elements based on multiple data quality dimensions including at least two of: validity, completeness, uniqueness, accuracy, consistency, and timeliness. The instructions also cause the computing device to provide an indication of the data quality of the multiple data elements via a graphical user interface (GUI) generated via a centralized data mart.

The invention provides an event prompting method and device, and the method is applied to a customer management platform, and the method comprises the steps: building a customer data mart based on the obtained bank internal system data and bank external system data; configuring a business event rule set for the customer data mart from at least one business participation object dimension; monitoring the customer data mart by using the business event rule set corresponding to each business participation object dimension, and identifying a target business event under each business participation object dimension; in response to the generation of the target business event, determining event prompt information, an information push mode and target business personnel for the target business event according to the target business event rule; and automatically distributing the event prompt information to the target business personnel through an information push channel included in the information push mode. In this way, timely and unified monitoring and targeted personalized information pushing of the business events can be achieved.

The application discloses a business index acquisition method and device based on distributed processing and a readable medium. The method comprises the following steps: collecting background transaction data from a source database to a distributed processing system as source layer data; partitioning and storing the source layer data to obtain partition storage data corresponding to different business requirements and / or functions as model layer data; processing the partition storage data corresponding to different business requirements and / or functions in the model layer data based on a distributed processing mode to obtain corresponding marketing indexes as data mart layer data; and finally outputting the data mart layer data to a downstream business system. The application can make more accurate and customer demand-conforming marketing recommendations by introducing background transaction data into a marketing scenario. By introducing a distributed processing system and designing a hierarchical structure of a database table for data storage and processing, the application solves the problems of slow calculation, poor accuracy and stability of indexes and the like in the traditional mode for large data volume.

In some implementations, an analytics platform may process a data lineage configuration to identify one or more data sources storing one or more data sets associated with a data analytics use case and identify one or more data relations associated with the one or more data sets. The analytics platform may parse a source code template defining one or more functions to build a data mart and a data pipeline that enables the data analytics use case. The source code template may include one or more tokens to specify elements that define the one or more data sources, data sets, and data relations identified in the data lineage configuration. The analytics platform may automatically generate source code that is executable to build a data mart and a data pipeline to enable the data analytics use case based on the data lineage configuration and the source code template.

The present invention discloses a scheduling task configuration method and apparatus, relating to the field of computer technology. A specific implementation manner of the method includes collecting relevant information of a task from a data mart historical execution log according to the id of a spark scheduling task, inputting same into the Actor network of a DDPG model to obtain an action space value; inputting the action space value and the information into the Critic network in the DDPG model to train and obtain a reward value, calculating the network loss function to obtain a DDPG model corresponding to the maximum reward value as a configuration model; obtaining the target information of the current spark scheduling task, obtaining spark configuration parameters through the configuration model, and executing the current spark scheduling task. Thereby, the problem that the processing performance of the scheduling task is affected due to the low precision and poor efficiency of setting Spark configuration parameters in the prior art is solved.

The invention discloses a financial risk rule automatic test method and device, equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: responding to a rule adjustment instruction, and determining a to-be-tested rule according to the rule adjustment instruction; determining external connection decision data required by testing from a data market according to the to-be-tested rule, wherein financial data of a user is stored in the data market; and testing the to-be-tested rule based on the external connection decision data to obtain a rule test result. According to the method, the external connection decision data required by the test is determined from the data mart according to the to-be-tested rule, and the to-be-tested rule is tested based on the external connection decision data, so that compared with the existing method that demand review and decision data generation need to be performed every time the rule is changed, the method can improve the rule test efficiency.

The present application provides a data processing method, comprising: constructing an operator set for each data layer in a data warehouse to obtain an operator set corresponding to each data layer; configuring an operator link based on the operator set corresponding to each data layer to obtain a data processing sub-link corresponding to each data layer; constructing a data processing link corresponding to the data warehouse based on the data processing sub-link corresponding to each data layer using a particle swarm algorithm; and processing source data based on the data processing link to obtain a data mart for the data warehouse. This method shortens data processing time, improves data processing efficiency, and thereby enhances the overall performance of the data processing link.

A system is provided.SOLUTION: A system comprising: means for analyzing input requirements using a production AI; means for selecting appropriate data marts based on the analyzed requirements; means for automatically generating SQL queries based on the selected data marts; means for presenting the generated SQL queries to a user and accepting user confirmations and modifications; and means for executing the confirmed or modified SQL queries and presenting the results to the user.SELECTED DRAWING: Figure 1

The invention discloses a BI enterprise-level report analysis system construction method, and belongs to the technical field of business intelligence and data analysis. According to the method, a five-layer architecture system is constructed and comprises a data source layer, a data processing layer, a data storage layer, an analysis layer and a display layer, so that comprehensive data management of bank retail business and public business is realized. According to the system, data cleaning, conversion and loading are carried out on a data processing layer by adopting an ETL / PLUS technology, and the data accuracy is ensured through metadata management and data quality management; a data warehouse and a data mart are established in the data storage layer, and large-scale data storage and rapid query are supported; and functions of multi-dimensional analysis, decision support, flexible query and the like are provided in the analysis layer. According to the method, the problems of data islands, low processing efficiency, insufficient safety and the like of a traditional report system are solved, centralized management, automatic processing and real-time analysis of the data are realized, and the data processing efficiency and the decision support capability are remarkably improved.

The invention discloses a semiconductor production data management method and system and a readable storage medium, and the method comprises the following steps: collecting target data of all machines on a production line according to a target, the target data comprising real-time operation parameters and detection data of the machines; processing the target data according to a preset data standard specification to obtain standardized data; storing the standardized data in a classified manner, and creating a data mart and a data resource directory according to requirements of a business department; and searching and outputting data required by a business department through the data mart and the data resource catalog, and generating a data analysis report. According to the invention, scattered data of semiconductor production data are managed in a centralized and unified manner, so that data quality is guaranteed, data sharing and collaboration are improved, data reutilization is realized, cost is reduced, and operation management is facilitated.

The embodiment of the application belongs to the field of base frame operation and maintenance, and relates to a data mart abnormality detection method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining task blood relationship information of a data mart; for each application in the data mart, determining a fluctuation detection link of the application according to the task blood relationship information; determining the task weight and the repetition degree of each task in the link according to the task blood relationship information; obtaining the task log of the task to extract the running index of the task; calculating the task fluctuation value of the task according to the task weight, the repetition degree and the running index; calculating the application fluctuation value of the application based on the task fluctuation value, and calculating the mart fluctuation value of the data mart according to the application fluctuation value; comparing the mart fluctuation value with a standard mart fluctuation value to obtain a data mart abnormality detection result. In addition, the application also relates to blockchain technology, and the task blood relationship information can be stored in the blockchain. The application can discover abnormalities in the data mart in advance and give a warning.

The invention discloses a data query request processing method and device, a storage medium and an electronic device, and relates to the technical field of data storage, and the data query request processing method comprises the following steps: obtaining marked data from a business source system, and storing the marked data to a big data platform data warehouse, the time label of the marked data represents a timestamp of newly added uploading time for the uploaded business data by the business source system, and the uploaded business data at least comprises real-time business data and historical business data; analyzing the data query request from the target object to obtain a time query condition corresponding to the data query request, the time query condition at least comprising data uploading time; and obtaining the target data conforming to the data uploading time from the big data platform data mart, and sending the target data to the target object, and by adopting the technical scheme, the technical problem of low data query accuracy caused by low historical data supplementary transmission efficiency is solved.

The embodiment of the present invention discloses a business data processing system, method, server and storage medium, which includes: a data warehouse mart data layer, which is used to synchronize the full data of each business data source to the data warehouse data mart according to a preset frequency, and process the full data to obtain the full data of the business wide table; an incremental data layer, which is used to connect with each business data source, obtain the real-time incremental data of each business data source, and obtain the incremental data of the business wide table; a business processing layer, which is used to verify the full data of the business wide table, and fuse the verified full data of the business wide table with the incremental data of the business wide table to obtain the target business wide table data. Through the technical solution of the embodiment of the present invention, the decoupling of the cumbersome docking process between multiple business departments can be achieved, and the consistency of data when docking multiple data sources is guaranteed.

The invention discloses an index batch derivation method and device based on a Cartesian product and computer equipment. The method comprises the steps of obtaining an existing product and a corresponding data table, and comprehensively analyzing the existing product and the corresponding data table to obtain an analysis result; designing a unified data mart based on the analysis result; splitting specific elements of the data mart; creating an index template according to the elements; generating Chinese and English names and SQL statements of the indexes by applying Cartesian product operation in combination with the elements and the index template so as to obtain a generation result; and outputting the generated result as an index list, and applying the index list to a big data platform for actual calculation. By implementing the method provided by the invention, batch derivation of indexes can be realized, the manual operation complexity is reduced, the efficiency is improved, the accuracy and consistency of index design are ensured, and the maintenance cost is reduced.

The invention provides a risk control system and method for a self-operated passenger car, and the system comprises a basic data collection module which is used for collecting risk control basic data of financial business of the self-operated passenger car; the risk data mart module is used for acquiring a characteristic factor of the transaction according to the risk control basic data; the risk modeling module is used for constructing a risk control model based on the characteristic factors of the transaction; the risk scoring module is used for scoring the transaction data in real time based on a risk control model and outputting a scoring result of the transaction; the real-time risk control module is used for judging the transaction risk according to the scoring result of the transaction and outputting an interception result; and the result feedback module is used for performing accuracy judgment on the interception result and feeding back the interception result to the real-time risk control module and the risk data mart module respectively. According to the technical scheme, the risk assessment accuracy, the model stability and the interpretability of the self-operation passenger car financial business are improved.

The present application relates to the technical field of environmental management big data, and discloses a multi-source heterogeneous environmental data asset construction method based on a computing power network, comprising the following steps: S1, multi-source heterogeneous environmental data processing; S2, data warehouse classification construction; S3, theme data mart construction; S4, knowledge query graph construction; and S5, cross-media retrieval application.The present application solves the problems of poor general standardization of environmental management data, lack of data correlation and integration between different business departments, and low application degree in actual business applications by means of multi-source heterogeneous environmental data processing, data warehouse classification construction, theme data mart construction, knowledge query graph construction, and cross-media retrieval application, efficiently completes the preprocessing and accurate classification of various environmental management data on the computing power network, meets the current application requirements for massive data management in the informationization construction of environmental protection work, and realizes the transformation of environmental protection informationization to digital environmental protection and smart environmental protection.

The invention provides an advertisement putting evaluation and optimization method and system based on big data, and relates to the technical field of advertisement optimization, and the method comprises the steps: obtaining full-link data, user behavior time sequence data and putting environment attribute data, and generating a structured unified data mart; fusing the extracted user characteristic parameters with the delivery environment attribute data by using an attention mechanism fusion algorithm, and constructing a user behavior graph based on a graph neural network; a multi-dimensional difference evaluation model is constructed to calculate a causal effect value of advertisement intervention, and a putting effect evaluation index is generated in combination with a periodic feature matching mechanism; based on the evaluation index, an optimization strategy adjustment scheme is generated through a reinforcement learning algorithm and is automatically executed; the system comprises an optimization strategy generation module, an optimization strategy execution module and the like. According to the method, the accuracy and optimization efficiency of advertisement effect evaluation caused by data islands, dynamic fluctuation of user behaviors and market environment interference are improved, and accurate evaluation and dynamic adaptive optimization of advertisement putting are realized.