293 results about "Datawarehouses" patented technology

The invention relates to the technical field of information processing and artificial intelligence, and discloses a multi-source fusion maintenance work order association early warning method and system. The method is used for solving the problem that the accuracy and timeliness of early warning are insufficient due to comprehensive modeling of multi-modal data in a traditional method. The method comprises the steps that firstly, a work order, a log event, a monitoring index and asset information are accessed, and after time caliber and field caliber unification and quality labeling are completed, the work order, the log event, the monitoring index and the asset information are written into a real-time data lake and a batch processing data warehouse respectively; normalizing the work order text, extracting a key identifier, and generating a feature index bound with the work order number; aligning the log and the index data according to a work order time window to form an event fragment and an index fragment, and associating an asset node; and constructing a dependency graph based on an asset ledger and a deployment link, forming a work order candidate pair, carrying out classification and hierarchical pushing, and recording feedback and version information for tracing.

The invention discloses a dialogue type business intelligent analysis method based on a natural language, and relates to the technical field of business management systems, and the method mainly comprises the steps: recognizing the intention type of a user intention contained in a natural language request through a general large language model; according to the identified specific intention type, calling a corresponding special analysis module to perform semantic analysis and parameter extraction on the natural language request so as to generate an operable analysis instruction; calling a business intelligent system data interface to execute the analysis instruction, and obtaining corresponding structured data from a business intelligent system data warehouse; and generating a comprehensive analysis report which is matched with the identified intention type and the corresponding structured data and comprises visual elements and analysis explanations based on the identified intention type and the corresponding structured data. According to the method, the intention type of the natural language request of the user is understood through the large language model, so that the user with a non-professional background can easily perform complex data exploration, and the use threshold is reduced.

The invention discloses a data warehouse table structure generation method and device, a medium and a product, and belongs to the technical field of data warehouses. According to the method, the Excel template configured with multiple pieces of source database table structure information is obtained, table field definition is automatically analyzed, the source data type is intelligently mapped and converted into the target data type according to the source database type, meanwhile, partition key fields are automatically recognized based on the preset strategy, and finally a complete and accurate data definition language (DDL) statement is generated. According to the method, remarkable technical breakthrough and value are achieved in practical application, the table building efficiency can be improved by 90% or above, batch processing of hundreds of tables is supported, and manual errors are reduced by 95% or above; mainstream database type conversion is comprehensively supported, and various data models such as Dores are adapted; the method has the capabilities of automatic type detection and conversion, intelligent key field selection and partitioning and bucket dividing strategy optimization.

The invention relates to the technical field of freight station scheduling, and particularly discloses an aviation freight station TV vehicle dynamic scheduling method and system based on multi-dimensional data, and the method comprises the steps: obtaining a multi-dimensional data source in real time, building a unified real-time data warehouse through the processing of a platform in the Internet of Things, and obtaining the load of a beating platform through a machine learning prediction model based on fusion data. A task sequence is generated in combination with a sorting plan, a dynamic scheduling algorithm is adopted to dispatch tasks, plan an optimal path, distribute instructions in real time and monitor the state of the TV vehicle, deviation is calculated based on feedback data, and parameters are dynamically adjusted to form a closed loop. According to the method, a unified real-time data warehouse is constructed, data islands of all systems are broken, seamless real-time sharing of data is achieved, accurate data support is provided for subsequent scheduling, task allocation and paths are optimized through multiple targets, the total empty driving distance and task delay of the TV vehicles are reduced, the resource utilization rate is increased, the TV vehicle scheduling efficiency is finally improved, and the scheduling efficiency of the TV vehicles is improved. And key indexes such as the empty driving rate and the punctuality rate are optimized.

Systems and methods are disclosed for converting natural language queries to a query instruction set for searching a data warehouse. To generate a query instruction set from a natural language query, a system iteratively uses a generative artificial intelligence (AI) model and database query tools to generate a query instruction set in a stepwise manner. The system and generative AI model do not require a priori knowledge of data table contents in the data warehouse, which may include sensitive information. In addition, the system does not require access to the data warehouse to generate the query instruction set. Instead, the system is implemented to use structure information from the data warehouse, including table lists (such as table names) and table format information (such as column names) of tables in the data warehouse, and the generative AI model is a generally trained model to generate the query instruction set.

The invention provides a data analysis method. The data analysis method comprises the steps of obtaining query information for data; according to the method, the query information is analyzed, the query requirement of the user is determined, the pre-constructed data warehouse is searched based on the query requirement, the target query content is determined, and the pre-constructed data warehouse is associated with databases storing different types, so that cross-database query can be performed based on the query requirement of the user, and the query efficiency is improved. Therefore, various types of data query of texts, images, videos and the like can be realized, and multi-dimensional and complex-dimensional data analysis can be realized. According to the query information, the generation template of the data is determined, and the analysis report is generated according to the generation template based on the target query content, so that automatic report generation is realized, the generated report conforms to the identity of the user, and the working efficiency of the user is improved.

The invention discloses a data warehouse intelligent construction and management method and system, and belongs to the technical field of data warehouse construction and management, and the data warehouse intelligent construction and management method comprises the steps: obtaining an enterprise data source, and extracting the feature parameters of the data source; constructing a data warehouse logic model; self-adaptive ETL configuration parameters are defined; constructing a three-layer physical architecture comprising a data access layer, a data storage layer and a data service layer; intelligent cache strategy configuration is executed based on a deep learning algorithm, a three-level cache architecture is established, and a pre-cache rule and a failure strategy are defined; and running management optimization is implemented through a multi-objective optimization function. According to the method, the construction automation level is remarkably improved, the cache hit rate is increased, the average query response time is shortened, the data consistency and the system performance are effectively balanced, the elastic adaptive capacity of the system is enhanced, the operation and maintenance complexity and cost are reduced, and meanwhile efficient sharing of cache resources is achieved on the premise that data security isolation is guaranteed.

The application discloses an industry data management method based on big data, and belongs to the technical field of industry data management, and specifically comprises the following steps: constructing an after-sales data warehouse based on historical after-sales maintenance records; extracting the correlation relationship of spare parts and restoring the level of finished parts from the after-sales data warehouse to construct a coupling correlation topology; screening abnormal samples deviating from the expected failure mode, extracting corresponding supplier production process parameters and batch information, and forming a process parameter set; deconstructing the coupling correlation topology and combining the process parameter set to establish an attribution mapping parameter set; constructing a causal inference model between the attribution mapping parameter set and the failure mode to obtain the attribution weight of each supplier batch to the failure mode; and marking the supplier batch with an attribution weight exceeding a preset threshold as a target traceability batch and outputting the same. The application realizes accurate attribution from after-sales failure data to a supplier batch, and provides a quantitative basis for supply chain quality control.

The invention discloses a data center system based on decision analysis. The data center system comprises a data acquisition layer, a data storage layer, a data analysis layer, a decision engine layer and a data service layer, the data acquisition layer is used for acquiring various heterogeneous data sources inside and outside an enterprise, supporting access of multiple data sources and calculating a null rate and a numerical range compliance rate of a data set; the data storage layer carries out data organization by adopting a mixed architecture combining a data lake and a data warehouse, and is used for storing various data from the data acquisition layer in an original format; and the data analysis layer works cooperatively with the real-time analysis unit through the batch processing unit. According to the invention, a flexible data access channel is constructed by integrating Flume, Sqoop and Kafka tools, efficient acquisition and cleaning of structured, semi-structured and unstructured data are realized, and meanwhile, the architecture design that Spark is adopted for batch processing analysis and Flink is adopted for real-time analysis is adopted, so that the problem of off-line batch processing and real-time stream processing splitting is solved.

The embodiment of the invention provides a business data processing method and device, a storage medium and a program product, and the method comprises the steps: obtaining business target data and actual business data of each employee; sending the business target data and the actual business data to a data warehouse after each preset synchronization time; and obtaining and calculating actual business data in the data warehouse by using a pre-obtained calculation model, comparing an actual business result obtained after calculation with the business target data, and generating a business index progress of each employee. According to the business data processing method provided by the embodiment of the invention, the business data of multiple employees can be processed in batches in real time, and the data processing efficiency is improved.

Live editing editable tables including receiving, via a client computing system by a table manager, a request for an editable table stored on a cloud-based data warehouse; determining, by the table manager, that an uncommitted edit to the editable table exists, wherein the uncommitted edit is an edit made to the editable table that has not been committed to the cloud-based data warehouse at a time when the request was received; redirecting, by the table manager, the request for the editable table to an edit queue accessible by the table manager, wherein the edit queue comprises the uncommitted edit; servicing by the table manager, the request for the editable table by combining the uncommitted edit from the edit queue with the editable table from the cloud-based data warehouse; and presenting, by the table manager, the editable table on the client computing system.

The invention relates to the technical field of data processing, and discloses a copper product full life cycle-oriented carbon footprint big data warehouse table design method, which comprises the following steps of: carrying out abnormal data filtering on an original carbon footprint data set of a copper product full life cycle to obtain standardized carbon footprint data; based on the standardized carbon footprint data, performing time series data storage optimization on the table structure of the big data warehouse to obtain a dynamic partition table structure; according to the standardized carbon footprint data, performing data mapping on the dynamic partition table structure to obtain an initial data table; traceability information embedding is carried out on the initial data table, and a carbon footprint big data warehouse table is obtained; performing storage structure reconstruction on the carbon footprint big data warehouse table to obtain an efficient storage data warehouse; carrying out life cycle carbon emission analysis on the efficient storage data warehouse to obtain a structured carbon footprint analysis report; according to the invention, the efficiency of carbon footprint data processing can be improved.

The invention relates to the technical field of medical informatization, in particular to a comprehensive rehabilitation service informatization integration method and system, and the method comprises the steps: obtaining multi-source heterogeneous patient rehabilitation data distributed in a hospital information system, a rehabilitation evaluation system, rehabilitation treatment equipment, wearable equipment and an electronic medical record; performing standardization processing on the multi-source heterogeneous patient rehabilitation data based on a preset unified rehabilitation data model; analyzing, cleaning and mapping the multi-source heterogeneous patient rehabilitation data into standardized patient rehabilitation data conforming to a unified rehabilitation data model in real time through a special data adapter deployed in a multi-protocol data interface layer; and by using a bidirectional synchronous data bus based on an event-driven architecture, issuing the standardized patient rehabilitation data to a specified rehabilitation device or client through the bidirectional synchronous data bus and an instruction generated by the central data warehouse. According to the invention, multi-source heterogeneous data from different systems and devices can be integrated to form a comprehensive, continuous and real-time health view for patients.

The invention discloses a family life event atlas construction method and system based on cooperation of multiple intelligent computing engines, and relates to the technical field of knowledge atlas construction.The method comprises the steps that a family environment continuous audio stream is collected through an end side, effective fragments are screened through voice activity detection, and original audio is destroyed after acoustic feature vectors are extracted; and then a plurality of parallel intelligent calculation engines respectively recognize non-language environment sounds, emotion and voiceprint identities, semantics and potential intentions, a standardized family event object is generated through multi-modal fusion, the standardized family event object is stored in an independent research and development data warehouse of a time sequence + graph mixed framework, a family life knowledge graph is constructed, and finally graph changes are monitored through an inference engine, so that the family life knowledge graph is obtained. And actively triggering home equipment adjustment or pushing suggestions. According to the invention, active perception and semantic understanding of family events are realized, the risk of privacy disclosure is avoided, and the problems of passive response, data islands and privacy safety hidden troubles of the existing smart home are solved.

Embodiments of the present application relate to the technical field of data warehouse, and more particularly, to a data processing method and device, a storage medium and an electronic device. The method comprises: performing hierarchical processing on a cluster of to-be-processed data tables to determine the hierarchical attribution of each to-be-processed data table in a data warehouse; performing blood relationship analysis on a DWD table corresponding to a data detail layer of the data warehouse, and performing filtering processing on the DWD table based on the blood relationship analysis result; and determining a substitution mapping model according to the blood relationship of each data table between a data operation layer model and a data detail layer model after filtering processing of the data warehouse. The scheme can solve the problem of unreasonable dependence on an ods table, promote the richness of the construction of the intermediate layer of the data warehouse, improve the recognizability of the data warehouse model, reduce the computing cost, and improve the data use efficiency.

The invention relates to a data restoration method and device for a data warehouse, equipment, a storage medium and a program product. Comprising the steps of receiving structured data input by a user; comprising to-be-repaired field information and screening conditions; generating an SQL statement according to the structured data and a preset SQL template; carrying out pre-verification on the SQL statement; after the pre-verification is passed, determining a to-be-repaired data magnitude corresponding to the SQL statement; if the magnitude of the to-be-repaired data is smaller than or equal to the preset magnitude, calling a first number repairing strategy, and executing ACID transaction operation according to the SQL statement so as to realize data repair for the data warehouse; and if the magnitude of the to-be-repaired data is greater than the preset magnitude, calling a second repair strategy, and performing data repair on the data warehouse by adopting an operation of overwriting data according to the SQL statement. By adopting the method, the number correction logic is automatically generated, misoperation is avoided, the labor cost can be saved, the repeated data correction probability can be reduced, and the situation that supervision early warning is triggered due to frequent rereporting is avoided.

A solution for transforming and managing data in a data lake environment is disclosed. A definition of a transformation stream is received via a graphical user interface. The transform stream specifies an operation to transform source data into target data stored in the first format object store. And generating a virtual process corresponding to the transformation stream and storing the virtual process in a memory cloud database. A virtual process is then executed such that metadata is retrieved from the first format object store and a transform job is triggered on the compute cluster. The transform job reads the source data from the first format object store, applies one or more transforms specified in the definition of the transform stream, and writes the result to the target data in the first format object store.

The embodiment of the application provides a kind of multi-data source integration method, device, computer equipment and storage medium, the method obtains the data source integration request of data warehouse, data source integration request includes to be handled business data and integration parameter, through service unit, data source integration request is checked, in the case where check passes, through analysis unit and integration parameter, to be handled business data is parsed, determines integration index and data source type, determines the target component matched with data source type from the multiple components of engine unit, according to integration index, through target component and service unit, to be handled business data is integrated, obtains standard integration data, realizes the normalization and unified processing of the data of different data sources, improves the integration efficiency of multiple data sources, improves the maintainability and expansibility of data warehouse, improves the ability of fast access and consumption to data.

The invention discloses a photovoltaic cell production and operation-oriented data warehouse modeling method, device and system, a storage medium, a computer product and a data warehouse. The method comprises the following steps: determining an associated system and equipment related to the whole process of photovoltaic cell production and operation; constructing a data source layer, an access layer, a model layer and an application layer for the associated system and equipment; wherein the model layer adopts a layered design; and forming the data warehouse by using the data source layer, the access layer, the model layer and the application layer. According to the method, the data integration efficiency is improved, and the processing performance of the multi-source data is improved, so that the service agility and the decision-making capability are optimized.

The application relates to the field of distributed databases in computer science, and particularly provides a distributed database recommended index verification method and device, which comprises a database node, a candidate index verification module and a data warehouse; the database node performs index verification analysis on a database system where an environment is located; the candidate index verification module finds relevant SQL statements according to indexes generated by an index recommendation module, calculates the benefits of a single index and the benefits of an index combination, verifies the effectiveness of the recommended indexes, and eliminates indexes that are useless to application load; and the data warehouse is used to collect and store running data in the database. Compared with the prior art, the application greatly reduces the calculation amount of index verification, and enhances the reliability of index verification, so as to avoid recommending indexes that are useless or even harmful to application load.

Creating data warehouse views for input tables including: selecting, by a workbook manager, a subset of columns of an input table stored in a cloud-based data warehouse, wherein the subset of columns excludes one or more metadata columns of the input table; generating, by the workbook manager, a view of the input table based on the subset of columns; and storing, by the workbook manager, the view of the input table in the cloud-based data warehouse.

The application discloses a data anomaly detection method and device, a processor and electronic equipment. The method comprises the following steps: obtaining the identification information of at least one preset calculation module and at least one preset storage module; filling the identification information into a preset adaptation table of a preset data warehouse; predicting the adaptation probability of the preset calculation module and the preset storage module according to a preset adaptation model, and filling the adaptation probability into the preset adaptation table, wherein the preset adaptation model is obtained by training a deep learning algorithm based on a self-attention mechanism; performing data caching of the preset adaptation table by calling stream data and a remote dictionary service through the preset calculation module, to obtain a data stream processing table, wherein the remote dictionary service and the preset calculation module establish a mapping relationship in the form of a key-value pair; and analyzing the data stream processing table to realize data anomaly detection. The application solves the technical problem of low data anomaly detection efficiency.

This invention relates to an intelligent monitoring system and deployment method for building module data interfaces based on the Internet of Things (IoT), belonging to the field of building information monitoring technology. The system consists of distributed sensing units, an edge computing gateway, a cloud platform, and a visualization terminal. The edge computing gateway incorporates an adaptive filtering algorithm and an anomaly pattern recognition model. The cloud platform uses a time-series database to construct a multi-dimensional data warehouse and performs structural health assessment by fusing LSTM neural networks and random forest algorithms. The deployment method includes: optimizing sensor placement strategies based on BIM models, establishing a wireless mesh self-organizing network communication architecture, and configuring a hierarchical early warning mechanism and fault tracing function. The innovation lies in proposing a dynamic threshold adjustment algorithm and an interface performance degradation prediction model to achieve real-time monitoring and lifespan prediction of building module connection status. This system has the advantages of flexible deployment, high detection accuracy, and low maintenance costs, effectively improving the intelligent level of building structural safety monitoring.

The application provides a cross-source synchronization table index creation method and device, equipment, storage medium and program product, which can be applied to the technical field of big data. The method comprises the following steps: receiving a query statement for obtaining data from a data warehouse; extracting a query field set and a condition field set from the query statement; obtaining data from the data warehouse according to the query statement, and synchronizing the obtained data to a target table in an online database, wherein the target table comprises data columns corresponding to the query field set; merging the condition field set into a candidate index field set of the target table; calculating a dispersion index of each field in the candidate index field set based on the condition value distribution corresponding to each field in the candidate index field set; selecting a target field set from the candidate index field set based on the dispersion index; and creating an index for each field in the target field set on the target table.

The application relates to the technical field of data processing, in particular to a data visualization method and device. The method comprises the following steps: receiving a data query instruction, and acquiring a target data object from a data warehouse based on the query instruction, wherein the data query instruction comprises a data query condition; performing rendering encapsulation processing on the target data object based on a preset data visualization rendering rule, so as to obtain a specified type of a visualization chart component; receiving a dashboard configuration instruction, and adding one or more target visualization charts into a dashboard based on the dashboard configuration instruction.

The application particularly relates to a large model-based home guest network industry collaborative planning and decision method. The large model-based home guest network industry collaborative planning and decision method collects network data, business data and customer data of a home guest network, integrates the data into a unified data warehouse after preprocessing, and establishes an association relationship; demand and target of planning and decision are self-defined, current real-time data are input into a large model after preprocessing for inference calculation to generate a plurality of candidate schemes; an optimal scheme is determined through comprehensive evaluation, the scheme is decomposed into specific implementation tasks, an implementation plan is formulated, and resources are allocated for implementation. The large model-based home guest network industry collaborative planning and decision method realizes accurate matching of network resources and business development, improves operation efficiency and service quality of the home guest network, reduces operation cost of the enterprise, enhances core competitiveness of the enterprise in the market, and provides more high-quality and more personalized network service and business experience for home guest users.

Automatic rewriting of subtotal calculations including presenting, by a table manager, a spreadsheet structure on a GUI of a client computing system, wherein the spreadsheet structure presents a data set from a cloud-based data warehouse; receiving, at a first column of the spreadsheet structure by the table manager, a formula that includes a reference to a second column of the spreadsheet structure, wherein the formula is received from the client computing system; detecting, by the table manager, that the second column includes a subtotal calculation of a third column of the spreadsheet structure; and rewriting, by the table manager, the formula by replacing the reference to the second column with the subtotal calculation of the third column.

This invention relates to the field of geotechnical engineering monitoring and slope stability analysis technology, specifically to a method and system for the layout of slope monitoring points throughout the entire construction and operation cycle. The method acquires multi-source monitoring data and constructs a unified data warehouse through cleaning, format conversion, and spatiotemporal registration. Based on this unified data warehouse, a dynamic risk assessment model coupled with the limit equilibrium method and the finite element method is used to identify target risk areas, and key monitoring variables are identified through global sensitivity analysis. Finally, a multi-objective optimization algorithm is used to solve for the Pareto optimal solution set to determine the location and density of monitoring points. This invention achieves adaptive matching of the monitoring network to changes in geological conditions and dynamic engineering needs throughout the entire cycle, improving the targeting of monitoring and the efficiency of resource allocation.

The embodiment of the application belongs to the technical field of financial science and technology, is applied to a financial data warehouse management scene, and relates to a data warehouse management method, device, equipment and storage medium thereof, comprising the following steps: obtaining characteristic indexes of all data source tables in a target data warehouse by analyzing metadata of the target data warehouse; identifying a target data source table according to the characteristic indexes and a preset evaluation strategy, and managing the target data source table according to a preset management strategy; and managing the target data warehouse in a timing task mode. According to the characteristic indexes and the preset evaluation strategy, the target data source table is identified, and the target data source table is managed according to the preset management strategy; a timing task is adopted, the target data warehouse can be managed regularly, the automatic and regular processing of the target data warehouse is ensured, the situation that a large amount of CPU resources are wasted due to chimney processing is avoided, and the standardization and scientific arrangement of financial business data in the data warehouse are facilitated.

By utilizing user-defined functions in the structured query language SQL, a system for performing entity resolution, specifically ranked personally identifiable information (PII) matching, can function directly within the data warehouse's execution environment, thereby eliminating the need to move data to external systems. This system uses SQL and user-defined function (UDF) implementations to utilize an inverted index system that can operate on the distributed processing capabilities of the data warehouse.