83 results about "Datawarehouses" patented technology

The application discloses an industry data management method based on big data, and belongs to the technical field of industry data management, and specifically comprises the following steps: constructing an after-sales data warehouse based on historical after-sales maintenance records; extracting the correlation relationship of spare parts and restoring the level of finished parts from the after-sales data warehouse to construct a coupling correlation topology; screening abnormal samples deviating from the expected failure mode, extracting corresponding supplier production process parameters and batch information, and forming a process parameter set; deconstructing the coupling correlation topology and combining the process parameter set to establish an attribution mapping parameter set; constructing a causal inference model between the attribution mapping parameter set and the failure mode to obtain the attribution weight of each supplier batch to the failure mode; and marking the supplier batch with an attribution weight exceeding a preset threshold as a target traceability batch and outputting the same. The application realizes accurate attribution from after-sales failure data to a supplier batch, and provides a quantitative basis for supply chain quality control.

The application relates to the field of distributed databases in computer science, and particularly provides a distributed database recommended index verification method and device, which comprises a database node, a candidate index verification module and a data warehouse; the database node performs index verification analysis on a database system where an environment is located; the candidate index verification module finds relevant SQL statements according to indexes generated by an index recommendation module, calculates the benefits of a single index and the benefits of an index combination, verifies the effectiveness of the recommended indexes, and eliminates indexes that are useless to application load; and the data warehouse is used to collect and store running data in the database. Compared with the prior art, the application greatly reduces the calculation amount of index verification, and enhances the reliability of index verification, so as to avoid recommending indexes that are useless or even harmful to application load.

This invention relates to an intelligent monitoring system and deployment method for building module data interfaces based on the Internet of Things (IoT), belonging to the field of building information monitoring technology. The system consists of distributed sensing units, an edge computing gateway, a cloud platform, and a visualization terminal. The edge computing gateway incorporates an adaptive filtering algorithm and an anomaly pattern recognition model. The cloud platform uses a time-series database to construct a multi-dimensional data warehouse and performs structural health assessment by fusing LSTM neural networks and random forest algorithms. The deployment method includes: optimizing sensor placement strategies based on BIM models, establishing a wireless mesh self-organizing network communication architecture, and configuring a hierarchical early warning mechanism and fault tracing function. The innovation lies in proposing a dynamic threshold adjustment algorithm and an interface performance degradation prediction model to achieve real-time monitoring and lifespan prediction of building module connection status. This system has the advantages of flexible deployment, high detection accuracy, and low maintenance costs, effectively improving the intelligent level of building structural safety monitoring.

The application provides a cross-source synchronization table index creation method and device, equipment, storage medium and program product, which can be applied to the technical field of big data. The method comprises the following steps: receiving a query statement for obtaining data from a data warehouse; extracting a query field set and a condition field set from the query statement; obtaining data from the data warehouse according to the query statement, and synchronizing the obtained data to a target table in an online database, wherein the target table comprises data columns corresponding to the query field set; merging the condition field set into a candidate index field set of the target table; calculating a dispersion index of each field in the candidate index field set based on the condition value distribution corresponding to each field in the candidate index field set; selecting a target field set from the candidate index field set based on the dispersion index; and creating an index for each field in the target field set on the target table.

The application particularly relates to a large model-based home guest network industry collaborative planning and decision method. The large model-based home guest network industry collaborative planning and decision method collects network data, business data and customer data of a home guest network, integrates the data into a unified data warehouse after preprocessing, and establishes an association relationship; demand and target of planning and decision are self-defined, current real-time data are input into a large model after preprocessing for inference calculation to generate a plurality of candidate schemes; an optimal scheme is determined through comprehensive evaluation, the scheme is decomposed into specific implementation tasks, an implementation plan is formulated, and resources are allocated for implementation. The large model-based home guest network industry collaborative planning and decision method realizes accurate matching of network resources and business development, improves operation efficiency and service quality of the home guest network, reduces operation cost of the enterprise, enhances core competitiveness of the enterprise in the market, and provides more high-quality and more personalized network service and business experience for home guest users.

This invention relates to the field of geotechnical engineering monitoring and slope stability analysis technology, specifically to a method and system for the layout of slope monitoring points throughout the entire construction and operation cycle. The method acquires multi-source monitoring data and constructs a unified data warehouse through cleaning, format conversion, and spatiotemporal registration. Based on this unified data warehouse, a dynamic risk assessment model coupled with the limit equilibrium method and the finite element method is used to identify target risk areas, and key monitoring variables are identified through global sensitivity analysis. Finally, a multi-objective optimization algorithm is used to solve for the Pareto optimal solution set to determine the location and density of monitoring points. This invention achieves adaptive matching of the monitoring network to changes in geological conditions and dynamic engineering needs throughout the entire cycle, improving the targeting of monitoring and the efficiency of resource allocation.

The embodiment of the application belongs to the technical field of financial science and technology, is applied to a financial data warehouse management scene, and relates to a data warehouse management method, device, equipment and storage medium thereof, comprising the following steps: obtaining characteristic indexes of all data source tables in a target data warehouse by analyzing metadata of the target data warehouse; identifying a target data source table according to the characteristic indexes and a preset evaluation strategy, and managing the target data source table according to a preset management strategy; and managing the target data warehouse in a timing task mode. According to the characteristic indexes and the preset evaluation strategy, the target data source table is identified, and the target data source table is managed according to the preset management strategy; a timing task is adopted, the target data warehouse can be managed regularly, the automatic and regular processing of the target data warehouse is ensured, the situation that a large amount of CPU resources are wasted due to chimney processing is avoided, and the standardization and scientific arrangement of financial business data in the data warehouse are facilitated.

By utilizing user-defined functions in the structured query language SQL, a system for performing entity resolution, specifically ranked personally identifiable information (PII) matching, can function directly within the data warehouse's execution environment, thereby eliminating the need to move data to external systems. This system uses SQL and user-defined function (UDF) implementations to utilize an inverted index system that can operate on the distributed processing capabilities of the data warehouse.

The application discloses a kind of based on multi-source data fusion and process evaluation's college student comprehensive performance automatic generation method and system, including S1 establishes ideological and political evaluation model, academic performance evaluation model, practical ability evaluation model, comprehensive quality evaluation model, development potential evaluation model comment model. S2 by middleware establishes standardization interface and automatically reads the heterogeneous data source data of educational administration system, school system, campus card system, library system, scientific research and teaching research practical training system, second classroom activity platform etc.S3 to behavior sequence is carried out pattern recognition and feature extraction, establishes student panoramic data warehouse, and the data of step S2 is labeled with S1 evaluation model label and storage.S4 user selects comment purpose.S5 dynamic weight distribution.S6 to student data is screened.S7 comment automatic generation.S8 artificial check comment.S9 generation visual report.The application improves work efficiency, and can provide accurate student performance in school for parents and employing units.

Disclosed are a traffic data warehouse construction method, a storage medium, and a terminal. The method includes: creating a target monitoring task based on a creation instruction; loading a monitoring object and spatial range corresponding to the target monitoring task, and obtaining a sampling unit set of the monitoring object; calculating a spatial attribution relationship between a spatial range of each sampling unit in the sampling unit set and the spatial range of the target monitoring task, and determining a spatial coupling relationship; setting a label of the monitoring object based on the spatial coupling relationship, generating a labeled monitoring object, inputting the labeled monitoring object to a preset monitoring calculation function, and outputting a calculation result; and configuring the calculation result in the labeled monitoring object, and distributing the configured monitoring object to a task database corresponding to the target monitoring task.

The application relates to a city design scheme generation method, device and equipment and a storage medium. The method comprises the following steps: obtaining multi-dimensional index data from a data source, performing a preprocessing operation to obtain a sample data set, constructing a data warehouse, obtaining related data of a city design scheme to be generated input by a user, dividing target plots according to the related data, calculating initial attribute indexes of each target plot, optimizing the initial attribute indexes by using predicted attribute indexes output by an index prediction model, obtaining target attribute indexes of each target plot, calculating building shape parameters of the city according to the target attribute indexes, generating an initial three-dimensional model of the city design scheme, and identifying the initial three-dimensional model by using different colors to obtain a target three-dimensional model of the city design scheme. The application can generate a city design scheme which is compliant, reasonable, has distinct city characteristics and designer will, and has high practicability.

The application discloses a lottery data warehouse table data checking method and device based on a sliding window and hierarchical aggregation, a medium and a product, and relates to the field of data quality management.The method comprises the following steps: defining a checking scheme for a lottery data warehouse table through configuration information, periodically reading the configuration by a master control program, starting a window checking task when the difference between the current time and the end time of the window time exceeds the sedimentation time, and comparing data by window and updating a completion identifier by using the sliding window.The checking task is executed in two layers: the first layer performs parallel aggregation counting according to units, tables and partitions, and the second layer aggregates source end data and target end data and compares them.If the comparison is successful, the state and the time are updated, and if the comparison fails for a preset number of times, an alarm is given.The application can guarantee data accuracy and real-time performance, reduce delay time, improve the stability of the checking process, reduce checking resource consumption, and flexibly adjust the balance between comparison efficiency and business delay.

A data lifecycle management method, device, equipment, storage medium and program product are disclosed, access heat of all data of a first storage layer in a data warehouse is acquired; wherein the data warehouse comprises at least two storage layers, and there is an association relationship between the storage layers, in which upstream data supports downstream data; cold data in the first storage layer is determined based on the access heat; associated data related to the cold data in the remaining storage layers is determined based on the association relationship; and a data cleaning operation is performed on the cold data and the associated data. By adopting the embodiment of the present application, the problems of hot data misdeletion and cold data redundancy in the data warehouse can be avoided, fine and differentiated management of data is realized, and resource waste of the data warehouse is reduced.

This invention discloses a method and system for intelligent scheduling of logistics vehicles based on big data, belonging to the field of logistics big data scheduling technology. It includes establishing a logistics transportation scheduling big data warehouse, collecting and storing raw data such as historical orders, vehicle files, real-time traffic, weather, and warehouse operation status; standardizing, cleaning, and integrating the data to form a multi-dimensional scheduling decision feature set; identifying high-frequency delivery areas and frequently congested road sections through spatiotemporal clustering algorithms; generating regional heat maps and road network risk layers; dynamically correcting the road network risk layers by combining real-time traffic and weather information to obtain a real-time road network status layer; and performing multi-objective programming operations by integrating orders to be scheduled, available vehicle pools, regional heat maps, and real-time road network status layers to generate a preliminary scheduling scheme. This invention can accurately present the spatiotemporal distribution characteristics of orders and road networks, dynamically adapt to the real-time transportation environment, and improve the multi-dimensional data support and computational adaptability of scheduling decisions.

This application discloses a method for constructing a data model, a method for data querying, an apparatus, and a storage medium, relating to the field of data processing technology. The method includes: obtaining field information from a dimensional model and at least two dimensions of a dimension table, where the field information indicates a mapping relationship between at least two dimensions; constructing a full dimensional relationship graph based on the field information, where the field information also indicates the association relationship between each of the at least two dimensions and a fact table; and attaching the fact table to at least one dimension in the full dimensional relationship graph based on the field information, generating a directed acyclic connected graph centered on the fact table as a data model, used to determine the path from the query metric to the fact table based on the query metric and its dimensions. This method can construct a data model based on existing data warehouse construction achievements, thereby determining the path of the query metric in the data warehouse through the data model.

Evaluating stored query results including generating a database query using a state specification of a graphical user interface, wherein the database query is composed to retrieve a current query result from a cloud-based data warehouse; searching a cache local to the query execution engine for the current query result, wherein the cache comprises a first previous query result from the cloud-based data warehouse; determining, based on the searching, that the current query result may be generated using the first previous query result from the cache; and creating the current query result using at least a portion of the first previous query result from the cache.

The application discloses a data processing method and device, equipment, storage medium and program product, and relates to the computer technical field. The method comprises the following steps: acquiring a domain model constructed based on a business requirement, the domain model being used for graphically representing entities and attributes in a business domain to which the business requirement belongs, the domain model comprising M entities, and each entity comprising at least one attribute; determining a first mapping relationship and a second mapping relationship according to the domain model, the first mapping relationship being used for mapping each entity to a table corresponding to the entity, and the second mapping relationship being used for mapping the attribute to a field corresponding to the attribute, the field corresponding to the attribute being a field in the table corresponding to the entity in which the attribute is located; and storing, in a form of a table, business data in a business domain into a data warehouse based on the first mapping relationship and the second mapping relationship, the business data being data related to at least one entity in the M entities. The method improves the data processing efficiency.

This application relates to the field of data management technology and discloses a data storage method, apparatus, device, and storage medium based on a data warehouse. The method includes: responding to input source data, determining a target storage space in the data warehouse according to the data format of the source data; generating corresponding original data tables based on the data table field rules associated with the target storage space and the source data; summarizing data at different aggregation levels based on the original data tables to obtain multiple summarized data tables; and storing the original data tables and each summarized data table to a corresponding storage file in the target storage space. This application can effectively improve the consistency and accuracy of data in a data warehouse and achieve high-efficiency data storage.

The present application relates to the technical field of data warehouse, especially to a data warehouse correlation link cleaning method, device, equipment and storage medium, the method first carries out script retrieval to the obtained task path set based on the preset function name matching rule, respectively to each function call line data obtained denoising and deduplication processing, obtain a plurality of task scheduling functions, based on a plurality of task scheduling functions in the preset data warehouse dictionary table SQL content search, obtain the SQL definition corresponding to each task scheduling function, respectively to each SQL definition character cleaning processing and sorting deduplication processing, obtain the data source table set corresponding to each task scheduling function and the cleaning target table set, the task path set, a plurality of task scheduling functions, a plurality of data source table sets and a plurality of cleaning target table sets are associated and bound and written in the blood relationship table, obtain the standardized blood table, aims at improving the working efficiency of data warehouse blood relationship carding.

A system according to the present disclosure may include a processor, a centralized data warehouse, and a non-transitory computer readable medium storing thereon instructions that are executable by the processor to cause the system to perform operations. The operations may include receiving, from a user, identifying information, generating, based on the identifying information, a central profile for the user, generating, from the central profile, a first verification token configured to verify the user for a first service, and generating, from the central profile, a second verification token configured to verify the user for a second service. The first verification token and the second verification token are separately revokable and severable from the central profile, and neither the first verification token nor the second verification token comprise the identifying information of the user.

The application provides a nested JSON parent field inheritance flattening extraction method based on path mapping. By pre-configuring the mapping relationship of "parent path and target output field", during the traversal of the JSON tree, the parent node field is automatically identified and extracted based on the current traversal path, and it is written into the inheritance context. When the target extraction node is reached, the target node field and the inheritance context are merged according to the preset conflict processing strategy to generate a stable structure of the flattened output record, and finally a standardized result set output is formed. By configuration instead of hard coding, flexible definition and runtime automatic inheritance of parent-child field relationship are realized. Compared with existing solutions, the data semantic integrity and consistency are significantly improved, the development and maintenance cost is greatly reduced, and the method has strong universality and engineering landing value, and can be widely applied to API data access, ETL offline processing, real-time data middle platform, data warehouse construction and the like.

The application discloses a credit business processing method and device, electronic equipment and storage medium, and relates to the technical field of computers, which comprises the following steps: obtaining highway toll data authorized by an enterprise, wherein the toll data at least contains account state information, arrearage information and credit list information; performing multi-level credit verification based on the highway toll data, and determining that the credit limit is zero if any verification in the multi-level credit verification fails; when all the multi-level credit verification passes, calculating the credit limit based on a credit limit calculation model according to the cumulative toll fee of the enterprise within a preset time period; interacting the credit result data with an external traffic data platform, and synchronizing the toll data that passes the credit to a data warehouse for post-loan risk monitoring and early warning. The technical effect of improving the precision of credit business and strengthening the post-loan risk control capability is achieved.

This invention discloses a configuration and processing method, apparatus, and related media based on an energy data warehouse. The method includes acquiring incremental energy data and location information, configuring effective location data; performing threshold configuration rule judgment to output a cleaning result table; wherein the cleaning result table includes environmental data and electricity meter data; performing outlier processing on the electricity meter data and the environmental data respectively to output first processed data and second processed data, ultimately overwriting to obtain a final cleaning result table; performing data processing based on the final cleaning result table to obtain categorized storage data and statistical storage data respectively; receiving query instructions and returning an energy information list. This invention processes the final cleaning result table to obtain categorized storage data and statistical storage data, and then uses query instructions to obtain an energy information list for managing the energy data warehouse. This solves the problem of data anomalies in the energy data warehouse and greatly improves management efficiency.

The application discloses a kind of teaching behavior analysis method and system based on knowledge base teaching feature fusion and modeling, and the application relates to data analysis technical field, including the following steps: collecting multi-source heterogeneous ideological and political education data and being associated to specific student and time point, form original material, structured data is stored to data warehouse, unstructured data and language content are stored to data lake, construct lake warehouse integrated data pedestal, establish the mapping relationship of query language and structure data field, and the semantic vectorization of unstructured data, generate consciousness tendency score and behavior quantization index, to calculate ideological and political development deviation degree, when exceeding adaptive threshold, trigger early warning, and extract relevant evidence fragments, timely targeted education intervention, to effectively reduce the risk of student's ideological development deviation.

The embodiment of the application belongs to the technical field of data processing and is applied to the field of data model reconstruction. The embodiment relates to a data model reconstruction method, device and equipment and a storage medium thereof. The method comprises the following steps: receiving a data model reconstruction instruction; obtaining a theme table that can be selected for reconstruction from a source layer of a preset data warehouse; performing numbering processing on the theme table that can be selected according to a preset numbering rule to obtain a numbering processing result; obtaining an optimal combination mode between the theme tables that can be selected during reconstruction according to a preset constraint condition and a preset configuration file; generating a combination table corresponding to a target theme domain according to the optimal combination mode and the distinguished identification information; replacing the target theme domain, and cyclically executing the above steps until combination tables corresponding to all theme domains are generated, and the cycle is stopped, and data model reconstruction is completed. The data model is automatically reconstructed in a cyclic generation mode, and the coupling degree of the reconstruction result is reduced by directly obtaining the source layer data.

The application discloses a big data warehouse intelligent analysis method and device, equipment and medium, the method comprises the following steps: in response to user analysis demand, and from the initial entry layer of the preset obtains the target resource positioning information corresponding to the user analysis demand; wherein, the initial entry layer is used for storing the resource positioning information of the big data warehouse; the current context is constructed according to the user analysis demand and target resource positioning information;Based on the preset demand coverage inspection strategy, check whether the current context meets the sufficiency condition of analyzing the user analysis demand to obtain the inspection result;If it is determined that the inspection result is a satisfaction result, then the current context is simplified according to the preset context management strategy, and the minimum effective context is obtained;Based on the minimum effective context, analysis reasoning is executed and analysis result is output.The application actively explores data, dynamically constructs the minimum effective context, and significantly improves the accuracy and efficiency of analyzing the big data warehouse.

The application discloses a data query method and device, a storage medium and an electronic device, relates to the technical field of smart homes, and comprises the following steps: performing metadata analysis on first data in a data warehouse, and determining metadata information of the first data according to a result of the metadata analysis; adding different types of metadata carried by the first data to a retrieval database based on the metadata information; in the case that a positioning condition corresponding to to-be-queried data is acquired from the retrieval database, determining target metadata matched with the positioning condition from the retrieval database, so as to confirm a data query result fed back to a use object according to the target metadata. By adopting the technical solution, the problems that data cannot be quickly found and fed back to a use object in the related art are solved.

The application discloses a kind of decoupling methods, devices and storage medium of enterprise data architecture, it is related to the technical field of data management, the method of the application comprises: obtaining data object in target data architecture and the dependent relationship between the data object;According to the data object and the dependent relationship, data architecture directed graph is constructed;According to the data architecture directed graph, the multidimensional centrality index and weight value of each data object are calculated;According to the multidimensional centrality index and the weight value, determine target coupling node, and execute corresponding decoupling strategy for the target coupling node.The application determines the corresponding decoupling node by abstracting data architecture into directed graph and applying graph theory centrality principle, and executes corresponding decoupling scheme, solves the problem that data model coupling degree is high in enterprise-level data warehouse, link is rigid, improves the maintainability of data architecture and the reuse value of model.

This invention discloses a data query method, apparatus, device, and storage medium, comprising: creating a corresponding index for the primary key of each initial query data record; adding the index to the corresponding initial query data to obtain updated query data; and sending the updated query data to a transaction system; creating a cache table based on the correspondence between the primary key and the index; and receiving the latest query request generated by the transaction system based on the updated query data already in the data queue; querying the cache table based on the target index to obtain the target primary key; and scanning the database based on the target primary key and the latest query request to obtain the latest query data. By creating the cache table based on the index and the primary key, when the initial query data is lost in the transaction system, the data warehouse system can relocate the query position in the database based on the latest query request and the cache table, thereby achieving a complete and accurate query of the database.