51 results about "Dimension table" patented technology

The invention discloses a meteorological data processing and storage method and system, and relates to the technical field of intelligent meteorological data processing, and the method comprises the steps: collecting multi-source meteorological observation data, radar data and numerical forecasting data, carrying out the access judgment, obtaining an access data set, carrying out the standardization and elevation correction, and obtaining a data set; calculating a comprehensive quality score to obtain a quality mark data set; screening samples based on the mass label data set to obtain an alignment data set, obtaining grid fusion data through the alignment data set, and performing smooth processing to obtain a fusion data set; obtaining a sealing partition based on the fused data set, and revising by using a late sample to obtain a versioned sealing partition; and obtaining a fact table and a dimension table according to the versioned sealing partition, establishing a topic domain model and a multi-version query interface, and calculating a prediction deviation and a hit rate. According to the method, unit standardization, spatial elevation correction and comprehensive quality scoring are performed on the accessed data set, so that dynamic evaluation and reliable marking of the meteorological data quality are realized.

The invention provides a Text-to-SQL (Structured Query Language) conversion method based on semantic modeling, which relates to the technical field of natural language processing and data analysis, and comprises the following steps: designing a structured semantic model comprising a fact table and a dimension table by adopting a dimension modeling method, and defining business field division, data model definition and model dictionary construction; performing word segmentation processing on the natural language question input by the user, matching the natural language question with dimensions, indexes and dimension values in the structured semantic model based on the model dictionary library, and generating Schema information containing database types, table names, field definitions and filtering conditions; calling the large model, and converting the natural language problem into a preliminary semantic SQL in combination with Schema information and a preset rule; calling the large model again, performing logic verification and grammar optimization on the preliminary semantic SQL, and generating a corrected semantic SQL; based on Schema information and data model definition, the corrected semantic SQL is converted into an executable database SQL statement.

The invention relates to the technical field of electric digital data processing, in particular to a user question-based dimension model wide table generation method, which comprises the following steps of: converting a natural language question input by a user into a standardized question text; performing semantic analysis on the standardized question text, and extracting key elements of the dimension model; determining at least one target dimension and a hierarchical structure corresponding to the target dimension in a pre-constructed dimension knowledge graph based on the key elements; based on the target dimension and the hierarchical structure, constructing a dimension model comprising a fact table and a dimension table; and generating a physical wide table based on the dimension model. According to the scheme, through deep fusion of natural language processing and the knowledge graph, the questions of the user can be mapped to the dimension knowledge graph after semantic analysis, dimension extraction and hierarchy generation are automatically completed, and the manual modeling cost is reduced.

The invention relates to a metadata-driven data consanguinity analysis and influence evaluation method, system and device and a medium. The method comprises the following steps: performing parameter analysis on an original query language script, and constructing a parameter function model in combination with a parameter dimension table; performing parameter replacement on an original query language script according to the model, and analyzing the replaced script to obtain a table-level blood relationship, a field-level blood relationship and the like; constructing a record-level query template based on the field-level blood relationship and the conversion logic, and constructing a dynamic blood relationship map; and executing a record level query template, carrying out record node mounting on the atlas, obtaining and responding to a data change event, carrying out hierarchical influence propagation calculation, obtaining and responding to a data exception event, carrying out root cause node positioning based on the mounted dynamic blood relationship atlas, and generating a repair suggestion. According to the method, through dynamic parameter analysis and record-level blood relationship analysis, the accuracy and automation level of data blood relationship analysis are improved, and the efficiency of data governance is improved.

The invention discloses an equipment data unified integration and supply method based on dynamic metadata and intelligent routing and an Internet of Things home platform, and the method comprises the steps: obtaining to-be-updated equipment data, classifying the equipment data according to the data structure, data increment and change frequency of the equipment data, and respectively marking the equipment data as main data or transaction data; based on task characteristics of the main data and the transaction data, intelligent scheduling is carried out by utilizing dynamic node pressure sensing, and different types of data are distributed to optimally matched processing nodes; and performing corresponding update processing on the main data and the transaction data, uniformly storing update processing results in a dimension table mode and a fact table mode, and enabling the update processing results to be called by a smart home language model through a uniform data service interface.

The application discloses a meteorological data processing and storage method and system, and relates to the technical field of intelligent meteorological data processing. The method comprises the following steps: collecting multi-source meteorological observation data, radar data and numerical prediction data, performing access determination to obtain access data sets, performing standardization and elevation correction on the access data sets, calculating comprehensive quality scores to obtain quality marked data sets; filtering samples based on the quality marked data sets to obtain aligned data sets, obtaining grid fusion data through the aligned data sets and performing smoothing processing to obtain a fusion data set; obtaining a sealing partition based on the fusion data set, and revising the sealing partition by using a late sample to obtain a versioned sealing partition; obtaining a fact table and a dimension table according to the versioned sealing partition, establishing a subject domain model and a multi-version query interface, and calculating a prediction deviation and a hit rate. Through unit standardization, spatial elevation correction and comprehensive quality scoring on the access data sets, dynamic evaluation and reliable marking of meteorological data quality are realized.

The invention discloses a data processing method, device and equipment, a computer medium and a product, and is applied to the field of data processing.The method comprises the steps that in response to detected business operation of data change of an upstream business system, business data are collected from a database log; dividing the business data into real-time business data and dimension table business data based on the updating frequency of the business data; sending the real-time business data and the dimension table business data to a streaming computing platform through a database acquisition component; storing the dimension table service data in a preset initial increment dimension table cache through the streaming computing platform to obtain an updated increment dimension table cache; on the basis of the service identification information, associating the real-time service data with dimension table service data in the update increment dimension table cache to obtain service result data; and the streaming computing platform sends the service result data to a data subscription demander through the message middleware. According to the method, data processing efficiency is improved, and computing resources are saved.

This invention discloses a method and system for implementing role-playing dimensions, relating to the field of data processing technology. It includes: creating a meta-model of dimension models related to role-playing dimensions, the meta-model comprising five types of data tables; selecting role-playing dimension names from their sources according to requirements; when a role-playing dimension name differs from a physical dimension table name, the dimension corresponding to the role-playing dimension name is considered a role-playing dimension; when displaying the related dimensions of an indicator, using the role-playing dimension name to replace the physical dimension name to demonstrate the actual business meaning of the dimension; when summarizing the indicator using selected dimensions, constructing dynamic SQL using the role-playing dimension name; when analyzing composite indicators, if the facts of the multi-atomic indicators constituting the composite indicator are all associated with the same physical dimension, then the composite indicator is summarized using that physical dimension; when performing parallel analysis of multiple indicators, if the facts of each indicator involved in the analysis are all associated with the same physical dimension, then the multiple indicators are summarized using the physical dimension; and displaying the list of related dimensions using the physical dimension name to avoid confusion caused by different role names.

The invention discloses a data warehouse model design method, and belongs to the technical field of data warehouse modeling. According to the method, a dimension modeling method is adopted to construct and analyze a mart model, data is organized into a non-standardized relation model, and the relation between tables is defined through keywords and foreign keys. And designing a dimension table system comprising a product dimension, a calendar day dimension, an institution dimension, an insurance policy attribute dimension and a customer dimension, and a fact table system comprising an insurance claim settlement analysis fact table, a payment analysis fact table, a claim settlement efficiency analysis fact table and a corresponding intermediate fact table. The analysis subject domain data is formatted into a star model or snowflake model, where the analysis fact table is a lightly summarized transactional fact table, and the intermediate fact table is a versioned historical table. According to the method, repeated data construction is avoided through a unified modeling method, and the cost is reduced; an efficient query analysis function is provided by optimizing a model structure; and the data traceability is ensured through versioning management.

This application provides a method, device, and storage medium for dimension table joins in Flink, relating to the field of big data technology. The method identifies a target subtask among multiple subtasks of a unified dimension table join operator in the current task manager. After controlling the target subtask to send a query request to an external database and obtain the dimension table data sent by the external database, the method controls the target subtask to load the dimension table data into a static dimension table object set in the metadata space of the task manager. This avoids putting significant operational pressure on the external database in a short period. The method controls the target subtask and multiple other subtasks to perform dimension table join operations based on the obtained raw data and the dimension table data in the static dimension table object, allowing all subtasks in the same task manager to share the same dimension table data, thereby saving physical memory resources.

The embodiment of the application provides a data query method, system, device, equipment and storage medium, and relates to the technical field of data processing. The specific implementation scheme is: receiving a first statistical query statement, determining a specified statistical range, a specified statistical method and a specified measurement indicated by the first statistical query statement; if there is no aggregate table containing to-be-queried data in each aggregate table stored in advance, determining whether there is a target aggregate table in each aggregate table, the statistical range of which belongs to the specified statistical range, the statistical measurement of which is the specified measurement, and the statistical method of which is the specified statistical method based on the hierarchical structure between each dimension table stored in advance; if there is, obtaining the measurement value of the specified measurement corresponding to each statistical range belonging to the specified statistical range from the target aggregate table based on the hierarchical structure, and obtaining a query result based on the obtained measurement value and the specified statistical method. It can be seen that, by the scheme, the query efficiency can be improved.

The invention discloses a data warehouse construction and application method based on multi-source data fusion, and relates to the technical field of data fusion, and the method comprises the steps: multi-source data integration layer construction: obtaining structured, unstructured and semi-structured data through a heterogeneous data collection module, carrying out the automatic preprocessing and standardization processing of the data, and obtaining a multi-source data integration layer; the standardization processing comprises a unified naming rule, a coding rule and a data type; designing a core model of the data warehouse: constructing a star model based on a business theme, the star model comprising at least one fact table and a plurality of associated dimension tables, the fact table storing business index data, and the dimension tables storing descriptive attribute data; implementing an ETL automatic process: configuring a data extraction and conversion rule and a scheduling strategy through a visual ETL tool, automatically loading the data processed by the integration layer to a star model, and monitoring a task state in real time; and intelligent query and application recommendation.

The invention discloses a method and a device for generating a commodity pool based on a new machine recommendation algorithm of big data. The method comprises the following steps: constructing a basic view through a Spark framework; performing popularity quantitative evaluation through a commodity popularity scoring model, and respectively screening out hot-sell commodities and new commodities; performing data matching through an SQL linked table query operation, and generating an initial candidate commodity set and a quantity target value of each type of commodities; calculating to obtain a comprehensive score through a multi-objective optimization model; partition-level commodity filtering and optimization are carried out through a MapReduce programming model, and an optimized commodity candidate pool is obtained; performing commodity compliance verification according to items of the white list, and based on a forbidden list in the dimension list black list, adding just-needed categories according to scene requirements through reverse connection operation to obtain a final commodity candidate pool; and merging the commodity data in the final commodity candidate pool into a hive table for persistent storage. The problems that in the prior art, consumption behavior recognition is lacked, and the requirement of a special scene cannot be met are solved.

The invention provides a three-phase asynchronous motor database construction method and system, and the method comprises the steps: obtaining the source data of a three-phase asynchronous motor, and the source data comprises the common parameters, geometric dimensions and performance parameters of the motor; analyzing a data dependency relationship among the common parameters, the geometric dimensions and the performance parameters to determine a determination condition of each data set; creating a plurality of correlative data tables based on the data dependency relationship, wherein the data tables comprise a geometric dimension table group for storing geometric dimensions of the motor and a performance parameter table for storing performance parameters of the motor; and constructing an entity-relationship model of the database based on the association relationship among the plurality of data tables, and generating an SQL statement for data query. According to the technical scheme provided by the invention, structured and standardized storage of the motor data is realized, data redundancy is remarkably reduced, data reusability and query efficiency are improved, and industrial digital application is effectively supported.

The invention provides a wide table construction method, a wide table construction system and related equipment, and relates to the wide table construction method and the wide table construction system. A user with a wide table construction demand only needs to input a wide table construction instruction carrying wide table construction related information (such as an identifier of a fact table, an identifier of a dimension table, an identifier of a column of a to-be-added wide table in the fact table, an identifier of a column of the to-be-added wide table in the dimension table and the like) to a wide table construction system; in this way, the wide table construction system can automatically achieve wide table construction according to the wide table construction instruction input by the user and provide the wide table for the user, the whole wide table construction process is simple and rapid, and the construction efficiency is high.

In various examples, table classification based query join reordering for relational database systems and applications are provided. In some embodiments, a relational database system is provided that includes a join optimizer that evaluates a join clause of a query and categorizes relational database tables as either fact tables or dimension tables based on a normalized cardinality statistic. The join optimizer uses the fact and dimension tables to deconstruct the query into a plurality of deconstructed query join trees. Individual deconstructed query join trees may be generated for each respective fact table. The deconstructed query join trees may be joined to generate a reordered join solution representing a sequential join of the plurality of deconstructed query join trees. An updated query may be generated based on the reordered join solution, and a query response generated that answers the query based at least on the updated query.

The application provides a metadata analysis processing method and device, and relates to the technical field of big data analysis processing.The method comprises the following steps: collecting metadata in a data source; acquiring a business process, declaring the granularity of a metadata model according to the business process, and constructing a dimension table and a fact table according to the granularity; taking the business process as a modeling driver, and establishing a metadata model according to the metadata, the dimension table and the fact table; when a data analysis request is acquired, performing data analysis by using the metadata model to obtain an analysis result.The metadata model is established by collecting metadata, the metadata of enterprises scattered in multiple application systems is centrally managed, a complete and comprehensive data structure relationship is presented for the enterprises, the enterprises are assisted in centrally and efficiently managing data assets, the metadata analysis processing of different business requirements can be met, data value can be mined from the analysis result to guide business or product innovation, and the production of enterprises is empowered.

The application discloses a protective shoe AI generation method based on physical process constraints and safety standard verification, relates to the technical field of product design automation, and comprises the following steps: assembling the topological relationship of the protective component and the material process constraint based on the scale conversion benchmark and the determinable rule table, binding the rules according to the functional partition, forming a parameterized constraint set, generating a shoe scheme according to the parameterized constraint set and the appearance intention, performing semantic segmentation, key point positioning and edge extraction measurement and verifying the rules, when the verification result indicates that the shoe scheme does not conform to the determinable rule table and the parameterized constraint set, sequentially performing local shape correction, local redrawing generation and discarding regeneration, and obtaining a final scheme, rechecking the final scheme, outputting a multi-view design drawing, a partition annotation drawing, a vectorized contour line, a key dimension table, a material list and a process specification, and performing encryption and decryption, clearing the plaintext and recording the access log.

This application discloses a method for constructing a data model, a method for data querying, an apparatus, and a storage medium, relating to the field of data processing technology. The method includes: obtaining field information from a dimensional model and at least two dimensions of a dimension table, where the field information indicates a mapping relationship between at least two dimensions; constructing a full dimensional relationship graph based on the field information, where the field information also indicates the association relationship between each of the at least two dimensions and a fact table; and attaching the fact table to at least one dimension in the full dimensional relationship graph based on the field information, generating a directed acyclic connected graph centered on the fact table as a data model, used to determine the path from the query metric to the fact table based on the query metric and its dimensions. This method can construct a data model based on existing data warehouse construction achievements, thereby determining the path of the query metric in the data warehouse through the data model.

The application relates to the fields of smart medical treatment, artificial intelligence and financial technology, and discloses a business data query method and device, a storage medium and electronic equipment. The method comprises the following steps: performing data extraction based on pre-acquired source data tables, obtaining each business table corresponding to each business node of a business process and each dimension table corresponding to the business process; constructing a mapping relationship between each field of each business table and each theme keyword of each dimension table; performing splitting processing on each business table to generate a plurality of sub-business tables corresponding to each business table, and storing each sub-business table; and querying each mapping relationship and each sub-business table based on a user's query request to obtain a query result. The method can improve query efficiency and save query cost.

The invention discloses a method, device and equipment for generating an SQL based on a natural language and a storage medium, and relates to the technical field of artificial intelligence and databases. The method comprises the following steps: firstly, establishing a knowledge graph to clarify an incidence relation between an index and a fact table and an incidence relation between the fact table and a dimension table, then identifying and locking a target fact table through a target index, carrying out correlation retrieval to form a candidate dimension table set, and analyzing and screening a step-by-step flow of the target dimension table through semantics and limitation conditions. The context input into the large language model is always a minimum complete set related to query, the problem of context window limitation caused by a full-scale table structure in a large-scale database is effectively avoided, meanwhile, interference of irrelevant table structure information is reduced fundamentally, retrieval errors caused by same-name fields are avoided, and the retrieval efficiency is improved. The dimensionality semantic ambiguity in the natural language instruction is accurately solved, so that the generated SQL statement can be accurately matched with the query intention of the user, and good performability is achieved.

The invention provides a laboratory data analysis method, and belongs to the technical field of data analysis, and the method comprises the steps: obtaining multi-modal original data of a laboratory based on a locked edge node, carrying out the preprocessing of the multi-modal original data, mapping the multiple preprocessing results into a preset multi-dimensional table, and carrying out the mapping of the multi-dimensional table according to a theme correlation variable; performing false analysis on the action data and the result data after each preprocessing to generate a false label set, and labeling; constructing a project reference matrix of the same experiment project and a step reference matrix of each experiment step, and combining the global feature vector and the local feature vector of each experiment step to determine abnormal type distribution and abnormal result distribution of abnormal operation of each experiment step, and converting the abnormal type distribution and the abnormal result distribution into abnormal vectors; and inputting into a pre-trained analysis model to obtain an equipment regulation and control suggestion, and performing instruction application. And the reliability of experimental data, the comprehensiveness of anomaly recognition and the effectiveness of equipment regulation and control are remarkably improved, so that the experimental efficiency and the result accuracy are improved.

The application discloses a method, system, device and medium for custom multi-dimensional analysis configuration based on a constellation model, and belongs to the technical field of data analysis.The technical problem to be solved by the application is that the construction of basic data such as dimensions, indexes and data models in multi-dimensional analysis is complicated, and multiple mappings such as a main dimension table, a role-playing dimension and a dictionary dimension are adopted.The technical scheme is as follows: the method is configured by importing a model table, and the dimensions, indexes, fact table and dimension table are associated, the related tables, attributes and model relationships are automatically imported and mapped between the dimension tables, the model maintenance cost and time are reduced, and the ordered model construction, the multi-fact table dimension screening machine fact table attribute reverse mapping application configuration scene are realized through the dimensions, indexes, fact table, dimension table, data model and custom multi-dimensional query, meanwhile, the required indexes are flexibly selected from multiple fact tables through a drag operation, and the common dimensions are automatically screened out and displayed according to the cascading mode of the indexes and the dimensions.

The invention relates to the technical field of data processing, and discloses a data association method, device and equipment, a medium and a program product, the method comprises the following steps: obtaining a target data stream based on a real-time calculation engine, and determining to-be-processed data according to the target data stream; splicing the to-be-processed data with data in other data streams obtained by the real-time calculation engine according to the primary key of the to-be-processed data to obtain associated data; under the condition that the blank field does not exist in the associated data, taking the associated data as target data; under the condition that blank fields exist in the associated data, filling the blank fields according to the data lake to obtain target data; the data lake is used for performing multi-stream data splicing on the target data stream and other data streams; and updating a preset target dimension table according to the target data. The visibility of the dimension table data can be improved from the minute level to the second level, and the timeliness of the dimension table is effectively guaranteed.

The application discloses a kind of port multi-source business data quality governance method and system, it is related to data processing technical field.The kind of port multi-source business data quality governance method and system, comprising the following steps: S1, obtain original governance data, and carry out time axis reposition, structure normalization and encapsulation processing;S2, extract dimension candidate record, evaluate the freezing credibility of candidate dimension version;S3, execute candidate version recall, build time state association candidate set, evaluate time state mismatch degree;S4, the release state of current release cycle is evaluated, and the release state of current result is identified.The application effectively improves consistency, stability and traceability in cross-system convergence, solves the problem that dimension table version change and fact record processing are not synchronized, which is prone to produce silent mismatch, mismatch result is difficult to block in time, and abnormal result is easy to spread pollution downstream.

The application relates to a rule engine establishment method based on attribute identification, which comprises the following steps: establishing a unique number for any entity related to system operation and forming a dimension table, recording attribute information and attribute identification fields of the entity; establishing a link table to allocate a unique number for each received order and storing the unique number; establishing a characteristic table based on the unique number, which is used for storing transaction records of the order, and entities related to the order are stored as entity numbers in the characteristic table; linking the corresponding dimension table through the entity number to obtain the attribute information and the attribute identification field of the entity; combining the corresponding attribute information to form input data according to requirements; combining the attribute identification fields to make rule parameters superimposed to form a new rule parameter group; and comparing the input data with the rule parameter group to obtain a reasoning result and storing the reasoning result. According to the application, the dimension table of each entity is established in advance, and the rule parameters are recorded in the dimension table, so that the rule parameters of the entity can be automatically matched when the entity is used, and the matching efficiency and accuracy are improved.

This application discloses a data query method, a visualization data processing method and apparatus, as well as an electronic device and a storage medium. The data query method includes: pre-configuring the update frequency of dimension table data; pre-configuring the association relationship between the fact table dataset and the dimension table dataset; pre-querying the dimension table data at the update frequency, and storing the pre-queried dimension table data using a predetermined data structure; querying the fact table data in response to the data query; and mapping the field values ​​in the queried fact table data to the associated field values ​​in the dimension table data based on the association relationship, to obtain the data query result.

A method, apparatus and device for adaptively associating historical data based on date dimensions, and a computer readable storage medium, the method comprising: modifying a constructed intelligent date comparison dimension table according to an automated process to generate a target intelligent date comparison dimension table, the target intelligent date comparison dimension table comprises storage dates and comparison mode configuration information, and the target intelligent date comparison dimension table is used for managing switching logic of comparison modes; when it is determined that historical data need to be associated, a current date is obtained, a preset BI report data model is operated, so that the preset BI report data model is associated with the target intelligent date comparison dimension table, a target date corresponding to the current date is determined from comparison mode configuration information in the target intelligent date comparison dimension table, and the target date is stored in the target intelligent date comparison dimension table. According to the method, historical data are associated, so that comparison dimension self-adaptive switching of festival and non-festival scenes is realized, and the technical problems of complexity and error proneness in manual parameter adjustment in the prior art are solved.

The application provides a medical insurance business index data storage and query method and system, acquires definition index classification, definition index modifier and definition index attribute, constructs an index model based on the acquired information, constructs an index storage model in a star model based on dimension modeling theory, acquires index calculation parameters corresponding to the index model, calculates and generates index data through matching calculation rules, updates the index data to a fact table, calculates the correlation between indexes through correlation analysis algorithms according to the indexes and index data, generates correlation relationship analysis results between the indexes, forms an index system, and extracts and queries data related to the instruction content from the fact table and corresponding data in the related dimension table according to the index model and the index system in response to a medical insurance business information query instruction. The application can make the number of data tables and the amount of single table data grow within a controllable range, and ensure clear data storage logic.

The application provides a multi-dimensional data report determination method and system. The method comprises the following steps: obtaining a target data set input by a user; the target data set comprises a target dimension table; based on the target dimension table, concatenating and caching instances related to the target dimension table to determine an initial data cache report; based on the target data set, determining a target query statement; and based on the initial data cache report, determining a target multi-dimensional data report according to the target query statement. The multi-dimensional data report can be generated without the participation of a program coding personnel, the communication cost of demand transmission between a business analysis personnel and the program coding personnel can be effectively reduced, and the efficiency of multi-dimensional data analysis is improved.