A modeling method, server and storage medium

Abstract

The embodiment of the present application provides a modeling method, a server and a storage medium, wherein the method includes: obtaining a requirement description language; performing word segmentation processing on the requirement description language to determine the indicator words of the matching indicator elements; the indicator elements include time periods, modifiers and atomic indicators; translating the indicator words of the matching indicator elements into standard words of the matching indicator elements; if there are no entered derived indicators for the standard words of the matching indicator elements, defining derived indicators for the standard words of the matching indicator elements, and automatically entering the derived indicators; outputting the derived indicators as the constructed indicator model; the derived indicators include standard words that describe time periods, modifiers and atomic indicators. Further, the embodiment of the present application also provides a solution for quickly constructing a dimensional model, as well as a solution for reversely generating a dimensional model. The embodiment of the present application can improve the efficiency of model construction, and the constructed model includes an indicator model; further, the constructed model also includes a dimensional model.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of data warehouses, and in particular to a modeling method, a server, and a storage medium. Background Art

[0002] Data warehouse, also known as data warehouse for short, is a strategic set of data support for all types of decision-making processes at all levels of an enterprise. Data warehouses have indicator models and data warehouse models. Modeling for data warehouses involves building indicator models and data warehouse models. As an important model in data warehouses, how to improve the efficiency of model building has always been a technical problem that technicians in this field need to solve. Summary of the invention

[0003] In view of this, an embodiment of the present application provides a modeling method, a server and a storage medium to improve the efficiency of model building.

[0004] To achieve the above objectives, the embodiments of the present application provide the following technical solutions.

[0005] In a first aspect, an embodiment of the present application provides a modeling method, including:

[0006] Obtain the requirement description language for building the indicator model;

[0007] Performing word segmentation processing on the demand description language to determine indicator words matching indicator elements from the demand description language; the indicator elements include time periods, modifiers and atomic indicators;

[0008] translating the indicator word of the matching indicator element into a standard word of the matching indicator element;

[0009] If there is no entered derived indicator for the standard word of the matching indicator element, define a derived indicator for the standard word of the matching indicator element and automatically enter the derived indicator; output the derived indicator as the constructed indicator model; the derived indicator includes standard words that describe time periods, modifiers and atomic indicators.

[0010] Furthermore, the modeling method of the first aspect also includes:

[0011] According to the target dimension, select the main table of the business system of the meta warehouse;

[0012] Filtering the core attributes of the dimensional model corresponding to the target dimension from the main table;

[0013] Generate a basic model of the dimensional model according to the meta information of the main table and the core attributes;

[0014] Selecting an associated dimension, redundantly adding core attributes of the associated dimension based on the basic model, and establishing an associated relationship between the basic model and the associated dimension;

[0015] Based on the association relationship and data source, data development and code generation are automatically performed on the dimensional model.

[0016] Furthermore, the modeling method of the first aspect also includes:

[0017] Load the data warehouse table;

[0018] According to at least one configured hierarchical domain division rule, the table name of the data warehouse table is parsed, and a naming expression corresponding to each hierarchical domain division rule is parsed from the table name to obtain at least one naming expression; wherein a hierarchical domain division rule defines a table naming method corresponding to a design of hierarchical domain division information;

[0019] Determine a target named expression from the at least one named expression, and use hierarchical domain information corresponding to the target named expression as target hierarchical domain information;

[0020] According to the derivation features, the type of the dimensional model corresponding to the data warehouse table is deduced; the data warehouse table is reversed into a model table corresponding to the type of dimensional model, and the model table is associated with the target hierarchical domain information.

[0021] In a second aspect, an embodiment of the present application provides a server comprising at least one memory and at least one processor, wherein the memory stores one or more computer executable instructions, and the processor calls the one or more computer executable instructions to execute the modeling method as described above.

[0022] In a third aspect, an embodiment of the present application provides a storage medium, wherein the storage medium stores one or more computer executable instructions, and wherein the one or more computer executable instructions, when executed, implement the modeling method as described above.

[0023] The modeling method provided in the embodiment of the present application can obtain the demand description language for constructing the indicator model, perform word segmentation processing on the demand description language to determine the indicator words of the matching indicator elements from the demand description language, and the indicator elements include time periods, modifiers and atomic indicators; translate the indicator words of the matching indicator elements into standard words of the matching indicator elements; if there are no entered derived indicators for the standard words of the matching indicator elements, a derived indicator can be defined for the standard words of the matching indicator elements, and the derived indicators can be automatically entered, and the derived indicators can be output as the constructed indicator model; wherein the derived indicators include standard words describing time periods, modifiers and atomic indicators. It can be seen that the embodiment of the present application can produce standard words of matching indicator elements that meet the OneData theory through automatic intelligent word segmentation and translation parsing processing of the unstructured demand description language provided by the demand personnel, and automatically define derived indicators for the standard words of the matching indicator elements, and automatically enter them into the indicator management platform, which reduces the threshold for the application of the OneData theory, and the modeling personnel do not need to manually enter and manage. The embodiment of the present application realizes automatic decomposition of the demand description language through automatic word segmentation and translation processing of the demand description language, and realizes automatic definition and automatic entry of derived indicators, thereby lowering the threshold for the application of OneData theory and improving the efficiency of building the indicator model.

[0024] Furthermore, the embodiments of the present application connect data with the meta warehouse, directly reference the main table and core attributes of the meta warehouse's business system, and associate redundant attributes of related dimensions, thereby solving the problem of reduced modeling efficiency caused by the need to manually design the model tables (fact tables and dimension tables), model attributes, and association relationships of the dimensional model; at the same time, it avoids the problem of major subsequent model changes caused by the lack of redundant dimensional attributes for reference in the dimensional model; further, based on data sources such as fields and the relationship between associated dimensions and redundant dimensional attributes, simple codes can be automatically generated to simplify code development and improve R&D efficiency.

[0025] Furthermore, the embodiments of the present application can perform table naming resolution on the data warehouse table based on the configured hierarchical division rules, thereby determining the target hierarchical division information of the data warehouse table, and then after the data warehouse table is reversed to the model table of the dimensional model, the model table is associated with the target hierarchical division information to achieve automatic association of the dimensional model and data warehouse planning; further, based on the derivation features, the type of dimensional model corresponding to the data warehouse table is deduced, thereby directly reversing the model table of the corresponding type of dimensional model that can be used, without the need for manual reversal and secondary processing to determine the model type, thereby improving the efficiency of model construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0027] Figure 1 This is an example diagram of the relationship between the indicator model and the data warehouse model.

[0028] Figure 2 This is an example diagram of the indicator model construction process.

[0029] Figure 3 A flowchart of a modeling method provided in an embodiment of the present application.

[0030] Figure 4 An example diagram of an implementation of an indicator model constructed in accordance with an embodiment of the present application.

[0031] Figure 5 This is an example diagram of a dimensional model.

[0032] Figure 6 An example diagram for building a dimensional model.

[0033] Figure 7 Another flow chart of the modeling method provided in an embodiment of the present application.

[0034] Fig. 8A This is an example diagram of the process of building a dimensional model provided in an embodiment of the present application.

[0035] Figure 8B An example diagram for constructing a dimensional model provided in an embodiment of the present application.

[0036] Fig. 9 Another flow chart of the modeling method provided in the embodiment of the present application.

[0037] Fig.10 A block diagram of a modeling device provided in an embodiment of the present application.

[0038] Fig.11 A block diagram of the server. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] The indicator model can be considered as an indicator used to query data in the data warehouse model. Figure 1 The following is an example diagram showing the relationship between the index model of the data warehouse and the data warehouse model. Figure 1 As shown, the data warehouse may include an indicator model 110 and a data warehouse model 120. Among them, the indicator model 110 may be composed of indicators for data query. The data warehouse model 120 may be specifically used to store data; in some embodiments, the data warehouse model 120 may be a dimensional model, which may be constructed by dimensional modeling, and is mainly composed of basic elements such as fact tables and dimension tables. The indicators in the indicator model 110 may be associated with the fields in the data warehouse model 120, so that when querying data from the data warehouse model using the indicators, the corresponding data may be found from the fields associated with the indicators.

[0041] Modelers usually build indicator models based on the demand description language of demanders. For example, during the indicator development process, when the demander wants the modeler to develop an indicator that meets a certain requirement, the demander can provide a demand description language, and the modeler needs to convert the demand description language into indicators that can be used for data query and enter it into the indicator management platform. In an example, when the demander provides the demand description language of "querying the daily online sales of city A", the modeler needs to decompose the demand description language into indicators that can be used for data query, such as city A, daily, online, and sales, which can be used for data query, to achieve association with the fields in the data warehouse model. However, in the process of building the indicator model, it is not possible to achieve standardization of indicator definitions through arbitrary text descriptions. Therefore, it is particularly necessary to guide modelers to decompose the demand description language into indicators and implement indicator specification and standardized definitions.

[0042] OneData theory can meet the needs of indicator specification and standardized definition, and guide modelers to disassemble the demand description language. OneData theory breaks down indicators into indicator elements such as atomic indicators, time periods, and modifiers, and defines derived indicators through these indicator elements. In other words, in OneData theory, derived indicators = atomic indicators + time periods + modifiers.

[0043] It should be noted that atomic indicators are used to clarify the statistical caliber and calculation logic of the business. They are created based on the user's business activities and can be used to count the values ​​of a certain business status in the business activities. They are business processes and metrics that cannot be disassembled any further. For example, if the user's business process is "placing an order" and the metric is "payment amount", then the atomic indicator can be defined as "order payment amount";

[0044] A business process refers to the business activities performed by an enterprise in a specified business scenario. It is the logical subject that needs to be analyzed in data modeling. A business process will precipitate a business detail table in the big data system. For example, in a transaction scenario, there may be business processes such as adding to a shopping cart, placing an order, and paying, which respectively produce business flow detail tables and data for adding to a shopping cart, placing an order, and paying.

[0045] Metrics refer to specific numerical fields in the business details table produced by a business process. For example, the "order" business process produces an "order flow sheet", and the "payment amount" field in it is a metric.

[0046] The time period is used to define the time range or time window for data statistics, such as 1 day, 1 week, etc. The clock period is used to limit the time range of derived indicators when calculating derived indicators.

[0047] Modifiers are used to limit the business scope of derived indicators. For example, if you are counting "the sales of fresh food stores in city A in one day", then city A and fresh food stores are the limiting modifiers of the business scope.

[0048] Derived indicators are composed of atomic indicators, time periods, and modifiers, and are used to reflect the business status of a business activity of an enterprise in a specified time period and business scope. For example, if the derived indicator is: 1 day_City A_Sales, then 1 day corresponds to the time period, City A corresponds to the modifier, and Sales corresponds to the atomic indicator.

[0049] It should be further explained that atomic indicators define the basic calculation caliber. The time dimension and business scope of different business analyses are different. Therefore, the two elements of time dimension and business scope are structured into time periods and modifiers to limit the scope of the results of atomic indicator calculations. Under the underlying consistent calculation caliber, different derived indicators can be combined through different business and time scope requirements to meet the changing needs of the business.

[0050] In one example, the demand description language used by the demand personnel to construct an indicator is "query the online sales volume of city A every day or every month". Then the indicator words corresponding to the following indicator elements can be extracted from the demand description language: time period - every day, every month, modifier - city A, online, atomic indicator - sales volume. Based on the indicator words corresponding to these indicator elements, multiple derived indicators can be defined, such as: daily_online_sales volume, monthly_online_sales volume, daily_online and city A_sales volume, monthly_online and city A_sales volume, so as to develop and construct multiple derived indicators based on the demand description language of the demand personnel to realize the construction of the indicator model.

[0051] It can be seen that when constructing an indicator model in the form of derived indicators, it involves decomposing the requirement description language into indicator words corresponding to structured indicator elements (such as indicator words corresponding to indicator elements such as atomic indicators, time periods, and modifiers), and defining derived indicators by the indicator words corresponding to the indicator elements. This is a big challenge for modelers. In other words, under the standardized design and architecture of the OneData theory, the requirement description language provided by the requirement personnel is structured and then manually entered and managed. This poses relatively serious regulatory constraints to modelers such as business and R&D personnel. As an example, Figure 2 The following is an example diagram showing the construction process of the indicator model. Figure 2 As shown, after the demand personnel provide the demand description language of the indicator model, the modeler faces the problem of how to disassemble the demand description language based on the OneData theory to obtain the indicator words corresponding to the indicator elements such as time period, modifiers, and atomic indicators. This requires the modeler to be proficient in the OneData theory, resulting in a certain application threshold of the OneData theory. Based on the indicator words corresponding to the obtained indicator elements, if the indicator words corresponding to the indicator elements already have pre-entered derived indicators, the pre-entered derived indicators can be used as the indicator model for demand delivery; if the indicator words corresponding to the indicator elements do not have pre-entered derived indicators, the modeler needs to define the derived indicators composed of the indicator words corresponding to the indicator elements and enter them manually to develop the derived indicators, which involves manual operations by the modeler and has the problem of low efficiency. It should be noted that after obtaining the indicator words corresponding to the indicator elements, the structured management of the indicator elements based on the OneData theory can facilitate the management of each indicator element and subsequent data search.

[0052] It can be seen that under the specification of OneData theory, how to automatically and efficiently decompose the demand description language into indicator words corresponding to indicator elements, and realize the automatic entry of derived indicators, is a problem that needs to be solved to lower the application threshold of OneData theory and improve the efficiency of indicator model construction. Based on this, the embodiment of the present application performs intelligent and automatic parsing of the demand description language on the premise of meeting the OneData theoretical specifications and not affecting the R&D efficiency, so as to automatically decompose the demand description language into standard words corresponding to indicator elements, and realize the automatic entry of standard words corresponding to indicator elements and corresponding derived indicators into the indicator management platform, thereby improving the efficiency of indicator model construction.

[0053] Figure 3 The optional flow chart of the modeling method provided in the embodiment of the present application is shown as an example. Figure 3 The method flow shown in the embodiment of the present application can realize automatic and efficient construction of an indicator model based on derived indicators. Figure 3The method flow shown can be implemented by a server. Optionally, the server can be a big data computing engine server based on cloud computing. For example, the method flow is implemented by a data middle platform server in a big data computing engine. The data middle platform can extract data from various businesses, unify standards and calibers, and provide data services through data calculation and processing. Figure 3 The method flow may include the following steps.

[0054] In step S310, a requirement description language for constructing an indicator model is obtained.

[0055] When the demand personnel want to develop an indicator model that meets a certain demand, they can provide a demand description language for constructing the indicator model. The demand description language can be input into the server. For example, the demand personnel can use their terminal device to submit the demand description language for constructing the indicator model to the server. The demand description language can be based on the needs of business development. The demand description language can be a text language that describes the demand content for constructing the indicator model. For example, when the demand personnel have the indicator demand to develop "query the sales of fresh goods in the store for 1 day", they can submit the demand description language of "query the sales of fresh goods in the store for 1 day" to the server through the terminal device. The content of the demand description language can be organized by the demand personnel themselves, and the embodiments of the present application are not limited.

[0056] In step S311, the requirement description language is segmented to determine indicator words matching indicator elements in the requirement description language.

[0057] After acquiring the demand description language, the server may perform word segmentation processing on the demand description language, thereby determining the indicator words matching the indicator elements from the demand description language. In some embodiments, the indicator elements include time periods, modifiers, and atomic indicators, and the server may perform word segmentation processing on the demand description language, thereby determining the indicator words matching the time periods, the indicator words matching the modifiers, and the indicator words matching the atomic indicators from the demand description language.

[0058] In some embodiments, the server may use natural language recognition platforms such as NLP (Natural Language Processing) to perform intelligent word segmentation on the demand description language. As an optional implementation, the embodiment of the present application may perform word segmentation processing on the demand description language based on the candidate words corresponding to the preset indicator elements using the NLP platform to determine the indicator words that match the candidate words corresponding to the indicator elements from the demand description language. For example, based on the candidate words corresponding to the preset time period, the candidate words corresponding to the modifiers, and the candidate words corresponding to the atomic indicators, the demand description language is atomically segmented using the NLP platform, thereby determining the indicator words that match the candidate words of the time period, the indicator words that match the candidate words of the modifiers, and the indicator words that match the candidate words of the atomic indicators from the demand description language. Optionally, an embodiment of the present application may pre-enter candidate words corresponding to indicator elements on the server's modeling platform (for example, pre-enter candidate words corresponding to time periods, candidate words corresponding to modifiers, and candidate words corresponding to atomic indicators). Then, when using the NLP platform to perform word segmentation on the requirement description language, the NLP platform may perform atomic word segmentation on the requirement description language based on the pre-entered candidate words corresponding to the indicator elements, and determine the indicator words that match the indicator elements from the requirement description language.

[0059] In some embodiments, the candidate words corresponding to the indicator element may represent the words that the indicator element may use, for example, the words that may be used in the time period, the words that may be used in the modifier, the words that may be used in the atomic indicator, etc. Based on the preset words that may be used by each indicator element (i.e., the candidate words corresponding to each indicator element), the embodiment of the present application can filter the words that are irrelevant to the indicator element in the demand description language through intelligent word segmentation technology, and separate the indicator words that match the candidate words of the indicator element in the demand description language, thereby determining the indicator words that match the indicator element from the demand description language. For example, the demand description language input by the demand personnel is "I want to see the sales of fresh food for one day", then based on the preset candidate words of the time period, the candidate words of the modifier, and the candidate words of the atomic indicator, the words that are irrelevant to the indicator elements such as the time period, the modifier, and the atomic indicator (such as filtering the words "I want to see") can be filtered out from the demand description language, and the indicator word "1 day" matching the time period, the indicator word "fresh food" matching the modifier, and the indicator word "sales" matching the atomic indicator are segmented.

[0060] It should be noted that word segmentation is a function in data search and language analysis. Word segmentation can be based on a natural language word library and a customized word library in the industry, and a section of language is atomically cut and segmented, and the words corresponding to each part of speech are output. For example, for the language "1 day A city sales", the reasonable word segmentation result is: "1 day" is the time period, "A city" is a modifier, and "sales" is an atomic indicator; atomic cutting mainly matches the coherent words in the language with the candidate words corresponding to the time period, modifiers, and atomic indicators in the word library, and then cuts them out. In some further embodiments, the candidate words corresponding to the preset indicator elements can be recorded in the indicator word library (for example, the indicator word library can record the candidate words corresponding to the time period, the candidate words corresponding to the modifiers, the candidate words corresponding to the atomic indicators, etc.), and the indicator word library can be located in the intelligent word library. As an optional implementation, the embodiment of the present application can be based on the candidate words corresponding to the indicator elements recorded in the indicator word library, and the NLP platform can be used to perform word segmentation on the demand description language to obtain the indicator words matching the indicator elements from the demand description voice.

[0061] As an optional implementation, the candidate words corresponding to the indicator elements recorded in the indicator vocabulary can be determined based on the industry vocabulary of multiple industries. In one example, the candidate words corresponding to the indicator elements can be pre-entered into the indicator vocabulary by the modeler, and / or extracted from the industry vocabulary of different industries. For example, when the modeler enters the candidate words corresponding to each indicator element in the indicator vocabulary, he can refer to the industry vocabulary of e-commerce, logistics, new retail and other industries to determine the candidate words belonging to atomic indicators, candidate words belonging to time periods, candidate words belonging to modifiers, etc. For another example, the embodiment of the present application can support the automatic extraction of candidate words corresponding to each indicator element such as time period, modifiers, atomic indicators, etc. from the industry vocabulary of e-commerce, logistics, new retail and other industries.

[0062] In step S312, the indicator word of the matching indicator element is translated into a standard word of the matching indicator element.

[0063] After the demand description language is segmented and the indicator words of the matching indicator elements are determined from the demand description language, since the indicator elements have a normative constraint on the data research and development to build table fields, the final wording of the indicator elements should be standardized, that is, the wording of the indicator elements used to define the derived indicators should be standard words. Thus, the embodiment of the present application can translate the indicator words that do not meet the standard words in the indicator words of the matching indicator elements into standard words, so that the words of the matching indicator elements are all standard words, thereby obtaining the standard words of the matching indicator elements. In an example, assuming that data research and development hopes that the goods should be uniformly translated into sku, without including words such as item and shangpin, if the indicator word of a certain indicator element expressing the goods is shangpin, it needs to be translated into the standard word sku of the goods. The translation referred to in the embodiment of the present application can be understood as a unified automatic translation of the words of the indicator elements such as time period, modifiers, and atomic indicators, and an English code is generated to assist data research and development in naming table fields based on indicator elements.

[0064] In some embodiments, the present application embodiment may use a translation platform to translate the indicator word of the matching indicator element into a standard word of the matching indicator element. Optionally, the present application embodiment may use a translation platform to determine an indicator word that has the same attribute as the standard word but is not a standard word from the indicator words of the matching indicator element based on the attribute of the standard word of the indicator element, and replace the determined indicator word with the standard word to obtain the standard word of the matching indicator element.

[0065] It should be noted that translation is the translation of the input source language into the target language, which includes natural language translation and industry-specific word library translation, including abbreviations, etc. Based on this, as an optional implementation, the embodiment of the present application can define the standard words of each indicator element in the intelligent word library (for example, the standard words of the time period, the standard words of the modifiers, the standard words of the atomic indicators, etc., defined in the intelligent word library), and the attributes of each standard word (for example, the part of speech, the meaning of each standard word, etc.), so as to use the translation platform, based on the attributes of the standard words of the indicator elements defined in the intelligent word library, from the indicator words of the matching indicator elements, determine the indicator words with the same attributes as the standard words but not the standard words, and replace the determined indicator words with the standard words, so as to achieve the translation of the indicator words that do not meet the standard words into standard words, thereby avoiding the repeated definition of the final words of the indicator elements (for example, there are repeated synonyms, synonyms, etc. in the final words of the indicator elements), so that the final words of the indicator elements can be standardized.

[0066] Based on step S311 and step S312, the embodiment of the present application can determine the standard words that match the indicator elements from the demand description language by performing intelligent segmentation and intelligent translation processing on the demand description language, thereby realizing the extraction of indicator elements with standard wording from the demand description language. In one example, taking the demand description language as "I want to see the sales of fresh food for 1 day" as an example, after the demand description language is intelligently segmented, indicator words that match the indicator elements of time period-1 day, modifier-fresh food, and atomic indicator-sales can be obtained. These indicator words are matched with standard words in the intelligent word library (for example, the indicator words are replaced with standard words with the same attributes such as meaning and part of speech in the intelligent word library), so as to standardize and reorganize the segmentation results (i.e., the indicator words that match the indicator elements) to form a standard indicator description (i.e., the standard words that match the indicator elements).

[0067] In step S313, it is determined whether the standard word of the matching indicator element has an entered derived indicator. If not, step S314 is executed. If so, step S315 is executed.

[0068] In step S314, a derived indicator is defined for the standard word of the matching indicator element to create a derived indicator; and the derived indicator is automatically entered.

[0069] In step S315, the derived indicator is output as the constructed indicator model.

[0070] In some embodiments, the indicator management platform may pre-enter derived indicators, and the pre-entered derived indicators may be composed of different standard words of different indicator elements. After obtaining the standard words of the matching indicator elements, the server may determine whether the standard words of the matching indicator elements have entered derived indicators in the indicator management platform; if so, the derived indicators entered by the indicator management platform may be output as a constructed indicator model to deliver the requirements; if not, the server may automatically define the derived indicators composed of the standard words of the matching indicator elements to create derived indicators, thereby automatically entering the derived indicators into the indicator management platform, and then outputting the derived indicators as a constructed indicator model to deliver the requirements.

[0071] In some embodiments, the derived indicators include standard words that describe time periods, modifiers, and atomic indicators. The embodiment of the present application can determine whether there are already entered derived indicators for the standard words that match the time period, the standard words that match the modifiers, and the standard words that match the atomic indicators; if so, the derived indicators that have been entered by the indicator management platform can be output as the constructed indicator model to deliver the demand; if not, the server can automatically define derived indicators for the standard words that match the time period, the standard words that match the modifiers, and the standard words that match the atomic indicators, thereby automatically entering the defined derived indicators into the indicator management platform, and outputting the derived indicators as the constructed indicator model to deliver the demand.

[0072] In an example, assuming that the demand description language is "I want to see the sales of fresh food in one day", the standard words of the matching indicator elements extracted from the demand description language are: time period - 1 day, modifier - fresh food, atomic indicator - sales, then the derived indicator formed by the standard words of the matching indicator element is: 1 day - fresh food - sales; the server can determine whether there is a derived indicator of "1 day - fresh food - sales" that has been entered on the indicator management platform; if so, output the derived indicator to deliver the demand; if not, define the derived indicator of "1 day - fresh food - sales" for the indicator elements of time period - 1 day, modifier - fresh food, atomic indicator - sales, and automatically enter it into the indicator management platform, and at the same time, output the derived indicator to deliver the demand.

[0073] The model building method provided in the embodiment of the present application can obtain the demand description language for building the indicator model, perform word segmentation processing on the demand description language to determine the indicator words of the matching indicator elements from the demand description language, and the indicator elements include time periods, modifiers and atomic indicators; translate the indicator words of the matching indicator elements into standard words of the matching indicator elements; if there are no entered derived indicators for the standard words of the matching indicator elements, a derived indicator can be defined for the standard words of the matching indicator elements, and the derived indicators can be automatically entered, and the derived indicators can be output as the constructed indicator model; wherein the derived indicators include standard words describing time periods, modifiers and atomic indicators. It can be seen that the embodiment of the present application can produce standard words of matching indicator elements that meet the OneData theory through automatic intelligent word segmentation and translation parsing processing of the unstructured demand description language provided by the demand personnel, and automatically define derived indicators for the standard words of the matching indicator elements, and automatically enter them into the indicator management platform, which reduces the threshold for the application of the OneData theory, and the modeling personnel do not need to manually enter the management. The embodiment of the present application realizes automatic decomposition of the demand description language through automatic word segmentation and translation processing of the demand description language, and realizes automatic definition and automatic entry of derived indicators, thereby lowering the threshold for the application of OneData theory and improving the efficiency of building the indicator model.

[0074] In some embodiments, Figure 4 The following is an exemplary diagram showing an implementation example of constructing an indicator model in an embodiment of the present application. Figure 4 As shown, for the unstructured demand description language provided by the demand personnel, the server can utilize the indicator intelligent parsing engine based on the OneData theory, and use the intelligent vocabulary to perform intelligent word segmentation and intelligent translation processing on the demand description language, so as to obtain standard words that match the indicator elements from the demand description language; it can be understood that the obtained standard words for the matching indicator elements are in line with the OneData theory, and are standard descriptions of each indicator element after synonyms and similar words are removed based on the standard words. Among them, intelligent word segmentation is utilized in the NLP platform, and intelligent translation is utilized in the translation platform; specifically, the indicator vocabulary in the intelligent vocabulary can be used for the intelligent word segmentation process, and the candidate words corresponding to each indicator element recorded in the indicator vocabulary can be obtained based on the industry vocabulary of each industry; the intelligent vocabulary further records the standard words of each indicator element (such as standard Chinese nouns, English nouns, English abbreviations, etc.), the part of speech of each standard word (the part of speech can express the word type of the standard word, such as expressing time periods, modifiers, indicator elements of atomic indicator types, and can also express the standard words as industry terms, daily words, natural languages, etc.), and the meaning of each standard word (such as synonyms, similar words, associated words, etc.). The standard words, parts of speech and meanings of the standard words recorded in the intelligent vocabulary can be used in the intelligent translation process. Combined with Figure 4 As shown, for example, the demand personnel provide demand description languages ​​such as "1-day DAU (Daily Active User) of fresh food stores" and "1-day GMV (Gross Merchandise Volume) of fresh food stores". After intelligent word segmentation and intelligent translation processing, the standard word "1 day" matching the time period, the standard word "fresh food store" matching the modifier, the standard word "DAU" and "GMV" matching the atomic indicator, and other standard words matching indicator elements can be obtained. Therefore, the server can automatically define derived indicators such as "1 day_fresh food store_DAU" and "1 day_fresh food store_GMV" and automatically enter them into the indicator management platform of the server, so that an indicator forest formed by multiple derived indicators is precipitated on the indicator management platform to realize a structured, standardized, reusable and extensible indicator model.

[0075] In some further embodiments, in combination Figure 4As shown, the embodiment of the present application can also synchronize the candidate words of the indicator elements to the NLP platform in real time, and perform semantic offline training on the standard words of the indicator elements to the translation platform, so as to improve the accuracy of the parsing process of intelligent word segmentation and intelligent translation. As an optional implementation, for modelers, modelers can enter candidate words of indicator elements such as time period, modifiers, atomic indicators, etc. on the modeling platform (the candidate words of the entered indicator elements can be derived from the modeling process of the dimensional model, the root management of the indicator, etc.), and the modeling platform can call the interface of the NLP platform to synchronize the candidate words of the entered indicator elements to the NLP platform, so that the NLP platform can synchronize the candidate words of the indicator elements entered by the modeler in real time, so as to perform atomic word segmentation operations on the demand description language and improve the accuracy of intelligent word segmentation. When performing offline semantic training on the standard words of indicator elements on the translation platform, the embodiment of the present application can utilize the development platform of the data middle platform, integrate the PI algorithm platform (a large real-time database platform) into the development platform, and then use the PI algorithm platform to perform offline semantic optimization calculations on the standard words of the indicator elements (such as removing synonyms, correcting wrong words, etc.), and then synchronize the semantic offline training results to the translation platform.

[0076] The advantages of the indicator model construction solution provided in the embodiment of the present application are explained below in combination with the solution of manually entering indicators and the solution of semi-automatic identification of indicators.

[0077] The manual entry of indicators requires modelers to manually analyze and decompose the demand description language based on their understanding of the business and OneData theory, and then manually enter it into the indicator management platform. The disadvantage is that the efficiency is slow and indicators may be repeatedly defined (such as repeated definitions of synonyms and synonyms). Although indicators are expected to be standardized, the error rate and repeated definition rate of manual combing will increase as the business expands, which requires a lot of manpower and management costs, affecting R&D efficiency and extending the demand delivery time.

[0078] The semi-automatic identification indicator solution uses a non-natural language word segmentation and recognition solution. For example, it sets a pre-entry constraint on the word position of the indicator element according to a specific separator, such as the format of "time period_modifier_atomic indicator". This places high demands on the input personnel, who must have a good understanding of the business and OneData theory before they can disassemble and reorganize the demand description language in advance. In addition, it cannot solve the problem of repeated definitions of synonyms and antonyms.

[0079] The solution provided in the embodiment of the present application automatically processes the demand description voice provided by the demand personnel through intelligent word segmentation and intelligent translation based on OneData theory, which solves the requirement that modelers need to have an in-depth understanding of the business and OneData theory, and lowers the threshold for the application of OneData theory. At the same time, a unified indicator management platform is provided to automatically enter and manage indicator elements, solving the problem of reduced R&D efficiency due to manual entry, while also reducing the cost of manpower maintenance. Furthermore, the embodiment of the present application can perform real-time synchronization and offline semantic training on the NLP platform and the translation platform, improving the parsing accuracy of intelligent word segmentation and intelligent translation, and further realizing the structured and standardized management of indicators.

[0080] The embodiment of the present application performs intelligent analysis of indicators based on the OneData theory, and can rely on technologies such as intelligent real-time word segmentation, intelligent real-time translation, natural language training, and an indicator management platform. By combining the OneData theory with word segmentation, translation, and natural language training, it is possible to automatically decompose the demand description language into standard words corresponding to the indicator elements, and automatically enter the standard words corresponding to the indicator elements and the corresponding derived indicators into the indicator management platform, thereby improving the efficiency of building the indicator model.

[0081] In some further embodiments, the embodiments of the present application can query data in the data warehouse model of the data warehouse based on the derived index. As an optional implementation, the embodiments of the present application can be applied to the data query background of big data and massive data. In the case of big data and massive data, based on the derived index constructed by the embodiments of the present application, accurate data query matching under big data and massive data is achieved.

[0082] In some embodiments, the data warehouse model used to query data in the data warehouse can be a dimensional model, and the dimensional model can be constructed using a dimensional modeling method. Dimensional modeling is a data modeling method in data warehouse construction, and is a logical design method for structuring data into a dimensional model. The dimensional model is mainly composed of basic elements such as fact tables and dimension tables, where measurements are called "facts" and environmental descriptions are called "dimensions."

[0083] A dimension table is a detailed description of a dimension, which is used to reflect a type of business attribute. Optionally, a dimension table can correspond to a detailed description of a dimension. Dimensions are measurement environments (e.g., diverse environments for analyzing facts). Dimensions can represent the angles of statistics and analysis of data. Optionally, a dimension can belong to a data domain, such as a geographic dimension (including content at the country, region, province, and city levels), a time dimension (including content at the year, quarter, month, week, and day levels), etc. In an example, when analyzing transaction data, analysis can be performed using multiple dimensions such as geography, commodity, and time, and a detailed description of each dimension can be carried by a dimension table.

[0084] The fact table is the basic table of the dimensional model. A data warehouse can contain one or more fact tables to record the full amount of data information. In an example, a fact table may contain business sales data, such as the sales volume of a product.

[0085] The combination of dimension tables and fact tables can realize the analysis and query of data content from multiple dimensions. For example, a fact table can store multiple dimension labels and full data information (such as business sales data). Multiple dimension tables can carry multiple dimensions of measurement data. Multiple dimension tables can be associated with multiple dimension labels in the fact table. The fact table can be combined with multiple dimension tables to query data, realizing data query and analysis from multiple dimensions. As an optional implementation, Figure 5 An example diagram of a dimensional model is shown as an example. Figure 5 As shown, the dimensional model may include a fact table 510, and multiple dimension tables 521 to 52n, where n represents the specific number of dimensions, which may be determined according to actual conditions; the fact table 510 may include multiple dimension tags 511 to 51n, and full information of the data; one dimension tag corresponds to one dimension table, for example, dimension tag 511 corresponds to dimension table 521, and so on, until dimension tag 51n corresponds to dimension table 52n. Thus, through the detailed description of multiple dimension tables 521 to 52n in multiple dimensions, and the full information of the data recorded in the fact table 510, data query in multiple dimensions can be realized in the dimensional model.

[0086] In an example, multiple dimension tables may be, for example, a geographic dimension table and a time dimension table. The geographic dimension table may provide a detailed description of geography (country, region, province, city, etc.); the time dimension table may provide a detailed description of the time range; the fact table may store geographic tags, time tags, and business sales data; thus, after the geographic tags of the fact table are associated with the corresponding geographic dimension table and the time tags are associated with the corresponding time dimension table, data queries of the geographic and time dimensions may be performed in the dimensional model.

[0087] It can be seen that the key to building a dimensional model is to construct the tables that make up the dimensional model (such as dimension tables and fact tables) and the relationships between them. Figure 6 An example diagram of constructing a dimensional model is shown as follows: Figure 6 As shown, building a dimensional model includes the following stages: dimension table design stage 610, fact table design stage 620, relationship design stage 630, and data development and code generation stage 640. Among them, the dimension table design stage 610 is mainly used to design the dimension table of the dimensional model, and this stage is completed manually by the developer; the fact table design stage 620 is mainly used to design the fact table of the dimensional model, and this stage is completed manually by the developer; the relationship design stage 630 is mainly used to design the relationship between the fact table and the dimension table and the relationship between the dimension table and the dimension table, for example, after designing a transaction table and a member table, the association relationship between the transaction table and the member table is designed, such as associating the transaction table and the member table through the member ID, and this stage is completed manually by the developer; the data development and code generation stage 640 is mainly used to make the model table (dimension table and fact table) effective in the big data engine, and develop data and generate code at the same time, and this stage is completed manually by the developer. The construction process of the above-mentioned dimensional model relies on the manual work of developers. The modeling process is relatively cumbersome and the model construction efficiency is relatively low. Therefore, the embodiment of the present application provides a new dimensional model construction solution, which is based on the meta warehouse (a unified metadata warehouse that records core information such as tables, fields, lineages, and dependencies) to abstractly form a set of efficient dimensional model modeling processes, while supporting data development and generating simple codes.

[0088] As an optional implementation, Figure 7 Another optional flow chart of the modeling method provided by the embodiment of the present application is shown as an example. The method flow can be implemented by the server, such as Figure 7 As shown, the method flow may include the following steps.

[0089] In step S710, the main table of the business system of the meta warehouse is selected according to the target dimension.

[0090] The target dimension is the dimension of the dimensional model that needs to be constructed in the embodiment of the present application. The target dimension can be selected according to business needs, and the embodiment of the present application is not limited. Based on the target dimension of the dimensional model that needs to be constructed, the embodiment of the present application can select the corresponding master table from the business system of Yuancang. For example, if the embodiment of the present application needs to construct a store commodity dimensional model, then there is a master table of a store commodity table in the business system of Yuancang, and the prototype of the store commodity dimensional table in the store commodity dimensional model can come from the master table of this store commodity table in the business system of Yuancang. In an example, assuming that the dimensional model dim_shop_sku of the store commodity needs to be constructed, the embodiment of the present application can select the master table ods_shop_sku from the business system of Yuancang.

[0091] In step S711, core attributes of the dimensional model corresponding to the target dimension are screened from the main table.

[0092] In some embodiments, the embodiments of the present application mainly screen the core attributes of the dimensional model from the master table based on the target dimension of the dimensional model to be constructed, that is, for the target dimension of the dimensional model, the embodiments of the present application can screen out the core attributes of the dimensional model corresponding to the target dimension from the master table. For example, if a dimensional model of store merchandise needs to be constructed, the core attributes of the store merchandise can be screened out from the master table. In some embodiments, based on the fact that the dimensional model includes a dimension table and a fact table, the embodiments of the present application can screen out the core attributes of the dimension table in the master table corresponding to the target dimension, or the core attributes of the fact table. As an optional implementation, the core attributes of the dimension table may include the dimension primary key and dimension attributes of the dimension table, and the core attributes of the fact table may include granularity, redundant attributes, and metrics.

[0093] In step S712, a basic model of a dimensional model is generated according to the meta information of the main table and the core attributes.

[0094] In some embodiments, based on the selected master table, the embodiments of the present application may use the meta information and core attributes of the master table to generate a basic model of the dimensional model. Optionally, the basic model of the dimensional model may include basic field attributes, field types, and description information; wherein the basic field attributes, types, and description information may be determined from the core attributes and meta information, and in other possible implementations, the description information may be directly inherited from the master table of the business system of the meta warehouse.

[0095] In step S713, an associated dimension is selected, core attributes of the associated dimension are redundant on the basis of the basic model, and an associated relationship between the basic model and the associated dimension is established.

[0096] After generating the base model of the dimensional model, the embodiment of the present application may, based on the base model, redundant core attributes of other associated dimensions that need to be associated, and establish an association relationship between the base model and the associated dimensions; for example, based on the base model, redundant core attributes of other dimension tables that need to be associated, and establish an association relationship with the base model. Optionally, the embodiment of the present application may select a dimension table of an associated dimension based on the ID of the associated dimension, redundant core attributes of the dimension table of the associated dimension based on the base model, and establish an association relationship between the base model and the associated dimension, and the associated dimension may be at least one dimension related to the target dimension.

[0097] As an optional implementation, when constructing a store product dimension model, the dimensions related to the store product dimension may be categories and brands, so that the embodiment of the present application may select categories and brands as associated dimensions. After generating the basic model of the store product dimension model, the embodiment of the present application may, based on the basic model, redundant core attributes of the associated dimensions of categories and brands, and establish an associated relationship with the basic model. In one example, the embodiment of the present application may select the associated dimensions as category dimensions and brand dimensions, thereby selecting a category dimension table based on the category ID, redundant core category attributes (such as category name, category level, etc.) of the category dimension table based on the basic model, selecting a brand dimension table based on the brand ID, redundant core brand attributes of the brand dimension table based on the basic model, and establishing an associated relationship between the basic model and the categories and brands. Optionally, on the basis of the basic model, one implementation method of redundant core attributes of associated dimensions may be to directly load the core attributes of the associated dimensions in the basic model, for example, directly loading the core category attributes into the redundant attributes of the store product dimension table of the dimensional model.

[0098] In step S714, data development and code generation are automatically performed on the dimensional model based on the association relationship and data source.

[0099] After constructing the basic model of the dimensional model, redundantly associating the core attributes of the dimensions, and establishing the associative relationship between the basic model and the associated dimensions, the embodiment of the present application can automatically develop the data of the dimensional model and generate the code based on the associative relationship and the data source of the dimensional model to realize the automatic construction of the dimensional model. It should be noted that, unlike the code of the dimensional model which is manually developed, the implementation of the present application can realize the automatic generation of the simple code of the dimensional model based on the associative relationship between the basic model and the associated dimensions, thereby saving manual development work, and the developer can directly modify the simple code to obtain the final code of the dimensional model.

[0100] The embodiment of the present application connects data with the meta warehouse, directly references the main table and core attributes of the meta warehouse's business system, and associates the redundant attributes of the associated dimensions, thereby solving the problem of reduced modeling efficiency caused by the need to manually design the model tables (fact tables and dimension tables), model attributes, and associated relationships of the dimensional model; at the same time, it avoids the problem of large subsequent model changes caused by the lack of redundant dimensional attributes for reference in the dimensional model. Furthermore, based on data sources such as fields and the relationship between associated dimensions and redundant dimensional attributes, simple codes can be automatically generated to simplify code development and improve R&D efficiency.

[0101] Fig. 8A The following is an exemplary diagram showing the process of constructing a dimensional model provided in an embodiment of the present application. Figure 8BAn example diagram of constructing a dimensional model provided by an embodiment of the present application is shown as an example. Fig. 8A As shown, the process of constructing a dimensional model in the embodiment of the present application may include the following stages: a stage 810 of selecting a main table of a business system, a stage 820 of screening main table attributes, a stage 830 of generating a basic model, a stage 840 of selecting associated dimensions, a stage 850 of generating associated relationships and redundant attributes, and a stage 860 of automatic data development and code generation.

[0102] Taking the generation of a store product dimensional model as an example, the store product dimensional model mainly includes the core attributes of store products, product category attributes, and product brand attributes. Fig. 8A and Figure 8B As shown, in the stage 810 of selecting the main table of the business system, the stage 820 of filtering the main table attributes, and the stage 830 of generating the basic model, the embodiment of the present application can select the store commodity table of the meta warehouse business system (for example Figure 8B ods_shop_sku) as the main table, and filter out the core attributes of the store products in the store product table, and build the basic model of the store product dimensional model based on the core attributes of the store products (for example Figure 8B The base model of dim_shop_sku shown).

[0103] In the phase 840 of selecting associated dimensions and the phase 850 of generating associated relationships and redundant attributes, the embodiment of the present application may select a category dimension table (eg, Figure 8B dim_category as shown), redundant core category attributes in the base model, select the brand dimension table based on brand ID (for example Figure 8B As shown in dim_brand, the core brand attributes are redundant in the basic model, and the relationship between the basic model, category and brand is established.

[0104] In the automatic data development and code generation stage 860, the embodiment of the present application can automatically perform data development and simple code generation for the store product dimensional model.

[0105] The embodiment of the present application greatly simplifies the process of building a dimensional model by connecting the meta-warehouse of the business system. At the same time, by generating simple codes based on the sources and associations of redundant attributes accumulated during model construction, the efficiency of code development and writing is greatly improved.

[0106] Another way to generate a dimensional model is to reverse the existing data warehouse model into a dimensional model through reverse engineering. For example, there are a large number of data warehouse models based on Kimball theory in the public cloud. These existing data warehouse models can be reversed into dimensional models through reverse engineering.

[0107] At present, the mainstream reverse engineering mainly parses the tables, fields, constraints and other information from the data source of the data warehouse model, and then generates the tables (fact tables, dimension tables, etc.) corresponding to the dimensional model based on the parsed tables, fields, constraints and other information. However, the table corresponding to the dimensional model is constructed based on the tables, fields, constraints and other information parsed from the data source, and the hierarchical domain information of the table cannot be parsed, which makes the final constructed dimensional model unable to be associated with the data warehouse planning, resulting in the performance of the constructed dimensional model being lacking.

[0108] It should be noted that hierarchical domain information is the top-level design of data warehouse planning, which is used for model classification and management, including but not limited to data warehouse layering, business classification, data domain and other hierarchical domain information. It can be seen that hierarchical domain is one of the most critical parts of data warehouse design, the core of model classification management, the focus of data warehouse business systematization, and the core of architects' top-down planning of data warehouses; if the constructed dimensional model does not have hierarchical domain information, the constructed dimensional model will not be associated with the data warehouse planning, which will affect the classification and management of the dimensional model.

[0109] Furthermore, the table corresponding to the dimensional model is constructed based on the tables, fields, constraints and other information parsed from the data source, and the type of the table cannot be directly deduced (for example, it is impossible to deduce whether the table is a dimension table or a fact table). This is because the core of dimensional modeling is the dimension table and the fact table, and the data source of the data warehouse model generally uses the ER (Entity Relationship) model, which does not distinguish between table types. Therefore, the table corresponding to the dimensional model is constructed based on the tables, fields, constraints and other information parsed from the data source of the data warehouse model, and the type of the table cannot be deduced. Instead, it is necessary to manually confirm whether the table type is a dimension table or a fact table, which is undoubtedly inefficient.

[0110] To solve the above problems, the present application embodiment provides a novel solution for reversely generating a dimensional model. As an optional implementation, Fig. 9 Another optional flow chart of the modeling method provided by the embodiment of the present application is shown as an example. The method flow can be implemented by a cloud server. As an optional implementation, the method flow can be implemented by a big data computing engine based on cloud computing. For example, the method flow is implemented by a data middle station server in the big data computing engine. Fig. 9 The method flow may include the following steps.

[0111] In step S910, the data warehouse table is loaded.

[0112] In some embodiments, the data warehouse table may be a table in an existing data warehouse model, and the data warehouse model may have multiple data warehouse tables. The embodiment of the present application may reverse the data warehouse table into a table of a dimensional model through reverse engineering to realize the construction of a dimensional model. In the process of reverse engineering the data warehouse table into a table of a dimensional model, the embodiment of the present application aims to associate the table with hierarchical domain information and realize automatic derivation of the table type.

[0113] In step S911, the table name of the data warehouse table is parsed according to at least one configured hierarchical division rule, and the naming expression corresponding to each hierarchical division rule is parsed from the table name to obtain at least one naming expression; wherein a hierarchical division rule defines a table naming method corresponding to a design of hierarchical division information.

[0114] Hierarchical domain information can have a variety of designs. For example, hierarchical domain information such as data warehouse stratification, business classification, and data domain can have different arrangement orders in table naming. For example, data warehouse stratification_business classification_data domain is hierarchical domain information of one design, and data warehouse stratification_data domain_business classification is hierarchical domain information of another design. Hierarchical domain information of different orders can correspond to different designs of hierarchical domain information. The embodiment of the present application can configure different hierarchical domain rules for hierarchical domain information of different designs, and match them with the table naming of the data warehouse table. A hierarchical domain rule can define a table naming method corresponding to a design of hierarchical domain information; thus, based on at least one configured hierarchical domain rule, the embodiment of the present application can parse the table naming of the data warehouse table to parse out the naming expressions corresponding to each hierarchical domain rule from the table naming, wherein a hierarchical domain rule can correspond to a naming expression parsed from the table naming, so that based on at least one hierarchical domain rule, the embodiment of the present application can obtain at least one naming expression.

[0115] In an example, assuming that a configured hierarchical zoning rule is {data warehouse tiering}_{business classification}_{data domain}_{business process}_{custom}_{storage policy}, then the hierarchical zoning rule expresses the naming method of hierarchical zoning information designed in the order of {data warehouse tiering}_{business classification}_{data domain}_{business process}_{custom}_{storage policy} in table naming.

[0116] Further, assuming that the data warehouse table is named dwd_csn_crm_hotline_outbound_touch_df, based on the above hierarchical domain division rules, the embodiment of the present application can parse the following naming expressions shown in Table 1 from the table naming. It can be seen that based on different hierarchical domain division rules, the embodiment of the present application can parse different naming expressions from the table naming.

[0117]

[0118] Table 1

[0119] In step S912, a target named expression is determined from the at least one named expression, and hierarchical domain information corresponding to the target named expression is used as the target hierarchical domain information.

[0120] After parsing at least one naming expression from the table naming of the data warehouse table, the embodiment of the present application can determine the target naming expression from at least one naming expression based on the election strategy, such as giving priority to the most hierarchical domain information hits, giving priority to the most critical hierarchical domain information hits, and other election strategies; as an optional implementation, the embodiment of the present application can use the naming expression that hits the most hierarchical domain information or the most critical hierarchical domain information in at least one naming expression as the target naming expression. Thus, the hierarchical domain information corresponding to the target naming expression is used as the target hierarchical domain information. After the data warehouse table is reversed to the model table of the dimensional model, the target hierarchical domain information is associated with the model table, so that the constructed dimensional model can be associated with the data warehouse planning.

[0121] In step S913, the type of the dimensional model corresponding to the data warehouse table is derived according to the derived features; the data warehouse table is reversed into a model table corresponding to the type of dimensional model, and the model table is associated with the target hierarchical domain information.

[0122] In some further embodiments, after determining the target hierarchical domain information, the embodiments of the present application may determine the derived features of the data warehouse table, thereby inferring whether the type of the dimensional model corresponding to the data warehouse table is a fact table or a dimension table based on the derived features. The derived features may include the target hierarchical domain information, the number of field aggregation calculations in the table, the number of numeric field types in the table, the number of times the field appears in GROUP BY (aggregation), the number of times JOIN (join) appears in the primary and secondary positions, etc.; wherein, the target hierarchical domain information may be determined by parsing the table name and through the election strategy (for example, based on at least one hierarchical domain rule, deriving from the first position of the dividing line of the table name, at least one naming expression is obtained, and the target naming expression is determined through the election strategy, and then the target hierarchical domain information corresponding to the target naming expression is determined), and the remaining features may be extracted based on the ETL (data extraction, cleaning, transformation, loading) code of the data warehouse. As an optional implementation, the embodiment of the present application can determine the derived features based on the target hierarchical domain information and the features extracted from the data warehouse code, so as to derive the type of dimensional model corresponding to the data warehouse table according to the derived features, and the type of dimensional model corresponding to the data warehouse table includes fact table and dimension table. As an optional implementation, the derivation of the model type based on the derived features can be regarded as a binary classification problem, which can be implemented using algorithms such as GBDT (GradientBoosting Decision Tree).

[0123] Based on the type of dimensional model corresponding to the derived data warehouse table, the embodiment of the present application can reverse the data warehouse table to a model table corresponding to the type of dimensional model. For example, if the type of dimensional model corresponding to the data warehouse table is a fact table, the data warehouse table is reversed to a fact table of the dimensional model; if the type of dimensional model corresponding to the data warehouse table is a dimension table, the data warehouse table is reversed to a dimension table of the dimensional model. The model table obtained by reverse engineering can be associated with the target hierarchical domain information so that the constructed dimensional model is associated with the data warehouse planning.

[0124] The embodiment of the present application can perform table naming resolution on the data warehouse table based on the configured hierarchical division rules, so as to determine the target hierarchical division information of the data warehouse table, and then after the data warehouse table is reversed to the model table of the dimensional model, the model table is associated with the target hierarchical division information to realize automatic association of the dimensional model and the data warehouse planning; further, based on the derivation features, the type of the dimensional model corresponding to the data warehouse table is deduced, so as to directly reverse the model table of the corresponding type of dimensional model that can be used, without the need for manual reversal and secondary processing to determine the model type, thereby improving the efficiency of model construction.

[0125] The above describes multiple implementation schemes provided by the embodiments of the present application. The various optional methods introduced in each implementation scheme can be combined and cross-referenced with each other without conflict, thereby extending a variety of possible implementation schemes, which can all be considered as implementation schemes disclosed and open in the embodiments of the present application.

[0126] The modeling device provided in the embodiment of the present application is introduced below. The modeling device described below can be considered as a functional module required for the server to implement the modeling method provided in the embodiment of the present application. The content of the modeling device described below can be referenced to the content of the modeling method described above.

[0127] As an optional implementation, Fig.10 An optional block diagram of a modeling device provided in an embodiment of the present application is shown. The modeling device can be applied to a server. Fig.10 , the modeling device may include:

[0128] Requirements acquisition module 01 is used to obtain the requirements description language for building the indicator model;

[0129] The word segmentation module 02 is used to perform word segmentation processing on the demand description language to determine the indicator words matching the indicator elements from the demand description language; the indicator elements include time period, modifiers and atomic indicators;

[0130] A translation module 03, used to translate the indicator word of the matching indicator element into a standard word of the matching indicator element;

[0131] The definition and entry module 04 is used to define a derived indicator for the standard word of the matching indicator element and automatically enter the derived indicator if there is no entered derived indicator for the standard word of the matching indicator element; output the derived indicator as a constructed indicator model; the derived indicator includes standard words that describe time periods, modifiers and atomic indicators.

[0132] In some embodiments, the word segmentation module 02 is used to perform word segmentation processing on the requirement description language to determine the indicator words matching the indicator elements from the requirement description language, including:

[0133] Based on the candidate words corresponding to the preset indicator elements, the NLP platform is used to perform word segmentation processing on the demand description language to determine the indicator words that match the candidate words corresponding to the indicator elements from the demand description language.

[0134] In some embodiments, the translation module 03, for translating the indicator word of the matching indicator element into a standard word of the matching indicator element, comprises:

[0135] Using the translation platform, based on the attributes of the standard words of the indicator element, an indicator word with the same attributes as the standard word but not a standard word is determined from the indicator words of the matching indicator element, and the determined indicator word is replaced by the standard word to obtain the standard word of the matching indicator element.

[0136] In some embodiments, the candidate words corresponding to the indicator elements are recorded in the indicator vocabulary of the intelligent vocabulary, and the candidate words corresponding to the indicator elements recorded in the indicator vocabulary are determined based on industry vocabulary of multiple industries, and / or the candidate words corresponding to the indicator elements recorded in the indicator vocabulary are pre-entered; the intelligent vocabulary also records the attributes of the standard words of the indicator elements, and the attributes of the standard words include the meaning and part of speech of the standard words.

[0137] In some embodiments, the device may also be used to:

[0138] Calling the interface of the NLP platform to synchronize the candidate words corresponding to the input indicator elements to the NLP platform in real time;

[0139] Perform semantic offline training on the standard words of the indicator elements, and synchronize the semantic offline training results to the translation platform.

[0140] In some embodiments, the device may also be used to:

[0141] Determine whether there are already entered derived indicators for the standard word matching the time period, the standard word matching the modifier, and the standard word matching the atomic indicator;

[0142] In some embodiments, the definition and entry module 04 is used to define a derived indicator for the standard word of the matching indicator element if there is no entered derived indicator for the standard word of the matching indicator element, and automatically enter the derived indicator, including:

[0143] If there is no entered derived indicator for the standard word matching the time period, the standard word matching the modifier, and the standard word matching the atomic indicator, automatically define a derived indicator for the standard word matching the time period, the standard word matching the modifier, and the standard word matching the atomic indicator, and automatically enter the derived indicator;

[0144] In some embodiments, the device may also be used to:

[0145] If there are already entered derived indicators for the standard words matching the time period, the standard words matching the modifiers, and the standard words matching the atomic indicators, the entered derived indicators are output as the constructed indicator model.

[0146] In some embodiments, the device may also be used to:

[0147] Data is queried in the data warehouse model based on the derived index.

[0148] In some embodiments, the data warehouse model includes a dimensional model; the apparatus may also be used to:

[0149] According to the target dimension, select the main table of the business system of the meta warehouse;

[0150] Filtering the core attributes of the dimensional model corresponding to the target dimension from the main table;

[0151] Generate a basic model of the dimensional model according to the meta information of the main table and the core attributes;

[0152] Selecting an associated dimension, redundantly adding core attributes of the associated dimension based on the basic model, and establishing an associated relationship between the basic model and the associated dimension;

[0153] Based on the association relationship and data source, data development and code generation are automatically performed on the dimensional model.

[0154] In some embodiments, the dimensional model includes a dimension table and a fact table; the device is used to filter the core attributes of the dimensional model corresponding to the target dimension from the main table, including:

[0155] The core attributes of the dimension table corresponding to the target dimension in the main table or the core attributes of the fact table are screened; the core attributes of the dimension table include the dimension primary key and dimension attributes of the dimension table, and the core attributes of the fact table include granularity, redundant attributes and measurement.

[0156] The device is used to select an associated dimension, redundantly add a core attribute of the associated dimension based on the basic model, and establish an associated relationship between the basic model and the associated dimension, including:

[0157] A dimension table of an associated dimension is selected based on an ID of the associated dimension, core attributes of the dimension table of the associated dimension are redundant on the basis of the basic model, and an association relationship between the basic model and the associated dimension is established, wherein the associated dimension is at least one dimension related to the target dimension.

[0158] In some embodiments, the data warehouse model includes a dimensional model; the apparatus may also be used to:

[0159] Load the data warehouse table;

[0160] According to at least one configured hierarchical domain division rule, the table name of the data warehouse table is parsed, and a naming expression corresponding to each hierarchical domain division rule is parsed from the table name to obtain at least one naming expression; wherein a hierarchical domain division rule defines a table naming method corresponding to a design of hierarchical domain division information;

[0161] Determine a target named expression from the at least one named expression, and use hierarchical domain information corresponding to the target named expression as target hierarchical domain information;

[0162] According to the derivation features, the type of the dimensional model corresponding to the data warehouse table is deduced; the data warehouse table is reversed into a model table corresponding to the type of dimensional model, and the model table is associated with the target hierarchical domain information.

[0163] In some embodiments, the apparatus for determining a target named expression from the at least one named expression comprises:

[0164] The named expression that hits the most hierarchical domain information or the named expression that hits the most key hierarchical domain information among the at least one named expression is used as the target named expression.

[0165] In some embodiments, the device is used to derive the type of the dimensional model corresponding to the data warehouse table according to the derived features, including:

[0166] Based on the target hierarchical domain information and the features extracted from the data warehouse code, the derived features are determined, and the type of dimensional model corresponding to the data warehouse table is derived according to the derived features. The type of dimensional model corresponding to the data warehouse table includes fact table and dimension table.

[0167] The embodiment of the present application also provides a server, which may be a big data computing engine server, and the server may implement the modeling method provided by the embodiment of the present application by configuring the modeling device provided by the embodiment of the present application. Fig.11 An alternative block diagram of a server is shown. Fig.11 As shown, the server may include: at least one processor 1 , at least one communication interface 2 , at least one memory 3 and at least one communication bus 4 .

[0168] In the embodiment of the present application, the number of the processor 1 , the communication interface 2 , the memory 3 , and the communication bus 4 is at least one, and the processor 1 , the communication interface 2 , and the memory 3 communicate with each other through the communication bus 4 .

[0169] Optionally, the communication interface 2 may be an interface of a communication module for performing network communication.

[0170] Optionally, processor 1 may be a CPU, a GPU (Graphics Processing Unit), an NPU (embedded neural network processor), an FPGA (Field Programmable Gate Array), a TPU (tensor processing unit), an AI chip, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement an embodiment of the present application.

[0171] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0172] Among them, the memory 3 stores one or more computer-executable instructions, and the processor 1 calls the one or more computer-executable instructions to execute the modeling method provided in the embodiment of the present application.

[0173] An embodiment of the present application also provides a storage medium, which stores one or more computer-executable instructions, and when the one or more computer-executable instructions are executed, the modeling method provided in the embodiment of the present application is implemented.

[0174] The embodiment of the present application also provides a computer program, which, when executed, implements the modeling method provided in the embodiment of the present application.

[0175] Although the embodiments of the present application are disclosed above, the present application is not limited thereto. Any person skilled in the art may make various changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be subject to the scope defined by the claims.

Claims

1. A modeling method, wherein: include: Obtain the requirement description language for building the indicator model; Performing word segmentation processing on the demand description language to determine indicator words matching indicator elements from the demand description language; the indicator elements include time periods, modifiers and atomic indicators; translating the indicator word of the matching indicator element into a standard word of the matching indicator element; If there is no entered derived indicator for the standard word of the matching indicator element, define a derived indicator for the standard word of the matching indicator element and automatically enter the derived indicator; output the derived indicator as the constructed indicator model; the derived indicator includes a standard word describing a time period, a modifier, and an atomic indicator; Querying data in the data warehouse model based on the derived indicator; Wherein, the data warehouse model includes a dimensional model; and the method further includes: According to the target dimension, select the main table of the business system of the meta warehouse; Filtering the core attributes of the dimensional model corresponding to the target dimension from the main table; Generate a basic model of the dimensional model according to the meta information of the main table and the core attributes; Selecting an associated dimension, redundantly adding core attributes of the associated dimension based on the basic model, and establishing an associated relationship between the basic model and the associated dimension; Based on the association relationship and data source, automatically develop data and generate code for the dimensional model; or, Load the data warehouse table; According to at least one configured hierarchical domain division rule, the table name of the data warehouse table is parsed, and a naming expression corresponding to each hierarchical domain division rule is parsed from the table name to obtain at least one naming expression; wherein a hierarchical domain division rule defines a table naming method corresponding to a design of hierarchical domain division information; Determine a target named expression from the at least one named expression, and use hierarchical domain information corresponding to the target named expression as target hierarchical domain information; According to the derivation features, the type of the dimensional model corresponding to the data warehouse table is deduced; the data warehouse table is reversed into a model table corresponding to the type of dimensional model, and the model table is associated with the target hierarchical domain information.

2. The modeling method according to claim 1, wherein: The performing word segmentation processing on the requirement description language to determine the indicator words matching the indicator elements from the requirement description language includes: Based on the candidate words corresponding to the preset indicator elements, the natural language processing (NLP) platform is used to perform word segmentation processing on the demand description language to determine the indicator words that match the candidate words corresponding to the indicator elements from the demand description language; The translating the indicator word of the matching indicator element into a standard word of the matching indicator element comprises: Using the translation platform, based on the attributes of the standard words of the indicator element, an indicator word with the same attributes as the standard word but not a standard word is determined from the indicator words of the matching indicator element, and the determined indicator word is replaced by the standard word to obtain the standard word of the matching indicator element.

3. The modeling method according to claim 2, wherein: The candidate words corresponding to the indicator elements are recorded in the indicator vocabulary of the intelligent vocabulary. The candidate words corresponding to the indicator elements recorded in the indicator vocabulary are determined based on industry vocabulary of multiple industries, and / or the candidate words corresponding to the indicator elements recorded in the indicator vocabulary are pre-entered; the intelligent vocabulary also records the attributes of the standard words of the indicator elements, and the attributes of the standard words include the meaning and part of speech of the standard words.

4. The modeling method according to claim 3, wherein: Also includes: Calling the interface of the NLP platform to synchronize the candidate words corresponding to the input indicator elements to the NLP platform in real time; Perform semantic offline training on the standard words of the indicator elements, and synchronize the semantic offline training results to the translation platform.

5. The modeling method according to claim 1, wherein: Also includes: Determine whether there are already entered derived indicators for the standard word matching the time period, the standard word matching the modifier, and the standard word matching the atomic indicator; If the standard word of the matching indicator element does not have an already entered derived indicator, defining a derived indicator for the standard word of the matching indicator element and automatically entering the derived indicator includes: if the standard word of the matching time period, the standard word of the matching modifier, and the standard word of the matching atomic indicator do not have an already entered derived indicator, automatically defining a derived indicator for the standard word of the matching time period, the standard word of the matching modifier, and the standard word of the matching atomic indicator, and automatically entering the derived indicator; The method further includes: if there are already entered derived indicators for the standard words matching the time period, the standard words matching the modifiers, and the standard words matching the atomic indicators, the entered derived indicators are output as the constructed indicator model.

6. The modeling method according to claim 1, wherein: The dimensional model includes a dimension table and a fact table; the core attributes of the dimensional model corresponding to the target dimension are selected from the main table and include: Screening the core attributes of the dimension table corresponding to the target dimension in the main table, or the core attributes of the fact table; the core attributes of the dimension table include the dimension primary key and dimension attributes of the dimension table, and the core attributes of the fact table include granularity, redundant attributes, and measurement; The step of selecting the associated dimension, redundancies the core attributes of the associated dimension based on the basic model, and establishing the association relationship between the basic model and the associated dimension includes: A dimension table of an associated dimension is selected based on an ID of the associated dimension, core attributes of the dimension table of the associated dimension are redundant on the basis of the basic model, and an association relationship between the basic model and the associated dimension is established, wherein the associated dimension is at least one dimension related to the target dimension.

7. The modeling method according to claim 1, wherein: Determining a target named expression from the at least one named expression comprises: The named expression that hits the most hierarchical domain information or the named expression that hits the most key hierarchical domain information among the at least one named expression is used as the target named expression.

8. The modeling method according to claim 1, wherein: The type of dimensional model corresponding to the data warehouse table derived according to the derivation feature includes: Based on the target hierarchical domain information and the features extracted from the data warehouse code, the derived features are determined, and the type of dimensional model corresponding to the data warehouse table is derived according to the derived features. The type of dimensional model corresponding to the data warehouse table includes fact table and dimension table.

9. A server, wherein: The system comprises at least one memory and at least one processor, wherein the memory stores one or more computer executable instructions, and the processor calls the one or more computer executable instructions to execute the modeling method according to any one of claims 1 to 8.

10. A storage medium, wherein: The storage medium stores one or more computer executable instructions, and when the one or more computer executable instructions are executed, the modeling method according to any one of claims 1 to 8 is implemented.

Images