Data index generation method and device, storage medium and electronic device

Abstract

The invention discloses a data index generation method and device, a storage medium and an electronic device, and relates to the technical field of smart home, the data index generation method comprises the following steps: determining a target data theme in a business demand, and obtaining metadata information corresponding to the target data theme; determining an index rule corresponding to the metadata information according to a statistical rule and a predicate rule preset for the metadata information, and storing the index rule in a database; the target index rule corresponding to the target data theme is read from the database, the data index is generated according to the target index rule, and by adopting the technical scheme, the technical problem that the index generation efficiency is too low is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart home, in particular to a data index generation method and device, a storage medium and an electronic device. BACKGROUND

[0002] Currently, in the traditional data warehouse job, the calculation index involves the integration and processing of data in different fields, which is a complex and time-consuming process. In this process, if the data source currently providing business data is not covered by the current business model, the corresponding index calculation cannot be performed. When calculating the index, the index calculation job is often started according to the latest ready data, which causes delay in calculation time, reduces the timeliness of index calculation, and results in low efficiency of index generation.

[0003] Therefore, in the related art, there is a technical problem of low efficiency of index generation.

[0004] For the technical problem of low efficiency of index generation in the related art, an effective solution has not been proposed. SUMMARY

[0005] Embodiments of the present application provide a data index generation method and device, a storage medium and an electronic device to at least solve the technical problem of low efficiency of index generation in the related art.

[0006] According to an embodiment of the present application, a data index generation method is provided, including: determining a target data theme in a business requirement, and obtaining metadata information corresponding to the target data theme; determining an index rule corresponding to the metadata information according to a statistical rule and a predicate rule pre-set for the metadata information, and storing the index rule in a database; reading a target index rule corresponding to the target data theme from the database, and generating a data index according to the target index rule.

[0007] In one exemplary embodiment, before obtaining the metadata information corresponding to the target data theme, the method further includes: receiving business data sent by the target object; in a case where it is determined that a data theme corresponding to the business data exists in the database, obtaining metadata information corresponding to the data theme from the database; in a case where it is determined that a data theme corresponding to the business data does not exist in the database, performing a theme configuration operation on the business data to obtain a new data theme corresponding to the business data and metadata information of the new data theme.

[0008] In an example embodiment, the subject configuration operation on the business data obtains a new data subject corresponding to the business data and metadata information of the new data subject, including: configuring a data subject number for the business data, generating a subject name corresponding to the data subject number, and generating a life cycle corresponding to the data subject number; generating a first information table of the new data subject based on the data subject number, the subject name corresponding to the data subject number, and the life cycle corresponding to the data subject number; adding a data field to the business data, and generating a second information table based on the data field; setting an enumeration value of the data field, and generating a third information table based on the enumeration value; and generating the metadata information of the new data subject according to the first information table, the second information table, and the third information table.

[0009] In an example embodiment, before determining the index rule corresponding to the metadata information according to the statistical rule and the predicate rule previously set for the metadata information, the method further includes: using a statistical function to count a current occurrence frequency of the metadata information, setting a statistical rule for using the statistical function to count the occurrence frequency of the metadata information for the metadata information in a case where the current occurrence frequency belongs to a preset range; and setting a predicate rule for filtering information meeting a predicate condition from the metadata information for the metadata information.

[0010] In an example embodiment, before obtaining the metadata information corresponding to the target data subject, the method further includes: receiving business data sent by the target object; using a detection algorithm to perform a cleaning operation on the business data to obtain cleaned data; adding a dimension field to the cleaned data, grouping the cleaned data according to the dimension field to obtain grouped data, and updating the business data to the grouped data.

[0011] In an example embodiment, reading the target index rule corresponding to the target data subject from the database and generating a data index according to the target index rule includes: determining an index calculation function corresponding to the target data subject from a preset index function, wherein the target data subject corresponds to a target index rule in the database; calling the index calculation function to restore the target index rule to a database query statement, obtaining a query result obtained after executing the database query statement, and generating a data index according to the query result.

[0012] In an example embodiment, after the data indicator is generated according to the target indicator rule, the method further comprises: visually presenting the data indicator to the target object; and determining a display order of a plurality of data indicators corresponding to a plurality of data subjects according to a generation order of the plurality of data indicators.

[0013] According to another aspect of the embodiments of the present application, a data indicator generation apparatus is further provided, comprising: an acquisition module configured to determine a target data subject in a business requirement and acquire metadata information corresponding to the target data subject; a storage module configured to determine an indicator rule corresponding to the metadata information according to a statistical rule and a predicate rule pre-set for the metadata information and store the indicator rule into a database; and a generation module configured to read a target indicator rule corresponding to the target data subject from the database and generate a data indicator according to the target indicator rule.

[0014] According to still another aspect of the embodiments of the present application, a computer readable storage medium is further provided, which stores a computer program, wherein the computer program is configured to execute the data indicator generation method when running.

[0015] According to still another aspect of the embodiments of the present application, an electronic apparatus is further provided, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the data indicator generation method through the computer program.

[0016] In the embodiments of the present application, the target data subject in the business requirement is determined and the metadata information corresponding to the target data subject is acquired; the indicator rule corresponding to the metadata information is determined according to the statistical rule and the predicate rule pre-set for the metadata information and stored into the database; the target indicator rule corresponding to the target data subject is read from the database and the data indicator is generated according to the target indicator rule; by using the above technical solution, the indicator rule can be generated according to the metadata information corresponding to the data subject in advance, which facilitates subsequent reading without considering the influence of the inability to perform indicator calculation on the overall business model when there is no corresponding indicator rule, and the indicator rule required for indicator generation is stored into the database in advance, which can realize the division and governance of indicators and data, improve the efficiency and accuracy of data processing, and reduce the risk of business operation. The technical problem of low indicator generation efficiency is solved, and the indicator generation efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles behind the present application.

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0019] Figure 1 Fig. 1 is a hardware environment schematic diagram of a data index generation method according to an embodiment of the present application;

[0020] Figure 2 Fig. 2 is a flowchart of a data index generation method according to an embodiment of the present application;

[0021] Figure 3 Fig. 3 is a schematic diagram of statistical rules and predicate rules according to an embodiment of the present application;

[0022] Figure 4 Fig. 4 is a flowchart of adding data subjects according to an embodiment of the present application;

[0023] Figure 5 Fig. 5 is a flowchart of generating metadata information according to an embodiment of the present application;

[0024] Figure 6 Fig. 6 is a flowchart of running rules according to an embodiment of the present application;

[0025] Figure 7 Fig. 7 is a structural block diagram of a data index generation device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort should belong to the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] According to one aspect of the embodiments of this application, a method for generating data indicators is provided. This method for generating data indicators is widely applicable to whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligence house ecosystems. Optionally, in this embodiment, the above-mentioned method for generating data indicators can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0029] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0030] The embodiment provides a data index generation method applied to the terminal device, Figure 2 is a flowchart of the data index generation method according to the embodiment of the application, and the flowchart comprises the following steps:

[0031] In step S202, a target data theme in a business requirement is determined, and metadata information corresponding to the target data theme is acquired.

[0032] In step S204, an index rule corresponding to the metadata information is determined according to a statistical rule and a predicate rule that are set in advance for the metadata information, and the index rule is stored into a database.

[0033] Optionally, the index rule is generated through a logical relationship between the statistical rule and the predicate rule. The statistical rule and the predicate rule are two key components of the index rule. The statistical rule defines how to perform a statistical operation on data to generate an index value, and the predicate rule defines a condition of a database query, that is, the data is filtered by using the predicate rule.

[0034] In an optional embodiment, the combination of the predicate rule and the statistical rule to generate the index rule can customize the logic between the predicate rule and the statistical rule in the system. For example, the statistical rule is set to include sales amount statistics and sales quantity statistics. The predicate rule is set to include time, region, and product.

[0035] The sales amount statistics is SUM (sales amount) - statistics of total sales amount of a certain product in a certain region. The sales quantity statistics is COUNT (*) - statistics of sales times of the certain product in the certain region.

[0036] The time limit is WHERE sales date BETWEEN '2023-01-01' AND '2023-12-31' - only statistics of sales data in 2023.

[0037] The region limit is WHERE region ID = '123' - only statistics of sales data of a specific region.

[0038] The product limit is WHERE product ID = '456' - only statistics of sales data of a specific product.

[0039] Therefore, the index rule is generated by combining the statistical rule and the predicate rule, for example:

[0040] The index rule is: statistics of total sales amount of the specific product '456' in the region ID '123' in 2023.

[0041] The statistical rule is SUM (sales amount).

[0042] The predicate rule is: WHERE sale date BETWEEN '2023-01-01' AND '2023-12-31' AND region ID = '123' AND product ID = '456'.

[0043] The SQL statement is, for example, "sql SELECT SUM (sales) AS total sales FROM sales data table WHERE sale date BETWEEN '2023-01-01' AND '2023-12-31' AND region ID = '123' AND product ID = '456'".

[0044] The above index rule represents a rule generated by accessing metadata information, and can subsequently generate index tree or index similarity information, and further implement the division and governance of indexes and data.

[0045] In step S206, the target index rule corresponding to the target data subject is read from the database, and a data index is generated according to the target index rule.

[0046] Through the above steps, the target data subject in the business requirement is determined, and the metadata information corresponding to the target data subject is obtained; the index rule corresponding to the metadata information is determined according to the statistical rule and the predicate rule set in advance for the metadata information, and the index rule is stored in the database; the target index rule corresponding to the target data subject is read from the database, and a data index is generated according to the target index rule; by using the above technical solution, the index rule can be generated according to the metadata information corresponding to the data subject, which facilitates subsequent index generation and direct reading, and eliminates the influence of the inability to perform index calculation without corresponding index rule on the overall business model. By storing the index rule required for index generation in the database in advance, the division and governance of indexes and data can be realized, the efficiency and accuracy of data processing are improved, the risk of business operation is reduced, the technical problem of low index generation efficiency is solved, and the index generation efficiency is improved.

[0047] In one example embodiment, before obtaining the metadata information corresponding to the target data subject, the business data sent by the target object can be further received; if it is determined that the database contains a data subject corresponding to the business data, the metadata information corresponding to the data subject is obtained from the database; if it is determined that the database does not contain a data subject corresponding to the business data, a subject configuration operation is performed on the business data to obtain a new data subject corresponding to the business data and metadata information of the new data subject.

[0048] In the embodiment, by performing the subject configuration operation, the business data can be classified and organized according to specific business logic or analysis requirements, for example, a star model or a snowflake model can be created in a data warehouse, the data structure can be optimized, data redundancy can be reduced, and queries can be faster. At the same time, the data in different data sources is ensured to be consistent under the subject configuration, and data conflicts are reduced. In addition, through the predefined subject, multi-table join queries and complex analysis can be more easily performed. Furthermore, the data administrator can more easily manage and maintain the data through the subject configuration. Users can also quickly find related data sets according to their own needs, improving the availability of data.

[0049] In an example embodiment, in the implementation process of subject configuration operation on the business data to obtain a new data subject corresponding to the business data and metadata information of the new data subject, the following steps are included: configuring a data subject number for the business data, generating a subject name corresponding to the data subject number, and generating a life cycle corresponding to the data subject number; generating a first information table of the new data subject based on the data subject number, the subject name corresponding to the data subject number, and the life cycle corresponding to the data subject number; adding a data field to the business data, and generating a second information table based on the data field; setting an enumeration value of the data field, and generating a third information table based on the enumeration value; and generating the metadata information of the new data subject according to the first information table, the second information table, and the third information table.

[0050] Optionally, the first information table can include subject name, data subject number, and life cycle information. The second information table can include field name and field type of each data field. The third information table can include enumeration values of enumeration type data fields. In the embodiment, each information table can be completely decoupled, improving data query efficiency.

[0051] In an example embodiment, the process of setting a statistical rule and a predicate rule for the metadata information before determining an index rule corresponding to the metadata information according to the statistical rule and the predicate rule pre-set for the metadata information can be implemented by the following steps: using a statistical function to count the current occurrence frequency of the metadata information, and in the case of determining that the current occurrence frequency belongs to a pre-set range, setting a statistical rule for the metadata information for using the statistical function to count the occurrence frequency of the metadata information; and setting a predicate rule for the metadata information for filtering information meeting a predicate condition from the metadata information.

[0052] Further, storing the index rule to a database includes: persistently saving the statistical rule and the predicate rule to the database.

[0053] The embodiment can support structured and semi-structured data by setting statistical rules and predicate rules, release the constraint on data granularity and primary keys, retain business real data to the greatest extent, facilitate user-defined rules, and improve user autonomy.

[0054] Optionally, as shown in FIG. 8, the calculation logic of the index can be split into a limited number of statistical rules (such as the content selected by the red box in FIG. 8) and predicate restriction rules (such as the content selected by the blue box in FIG. 8), so as to rewrite the artificial definition logic into background calculation logic, thereby realizing automatic generation of the index. Figure 3 Figure 3 Figure 3

[0055] In one example embodiment, before obtaining the metadata information corresponding to the target data subject, the target object can further send business data; a detection algorithm is used to clean the business data to obtain cleaned data; a dimension field is added to the cleaned data, the cleaned data is grouped according to the dimension field to obtain grouped data, and the business data is updated to the grouped data.

[0056] It can be understood that the embodiment can clean and supplement the dimension field of the business data in the data preparation stage, exclude abnormal data, improve the independence of the data, ensure that the data processing process is not affected by other businesses, and can more quickly complete the development of the new index.

[0057] Optionally, the detection algorithm is used to clean the business data to obtain cleaned data, and the data quality can be improved based on statistics.

[0058] ​​​In an example embodiment, it is further proposed to read the target indicator rule corresponding to the target data subject from the database, and generate an implementation scheme of a data indicator according to the target indicator rule: determine an indicator calculation function corresponding to the target data subject from preset indicator functions, wherein the target data subject corresponds to a target indicator rule in the database; call the indicator calculation function to restore the target indicator rule into a database query statement, obtain a query result obtained after executing the database query statement, and generate a data indicator according to the query result. In this embodiment, the generation process of the indicator can be automated and optimized by writing and calling the function to execute the database query statement, the error rate is reduced by automated query and calculation, the query performance is improved and the response time is reduced by optimizing the SQL query and using the index of the database. By calling the indicator calculation function to restore the target indicator rule into a database query statement, the restoration logic can be encapsulated into a function, which is convenient for repeated use in different applications and services, improves the development efficiency, and can ensure that the same logic and data are used for each time of calculating the indicator, thereby ensuring the consistency of the results.

[0059] When the subsequent business logic changes, only the query statement in the function needs to be updated, without the need to modify each place where the logic is used. Moreover, using the function can better control the data access permission and enhance the data security. Not only the efficiency and accuracy of data processing are improved, but also the usability and security of data are enhanced, which is crucial for supporting the data analysis and decision-making of enterprises.

[0060] In an example embodiment, after generating the data indicator according to the target indicator rule, the data indicator can be displayed to the target object in a visual manner; for a plurality of data indicators corresponding to a plurality of data subjects, the display order of the plurality of data indicators is determined according to the generation order of the plurality of data indicators.

[0061] This embodiment can generate indicators in parallel for different data subjects, realize complete decoupling of multiple indicators, and reduce the influence between indicators in the generation process.

[0062] In an embodiment, the indicator system is gradually generated during the use of the user, and the characteristics corresponding to the indicator system include indicator popularity, relationship between indicators, similarity, ambiguity, etc. The indicator popularity can reflect the indicator subscription, the relationship between indicators indicates whether the rules are for the same subject, the similarity indicates the same predicate condition between indicators, and the ambiguity indicates the Chinese character rule public information of the indicator rule.

[0063] For better understanding of the process of the generation method of the above data index, the implementation method flow of the generation of the above data index will be described in combination with optional embodiments below, but not used to limit the technical solutions of the embodiments of the present application.

[0064] In the present embodiment, a data index generation method is provided, Figure 4 is a flow diagram of adding data topics according to the embodiments of the present application, as Figure 4 shown, the specific steps are as follows:

[0065] Step 1, data access. Data access can be achieved using datax, sqoop and other data table synchronization tools.

[0066] Step 2, data configuration. Add field name, enumeration value, life cycle and other configuration values to the accessed data.

[0067] It should be noted that steps 1 and 2 are initialization actions, that is, each topic will only be executed once when it is online.

[0068] Step 3, edit rules, that is, customize statistical rules and predicate rules of limit conditions.

[0069] Information tables for registering metadata information of topic data are set in the database, and the three tables can be accessed to obtain corresponding information during rule definition.

[0070] These information tables include topic basic information table, topic field table and topic field enumeration value table.

[0071] Among them, the topic basic information table contains table name and number (corresponding to data topic number) assigned to the table name, life cycle and other information. The topic field table contains field name and type of each field. The topic field enumeration value table contains enumeration values of enumeration type fields.

[0072] Optionally, as Figure 5 shown, after the user selects the topic, the following steps can also be implemented.

[0073] 1, establish the topic basic information table.

[0074] 2, establish the topic field table.

[0075] 3, filter whether the field is of enumeration type, if yes, execute 4, otherwise execute 5.

[0076] 4, establish the topic field enumeration value table.

[0077] 5, re-enter variables or characters.

[0078] 6, save the filter condition.

[0079] 7. Add statistical functions and fields.

[0080] 8. Save the index rule to the database.

[0081] Step 4, run the rule, if successful, execute step 5, if failed, return error information, prompt rule exception.

[0082] Step 5, persist the compliant index rule.

[0083] Step 6, whether to realize historical index traceability. Yes, execute step 7, otherwise execute step 9.

[0084] Step 7, persist the index.

[0085] Step 8, display the index in a chart or other visual way. The indexes under different data topics can be displayed together. If the readiness times of various topics are different, first calculate the index corresponding to the topic that is ready first, realize the decoupling between indexes.

[0086] In actual scenarios, users can directly access the index table by subscribing to data topic numbers, and at the same time, various charts of multiple indexes can be displayed to users, and comparison and drilling down functions can be provided.

[0087] Step 9, run the index rule. As shown in the following steps: Figure 6

[0088] 1. Get the index ID by passing parameters.

[0089] 2. Read the predicate rule of the index ID.

[0090] 3. Read the statistical rule of the index ID.

[0091] 4. Run the real SQL to persist to the database.

[0092] 5. Whether to configure the drilling down of the index. If yes, execute 6, otherwise end.

[0093] 6. Get the drilling down dimension field.

[0094] 7. Run the real SQL to persist to the database.

[0095] In this step, by using the index calculation function customized by the database, the persisted index rule can be restored to the SQL statement of the database. Each call of the index calculation function will generate a corresponding index. When calling the index calculation function, if the date of the parameter is a historical date, the traceability function is realized.

[0096] Further, as shown in the following, when updating the topic data, the index rule under the corresponding topic is also run. Figure 4 ​​

[0097] Optionally, in other embodiments, the OneDate scheme can also be used to generate the indicators, specifically by layering the data model, constructing consistent dimensions and facts (i.e. dimension tables and fact tables), establishing a set of standardized indicator system, including atomic indicators, derived indicators, business restrictions, statistical periods, etc., and ensuring the accuracy and interpretability of the indicators through unified scheduling. However, based on the above, the indicator calculation of the present application is only related to the data of the theme table, and the calculation of the indicators is converged to the theme table, so that the indicators can be directly generated based on the data of the theme table without considering the impact on the overall business model. Compared with the OneDate scheme, the present application does not need to consider the indicator system, realizes the division and rule of indicators, improves the development efficiency, and simplifies the design of the indicator system.

[0098] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the methods of various embodiments of the present application.

[0099] Figure 7 is a structural block diagram of a data indicator generation device according to an embodiment of the present application; as shown in Figure 7 , comprising:

[0100] The acquisition module 72 is configured to determine a target data theme in a business requirement, and acquire metadata information corresponding to the target data theme.

[0101] The storage module 74 is configured to determine an indicator rule corresponding to the metadata information according to a statistical rule and a predicate rule pre-set for the metadata information, and store the indicator rule to a database.

[0102] Optionally, the above indicator rule represents a rule generated by accessing the metadata information, which can be used to generate an indicator tree or an indicator similarity information subsequently.

[0103] The generation module 76 is configured to read a target indicator rule corresponding to the target data theme from the database, and generate a data indicator according to the target indicator rule.

[0104] By the above device, by determining a target data theme in a service requirement, and obtaining metadata information corresponding to the target data theme; determining an index rule corresponding to the metadata information according to a statistical rule and a predicate rule pre-set for the metadata information, and storing the index rule into a database; reading a target index rule corresponding to the target data theme from the database, and generating a data index according to the target index rule; by the above technical solution, the index rule can be generated according to the metadata information corresponding to the data theme, which facilitates subsequent index generation and direct reading, without considering the influence of the inability to perform index calculation on the overall business model, and by storing the index rule required for index generation into the database in advance, the index and data can be divided and conquered, the efficiency and accuracy of data processing are improved, and the risk of business operation is reduced. The technical problem of low index generation efficiency is solved, and the index generation efficiency is improved.

[0105] In one example embodiment, the obtaining module 72 is further configured to, before obtaining the metadata information corresponding to the target data theme, receive service data sent by the target object; obtain metadata information corresponding to a data theme corresponding to the service data from the database, if it is determined that the data theme corresponding to the service data exists in the database; and perform a theme configuration operation on the service data to obtain a new data theme corresponding to the service data and metadata information of the new data theme, if it is determined that the data theme corresponding to the service data does not exist in the database.

[0106] In one example embodiment, the obtaining module 72 is further configured to: configure a data theme number for the service data, and generate a theme name corresponding to the data theme number and a life cycle corresponding to the data theme number; generate a first information table of the new data theme based on the data theme number, the theme name corresponding to the data theme number, and the life cycle corresponding to the data theme number; add a data field to the service data, and generate a second information table based on the data field; set an enumeration value of the data field, and generate a third information table based on the enumeration value; and generate the metadata information of the new data theme according to the first information table, the second information table, and the third information table.

[0107] Optionally, the first information table can include theme name, data theme number, and life cycle information. The second information table can include field name and field type of each data field. The third information table can include enumeration value of the enumeration type data field. In this embodiment, each information table can be completely decoupled, and the data query efficiency is improved.

[0108] In an example embodiment, the storage module 74 is further configured to set the statistical rule and the predicate rule for the metadata information before determining the index rule corresponding to the metadata information according to the statistical rule and the predicate rule pre-set for the metadata information, by using a statistical function to count the current occurrence frequency of the metadata information, setting a statistical rule for the metadata information for counting the occurrence frequency of the metadata information using the statistical function if the current occurrence frequency is determined to be in a preset range, and setting a predicate rule for the metadata information for filtering information meeting a predicate condition from the metadata information.

[0109] In an example embodiment, the storage module 74 is further configured to persistently save the statistical rule and the predicate rule into the database.

[0110] The embodiment can support structured and semi-structured data by setting the statistical rule and the predicate rule, release the constraint on data granularity and primary key, retain the business real data to the greatest extent, facilitate user-defined rules, and improve the autonomy of users.

[0111] In an example embodiment, the acquisition module 72 is further configured to, before acquiring the metadata information corresponding to the target data subject, further receive business data sent by the target object, perform a cleaning operation on the business data using a detection algorithm to obtain cleaned data, add a dimension field to the cleaned data, group the cleaned data according to the dimension field to obtain grouped data, and update the business data to the grouped data.

[0112] It can be understood that the embodiment can perform cleaning and dimension field supplementing operations on the business data in the data preparation stage, exclude abnormal data, improve the independence of data, ensure that the data processing process is not affected by other businesses, and can complete the development of new indexes more quickly.

[0113] Optionally, the detection algorithm is used to clean the business data to obtain cleaned data, and the data quality can be improved based on statistics.

[0114] In an example embodiment, the generation module 76 is further configured to determine an index calculation function corresponding to a target data subject from preset index functions, wherein the target data subject corresponds to a target index rule in the database, call the index calculation function to restore the target index rule to a database query statement, acquire a query result obtained by executing the database query statement, and generate a data index according to the query result.

[0115] In one example embodiment, after the data indicator is generated according to the target indicator rule, further, the generation module 76 is further configured to: visually display the data indicator to the target object; and determine a display order of a plurality of data indicators corresponding to a plurality of data subjects according to a generation order of the plurality of data indicators.

[0116] The embodiment can generate indicators in parallel for different data subjects, realize complete decoupling of multiple indicators, and reduce mutual influence in the indicator generation process.

[0117] In one embodiment, the indicator system is gradually generated during the use of the user, and the features corresponding to the indicator system include indicator popularity, indicator relationship, similarity, ambiguity, etc. The indicator popularity can reflect the indicator subscription, the indicator relationship can represent whether the rule is for the same subject, the similarity can represent the same predicate condition between indicators, and the ambiguity can represent the Chinese character rule public information of the indicator rule.

[0118] The embodiment of the present application further provides a storage medium including a stored program, wherein the program performs the method of any one of the above when running.

[0119] Optionally, in the embodiment, the storage medium can be configured to store program code for performing the following steps:

[0120] S1, determining a target data subject in a business requirement, and obtaining metadata information corresponding to the target data subject;

[0121] S2, determining an indicator rule corresponding to the metadata information according to a statistical rule and a predicate rule pre-set for the metadata information, and storing the indicator rule into a database;

[0122] S3, reading a target indicator rule corresponding to the target data subject from the database, and generating a data indicator according to the target indicator rule.

[0123] The embodiment of the present application further provides an electronic device including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in the method embodiment of any one of the above.

[0124] Optionally, the electronic device can further include a transmission device and an input and output device, wherein the transmission device is connected with the processor, and the input and output device is connected with the processor.

[0125] Optionally, in the embodiment, the processor can be configured to perform the following steps through the computer program:

[0126] S1, determine a target data theme in a service requirement, and acquire metadata information corresponding to the target data theme;

[0127] S2, determine an index rule corresponding to the metadata information according to a statistical rule and a predicate rule pre-set for the metadata information, and store the index rule to a database;

[0128] S3, read a target index rule corresponding to the target data theme from the database, and generate a data index according to the target index rule.

[0129] Optionally, in the embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0130] Optionally, specific examples in the embodiment can refer to examples described in the above embodiments and optional implementation manners, and the embodiment will not be described here.

[0131] Obviously, those skilled in the art should understand that each module or each step of the present application described above can be realized by a general computing device, which can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and can be realized by program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order, or they can be manufactured into each integrated circuit module, or multiple modules or steps among them can be manufactured into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.

[0132] The above is only the preferred embodiment of the present application, and it should be noted that, for those skilled in the art, without departing from the principle of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method of generating a data indicator, characterized by, The method comprises the following steps: determining a target data theme in a business requirement, and obtaining metadata information corresponding to the target data theme; determining an index rule corresponding to the metadata information according to a statistical rule and a predicate rule preset for the metadata information, and storing the index rule in a database; reading a target index rule corresponding to the target data theme from the database, and generating a data index according to the target index rule.

2. The method of claim 1, wherein, Before obtaining the metadata information corresponding to the target data theme, the method further comprises: receiving business data sent by a target object; in a case where it is determined that a data theme corresponding to the business data exists in the database, obtaining metadata information corresponding to the data theme from the database; in a case where it is determined that a data theme corresponding to the business data does not exist in the database, performing a theme configuration operation on the business data to obtain a new data theme corresponding to the business data and metadata information of the new data theme.

3. The method of claim 2, wherein, The theme configuration operation on the business data to obtain a new data theme corresponding to the business data and metadata information of the new data theme comprises: configuring a data theme number for the business data, and generating a theme name corresponding to the data theme number and a life cycle corresponding to the data theme number; generating a first information table of the new data theme based on the data theme number, the theme name corresponding to the data theme number, and the life cycle corresponding to the data theme number; adding a data field to the business data, and generating a second information table based on the data field; setting an enumeration value of the data field, and generating a third information table based on the enumeration value; generating the metadata information of the new data theme according to the first information table, the second information table, and the third information table.

4. The method of claim 1, wherein, Before determining the index rule corresponding to the metadata information according to the statistical rule and the predicate rule preset for the metadata information, the method further comprises: using a statistical function to count a current occurrence frequency of the metadata information, and in a case where it is determined that the current occurrence frequency belongs to a preset range, setting a statistical rule for using the statistical function to count the occurrence frequency of the metadata information for the metadata information; and setting a predicate rule for filtering information meeting a predicate condition from the metadata information for the metadata information.

5. The method of claim 1, wherein, Before obtaining the metadata information corresponding to the target data theme, the method further comprises: receiving business data sent by a target object; performing a cleaning operation on the business data using a detection algorithm to obtain cleaned data; adding a dimension field to the cleaned data, grouping the cleaned data according to the dimension field to obtain grouped data, and updating the business data to the grouped data.

6. The method of generating data indicators of claim 1, wherein, reading a target index rule corresponding to the target data theme from the database, and generating a data index according to the target index rule, comprises: determine an index calculation function corresponding to the target data subject from a preset index function, wherein the target data subject corresponds to a target index rule in the database; invoke the index calculation function to restore the target index rule into a database query statement, obtain a query result obtained after the database query statement is executed, and generate a data index according to the query result.

7. The method of generating data indicators of claim 1, wherein, After the data index is generated according to the target index rule, the method further comprises: visually display the data index to the target object; for a plurality of data indexes corresponding to a plurality of data subjects, determine a display order of the plurality of data indexes according to a generation order of the plurality of data indexes.

8. An apparatus for generating a data indicator, the apparatus comprising: comprise: an acquisition module configured to determine a target data subject in a business requirement and acquire metadata information corresponding to the target data subject; a storage module configured to determine an index rule corresponding to the metadata information according to a statistical rule and a predicate rule preset for the metadata information, and store the index rule into a database; a generation module configured to read a target index rule corresponding to the target data subject from the database, and generate a data index according to the target index rule.

9. A computer readable storage medium, characterized in that, The computer-readable storage medium comprises a stored program, wherein the program executes the method described in any one of claims 1 to 7 when running. 10.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 7 by using the computer program.

Images