Shared dimension optimization method, device and equipment of data warehouse model and storage medium

Abstract

The invention relates to a shared dimension optimization method for a data warehouse model. The method comprises the following steps of: collecting a business data table, generating Chinese description of a field, disassembling and counting the Chinese description by adopting a text analysis method, tagging part-of-speech and generating an analysis result. And carrying out optimization processing on an analysis result, filtering part-of-speech irrelevant to the shared dimension, and screening out time period part-of-speech and measurement part-of-speech. And carrying out statistical analysis on an optimized analysis result, and identifying dimensions with common characteristics. And processing the identified common dimension based on business logic to generate a shared dimension. Through the generated shared dimension, the structure of the number bin model is adjusted, the structural design of the dimension table and the fact table is perfected, and the data query efficiency is optimized. According to the method, the workload of manual investigation and data analysis is reduced, the efficiency of dimension identification and sharing is improved through automatic processing, the period of data sorting is shortened, the labor cost is reduced, and the method adapts to data processing requirements of complex business scenes.

Description

Technical Field

[0001] The present invention relates to the fields of big data technology and financial technology, and in particular to a method, device, equipment and storage medium for optimizing shared dimensions of a data warehouse model. Background Art

[0002] In finance and other data-intensive industries, the organization and sharing of data dimensions are important components of data analysis and business decision-making. Shared dimensions usually include primary dimensions that are highly relevant to the business (such as products, customers, institutions, etc.), other analytical dimensions (such as business type, payment method, etc.), and labels used to modify or limit business characteristics (such as whether it is self-purchased, whether it is a new energy vehicle, etc.). These dimensions provide a basis for enterprise data analysis and support cross-departmental data sharing and decision optimization.

[0003] However, existing dimension combing methods face many challenges and shortcomings in practical applications. Traditional methods mainly rely on manually initiated business surveys and demand analysis processes, or manually analyzing data processing logic one by one. These methods have the following obvious defects:

[0004] Cross-departmental research is difficult: Sharing data dimensions requires collaboration and communication between different departments. However, cross-departmental data research is usually difficult to coordinate, and there are differences in data standards and analysis requirements between different departments, which makes the research process complicated and inefficient.

[0005] Long research and analysis cycle: Manual research and analysis of dimensional sharing require a lot of time and effort. Especially when multiple business scenarios are involved, the amount of data processed manually is huge, resulting in a long analysis cycle and difficulty in responding to changes in business needs in a timely manner.

[0006] The burden of data processing logic analysis is heavy: the existing method requires manual analysis of each data processing logic one by one. This not only requires the input of professionals with high technical thresholds, but also requires a large amount of human resources, making the data processing logic combing an extremely time-consuming and costly process.

[0007] These shortcomings make the existing dimension combing methods inefficient when dealing with complex data environments and rapidly changing business needs, and difficult to quickly implement in actual business applications. Therefore, how to use a more intelligent way to comb shared dimensions, improve cross-departmental collaboration efficiency, and reduce manpower investment has become an urgent problem to be solved. Summary of the invention

[0008] The main purpose of the present invention is to provide a method, device, equipment and storage medium for optimizing the shared dimensions of a data warehouse model, aiming to solve the technical problem that the existing technology relies on manual business research and one-by-one analysis of data processing logic, and it is difficult to efficiently identify the sharing of dimensions.

[0009] To achieve the above object, the present invention provides a shared dimension optimization method for a data warehouse model, comprising:

[0010] Collecting business data tables, including data list reports and data summary tables, and generating Chinese descriptions of fields in the business data tables;

[0011] The Chinese description is disassembled and counted by a text parsing method, and parts of speech are marked to generate a parsing result;

[0012] Optimizing the parsing results, filtering out parts of speech that are irrelevant to the shared dimension, wherein the irrelevant parts of speech include time period parts of speech and measurement parts of speech;

[0013] Conduct statistical analysis on the optimized analysis results to identify common dimensions;

[0014] Each identified common dimension is processed in combination with business logic to generate a shared dimension, and the structure of the data warehouse model is adjusted based on the shared dimension.

[0015] In one embodiment, statistical analysis is performed on the filtered parsing results to identify common dimensions, including:

[0016] Performing word frequency statistics on the words in the parsing result to determine the frequency of occurrence of each word;

[0017] Based on the set word frequency threshold, high-frequency words with a frequency higher than the word frequency threshold are selected from all words;

[0018] Matching the high-frequency words in the Chinese description of the field, and determining whether the high-frequency words have the same business meaning in multiple Chinese descriptions;

[0019] High-frequency words with the same business meaning are marked as common dimensions.

[0020] In one embodiment, each identified common dimension is processed in combination with business logic to generate a shared dimension, including:

[0021] Classify the identified common dimensions, analyze the usage and definition of common dimensions in multiple business scenarios, and extract representative scenarios from multiple business scenarios;

[0022] In representative scenarios, the dimension names, data types, and processing methods in each scenario are analyzed in detail to compare whether there are differences in the use of common dimensions in different scenarios;

[0023] Compare and analyze the differences in common dimensions, adjust and process the differences by establishing unified standards, and generate shared dimensions.

[0024] In one embodiment, the Chinese description is disassembled and counted by a text parsing method, and parts of speech are marked to generate parsing results, including:

[0025] Performing word segmentation processing on the Chinese description to break down the continuous text into independent words;

[0026] Filter the words after word segmentation, exclude words whose length is less than the set threshold, and retain other words as candidate words;

[0027] Perform part-of-speech tagging on the retained candidate words, and mark the grammatical attributes of each candidate word to distinguish different types of words;

[0028] Perform word frequency statistics on the candidate words, and record the frequency of occurrence of each candidate word in the Chinese description;

[0029] A parsing result is generated, wherein the parsing result includes candidate words and corresponding word frequencies and part-of-speech tags.

[0030] In one embodiment, the parsing results are optimized and parts of speech that are irrelevant to the shared dimension are filtered, wherein the irrelevant parts of speech include time period parts of speech and measurement parts of speech, including:

[0031] Classifying the time words in the analysis results, and dividing the time words into long-term, medium-term and short-term time dimensions according to the length of the time period;

[0032] Evaluate the use of time dimensions in business scenarios, and select the first target time dimension for business analysis based on the frequency of occurrence of time dimensions in different business scenarios;

[0033] Evaluate the cross-scenario applicability of the time dimension, determine whether each time dimension is used in multiple business scenarios, and retain the second target time dimension used in multiple business scenarios;

[0034] Evaluate the historical data accumulation capability of the time dimension, determine whether the time dimension supports cross-period data accumulation, and retain the third target time dimension that supports cross-period accumulation;

[0035] Evaluate the trend analysis capability of the time dimension, and select the fourth target time dimension that supports long-term trend analysis based on the performance of the time dimension in trend analysis;

[0036] The time period parts of speech that match the target time dimension are retained, and other time period parts of speech that are not related to the shared dimension are filtered out.

[0037] In one embodiment, adjusting the structure of the data warehouse model based on the shared dimension includes:

[0038] Analyze the relationship between the shared dimension and the data warehouse model, determine the application scenario of the shared dimension in the data warehouse model, and identify the existing dimension that needs to be replaced, merged or expanded;

[0039] Adjust the dimension table structure of the data warehouse model, add new fields and delete duplicate dimensions according to the definition of shared dimensions;

[0040] Reconstruct the fact table structure of the data warehouse model, adjust the fields and join methods in the fact table, and associate the shared dimensions with the metrics in the fact table;

[0041] Update the relationship between dimension tables and fact tables, and optimize the connection method between shared dimensions and fact tables.

[0042] In one embodiment, after adjusting the structure of the data warehouse model based on the shared dimension, the method further includes:

[0043] Import business data into the adjusted data warehouse model to verify data integrity and loading accuracy;

[0044] Test whether the adjusted data warehouse model structure meets the query requirements in various business scenarios, and evaluate the performance of shared dimensions in actual applications;

[0045] Verify the relationship between dimension tables and fact tables in the data warehouse model, and evaluate the data consistency in the adjusted data warehouse model;

[0046] Evaluate the response efficiency of the adjusted data warehouse model in data queries in different scenarios;

[0047] Generate a data validation report, record the validation results, and evaluate the usability and performance of the adjusted data warehouse model.

[0048] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a shared dimension optimization device for a data warehouse model, wherein the shared dimension optimization device for a data warehouse model comprises a memory, a processor, and a shared dimension optimization program for a data warehouse model stored in the memory and executable on the processor, wherein the shared dimension optimization program for a data warehouse model implements the steps of the shared dimension optimization method for a data warehouse model as described above when executed by the processor.

[0049] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer storage medium, on which a shared dimension optimization program of a data warehouse model is stored, and when the shared dimension optimization program of the data warehouse model is executed by a processor, the steps of the shared dimension optimization method of the data warehouse model as described above are implemented.

[0050] Beneficial effects: The present invention relates to a shared dimension optimization method for a data warehouse model, which collects business data tables, generates Chinese descriptions of fields, uses text parsing methods to disassemble and count the Chinese descriptions, annotates parts of speech, and generates parsing results. The parsing results are optimized, parts of speech that are not related to shared dimensions are filtered out, and time period parts of speech and measurement parts of speech are screened out. Statistical analysis is performed on the optimized parsing results to identify dimensions with common characteristics. The identified common dimensions are processed based on business logic to generate shared dimensions. Through the generated shared dimensions, the structure of the data warehouse model is adjusted, the structural design of the dimension table and the fact table is improved, and the data query efficiency is optimized. The present invention reduces the workload of manual research and data analysis, improves the efficiency of dimension identification and sharing through automated processing, shortens the cycle of data combing, reduces labor costs, and adapts to the data processing needs of complex business scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0052] Figure 1 It is a flow chart of an embodiment of a method for optimizing shared dimensions of a data warehouse model of the present invention;

[0053] Figure 2 It is a functional module diagram of a preferred embodiment of the shared dimension optimization device of the data warehouse model of the present invention;

[0054] Figure 3 It is a structural schematic diagram of the device hardware operating environment involved in the shared dimension optimization device implementation scheme of the data warehouse model of the present invention. DETAILED DESCRIPTION

[0055] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0056] In finance and other data-intensive industries, the organization and sharing of data dimensions are important components of data analysis and business decision-making. Shared dimensions usually include primary dimensions that are highly relevant to the business (such as products, customers, institutions, etc.), other analytical dimensions (such as business type, payment method, etc.), and labels used to modify or limit business characteristics (such as whether it is self-purchased, whether it is a new energy vehicle, etc.). These dimensions provide a basis for enterprise data analysis and support cross-departmental data sharing and decision optimization.

[0057] However, existing dimension combing methods face many challenges and shortcomings in practical applications. Traditional methods mainly rely on manually initiated business surveys and demand analysis processes, or manually analyzing data processing logic one by one. These methods have the following obvious defects:

[0058] Cross-departmental research is difficult: Sharing data dimensions requires collaboration and communication between different departments. However, cross-departmental data research is usually difficult to coordinate, and there are differences in data standards and analysis requirements between different departments, which makes the research process complicated and inefficient.

[0059] Long research and analysis cycle: Manual research and analysis of dimensional sharing require a lot of time and effort. Especially when multiple business scenarios are involved, the amount of data processed manually is huge, resulting in a long analysis cycle and difficulty in responding to changes in business needs in a timely manner.

[0060] The burden of data processing logic analysis is heavy: the existing method requires manual analysis of each data processing logic one by one. This not only requires the input of professionals with high technical thresholds, but also requires a large amount of human resources, making the data processing logic combing an extremely time-consuming and costly process.

[0061] These shortcomings make the existing dimension combing methods inefficient when dealing with complex data environments and rapidly changing business needs, and difficult to quickly implement in actual business applications. Therefore, how to use a more intelligent way to comb shared dimensions, improve cross-departmental collaboration efficiency, and reduce manpower investment has become an urgent problem to be solved.

[0062] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of a shared dimension optimization method for a data warehouse model provided by the present invention. It should be noted that although a logical sequence is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0063] like Figure 1 As shown, the shared dimension optimization method of the data warehouse model proposed in the present invention includes the following steps:

[0064] S10, collecting business data tables, the business data tables including data list reports and data summary tables, and generating Chinese descriptions of fields in the business data tables;

[0065] In this embodiment, business data tables related to analysis are obtained from different business systems. The data tables include different structured data, such as transaction data, user information, product information, etc. The source of the data table can be the internal database, ERP system or external cooperation platform of the enterprise. Data collection is performed through data interface (API), batch import, or real-time data stream to ensure the integrity and real-time performance of the data table.

[0066] The data list report is a data collection that records the details of business operations, such as customer transaction details, product sales details, etc. The data summary table is the result data after summarizing the list data, such as daily sales totals, customer distribution, etc. The data list report usually has a higher granularity, while the data summary table is used for higher-level business analysis. The collection of these two tables is usually completed through SQL queries, ETL (data extraction, transformation, loading) tools or other data analysis platforms.

[0067] Generate descriptions for the collected data table fields. These fields may exist in the form of codes, abbreviations, or other forms. For subsequent analysis, it is necessary to generate natural language Chinese descriptions for these fields. The generation method can be through automated scripts (such as translation tools or dictionaries combined with field names), rule bases, or generated by data dictionaries and historical business rules.

[0068] In one specific implementation, an enterprise data center is used to obtain data tables distributed in multiple business systems in real time through an API interface. The data tables contain customer account information, transaction records, and compliance review data. The data center uses a microservice architecture to ensure seamless data collection between different business systems. Combined with a big data processing framework (such as Apache Kafka), it ensures real-time data flow and can support high-concurrency data requests.

[0069] By automatically collecting business data tables and generating Chinese descriptions, errors and inconsistencies in manually generated descriptions are avoided, greatly improving the efficiency and accuracy of data processing.

[0070] S20, disassembling and counting the Chinese description by a text parsing method, and marking the parts of speech to generate a parsing result;

[0071] In this embodiment, the Chinese description is processed using a specific text parsing algorithm through natural language processing (NLP) technology. Common methods include rule-based word segmentation algorithms (such as Jieba word segmentation) or deep learning models (such as BERT and other pre-trained models). The core goal of text parsing is to break down continuous text into independent vocabulary units that are easier to handle.

[0072] Disassembly means breaking down complex sentences in Chinese descriptions into words through word segmentation technology. For example, "customer account balance" can be broken down into "customer", "account" and "balance". After word segmentation, the frequency of each word is counted and the number of times it appears in different descriptions is recorded for subsequent analysis.

[0073] Part-of-speech tagging is the process of performing lexical analysis on each word to mark its role in the grammar (such as noun, verb, adjective, etc.). Through part-of-speech tagging, the system can better understand the meaning of each word in the business logic. Part-of-speech tagging can be done using either a statistical-based model or a rule-based system.

[0074] After the disassembly, statistics and part-of-speech tagging are completed, a parsing result containing all vocabulary, word frequency and part-of-speech tags is generated. These results will be used for subsequent common dimension identification and model adjustment.

[0075] In one specific implementation, the Chinese description is segmented by using an industry-standard word segmentation library (such as the Jieba library). The word segmentation library is customized and optimized based on the terminology list in the financial field to ensure that professional terms can be correctly parsed. After word segmentation, the disassembled words are counted and sorted by frequency of occurrence to facilitate subsequent screening of high-frequency words. A rule-based part-of-speech tagging system is used to mark the disassembled words as nouns, verbs, etc. to ensure accurate part-of-speech tagging of financial terms. A result set containing vocabulary, word frequency, and part-of-speech tags is generated for subsequent analysis.

[0076] Through text parsing methods, complex Chinese descriptions can be efficiently disassembled and marked with parts of speech, and parsing results can be automatically generated, reducing human intervention, improving the accuracy and efficiency of data processing, and providing high-quality input data for subsequent common dimension identification.

[0077] S30, optimizing and processing the parsing result, filtering out parts of speech that are irrelevant to the shared dimension, wherein the irrelevant parts of speech include time period parts of speech and measurement parts of speech;

[0078] In this embodiment, the results generated by text parsing are optimized. The optimization process is mainly aimed at the vocabulary data obtained by previous parsing and statistics, with the purpose of improving the recognition accuracy of shared dimensions. Through the optimization process, the interference of noise data is reduced, so that highly business-related words can be correctly recognized. Which words need to be optimized can be defined through a rule base, manual settings or automated models. The optimization process can dynamically adjust the vocabulary filtering rules based on business needs.

[0079] In the parsing results, some words have no direct relationship with the construction of shared dimensions, such as words representing time and measurement. The existence of these words may affect the recognition of shared dimensions, so they need to be filtered. First, the words are tagged with part of speech, and then the words that are not related to the shared dimension are removed according to business rules or part of speech classification rules. Common methods include automatic filtering algorithms based on part of speech tagging, or manually set word category libraries.

[0080] Time period parts of speech refer to time-related words, such as "year", "month", and "day". These words often represent time spans in data analysis, but do not directly contribute to shared dimensions. Measurement parts of speech include measurement-related words such as "amount" and "number of times". These words are related to specific values, but are not used as shared dimensions. These words can be pre-defined through the part-of-speech tagging tool, and automatically filtered out in combination with the business rules in the vocabulary library.

[0081] By optimizing the parsing results and filtering irrelevant parts of speech, the noise in the data is reduced and the recognition accuracy of shared dimensions is improved. Especially in big data analysis scenarios, it can significantly improve processing efficiency and reduce manual intervention.

[0082] S40, performing statistical analysis on the optimized analysis results to identify common dimensions;

[0083] In this embodiment, the parsing results after filtering out irrelevant parts of speech retain high-quality words related to the shared dimension. After preliminary screening, these words have stronger business relevance and are suitable for further statistical analysis. After removing irrelevant words from the results generated by text parsing, a more streamlined vocabulary set is formed. The optimized results are the basic data for subsequent dimension identification.

[0084] Perform statistical analysis on the quantity and distribution characteristics of the optimized vocabulary to determine which words appear frequently in multiple data sets and business scenarios, and try to identify common dimensions from these words. Statistical analysis can include word frequency statistics, business relevance analysis of words, and word combination analysis in context. Commonly used technical means include word frequency statistics, n-gram models, TF-IDF algorithms, etc., which are used to calculate the frequency, weight, and importance of words in business scenarios. At the same time, statistical analysis can be combined with historical data to further screen out words with common characteristics.

[0085] Common dimensions refer to dimensions that appear frequently and have consistent business meanings in multiple business scenarios and data sets. Through statistical analysis of vocabulary, we can identify which dimensions are common in multiple scenarios, and these dimensions can be used as shared dimensions. The identification of common dimensions can be completed through frequency statistics and cross-validation of business application scenarios of vocabulary. For example, if certain words appear frequently in multiple data tables and have the same business meaning, they can be identified as common dimensions. In addition, semantic analysis tools can be combined to further improve the recognition accuracy of common dimensions.

[0086] By performing statistical analysis on the optimized parsing results, common dimensions in multiple business scenarios can be efficiently identified, reducing the complexity and time-consuming of manual analysis. Dimensions with common characteristics in multiple data tables can be automatically screened out, improving the accuracy and consistency of data sharing, supporting data analysis and decision-making across business departments, improving the automation of dimension identification, and adapting to data processing requirements in complex business scenarios.

[0087] S50, processing each identified common dimension in combination with business logic to generate a shared dimension, and adjusting the structure of the data warehouse model based on the shared dimension.

[0088] In this embodiment, the common dimensions are dimensions with universal applicability obtained through statistical analysis of multiple data tables and business scenarios. The process of identifying these common dimensions includes frequency analysis of vocabulary, business logic verification, etc., and finally screening out dimensions that can be used across scenarios as the basis for shared dimensions. Optimized and verified vocabulary is extracted from the parsing results and used as the basis for common dimensions. These dimensions have cross-scenario business relevance and are easy to share.

[0089] Based on the identified common dimensions, they are further processed and refined in combination with specific business logic. The processing process includes adjusting the definition of the dimension, adding or modifying the dimension attributes, and ensuring that the dimension matches the business needs. By processing the dimensions using rule bases, manual adjustments, or automated tools, the dimensions can be expanded or optimized according to different business needs so that these dimensions can more accurately reflect the key indicators in the business scenario.

[0090] After processing, a shared dimension that can be used in multiple business scenarios is finally generated. A shared dimension is a dimension that has been adjusted by business logic and has the ability to be used across departments and systems to support unified data analysis and decision-making. The generation of shared dimensions can be done through automated tools or scripts, adding the processed dimensions to the data model to ensure that each business department can perform data analysis and sharing under the same dimension framework.

[0091] The structural adjustment of the data warehouse model is to better adapt to the generated shared dimensions. The introduction of shared dimensions requires the optimization of the dimension tables and fact tables of the data warehouse to support cross-scenario data query and analysis. The structural adjustment of the data warehouse model also needs to consider factors such as data consistency and query efficiency. By modifying the dimension table structure in the data warehouse model, adjusting or adding shared dimension fields, and optimizing the connection method between dimensions and fact tables, the flexibility and query efficiency of the data warehouse model can be improved.

[0092] In one implementation, the identified common dimensions are adjusted and expanded according to the business rule library. For the customer dimension, the customer type can be further refined according to the customer's transaction behavior, preferences and other attributes. The processed customer dimension is defined as a standard shared dimension, which is suitable for multiple business scenarios of customer segmentation analysis. The customer dimension table in the data warehouse model adds a shared field "customer type" and supports fast query of customer behavior analysis by joining the fact table. The dimension processing process based on business logic makes the shared dimension more accurate, and the optimization of the data warehouse model improves the accuracy of customer behavior analysis.

[0093] By combining business logic to process the identified common dimensions and generate shared dimensions, cross-departmental sharing and unified use of dimensions can be achieved, improving the accuracy and consistency of data analysis. After adjusting the data warehouse model structure, the data warehouse model can adapt to more business scenarios, improving the efficiency of data query and the flexibility of the system.

[0094] The present invention relates to a shared dimension optimization method for a data warehouse model. By collecting business data tables, Chinese descriptions of fields are generated, and a text parsing method is used to disassemble and count the Chinese descriptions, mark the parts of speech, and generate parsing results. The parsing results are optimized, parts of speech that are not related to the shared dimensions are filtered, and time period parts of speech and measurement parts of speech are screened out. Statistical analysis is performed on the optimized parsing results to identify dimensions with common characteristics. The identified common dimensions are processed based on business logic to generate shared dimensions. Through the generated shared dimensions, the structure of the data warehouse model is adjusted, the structural design of the dimension table and the fact table is improved, and the data query efficiency is optimized. The present invention reduces the workload of manual research and data analysis, improves the efficiency of dimension identification and sharing through automated processing, shortens the cycle of data combing, reduces labor costs, and adapts to the data processing requirements of complex business scenarios.

[0095] In one embodiment, the above S40 includes:

[0096] S401, performing word frequency statistics on the words in the parsing result to determine the frequency of occurrence of each word;

[0097] S402, based on the set word frequency threshold, selecting high-frequency words whose occurrence frequency is higher than the word frequency threshold from all words;

[0098] S403, matching the high-frequency words in the Chinese description of the field, and determining whether the high-frequency words have the same business meaning in multiple Chinese descriptions;

[0099] S404, marking high-frequency words with the same business meaning as common dimensions.

[0100] In this embodiment, word frequency statistics refers to counting the parsed words and calculating the number of times each word appears in the data set or field description. By counting the word frequency, it is possible to determine which words have a higher frequency of use in the business data and provide a basis for subsequent high-frequency word screening. Through statistical tools or scripts, all words in the parsing results are traversed and the number of occurrences of each word is recorded. Collections.Counter or other similar tools in Python can be used for quick statistics.

[0101] The word frequency threshold is a pre-set value used to filter out words that appear frequently. Words that appear more frequently than the threshold are considered high-frequency words, and words that appear less frequently than the threshold are ignored. This step can help eliminate low-frequency words that appear occasionally in the business, ensuring that only words that are highly relevant to the business are retained. After the word frequency statistics are performed, words with frequencies exceeding the threshold are filtered out based on the pre-set word frequency threshold. The filtering results can be optimized by adjusting the size of the word frequency threshold to ensure that the filtered words have high business value

[0102] For the high-frequency words that have been filtered out, further matching is performed to determine whether they have the same business meaning in the field descriptions of different data tables. This is to ensure that the words can represent the same or similar business concepts in different business scenarios, so that they have the potential to be shared dimensions. Through the semantic matching or word meaning matching tools in the Chinese description of the field, determine whether the high-frequency words have the same business meaning in different field descriptions. Natural language processing (NLP) tools or regular expressions can be used for matching.

[0103] When a high-frequency word has the same business meaning in multiple business scenarios or fields, the word can be marked as a common dimension. Common dimensions are applicable across scenarios and can be shared and reused in different data tables and business scenarios. Through annotation or data model tools, high-frequency words that have the same business meaning are marked as common dimensions. These dimensions can serve as the basis for subsequent data analysis and business decisions.

[0104] This embodiment can quickly filter out representative high-frequency words in business scenarios by counting the word frequency and filtering high-frequency words on the filtered parsing results, and further match their business meanings, and finally mark them as common dimensions. This step improves the automation of dimension recognition, reduces human intervention, and makes data processing and analysis more accurate.

[0105] In one embodiment, in the above S50, each identified common dimension is processed in combination with business logic to generate a shared dimension, including:

[0106] S501, classifying the identified common dimensions, analyzing usage and definitions of the common dimensions in multiple business scenarios, and extracting representative scenarios from the multiple business scenarios;

[0107] S502, in representative scenarios, conduct a detailed analysis of the dimension name, data type, and processing method in each scenario, and compare whether there are differences in the use of common dimensions in different scenarios;

[0108] S503, performing comparative analysis on the differences in common dimensions, adjusting and processing the differences by formulating unified standards, and generating shared dimensions.

[0109] In this embodiment, after the preliminary word frequency statistics and business meaning matching, multiple dimensions with common characteristics have been identified. These dimensions are further classified according to their business nature and usage scenarios. For example, customer dimension, product dimension, time dimension, etc. The purpose of classification is to better understand the usage characteristics of dimensions in different business scenarios. Dimensions are classified by business logic and dimension attributes. Dimensions can be divided into primary dimensions (such as customers, products, etc.) and auxiliary dimensions (such as time, region, etc.) based on business needs or system requirements. This classification process can be rule-driven or automatically classified using a classification model.

[0110] Analyze how common dimensions are used in different business scenarios, including the frequency of dimension use, data processing logic, and definitions in actual applications. The goal is to gain a deep understanding of the application of dimensions in multiple scenarios to ensure the universality and accuracy of shared dimensions. Analyze the specific usage details of common dimensions in multiple business scenarios. Understand the use of dimensions through system logs, query records and other data, and analyze whether they are applicable across scenarios.

[0111] Extract the most representative specific scenarios for different business scenarios for further dimension analysis. For example, in a banking system, you can select credit card business, loan business, and payment business as representative scenarios and analyze the use of dimensions in these scenarios. Determine representative scenarios by analyzing business flows and data flows. You can use data analysis tools or manually select key business scenarios.

[0112] In representative scenarios, analyze the names, data types, and processing methods of common dimensions in detail. By analyzing these details, understand whether the definitions of dimensions in different business scenarios are consistent and whether the processing methods are the same. Use data query, system logs, and other means to view the names and data types of dimensions in each scenario, as well as their specific usage in the data processing process. By comparing the dimension definitions in different scenarios, find out the inconsistencies.

[0113] By comparing and analyzing the dimension definitions in multiple business scenarios, you can find the differences. These differences may be reflected in the naming method, data type or processing logic of the dimension. Use analysis tools or scripts to compare the differences in dimensions in different scenarios, generate a difference analysis report, and identify the areas that need to be adjusted and unified.

[0114] Based on the results of the difference analysis, by formulating unified business standards, the dimensions are adjusted and processed, and finally a shared dimension that can be used across scenarios is generated. The unified standards include unified specifications for dimension names, data types, and business processing logic. Dimension unified standards are formulated in combination with business requirements, technical specifications, and difference analysis reports. By adjusting the definition and use of dimensions in different scenarios, ensure that shared dimensions can be applied to various business scenarios.

[0115] This embodiment can effectively improve the versatility of dimensions by classifying, analyzing and adjusting differences of common dimensions, enabling them to be shared and used across business scenarios, reducing duplicate definitions and inconsistencies in data analysis, and improving the efficiency and accuracy of data analysis.

[0116] In one embodiment, the above S20 includes:

[0117] S201, performing word segmentation processing on the Chinese description to break down the continuous text into independent words;

[0118] S202, screening the words after word segmentation, excluding words whose length is less than a set threshold, and retaining other words as candidate words;

[0119] S203, performing part-of-speech tagging on the retained candidate words, and marking grammatical attributes for each candidate word to distinguish different types of words;

[0120] S204, performing word frequency statistics on the candidate words, and recording the frequency of occurrence of each candidate word in the Chinese description;

[0121] S205, generating a parsing result, wherein the parsing result includes candidate words and corresponding word frequencies and part-of-speech tags.

[0122] In the present embodiment, word segmentation is one of the basic steps in natural language processing, and its main purpose is to break down continuous Chinese text into independent words. Chinese text does not have natural spaces like English, so it is necessary to use word segmentation tools to identify word boundaries in the text. Common Chinese word segmentation tools (such as Jieba word segmentation, NLPIR, etc.) can be used to segment the Chinese description text. The word segmentation tool is based on a dictionary and a statistical model, and can identify a single word and break the text into independent words.

[0123] After word segmentation, there are usually some meaningless short words, such as punctuation marks or single letters. These words are not helpful for business analysis, so it is necessary to set a word length threshold to exclude words shorter than the threshold. Set a word length threshold (for example, 2 characters), select words that meet the length through programming, retain them as candidate words, and filter out short words to reduce the interference of noise data.

[0124] Part-of-speech tagging is to attach grammatical attributes to each word, such as noun, verb, adjective, etc. Through part-of-speech tagging, we can further understand the role of words in sentences, which is convenient for subsequent semantic analysis and business relevance judgment. Common part-of-speech tagging tools (such as Stanford NLP, HanLP, etc.) can assign part-of-speech tags to each word according to the context. Based on the pre-trained model, the tool can identify the grammatical attributes of words and mark the part-of-speech information.

[0125] Word frequency statistics record the frequency of each candidate word in the text to determine the importance of the word. Frequently occurring words usually have a high relevance in business scenarios and can provide a basis for subsequent identification of common dimensions. The frequency of occurrence of each word can be counted through programming. You can use collections.Counter or other similar tools in Python to quickly count word frequencies and provide support for subsequent screening of high-frequency words.

[0126] The parsing result is the final output of word segmentation and part-of-speech tagging, including a list of candidate words, word frequency information for each word, and part-of-speech tags. These parsing results will serve as input for subsequent steps (such as common dimension identification). The segmented words, their corresponding word frequencies, and part-of-speech information are integrated into a structured result, such as a vocabulary, and exported to a format that can be used for subsequent analysis (such as JSON, CSV, etc.).

[0127] This embodiment uses automated word segmentation, vocabulary screening, and part-of-speech tagging to generate parsing results that accurately reflect the vocabulary structure and usage frequency in the business data, providing a high-quality foundation for subsequent dimension identification and data analysis, reducing manual processing errors, and improving efficiency.

[0128] In one embodiment, the above S30 includes:

[0129] S301, classifying the time words in the analysis result, and dividing the time words into long-term, medium-term and short-term time dimensions according to the length of the time period;

[0130] S302, evaluating the usage of the time dimension in business scenarios, and selecting a first target time dimension for business analysis based on the frequency of occurrence of the time dimension in different business scenarios;

[0131] S303, evaluating the cross-scenario applicability of the time dimension, determining whether each time dimension is used in multiple business scenarios, and retaining the second target time dimension used in multiple business scenarios;

[0132] S304, evaluating the historical data accumulation capability of the time dimension, determining whether the time dimension supports cross-period data accumulation, and retaining the third target time dimension that supports cross-period accumulation;

[0133] S305, evaluating the trend analysis capability of the time dimension, and selecting a fourth target time dimension that supports long-term trend analysis based on the performance of the time dimension in the trend analysis;

[0134] S306, retaining the time period parts of speech that match the target time dimension, and filtering other time period parts of speech that are not related to the shared dimension.

[0135] In this embodiment, all time-related words (such as "year", "month", "day", etc.) are extracted from the parsing results and divided into three categories: long-term, medium-term and short-term according to the length of the time period. The long-term dimension usually represents years, quarters, etc., the medium-term dimension may involve months, weeks, and the short-term dimension includes days, hours, etc. Time words can be classified through a rule base or an automated algorithm. These words are divided into different time dimension categories based on the time span or business scenario requirements.

[0136] Perform a usage frequency analysis on each time dimension to evaluate its importance in various business scenarios. The key is to identify which time dimensions are used more frequently in business analysis, thereby screening out time dimensions with business value. By counting the usage frequency of time dimensions in various business scenarios, using system logs, query records and other data, analyze the usage frequency of each time dimension in various scenarios.

[0137] According to the frequency of occurrence of time dimensions in different business scenarios, the first target time dimension for business analysis is selected. The first target time dimension refers to the time dimension with the highest frequency of use in a specific business scenario and the strongest relevance to business analysis. Based on the frequency of use, the time dimensions are selected through the preset screening threshold to ensure that business analysis can focus on using these high-frequency time dimensions.

[0138] Analyze whether the time dimension is applicable across scenarios and determine whether a time dimension can be used in multiple business scenarios. If a time dimension can be applied in multiple scenarios, its shared value is higher. By analyzing the usage data of the time dimension in each business scenario, determine whether a time dimension appears in multiple business analysis scenarios, and filter out time dimensions with strong cross-scenario applicability through threshold setting.

[0139] Keep the time dimension used across scenarios, that is, the second target time dimension. Through screening, ensure that these dimensions are not only useful in one scenario, but can also be used in multiple business scenarios and have high shared value. Through cross-scenario usage frequency analysis, screen out the time dimensions that can be applied to multiple business scenarios and keep them as the second target time dimension.

[0140] Analyze whether the time dimension can support data accumulation across time periods to ensure the comparability of data across multiple time periods. For example, the annual and quarterly time dimensions can accumulate data across years, while dimensions such as "day" are more suitable for short-term analysis. Analyze the accumulation cycle of historical data, evaluate whether the time dimension can support historical data accumulation across time periods, and select time dimensions that can accumulate data for a long time.

[0141] The time dimension that supports cross-cycle accumulation is retained, that is, the third target time dimension. These dimensions are suitable for long-term data analysis and historical comparison. By analyzing the effectiveness of historical data accumulation, the time dimension that supports cross-cycle accumulation is selected as the third target time dimension.

[0142] Analyze the performance of the time dimension in trend analysis and determine which time dimensions are suitable for long-term trend analysis. For example, annual and quarterly dimensions are more suitable for long-term trend analysis. Through data trend analysis, determine whether the time dimension can support long-term trend analysis and ensure that the time dimension can be used for trend judgment in business decisions.

[0143] Filter out the time dimension that can support long-term trend analysis, that is, the fourth target time dimension. By analyzing trend data, filter out the time dimension that can provide long-term data trends to ensure that it is used for long-term trend forecasting.

[0144] Finally, the target time dimensions (the first to fourth target time dimensions) that have passed the evaluation are retained to ensure that these time dimensions are highly relevant to business analysis and can be used as shared dimensions across scenarios. The time dimensions that meet the above screening criteria are marked and retained in the vocabulary library as part of the shared dimensions.

[0145] Filter out time period words that are not related to the shared dimension. These words are used less frequently in data analysis or do not have cross-scenario applicability and historical data accumulation capabilities. Based on the results of the above screening, filter out irrelevant time period part-of-speech words automatically or through the rule base to improve analysis efficiency and the quality of shared dimensions.

[0146] This embodiment optimizes the analysis results and performs fine screening and analysis on the time dimension, which can effectively retain the time dimension related to the business and filter out irrelevant time period words, thereby improving the sharing of dimensions and the efficiency of data analysis. The classification, screening and evaluation of the time dimension are automatically processed, reducing manual intervention and ensuring the accuracy of business analysis.

[0147] In one embodiment, in the above S50, adjusting the structure of the data warehouse model based on the shared dimension includes:

[0148] S501, analyzing the relationship between the shared dimension and the data warehouse model, determining the application scenario of the shared dimension in the data warehouse model, and identifying the existing dimension that needs to be replaced, merged or expanded;

[0149] S502: Adjust the dimension table structure of the data warehouse model, add new fields and delete duplicate dimensions according to the definition of shared dimensions;

[0150] S503, reconstructing the fact table structure of the data warehouse model, adjusting the fields and join methods in the fact table, and associating the shared dimensions with the metrics in the fact table;

[0151] S504, updating the association relationship between the dimension table and the fact table, and optimizing the connection method between the shared dimension and the fact table.

[0152] In this embodiment, the compatibility of shared dimensions with existing data warehouse models and their application in different business scenarios are analyzed. It is necessary to identify whether the dimensions in the existing dimension table overlap with the shared dimensions or whether they can be optimized through the shared dimensions. Through automated scripts or manual analysis, the shared dimensions are compared with the existing dimensions of the data warehouse model to analyze the specific application scenarios of the shared dimensions in the data warehouse model. For example, customer dimensions or product dimensions may already exist in the existing model, but may need to be merged or optimized.

[0153] After analyzing the relationship between shared dimensions and the data warehouse model, determine which existing dimensions need to be replaced, merged, or expanded to better support the application of shared dimensions. The goal is to optimize the structure of existing dimensions through shared dimensions, reduce redundancy, and improve the efficiency of the data warehouse model. Perform structural analysis on the dimension tables in the data warehouse model, and based on business needs, confirm which existing dimensions are duplicated with shared dimensions or need further optimization, and formulate specific merging or expansion strategies.

[0154] Adjust the dimension table in the data warehouse model according to the definition of shared dimensions. You may need to add new shared dimension fields and delete existing duplicate dimensions. By simplifying the dimension table structure, reduce data redundancy and improve the query efficiency and consistency of the model. Modify the metadata definition of the data warehouse model and use SQL or ETL tools to ensure that the newly added shared dimension fields can be seamlessly integrated into the existing model and remove redundant dimension fields.

[0155] The fact table is the core data table in the data warehouse model, storing the metrics of business data. The purpose of adjusting the fact table structure is to ensure that the shared dimensions can be correctly associated with the metrics in the fact table. This includes adjusting the field definitions and join methods in the fact table to support the efficient application of shared dimensions. By modifying the field definitions of the fact table, ensure that the metrics can be correctly connected to the shared dimensions, and optimize the join method to improve query performance. This operation usually needs to be performed simultaneously with the adjustment of the dimension table to ensure data consistency and integrity.

[0156] After adjusting the dimension table and fact table, you need to update the relationship between them. The purpose is to optimize the connection between the dimension table and the fact table and improve the performance of data query and analysis. By adjusting the SQL query and the connection conditions of the data model, ensure that the relationship between the dimension table and the fact table is correct and improve the performance of cross-table queries. You can use database optimization tools or SQL tuning to optimize the connection efficiency.

[0157] This embodiment optimizes the dimension table and fact table structure of the data warehouse model by adjusting the data warehouse model based on shared dimensions, reduces duplicate dimensions, and improves data consistency. The introduction of shared dimensions makes the data warehouse model more unified in different business scenarios, improving the performance of data queries and the efficiency of analysis.

[0158] In one embodiment, in the above S50, after adjusting the structure of the data warehouse model based on the shared dimension, the method further includes:

[0159] S601: Import the business data into the adjusted data warehouse model to verify the integrity of the data and the accuracy of the loading;

[0160] S602: Test whether the adjusted data warehouse model structure meets the query requirements in various business scenarios and evaluate the performance of shared dimensions in actual applications.

[0161] S603, verifying the association between the dimension table and the fact table in the data warehouse model, and evaluating the data consistency in the adjusted data warehouse model;

[0162] S604, evaluating the response efficiency of the adjusted data warehouse model in data queries in different scenarios;

[0163] S605, generate a data verification report, record the verification results, and evaluate the availability and performance of the adjusted data warehouse model.

[0164] In this embodiment, after the data warehouse model structure adjustment is completed, the business data (such as transaction records, customer information, etc.) is imported into the new model to ensure that all data can be loaded completely and accurately. This step requires verification that all shared dimensions and measurement data can be correctly mapped to the new data warehouse model without loss or error. When data is imported through the ETL tool, the system will automatically check for anomalies in the data loading process, such as data loss, format inconsistency, etc., and generate a completeness report.

[0165] The adjusted data warehouse model needs to be tested for queries in actual business scenarios to evaluate its applicability in different business scenarios (such as customer analysis, sales data analysis, etc.). Focus on testing whether the shared dimensions can effectively support the query requirements of multiple scenarios. Use common SQL query statements in business scenarios to test the model and verify whether the model can efficiently and accurately return results in actual queries.

[0166] After adjusting the model, ensure that the relationship between the dimension table and the fact table is correct and that data consistency is not affected. Data consistency means that the information in the dimension table (such as customer information) and the measurement data in the fact table (such as sales amount) can correctly correspond and there is no conflict. Use the data consistency verification tool to check whether the relationship between the dimension table and the fact table is complete, and ensure consistency by comparing the data source with the data warehouse.

[0167] The adjusted data warehouse model needs to be tested for query performance to ensure that the response time of data queries can meet business requirements in different business scenarios. Evaluate the impact of shared dimensions on query efficiency and the overall performance of the data warehouse model. Execute complex queries in different business scenarios and measure query response time. Evaluate and optimize using performance optimization tools (such as index optimization, query tuning, etc.).

[0168] After completing data loading, query requirements testing, data consistency verification, and response efficiency evaluation, a data verification report is generated to record the results of all verification steps. The report will describe in detail the availability and performance of the data warehouse model after adjustment, and point out potential optimization points or improvement suggestions. The verification report is automatically generated, and the report includes the results of data loading, query testing, data consistency checking, and performance evaluation, which serves as the basis for subsequent optimization.

[0169] This embodiment ensures that the model can run efficiently and accurately in different business scenarios by performing complete data import, query requirement testing, data consistency verification and performance evaluation on the adjusted data warehouse model. The generated data verification report provides a reliable basis for subsequent model optimization and business analysis, while reducing business risks caused by inconsistent data or insufficient query performance.

[0170] The present invention also provides a shared dimension optimization device for a data warehouse model, referring to Figure 2 , Figure 2 The figure is a functional module diagram of a preferred embodiment of a shared dimension optimization device for a data warehouse model of the present invention. The shared dimension optimization device for a data warehouse model comprises:

[0171] A data collection module collects business data tables, including data list reports and data summary tables, and generates Chinese descriptions of fields in the business data tables;

[0172] A text analysis module, which uses a text analysis method to disassemble and count the Chinese description, annotate the parts of speech, and generate analysis results;

[0173] A data filtering module, which optimizes and processes the parsing results and filters out parts of speech that are irrelevant to the shared dimension, wherein the irrelevant parts of speech include time period parts of speech and measurement parts of speech;

[0174] The dimension identification module performs statistical analysis on the optimized parsing results to identify common dimensions;

[0175] The shared dimension generation and adjustment module processes each identified common dimension in combination with business logic to generate a shared dimension, and adjusts the structure of the data warehouse model based on the shared dimension.

[0176] The specific implementation of the shared dimension optimization device of the data warehouse model of the present invention is basically the same as the various embodiments of the shared dimension optimization method of the data warehouse model mentioned above, and will not be repeated here.

[0177] The present invention also provides a shared dimension optimization device for a data warehouse model, such as Figure 3As shown, the shared dimension optimization device of the data warehouse model may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0178] Those skilled in the art will understand that Figure 3 The hardware structure of the shared dimension optimization device of the data warehouse model shown in the figure does not constitute a limitation of the shared dimension optimization device of the data warehouse model, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0179] like Figure 3 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a shared dimension optimization program of a data warehouse model. The operating system is a program for managing and controlling the shared dimension optimization device and software resources of the data warehouse model, and supports the operation of the network communication module, the user interface module, the shared dimension optimization program of the data warehouse model, and other programs or software; the network communication module is used to manage and control the network interface 1004; and the user interface module is used to manage and control the user interface 1003.

[0180] exist Figure 3 In the hardware structure of the shared dimension optimization device of the data warehouse model shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client and communicate data with the client; the processor 1001 can call the shared dimension optimization program of the data warehouse model stored in the memory 1005, and perform the same operations as the shared dimension optimization method of the data warehouse model.

[0181] The specific implementation of the shared dimension optimization device of the data warehouse model of the present invention is basically the same as the various embodiments of the shared dimension optimization method of the data warehouse model mentioned above, and will not be repeated here.

[0182] In addition, an embodiment of the present invention also proposes a computer storage medium, on which a shared dimension optimization program for a data warehouse model is stored. When the shared dimension optimization program for a data warehouse model is executed by a processor, the steps of the shared dimension optimization method for a data warehouse model as described above are implemented.

[0183] The specific implementation of the computer storage medium of the present invention is basically the same as the various embodiments of the shared dimension optimization method of the above-mentioned data warehouse model, and will not be repeated here.

[0184] It should be noted that if software tools or components other than those of our company appear in the embodiments of the present application, they are only used for illustrative purposes and do not represent actual use.

Claims

1. A shared dimension optimization method for a data warehouse model, characterized in that: The following steps are involved: Collecting business data tables, including data list reports and data summary tables, and generating Chinese descriptions of fields in the business data tables; The Chinese description is disassembled and counted by a text parsing method, and parts of speech are marked to generate a parsing result; Optimizing the parsing results, filtering out parts of speech that are irrelevant to the shared dimension, wherein the irrelevant parts of speech include time period parts of speech and measurement parts of speech; Conduct statistical analysis on the optimized analysis results to identify common dimensions; Each identified common dimension is processed in combination with business logic to generate a shared dimension, and the structure of the data warehouse model is adjusted based on the shared dimension.

2. The shared dimension optimization method of the data warehouse model according to claim 1, characterized in that: Perform statistical analysis on the filtered analysis results to identify common dimensions, including: Performing word frequency statistics on the words in the parsing result to determine the frequency of occurrence of each word; Based on the set word frequency threshold, high-frequency words with a frequency higher than the word frequency threshold are selected from all words; Matching the high-frequency words in the Chinese description of the field, and determining whether the high-frequency words have the same business meaning in multiple Chinese descriptions; High-frequency words with the same business meaning are marked as common dimensions.

3. The shared dimension optimization method of the data warehouse model according to claim 1, characterized in that: Each identified common dimension is processed in combination with business logic to generate shared dimensions, including: Classify the identified common dimensions, analyze the usage and definition of common dimensions in multiple business scenarios, and extract representative scenarios from multiple business scenarios; In representative scenarios, the dimension names, data types, and processing methods in each scenario are analyzed in detail to compare whether there are differences in the use of common dimensions in different scenarios; Compare and analyze the differences in common dimensions, adjust and process the differences by establishing unified standards, and generate shared dimensions.

4. The shared dimension optimization method of the data warehouse model according to claim 1, characterized in that: The Chinese description is disassembled and counted by a text parsing method, and the parts of speech are marked to generate parsing results, including: Performing word segmentation processing on the Chinese description to break down the continuous text into independent words; Filter the words after word segmentation, exclude words whose length is less than the set threshold, and retain other words as candidate words; Perform part-of-speech tagging on the retained candidate words, and mark the grammatical attributes of each candidate word to distinguish different types of words; Perform word frequency statistics on the candidate words, and record the frequency of occurrence of each candidate word in the Chinese description; A parsing result is generated, wherein the parsing result includes candidate words and corresponding word frequencies and part-of-speech tags.

5. The shared dimension optimization method of the data warehouse model as claimed in claim 1, characterized in that: The parsing results are optimized and filtered out parts of speech that are irrelevant to the shared dimension, the irrelevant parts of speech including time period parts of speech and measurement parts of speech, including: Classifying the time words in the analysis results, and dividing the time words into long-term, medium-term and short-term time dimensions according to the length of the time period; Evaluate the use of time dimensions in business scenarios, and select the first target time dimension for business analysis based on the frequency of occurrence of time dimensions in different business scenarios; Evaluate the cross-scenario applicability of the time dimension, determine whether each time dimension is used in multiple business scenarios, and retain the second target time dimension used in multiple business scenarios; Evaluate the historical data accumulation capability of the time dimension, determine whether the time dimension supports cross-period data accumulation, and retain the third target time dimension that supports cross-period accumulation; Evaluate the trend analysis capability of the time dimension, and select the fourth target time dimension that supports long-term trend analysis based on the performance of the time dimension in trend analysis; The time period parts of speech that match the target time dimension are retained, and other time period parts of speech that are not related to the shared dimension are filtered out.

6. The shared dimension optimization method of the data warehouse model according to claim 1, characterized in that: Adjusting the structure of the data warehouse model based on the shared dimension includes: Analyze the relationship between the shared dimension and the data warehouse model, determine the application scenario of the shared dimension in the data warehouse model, and identify the existing dimension that needs to be replaced, merged or expanded; Adjust the dimension table structure of the data warehouse model, add new fields and delete duplicate dimensions according to the definition of shared dimensions; Reconstruct the fact table structure of the data warehouse model, adjust the fields and join methods in the fact table, and associate the shared dimensions with the metrics in the fact table; Update the relationship between dimension tables and fact tables, and optimize the connection method between shared dimensions and fact tables.

7. The shared dimension optimization method of the data warehouse model according to claim 1, characterized in that: After adjusting the structure of the data warehouse model based on the shared dimension, the method further includes: Import business data into the adjusted data warehouse model to verify data integrity and loading accuracy; Test whether the adjusted data warehouse model structure meets the query requirements in various business scenarios, and evaluate the performance of shared dimensions in actual applications; Verify the relationship between dimension tables and fact tables in the data warehouse model, and evaluate the data consistency in the adjusted data warehouse model; Evaluate the response efficiency of the adjusted data warehouse model in data queries in different scenarios; Generate a data validation report, record the validation results, and evaluate the usability and performance of the adjusted data warehouse model.

8. A shared dimension optimization device for a data warehouse model, characterized in that: The shared dimension optimization device of the data warehouse model includes: A data collection module is used to collect business data tables, including data list reports and data summary tables, and generate Chinese descriptions of fields in the business data tables; A text analysis module, used to disassemble and count the Chinese descriptions through a text analysis method, and to mark the parts of speech to generate analysis results; A data filtering module, used for optimizing the processing of the parsing results, filtering out parts of speech that are irrelevant to the shared dimension, wherein the irrelevant parts of speech include time period parts of speech and measurement parts of speech; Dimension identification module, used to perform statistical analysis on the optimized parsing results and identify common dimensions; The shared dimension generation and adjustment module is used to process each identified common dimension in combination with business logic to generate a shared dimension, and adjust the structure of the data warehouse model based on the shared dimension.

9. A shared dimension optimization device for a data warehouse model, characterized in that: The shared dimension optimization device of the data warehouse model includes a memory, a processor, and a shared dimension optimization program of the data warehouse model stored in the memory and executable on the processor. When the shared dimension optimization program of the data warehouse model is executed by the processor, the steps of the shared dimension optimization method of the data warehouse model as described in any one of claims 1 to 7 are implemented.

10. A computer storage medium, characterized in that: The storage medium stores a shared dimension optimization program for a data warehouse model, and when the shared dimension optimization program for a data warehouse model is executed by a processor, the steps of the shared dimension optimization method for a data warehouse model as described in any one of claims 1 to 7 are implemented.

Images