Shared dimension optimization method, device and equipment of data warehouse model and storage medium

Abstract

The application relates to a shared dimension optimization method of a data warehouse model. By collecting business data tables, Chinese descriptions of fields are generated, text analysis methods are used to disassemble and count the Chinese descriptions, and the parts of speech are marked and the analysis results are generated. The analysis results are optimized, the parts of speech irrelevant to the shared dimensions are filtered, and the time period parts of speech and the metric parts of speech are screened out. The optimized analysis results are statistically analyzed, and the dimensions with common characteristics are identified. The identified common dimensions are processed based on business logic to generate shared dimensions. The structure of the data warehouse model is adjusted through the generated shared dimensions, the structure design of the dimension table and the fact table is improved, and the data query efficiency is optimized. The application reduces the workload of manual research and data analysis, improves the efficiency of dimension identification and sharing through automatic processing, shortens the data sorting cycle, reduces the labor cost, and adapts to the data processing requirements of complex business scenarios.

Description

Technical Field

[0001] This invention relates to the fields of big data technology and fintech, and particularly to a method, apparatus, device, and storage medium for optimizing the shared dimensions of a data warehouse model. Background Technology

[0002] In finance and other data-intensive industries, the organization and sharing of data dimensions are crucial components of data analysis and business decision-making. Shared dimensions typically include primary dimensions highly relevant to the business (such as products, customers, and institutions), other analytical dimensions (such as business type and payment methods), and tags used to modify or define business characteristics (such as whether it is a self-purchased vehicle or a new energy vehicle). These dimensions provide the foundation for enterprise data analysis, supporting cross-departmental data sharing and decision optimization.

[0003] However, existing methods for dimension analysis face numerous challenges and shortcomings in practical applications. Traditional methods primarily rely on manually initiated business surveys and requirements analysis processes, or on manual analysis of data processing logic. These methods have the following significant drawbacks:

[0004] Cross-departmental research is challenging: sharing data dimensions requires collaboration and communication between different departments. However, cross-departmental data research is often difficult to coordinate, as different departments have different data standards and analytical needs, leading to a complex and inefficient research process.

[0005] Long research and analysis cycle: Manual research and analysis of shared dimensions require a lot of time and effort, especially when multiple business scenarios are involved. The amount of data processed manually is huge, resulting in a long analysis cycle and difficulty in responding to changes in business needs in a timely manner.

[0006] The burden of data processing logic analysis is heavy: existing methods require manual analysis of each data processing logic. This not only requires highly skilled professionals but also a large amount of human resources, making the process of sorting out data processing logic an extremely time-consuming and costly one.

[0007] These shortcomings make existing dimensional analysis methods inefficient when dealing with complex data environments and rapidly changing business needs, hindering their rapid implementation in real-world business applications. Therefore, how to utilize a more intelligent approach to analyze shared dimensions, improve cross-departmental collaboration efficiency, and reduce manpower investment has become an urgent problem to solve. Summary of the Invention

[0008] The main objective of this invention is to provide a method, apparatus, device, and storage medium for optimizing the shared dimensions of a data warehouse model, aiming to solve the technical problem that existing technologies rely on manual business research and data processing logic analysis, making it difficult to efficiently identify the shared nature of dimensions.

[0009] To achieve the above objectives, this invention provides a method for optimizing the shared dimension of a data warehouse model, comprising:

[0010] Collect business data tables, which include data list reports and data summary tables, and generate Chinese descriptions of the fields in the business data tables;

[0011] The Chinese description is broken down and statistically analyzed using text parsing methods, and its parts of speech are tagged to generate parsing results;

[0012] The parsing results are optimized by filtering out parts of speech that are irrelevant to the shared dimension. These irrelevant parts of speech include time-period parts of speech and metric parts of speech.

[0013] Statistical analysis was performed on the optimized parsing results to identify common dimensions;

[0014] For each identified common dimension, it is processed in conjunction with business logic to generate a shared dimension, and the structure of the data warehouse model is adjusted based on the shared dimension.

[0015] In one embodiment, statistical analysis is performed on the filtered parsing results to identify common dimensions, including:

[0016] The frequency of each word in the analysis results is determined by performing word frequency statistics.

[0017] Based on a set word frequency threshold, high-frequency words that appear more frequently than the threshold are selected from all words.

[0018] The high-frequency words are matched in the Chinese description of the field to determine whether the high-frequency words have the same business meaning in multiple Chinese descriptions;

[0019] High-frequency words with the same business meaning are marked as common dimensions.

[0020] In one embodiment, each identified common dimension is processed in conjunction with business logic to generate a shared dimension, including:

[0021] The identified common dimensions are classified, their usage and definition are analyzed in multiple business scenarios, and representative scenarios are extracted from multiple business scenarios.

[0022] In representative scenarios, we conduct a detailed analysis of the dimension names, data types, and processing methods for each scenario, and compare whether there are differences in the use of common dimensions in different scenarios.

[0023] By comparing and analyzing the differences in common dimensions, and by establishing unified standards to adjust and process the differences, shared dimensions are generated.

[0024] In one embodiment, the Chinese description is broken down and statistically analyzed using a text parsing method, and part-of-speech tags are assigned to generate parsing results, including:

[0025] The Chinese description is segmented into words, breaking down the continuous text into independent words;

[0026] After word segmentation, words are filtered out, words shorter than a set threshold are excluded, and other words are retained as candidate words.

[0027] Part-of-speech tagging is performed on the retained candidate words, and grammatical attributes are labeled for each candidate word to distinguish different types of words;

[0028] Frequency statistics were performed on the candidate words to record the frequency of each candidate word in the Chinese description.

[0029] Generate parsing results, which include candidate words and their corresponding word frequencies and part-of-speech tags.

[0030] In one embodiment, the parsing results are optimized by filtering out parts of speech that are unrelated to the shared dimension. These unrelated parts of speech include time-periodic and metric parts of speech, including:

[0031] The time-related terms in the analysis results are classified into long-term, medium-term, and short-term time dimensions based on the length of the time period.

[0032] Evaluate the use of the time dimension in business scenarios, and select the primary target time dimension for business analysis based on the frequency of the time dimension in different business scenarios.

[0033] Assess the cross-scenario applicability of the time dimension, determine whether each time dimension is used in multiple business scenarios, and retain the second target time dimension used in multiple business scenarios;

[0034] Assess the historical data accumulation capability of the time dimension, determine whether the time dimension supports cross-cycle data accumulation, and retain the third target time dimension that supports cross-cycle accumulation;

[0035] Assess the trend analysis capability of the time dimension, and select a fourth target time dimension to support long-term trend analysis based on the performance of the time dimension in trend analysis.

[0036] Retain time-period parts of speech that match the target time dimension, and filter out other time-period parts of speech that are unrelated to the shared dimension.

[0037] In one embodiment, adjusting the structure of the data warehouse model based on the shared dimension includes:

[0038] Analyze the relationship between the shared dimension and the data warehouse model, determine the application scenarios of the shared dimension in the data warehouse model, and identify existing dimensions that need to be replaced, merged, or expanded;

[0039] Adjust the dimension table structure of the data warehouse model, adding fields and deleting duplicate dimensions according to the definition of shared dimensions;

[0040] Reconstruct the fact table structure of the data warehouse model, adjust the fields and join methods in the fact table, and associate the shared dimensions with the metrics in the fact table;

[0041] Update the relationship between the dimension table and the fact table, and optimize the connection method between shared dimensions and the fact table.

[0042] In one embodiment, after adjusting the structure of the data warehouse model based on the shared dimension, the method further includes:

[0043] Import business data into the adjusted data warehouse model to verify the integrity of the data and the accuracy of the loading.

[0044] Test whether the adjusted data warehouse model structure meets the query requirements in various business scenarios, and evaluate the performance of shared dimensions in practical applications;

[0045] Verify the relationships between dimension tables and fact tables in the data warehouse model, and evaluate the data consistency in the adjusted data warehouse model;

[0046] Evaluate the response efficiency of the adjusted data warehouse model in data queries across different scenarios;

[0047] Generate a data validation report, record the validation results, and evaluate the availability and performance of the adjusted data warehouse model.

[0048] Furthermore, to achieve the above objectives, the present invention also provides a shared dimension optimization device for a data warehouse model, the shared dimension optimization device for a data warehouse model including a memory, a processor, and a shared dimension optimization program for a data warehouse model stored in the memory and executable on the processor, wherein when the shared dimension optimization program for a data warehouse model is executed by the processor, it implements the steps of the shared dimension optimization method for a data warehouse model as described above.

[0049] Furthermore, to achieve the above objectives, the present invention also provides a computer storage medium storing a shared dimension optimization program for a data warehouse model, wherein when the shared dimension optimization program for the data warehouse model is executed by a processor, the shared dimension optimization program for the data warehouse model implements the steps of the shared dimension optimization method for the data warehouse model as described above.

[0050] Beneficial Effects: This invention relates to a shared dimension optimization method for a data warehouse model. It involves collecting business data tables, generating Chinese descriptions of fields, and using text parsing methods to break down and statistically analyze these descriptions, tagging parts of speech, and generating parsing results. The parsing results are then optimized by filtering out parts of speech irrelevant to the shared dimensions, and eliminating time-period and metric-based parts of speech. Statistical analysis is performed on the optimized parsing results to identify dimensions with common characteristics. These common dimensions are then processed based on business logic to generate shared dimensions. The generated shared dimensions are used to adjust the structure of the data warehouse model, improving the structural design of the dimension table and fact table, and optimizing data query efficiency. This invention reduces the workload of manual surveys and data analysis, improves the efficiency of dimension identification and sharing through automated processing, shortens the data processing cycle, reduces labor costs, and adapts to the data processing needs of complex business scenarios. Attached Figure Description

[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0052] Figure 1 This is a flowchart illustrating an embodiment of the shared dimension optimization method for the data warehouse model of the present invention;

[0053] Figure 2 A schematic diagram of the functional modules of a preferred embodiment of the shared dimension optimization device for the data warehouse model of the present invention;

[0054] Figure 3 This is a schematic diagram of the hardware operating environment of the device involved in the embodiment of the shared dimension optimization device scheme of the data warehouse model of the present invention. Detailed Implementation

[0055] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0056] In finance and other data-intensive industries, the organization and sharing of data dimensions are crucial components of data analysis and business decision-making. Shared dimensions typically include primary dimensions highly relevant to the business (such as products, customers, and institutions), other analytical dimensions (such as business type and payment methods), and tags used to modify or define business characteristics (such as whether it is a self-purchased vehicle or a new energy vehicle). These dimensions provide the foundation for enterprise data analysis, supporting cross-departmental data sharing and decision optimization.

[0057] However, existing methods for dimension analysis face numerous challenges and shortcomings in practical applications. Traditional methods primarily rely on manually initiated business surveys and requirements analysis processes, or on manual analysis of data processing logic. These methods have the following significant drawbacks:

[0058] Cross-departmental research is challenging: sharing data dimensions requires collaboration and communication between different departments. However, cross-departmental data research is often difficult to coordinate, as different departments have different data standards and analytical needs, leading to a complex and inefficient research process.

[0059] Long research and analysis cycle: Manual research and analysis of shared dimensions require a lot of time and effort, especially when multiple business scenarios are involved. The amount of data processed manually is huge, resulting in a long analysis cycle and difficulty in responding to changes in business needs in a timely manner.

[0060] The burden of data processing logic analysis is heavy: existing methods require manual analysis of each data processing logic. This not only requires highly skilled professionals but also a large amount of human resources, making the process of sorting out data processing logic an extremely time-consuming and costly one.

[0061] These shortcomings make existing dimensional analysis methods inefficient when dealing with complex data environments and rapidly changing business needs, hindering their rapid implementation in real-world business applications. Therefore, how to utilize a more intelligent approach to analyze shared dimensions, improve cross-departmental collaboration efficiency, and reduce manpower investment has become an urgent problem to solve.

[0062] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the shared dimension optimization method for a data warehouse model provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0063] like Figure 1 As shown, the shared dimension optimization method for the data warehouse model proposed in this invention includes the following steps:

[0064] S10, collect business data tables, which include data list reports and data summary tables, and generate Chinese descriptions of the fields in the business data tables;

[0065] In this embodiment, relevant business data tables are obtained and analyzed from different business systems. These data tables include various structured data, such as transaction data, user information, and product information. The data tables can originate from the enterprise's internal database, ERP system, or external collaborative platform. Data collection is performed through APIs, batch import, or real-time data streaming to ensure the integrity and timeliness of the data tables.

[0066] A data list report is a collection of data recording details of business operations, such as customer transaction details and product sales details; a data summary report is the result of summarizing and processing the list data, such as daily sales totals and customer distribution. Data list reports typically have a higher granularity, while data summary reports are used for higher-level business analysis. The collection of these two tables is usually accomplished through SQL queries, ETL (Extract, Transform, Load) tools, or other data analysis platforms.

[0067] Descriptive descriptions are generated for the collected data table fields. These fields may exist in encoded, abbreviated, or other forms. For subsequent analysis, natural language Chinese descriptions need to be generated for these fields. This can be done through automated scripts (such as translation tools or dictionaries combined with field names), rule bases, or by using data dictionaries and historical business rules.

[0068] In one specific implementation, an enterprise data platform is employed to obtain data tables distributed across multiple business systems in real time via API interfaces. These data tables contain customer account information, transaction records, and compliance review data. The data platform uses a microservice architecture to ensure seamless data collection across different business systems. Combined with a big data processing framework (such as Apache Kafka), it ensures real-time data flow and supports high-concurrency data requests.

[0069] By automating the collection of business data tables and generating Chinese descriptions, errors and inconsistencies that occur when manually generating descriptions are avoided, greatly improving the efficiency and accuracy of data processing.

[0070] S20, The Chinese description is broken down and statistically analyzed using text parsing methods, and part-of-speech tags are assigned to generate parsing results;

[0071] In this embodiment, Natural Language Processing (NLP) techniques are used to process the Chinese description using specific text parsing algorithms. Commonly used methods include rule-based word segmentation algorithms (such as jieba segmentation) or deep learning models (such as pre-trained models like BERT). The core goal of text parsing is to break down continuous text into more easily processed independent lexical units.

[0072] Decomposition refers to breaking down complex Chinese descriptions into individual words using word segmentation technology. For example, "customer account balance" can be broken down into "customer," "account," and "balance." After word segmentation, frequency statistics are performed on each word to record its occurrence frequency in different descriptions, facilitating subsequent analysis.

[0073] Part-of-speech tagging (POS) involves performing lexical analysis on each word and labeling its grammatical role (e.g., noun, verb, adjective). Through POS tagging, the system can better understand the meaning of each word within the business logic. POS tagging can be performed using statistical models or rule-based systems.

[0074] After decomposition, statistics, and part-of-speech tagging, a parsing result containing all words, word frequencies, and part-of-speech tags is generated. These results will be used for subsequent common dimension identification and model tuning.

[0075] In one specific implementation, Chinese descriptions are segmented using an industry-standard word segmentation library (such as jieba). The word segmentation library is customized and optimized based on a financial terminology list to ensure correct parsing of specialized terms. After segmentation, the extracted words are statistically analyzed and sorted by frequency of occurrence for subsequent filtering of high-frequency words. A rule-based part-of-speech tagging system is used to label the extracted words as nouns, verbs, etc., ensuring accurate part-of-speech tagging of financial terms. A result set containing word information, word frequency, and part-of-speech tagging is generated for subsequent analysis.

[0076] By using text parsing methods, complex Chinese descriptions are efficiently decomposed and labeled with parts of speech, and parsing results are automatically generated, reducing manual intervention and improving the accuracy and efficiency of data processing. This provides high-quality input data for subsequent common dimension identification.

[0077] S30, optimize the parsing results and filter out parts of speech that are unrelated to the shared dimension, including time periodic parts of speech and metric parts of speech;

[0078] In this embodiment, the results generated after text parsing are optimized. The optimization primarily targets the previously parsed and statistically analyzed vocabulary data, aiming to improve the recognition accuracy of shared dimensions. By reducing interference from noisy data, highly business-relevant words can be correctly identified. Which words require optimization can be defined through rule bases, manual settings, or automated models. The optimization process can dynamically adjust the vocabulary filtering rules based on business needs.

[0079] In the analysis results, some words are not directly related to the construction of the shared dimensions, such as words representing time or measurement. The presence of these words may affect the identification of the shared dimensions, so filtering is necessary. First, words are tagged with parts of speech. Then, based on business rules or part-of-speech classification rules, words irrelevant to the shared dimensions are removed. Common methods include automatic filtering algorithms based on part-of-speech tagging, or manually defined vocabulary category libraries.

[0080] Time-related parts of speech refer to time-related words such as "year," "month," and "day." These words often represent time spans in data analysis but do not directly contribute to shared dimensions. Measurement-related parts of speech include measurement-related words such as "amount" and "number of times." These words are related to specific numerical values ​​but are not used as shared dimensions. These words can be predefined using part-of-speech tagging tools, and then automatically filtered out using business rules from a vocabulary database.

[0081] By optimizing the parsing results and filtering out irrelevant parts of speech, noise in the data is reduced, and the recognition accuracy of shared dimensions is improved. Especially in big data analysis scenarios, it can significantly improve processing efficiency and reduce manual intervention.

[0082] S40: Perform statistical analysis on the optimized parsing results to identify common dimensions;

[0083] In this embodiment, the parsing results after filtering out irrelevant parts of speech retain high-quality words related to the shared dimensions. These words, after initial screening, possess stronger business relevance and are suitable for further statistical analysis. The results generated through text parsing, after removing irrelevant words, form a more concise vocabulary set. The optimized results serve as the foundational data for subsequent dimension identification.

[0084] Statistical analysis of the optimized vocabulary, focusing on its quantity and distribution characteristics, is performed to determine which words frequently appear across multiple datasets and business scenarios, and to attempt to identify commonalities among these words. This statistical analysis can include word frequency statistics, business relevance analysis of words, and analysis of word combinations within context. Commonly used techniques include word frequency statistics, n-gram models, and the TF-IDF algorithm, used to calculate word frequency, weight, and importance in business scenarios. Furthermore, statistical analysis can be combined with historical data to further filter out words with common characteristics.

[0085] Common dimensions refer to dimensions that frequently appear and have consistent business meanings across multiple business scenarios and datasets. Statistical analysis of vocabulary can identify which dimensions are universal across multiple scenarios, and these dimensions can be used as shared dimensions. The identification of common dimensions can be accomplished through frequency statistics and cross-validation of vocabulary across business application scenarios. For example, if certain words frequently appear in multiple data tables and have the same business meaning, they can be identified as common dimensions. Furthermore, semantic analysis tools can be used to further improve the accuracy of common dimension identification.

[0086] By performing statistical analysis on the optimized parsing results, common dimensions across multiple business scenarios can be efficiently identified, reducing the complexity and time-consuming nature of manual analysis. It can automatically filter out dimensions with common characteristics across multiple data tables, improving the accuracy and consistency of data sharing, supporting cross-business department data analysis and decision-making, increasing the automation level of dimension identification, and adapting to the data processing needs of complex business scenarios.

[0087] S50: For each identified common dimension, process it in conjunction with business logic to generate a shared dimension, and adjust the structure of the data warehouse model based on the shared dimension.

[0088] In this embodiment, common dimensions are universally applicable dimensions derived from statistical analysis of multiple data tables and business scenarios. The process of identifying these common dimensions includes frequency analysis of vocabulary and business logic verification, ultimately selecting dimensions that can be used across scenarios as the basis for shared dimensions. Optimized and verified vocabulary is extracted from the parsing results and used as the foundation for common dimensions. These dimensions have cross-scenario business relevance, facilitating sharing.

[0089] Based on the identified common dimensions, they are further processed and refined in conjunction with specific business logic. This processing includes adjusting the dimension definitions, adding or modifying dimension attributes to ensure the dimensions match business needs. Utilizing rule bases, manual adjustments, or automated tools to process dimensions allows for their expansion or optimization according to different business requirements, enabling these dimensions to more accurately reflect key indicators in the business scenario.

[0090] After processing, a shared dimension is finally generated that can be used across multiple business scenarios. This shared dimension, adjusted according to business logic, has the capability to be used across departments and systems, supporting unified data analysis and decision-making. The generation of shared dimensions can be automated using tools or scripts, adding the processed dimensions to the data model to ensure that various business departments can perform data analysis and share within the same dimensional framework.

[0091] The structural adjustments to the data warehouse model are designed to better accommodate the generated shared dimensions. Introducing shared dimensions requires optimizing the data warehouse's dimension tables, fact tables, and other components to support cross-scenario data querying and analysis. The structural adjustments also need to consider factors such as data consistency and query efficiency. By modifying the dimension table structure within the data warehouse model, adjusting or adding shared dimension fields, and optimizing the join method between dimensions and fact tables, the flexibility and query efficiency of the data warehouse model can be improved.

[0092] In one implementation, the identified common dimensions are adjusted and expanded based on a business rule base. For the customer dimension, customer types can be further refined based on attributes such as customer transaction behavior and preferences. The processed customer dimensions are defined as standard shared dimensions, applicable to multiple business scenarios for customer segmentation analysis. A shared field "Customer Type" is added to the customer dimension table in the data warehouse model, and by joining the fact table, it supports rapid queries for customer behavior analysis. The dimension processing based on business logic makes the shared dimensions more accurate, and the optimization of the data warehouse model improves the accuracy of customer behavior analysis.

[0093] By processing the identified common dimensions in conjunction with business logic and generating shared dimensions, cross-departmental sharing and unified use of dimensions can be achieved, improving the accuracy and consistency of data analysis. After adjusting the data warehouse model structure, the data warehouse model can adapt to more business scenarios, improving the efficiency of data querying and the flexibility of the system.

[0094] This invention relates to a method for optimizing shared dimensions in a data warehouse model. It involves collecting business data tables, generating Chinese descriptions of fields, and using text parsing methods to break down and statistically analyze these descriptions, tagging parts of speech, and generating parsing results. The parsing results are then optimized by filtering out parts of speech irrelevant to the shared dimensions, and eliminating time-related and metric-related parts of speech. Statistical analysis is performed on the optimized parsing results to identify dimensions with common characteristics. These common dimensions are then processed based on business logic to generate shared dimensions. The generated shared dimensions are used to adjust the structure of the data warehouse model, improving the structural design of the dimension table and fact table, and optimizing data query efficiency. This invention reduces the workload of manual surveys and data analysis, improves the efficiency of dimension identification and sharing through automated processing, shortens the data processing cycle, reduces labor costs, and adapts to the data processing needs of complex business scenarios.

[0095] In one embodiment, S40 includes:

[0096] S401, Perform word frequency statistics on the words in the parsing results to determine the frequency of occurrence of each word;

[0097] S402, Based on the set word frequency threshold, select high-frequency words from all words that appear more frequently than the word frequency threshold;

[0098] S403, Match the high-frequency words in the Chinese description of the field to determine whether the high-frequency words have the same business meaning in multiple Chinese descriptions;

[0099] S404 marks high-frequency words with the same business meaning as common dimensions.

[0100] In this embodiment, word frequency statistics refer to counting the number of times each word appears in the dataset or field description after parsing. By counting word frequencies, we can determine which words have a higher usage frequency in the business data and provide a basis for subsequent high-frequency word selection. Using statistical tools or scripts, we iterate through all the words in the parsed results and record the occurrence frequency of each word. We can use Python's `collections.Counter` or other similar tools for fast statistics.

[0101] A word frequency threshold is a pre-set value used to filter words that appear frequently. Words with a frequency above this threshold are considered high-frequency words, while words with a frequency below the threshold are ignored. This step helps eliminate low-frequency words that occasionally appear in the business context, ensuring that only words with strong business relevance are retained. After word frequency statistics are performed, words with frequencies exceeding the pre-set threshold are filtered out. The filtering results can be optimized by adjusting the word frequency threshold to ensure that the selected words have high business value.

[0102] For the selected high-frequency words, further matching is performed to determine whether they have the same business meaning in the field descriptions of different data tables. The aim is to ensure that words can represent the same or similar business concepts in different business scenarios, thus possessing the potential to serve as a shared dimension. Semantic or word-meaning matching tools are used in the Chinese field descriptions to determine whether high-frequency words have the same business meaning in different field descriptions. Natural Language Processing (NLP) tools or regular expressions can be used for matching.

[0103] When a high-frequency term has the same business meaning across multiple business scenarios or fields, it can be marked as a common dimension. Common dimensions are applicable across scenarios and can be shared and reused across different data tables and business contexts. High-frequency terms with the same business meaning are identified and marked as common dimensions using annotation or data modeling tools. These dimensions can serve as the basis for subsequent data analysis and business decisions.

[0104] This embodiment performs word frequency statistics and high-frequency word screening on the filtered parsing results, which can quickly identify representative high-frequency words in the business scenario, further match their business meanings, and finally mark them as common dimensions. This step improves the automation of dimension identification, reduces human intervention, and makes data processing and analysis more accurate.

[0105] In one embodiment, in S50 above, each identified common dimension is processed in conjunction with business logic to generate a shared dimension, including:

[0106] S501 classifies the identified common dimensions, analyzes the usage and definition of common dimensions in multiple business scenarios, and extracts representative scenarios from multiple business scenarios.

[0107] S502, in representative scenarios, conduct a detailed analysis of the dimension names, data types and processing methods in each scenario, and compare whether there are differences in the use of common dimensions in different scenarios;

[0108] S503 compares and analyzes the differences in common dimensions, and adjusts and processes the differences by establishing unified standards to generate shared dimensions.

[0109] In this embodiment, after preliminary word frequency statistics and business meaning matching, several dimensions with common characteristics have been identified. These dimensions are further classified according to their business nature and usage scenarios. For example, customer dimension, product dimension, time dimension, etc. The purpose of classification is to better understand the usage characteristics of dimensions in different business scenarios. Dimensions are classified through business logic and dimension attributes. Based on business needs or system requirements, dimensions can be divided into primary dimensions (such as customer, product, etc.) and auxiliary dimensions (such as time, region, etc.). This classification process can be rule-driven or automatically classified using a classification model.

[0110] This analysis examines the usage of common dimensions across various business scenarios, including their frequency of use, data processing logic, and definitions in practical applications. The aim is to gain a deep understanding of dimension application in multiple scenarios to ensure the universality and accuracy of shared dimensions. The analysis also examines the specific usage details of common dimensions in multiple business scenarios. By utilizing data such as system logs and query records, the analysis clarifies the usage of dimensions and assesses their cross-scenario applicability.

[0111] Extract the most representative specific scenarios from different business contexts for further dimensional analysis. For example, in a banking system, credit card business, loan business, and payment business can be selected as representative scenarios to analyze the usage of dimensions within these scenarios. Representative scenarios are identified by analyzing business flows and data flows. Key business scenarios can be selected using data analysis tools or manually.

[0112] In representative scenarios, we analyze in detail the names, data types, and processing methods of common dimensions. By analyzing these details, we understand whether the definitions and processing methods of dimensions are consistent across different business scenarios. Using data queries and system logs, we examine the names and data types of dimensions in each scenario, as well as their specific usage in the data processing flow. By comparing the dimension definitions in different scenarios, we identify inconsistencies.

[0113] By comparing and analyzing the dimension definitions across multiple business scenarios, differences were identified. These differences may lie in the dimension naming conventions, data types, or processing logic. Analysis tools or scripts were used to compare these differences across different scenarios, generating a difference analysis report to identify areas requiring adjustment and standardization.

[0114] Based on the results of the variance analysis, unified business standards are established to adjust and process the dimensions, ultimately generating shared dimensions that can be used across different scenarios. The unified standards include standardized specifications for dimension names, data types, and business processing logic. These unified standards are developed by combining business requirements, technical specifications, and variance analysis reports. By adjusting the definition and usage of dimensions in different scenarios, it is ensured that the shared dimensions are applicable to various business scenarios.

[0115] This embodiment effectively improves the universality of dimensions by classifying, analyzing, and adjusting the differences in common dimensions, enabling them to be shared and used across business scenarios. This reduces redundant definitions and inconsistencies in data analysis, thereby improving the efficiency and accuracy of data analysis.

[0116] In one embodiment, S20 includes:

[0117] S201, The Chinese description is segmented into words to break down the continuous text into independent words;

[0118] S202, filter the words after word segmentation, exclude words whose length is less than the set threshold, and retain other words as candidate words;

[0119] S203, perform part-of-speech tagging on the retained candidate words and tag each candidate word with grammatical attributes to distinguish different types of words;

[0120] S204, Perform word frequency statistics on the candidate words and record the frequency of each candidate word in the Chinese description;

[0121] S205, Generate parsing results, which include candidate words and corresponding word frequencies and part-of-speech tags.

[0122] In this embodiment, word segmentation is one of the fundamental steps in natural language processing, primarily aimed at breaking down continuous Chinese text into individual words. Unlike English text, Chinese text does not contain natural spaces, thus requiring word segmentation tools to identify word boundaries within the text. Common Chinese word segmentation tools (such as Jieba, NLPIR, etc.) can be used to segment Chinese descriptive text. These tools, based on dictionaries and statistical models, can identify individual words and break down the text into independent terms.

[0123] After word segmentation, there are often some meaningless short words, such as punctuation marks or single letters. These words are not helpful for business analysis, so it is necessary to set a word length threshold to exclude words shorter than that threshold. A word length threshold (e.g., 2 characters) is set, and words that meet this length are selected programmatically and retained as candidate words, while short words are filtered out to reduce interference from noisy data.

[0124] Part-of-speech tagging (POS) involves attaching grammatical attributes, such as noun, verb, or adjective, to each word. This tagging helps to understand the role of words in a sentence, facilitating subsequent semantic analysis and business relevance assessment. Common POS tagging tools (such as Stanford NLP and HanLP) can assign POS labels to each word based on context. Based on pre-trained models, these tools can identify the grammatical attributes of words and annotate their part-of-speech information.

[0125] Word frequency statistics record the frequency of each candidate word in a text to determine its importance. Words with high frequency typically have high relevance in the business context and can provide a basis for subsequent identification of commonalities. This can be implemented programmatically to count the frequency of each word. Python's `collections.Counter` or similar tools can be used to quickly count word frequencies, supporting the subsequent filtering of high-frequency words.

[0126] The parsing results are the final output of word segmentation and part-of-speech tagging, containing a list of candidate words, word frequency information for each word, and part-of-speech tagging. These parsing results will serve as input for subsequent steps (such as commonality dimension identification). The segmented words, their corresponding word frequencies, and part-of-speech information are integrated into a structured result, such as a vocabulary list, and exported in a format that can be used for subsequent analysis (such as JSON, CSV, etc.).

[0127] This embodiment generates parsing results that accurately reflect the lexical structure and usage frequency in business data through automated word segmentation, vocabulary filtering, and part-of-speech tagging. This provides a high-quality foundation for subsequent dimension identification and data analysis, reduces errors from manual processing, and improves efficiency.

[0128] In one embodiment, S30 includes:

[0129] S301, Classify the time words in the parsing results, and divide the time words into long-term, medium-term and short-term time dimensions according to the length of the time period;

[0130] S302, Evaluate the use of the time dimension in business scenarios, and select the primary target time dimension for business analysis based on the frequency of the time dimension in different business scenarios.

[0131] S303, assess the cross-scenario applicability of the time dimension, determine whether each time dimension is used in multiple business scenarios, and retain the second target time dimension used in multiple business scenarios;

[0132] S304, assess the historical data accumulation capability of the time dimension, determine whether the time dimension supports cross-cycle data accumulation, and retain the third target time dimension that supports cross-cycle accumulation;

[0133] S305, Evaluate the trend analysis capability of the time dimension, and select a fourth target time dimension to support long-term trend analysis based on the performance of the time dimension in trend analysis.

[0134] S306 retains time-period parts of speech that conform to the target time dimension, and filters out other time-period parts of speech that are unrelated to the shared dimension.

[0135] In this embodiment, all time-related terms (such as "year," "month," and "day") are extracted from the parsing results and categorized into three types based on the length of the time period: long-term, medium-term, and short-term. Long-term dimensions typically represent years, quarters, etc., medium-term dimensions may involve months and weeks, and short-term dimensions include days and hours. Time-related terms can be classified using rule bases or automated algorithms. Based on the time span or business scenario requirements, these terms are divided into different time dimension categories.

[0136] Perform usage frequency analysis on each time dimension to assess its importance in various business scenarios. The key is to identify which time dimensions have a high usage frequency in business analysis, thereby filtering out time dimensions with business value. This is done by statistically analyzing the usage frequency of each time dimension in various business scenarios, utilizing data such as system logs and query records.

[0137] Based on the frequency of time occurrence in different business scenarios, the primary target time dimension for business analysis is selected. The primary target time dimension refers to the time dimension that is used most frequently in a specific business scenario and is most relevant to business analysis. Based on usage frequency, time dimensions are selected using preset screening thresholds to ensure that business analysis focuses on using these high-frequency time dimensions.

[0138] Analyze the applicability of the time dimension across different scenarios to determine if a particular time dimension can be used in multiple business scenarios. A time dimension with broad applicability across multiple scenarios has higher shared value. By analyzing the usage data of the time dimension in various business scenarios, determine if a particular time dimension appears in multiple business analysis scenarios, and filter out time dimensions with strong cross-scenario applicability through threshold settings.

[0139] Retain the time dimension that can be used across multiple scenarios, i.e., the second target time dimension. Through screening, ensure that these dimensions are not only useful in one scenario but also usable in multiple business scenarios, possessing high shared value. Through cross-scenario usage frequency analysis, screen out the time dimensions that can be applied to multiple business scenarios and retain them as the second target time dimensions.

[0140] Analyze whether the time dimension can support data accumulation across time periods to ensure data comparability across multiple time periods. For example, annual and quarterly time dimensions can accumulate data across years, while dimensions like "days" are more suitable for short-term analysis. Analyze the accumulation period of historical data to assess whether the time dimension supports the accumulation of historical data across time periods, and select time dimensions that can accumulate data over a long period.

[0141] We retain time dimensions that support cross-cycle accumulation, i.e., the third target time dimension. These dimensions are suitable for long-term data analysis and historical comparison. By analyzing the effectiveness of historical data accumulation, we select time dimensions that support cross-cycle accumulation as the third target time dimension.

[0142] Analyze the performance of the time dimension in trend analysis to determine which time dimensions are suitable for long-term trend analysis, such as annual and quarterly dimensions. Through data trend analysis, determine whether the time dimension can support long-term trend analysis, ensuring that the time dimension can be used to determine trends for business decision-making.

[0143] The fourth target time dimension is selected by screening out time dimensions that can support long-term trend analysis. This involves analyzing trend data to identify time dimensions that provide long-term data trends, ensuring their suitability for long-term trend prediction.

[0144] Ultimately, the target time dimensions that pass the evaluation (target time dimensions one through four) are retained, ensuring that these time dimensions are highly relevant to business analysis and can be used as shared dimensions across scenarios. Time dimensions that meet the above screening criteria are marked and retained in the vocabulary as part of the shared dimensions.

[0145] Filter out time-period terms irrelevant to the shared dimensions. These terms are used infrequently in data analysis or lack cross-scenario applicability and historical data accumulation capabilities. Based on the aforementioned filtering results, filter irrelevant time-period part-of-speech terms automatically or through a rule base to improve analysis efficiency and the quality of the shared dimensions.

[0146] This embodiment optimizes the parsing results, performing fine-grained filtering and analysis on the time dimension. This effectively retains business-relevant time dimensions while filtering out irrelevant time-period terms, improving dimension sharing and data analysis efficiency. The automated processing of time dimension classification, filtering, and evaluation reduces manual intervention and ensures the accuracy of business analysis.

[0147] In one embodiment, in S50 above, adjusting the structure of the data warehouse model based on the shared dimension includes:

[0148] S501, Analyze the relationship between the shared dimension and the data warehouse model, determine the application scenarios of the shared dimension in the data warehouse model, and identify existing dimensions that need to be replaced, merged, or expanded;

[0149] S502, Adjust the dimension table structure of the data warehouse model, add fields and delete duplicate dimensions according to the definition of shared dimensions;

[0150] S503 restructures the fact table structure of the data warehouse model, adjusts the fields and join methods in the fact table, and associates the shared dimensions with the metrics in the fact table;

[0151] S504 updates the relationship between the dimension table and the fact table, and optimizes the connection method between shared dimensions and the fact table.

[0152] This embodiment analyzes the compatibility of shared dimensions with existing data warehouse models and their application in different business scenarios. It is necessary to identify whether dimensions in existing dimension tables overlap with shared dimensions, or whether optimization can be achieved through shared dimensions. Through automated scripts or manual analysis, shared dimensions and existing dimensions in the data warehouse model are compared to analyze the specific application scenarios of shared dimensions within the data warehouse model. For example, customer or product dimensions may already exist in the existing model, but may need to be merged or optimized.

[0153] After analyzing the relationship between shared dimensions and the data warehouse model, we determine which existing dimensions need to be replaced, merged, or expanded to better support the application of shared dimensions. The aim is to optimize the structure of existing dimensions through shared dimensions, reduce redundancy, and improve the efficiency of the data warehouse model. We conduct structural analysis of the dimension tables in the data warehouse model, and in conjunction with business requirements, identify which existing dimensions overlap with shared dimensions or require further optimization, and formulate specific merging or expansion strategies.

[0154] Based on the definition of shared dimensions, adjust the dimension tables in the data warehouse model. This may require adding new shared dimension fields and deleting existing duplicate dimensions. By simplifying the dimension table structure, data redundancy is reduced, improving the model's query efficiency and consistency. Modify the metadata definition of the data warehouse model using SQL or ETL tools to ensure that newly added shared dimension fields are seamlessly integrated into the existing model, while removing redundant dimension fields.

[0155] The fact table is the core data table in a data warehouse model, storing metrics for business data. Adjusting the fact table structure ensures that shared dimensions are correctly associated with the metrics in the fact table. This includes adjusting field definitions and join methods in the fact table to support the efficient application of shared dimensions. By modifying the field definitions of the fact table, we ensure that metrics can be correctly joined with shared dimensions and optimize join methods to improve query performance. This operation usually needs to be performed simultaneously with adjustments to the dimension tables to ensure data consistency and integrity.

[0156] After adjusting the dimension tables and fact tables, the relationships between them need to be updated. The goal is to optimize the join method between the dimension tables and fact tables, improving the performance of data querying and analysis. This is achieved by adjusting SQL queries, join conditions in the data model, etc., ensuring the correct relationships between the dimension tables and fact tables, while also improving the performance of cross-table queries. Database optimization tools or SQL tuning can be used to optimize join efficiency.

[0157] This embodiment optimizes the structure of the dimension and fact tables of the data warehouse model by adjusting it based on shared dimensions, reducing duplicate dimensions and improving data consistency. The introduction of shared dimensions makes the data warehouse model more consistent across different business scenarios, improving the performance of data queries and the efficiency of analysis.

[0158] In one embodiment, after adjusting the structure of the data warehouse model based on the shared dimension in S50 above, the method further includes:

[0159] S601: Import business data into the adjusted data warehouse model to verify the integrity of the data and the accuracy of the loading.

[0160] S602, Test whether the adjusted data warehouse model structure meets the query requirements in various business scenarios, and evaluate the performance of shared dimensions in actual applications;

[0161] S603, Verify the relationship between dimension tables and fact tables in the data warehouse model, and evaluate the data consistency in the adjusted data warehouse model;

[0162] S604, Evaluate the response efficiency of the adjusted data warehouse model in data queries under different scenarios;

[0163] S605 generates a data validation report, records the validation results, and evaluates the availability and performance of the adjusted data warehouse model.

[0164] In this embodiment, after the data warehouse model structure is adjusted, business data (such as transaction records, customer information, etc.) is imported into the new model to ensure that all data can be loaded completely and accurately. This step requires verifying that all shared dimensions and metrics can be correctly mapped to the new data warehouse model without loss or errors. Data is imported using ETL tools, and the system automatically checks for anomalies during the data loading process, such as data loss or format mismatches, and generates an integrity report.

[0165] The adjusted data warehouse model needs to be tested in real-world business scenarios to evaluate its applicability across different scenarios (such as customer analysis and sales data analysis). The focus should be on testing whether the shared dimensions can effectively support query requirements across multiple scenarios. The model should be tested using commonly used SQL queries from business scenarios to verify its ability to return results efficiently and accurately in actual queries.

[0166] After model adjustments, ensure the relationships between dimension tables and fact tables are correct and that data consistency is not affected. Data consistency means that information in dimension tables (such as customer information) corresponds correctly to measurement data in fact tables (such as sales revenue) without conflict. Use data consistency verification tools to check the completeness of the relationships between dimension tables and fact tables, and ensure consistency by comparing data from the data source with data in the data warehouse.

[0167] The adjusted data warehouse model needs to undergo query performance testing to ensure that the response time for data queries meets business requirements in different business scenarios. The impact of shared dimensions on query efficiency and the overall performance of the data warehouse model should be evaluated. Complex queries should be executed under different business scenarios, and query response times measured. Evaluation and optimization should be performed using performance optimization tools (such as index optimization and query tuning).

[0168] After completing data loading, query requirement testing, data consistency verification, and response efficiency evaluation, a data verification report is generated, recording the results of all verification steps. This report will describe in detail the usability and performance of the data warehouse model after adjustments, and point out potential optimization points or improvement suggestions. The verification report is automatically generated and includes the results of data loading, query testing, data consistency checks, and performance evaluations, serving as the basis for subsequent optimizations.

[0169] This embodiment ensures that the adjusted data warehouse model can run efficiently and accurately in different business scenarios by performing a complete data import, query requirement testing, data consistency verification, and performance evaluation. The generated data verification report provides a reliable basis for subsequent model optimization and business analysis, while reducing business risks caused by data inconsistency or insufficient query performance.

[0170] The present invention also provides a shared dimension optimization device for a data warehouse model, referring to... Figure 2 , Figure 2 This is a functional module diagram of a preferred embodiment of the shared dimension optimization device for the data warehouse model of the present invention. The shared dimension optimization device for the data warehouse model includes:

[0171] The data acquisition module collects business data tables, which include data list reports and data summary tables, and generates Chinese descriptions of the fields in the business data tables.

[0172] The text parsing module breaks down and statistically analyzes the Chinese description using text parsing methods, tags its parts of speech, and generates parsing results.

[0173] The data filtering module optimizes the parsing results and filters out parts of speech that are unrelated to the shared dimension. These unrelated parts of speech include time-period parts of speech and metric parts of speech.

[0174] The dimension identification module performs statistical analysis on the optimized parsing results to identify common dimensions;

[0175] The shared dimension generation and adjustment module processes each identified common dimension in conjunction with business logic to generate a shared dimension, and adjusts the structure of the data warehouse model based on the shared dimension.

[0176] The specific implementation of the shared dimension optimization device for the data warehouse model of the present invention is basically the same as the embodiments of the shared dimension optimization method for the data warehouse model described above, and will not be repeated here.

[0177] This invention also provides a shared dimension optimization device for data warehouse models, such as... Figure 3As shown, the shared dimension optimization device of this data warehouse model may include: a processor 1001, such as a CPU; a communication bus 1002; a user interface 1003; a network interface 1004; and a memory 1005. The communication bus 1002 is used to implement communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or stable non-volatile memory, such as disk storage. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0178] Those skilled in the art will understand that Figure 3 The hardware structure of the shared dimension optimization device for the data warehouse model shown in the figure does not constitute a limitation on the shared dimension optimization device for the data warehouse model. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0179] like Figure 3 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a shared dimension optimization program for the data warehouse model. The operating system is a program that manages and controls the shared dimension optimization devices and software resources of the data warehouse model, supporting the operation of the network communication module, the user interface module, the shared dimension optimization program for the data warehouse model, and other programs or software. The network communication module manages and controls the network interface 1004; the user interface module manages and controls the user interface 1003.

[0180] exist Figure 3 In the hardware structure of the shared dimension optimization device for the data warehouse model shown, the network interface 1004 is mainly used to connect to the backend server and communicate with the backend server; the user interface 1003 is mainly used to connect to the client and communicate with the client; the processor 1001 can call the shared dimension optimization program of the data warehouse model stored in the memory 1005 and perform the same operation as the shared dimension optimization method of the data warehouse model.

[0181] The specific implementation of the shared dimension optimization device for the data warehouse model of the present invention is basically the same as the embodiments of the shared dimension optimization method for the data warehouse model described above, and will not be repeated here.

[0182] Furthermore, this embodiment of the invention also proposes a computer storage medium storing a shared dimension optimization program for a data warehouse model. When the shared dimension optimization program for the data warehouse model is executed by a processor, it implements the steps of the shared dimension optimization method for the data warehouse model as described above.

[0183] The specific implementation of the computer storage medium of the present invention is basically the same as the various embodiments of the shared dimension optimization method of the above data warehouse model, and will not be repeated here.

[0184] It should be noted that if any software tools or components not belonging to our company appear in the embodiments of this application, they are merely for illustrative purposes and do not represent actual use.

Claims

1. A method for optimizing the shared dimension of a data warehouse model, characterized in that, Includes the following steps: Collect business data tables, which include data list reports and data summary tables, and generate Chinese descriptions of the fields in the business data tables; The Chinese description is broken down and statistically analyzed using text parsing methods, and its parts of speech are tagged to generate parsing results; The parsing results are optimized by filtering out parts of speech that are irrelevant to the shared dimension. These irrelevant parts of speech include time-period parts of speech and metric parts of speech. Statistical analysis was performed on the optimized parsing results to identify common dimensions; For each identified common dimension, it is processed in conjunction with business logic to generate a shared dimension, and the structure of the data warehouse model is adjusted based on the shared dimension.

2. The shared dimension optimization method for the data warehouse model as described in claim 1, characterized in that, Statistical analysis was performed on the filtered parsing results to identify common dimensions, including: The frequency of each word in the analysis results is determined by performing word frequency statistics. Based on a set word frequency threshold, high-frequency words that appear more frequently than the threshold are selected from all words. The high-frequency words are matched in the Chinese description of the field to determine whether the high-frequency words have the same business meaning in multiple Chinese descriptions; High-frequency words with the same business meaning are marked as common dimensions.

3. The shared dimension optimization method for the data warehouse model as described in claim 1, characterized in that, For each identified common dimension, it is processed in conjunction with business logic to generate shared dimensions, including: The identified common dimensions are classified, their usage and definition are analyzed in multiple business scenarios, and representative scenarios are extracted from multiple business scenarios. In representative scenarios, we conduct a detailed analysis of the dimension names, data types, and processing methods for each scenario, and compare whether there are differences in the use of common dimensions in different scenarios. By comparing and analyzing the differences in common dimensions, and by establishing unified standards to adjust and process the differences, shared dimensions are generated.

4. The shared dimension optimization method for the data warehouse model as described in claim 1, characterized in that, The Chinese description is broken down and statistically analyzed using text parsing methods, and its parts of speech are tagged to generate parsing results, including: The Chinese description is segmented into words, breaking down the continuous text into independent words; After word segmentation, words are filtered out, words shorter than a set threshold are excluded, and other words are retained as candidate words. Part-of-speech tagging is performed on the retained candidate words, and grammatical attributes are labeled for each candidate word to distinguish different types of words; Frequency statistics were performed on the candidate words to record the frequency of each candidate word in the Chinese description. Generate parsing results, which include candidate words and their corresponding word frequencies and part-of-speech tags.

5. The shared dimension optimization method for the data warehouse model as described in claim 1, characterized in that, The parsing results are optimized by filtering out parts of speech that are irrelevant to the shared dimension. These irrelevant parts of speech include time-period parts of speech and metric parts of speech, including: The time-related terms in the analysis results are classified into long-term, medium-term, and short-term time dimensions based on the length of the time period. Evaluate the use of the time dimension in business scenarios, and select the primary target time dimension for business analysis based on the frequency of the time dimension in different business scenarios. Assess the cross-scenario applicability of the time dimension, determine whether each time dimension is used in multiple business scenarios, and retain the second target time dimension used in multiple business scenarios; Assess the historical data accumulation capability of the time dimension, determine whether the time dimension supports cross-cycle data accumulation, and retain the third target time dimension that supports cross-cycle accumulation; Assess the trend analysis capability of the time dimension, and select a fourth target time dimension to support long-term trend analysis based on the performance of the time dimension in trend analysis. Retain time-period parts of speech that match the target time dimension, and filter out other time-period parts of speech that are unrelated to the shared dimension.

6. The shared dimension optimization method for the data warehouse model as described in claim 1, characterized in that, Adjusting the structure of the data warehouse model based on the shared dimension includes: Analyze the relationship between the shared dimension and the data warehouse model, determine the application scenarios of the shared dimension in the data warehouse model, and identify existing dimensions that need to be replaced, merged, or expanded; Adjust the dimension table structure of the data warehouse model, adding fields and deleting duplicate dimensions according to the definition of shared dimensions; Reconstruct the fact table structure of the data warehouse model, adjust the fields and join methods in the fact table, and associate the shared dimensions with the metrics in the fact table; Update the relationship between the dimension table and the fact table, and optimize the connection method between shared dimensions and the fact table.

7. The method for optimizing the shared dimension of a data warehouse model as described in claim 1, characterized in that, After adjusting the structure of the data warehouse model based on the shared dimension, it also includes: Import business data into the adjusted data warehouse model to verify the integrity of the data and the accuracy of the loading. Test whether the adjusted data warehouse model structure meets the query requirements in various business scenarios, and evaluate the performance of shared dimensions in practical applications; Verify the relationships between dimension tables and fact tables in the data warehouse model, and evaluate the data consistency in the adjusted data warehouse model; Evaluate the response efficiency of the adjusted data warehouse model in data queries across different scenarios; Generate a data validation report, record the validation results, and evaluate the availability and performance of the adjusted data warehouse model.

8. A shared dimension optimization device for a data warehouse model, characterized in that, The shared dimension optimization device for the data warehouse model includes: The data acquisition module is used to collect business data tables, which include data list reports and data summary tables, and to generate Chinese descriptions of the fields in the business data tables. The text parsing module is used to break down and analyze the Chinese description using text parsing methods, and to tag parts of speech to generate parsing results; The data filtering module is used to optimize the parsing results and filter out parts of speech that are unrelated to the shared dimension. The unrelated parts of speech include time-period parts of speech and metric parts of speech. The dimension identification module is used to perform statistical analysis on the optimized parsing results and identify common dimensions. The shared dimension generation and adjustment module is used to process each identified common dimension in conjunction with business logic to generate a shared dimension, and adjust the structure of the data warehouse model based on the shared dimension.

9. A shared dimension optimization device for a data warehouse model, characterized in that, The shared dimension optimization device for the data warehouse model includes a memory, a processor, and a shared dimension optimization program for the data warehouse model stored in the memory and executable on the processor. When the shared dimension optimization program for the data warehouse model is executed by the processor, it implements the steps of the shared dimension optimization method for the data warehouse model as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, The storage medium stores a shared dimension optimization program for a data warehouse model. When the shared dimension optimization program for the data warehouse model is executed by the processor, it implements the steps of the shared dimension optimization method for the data warehouse model as described in any one of claims 1-7.

Images