Automatic statistical analysis method and system for economic information

Abstract

According to the economic information automatic statistical analysis method and system, a unified data foundation is laid through multi-source data integration and structured processing, and the invalid I / O operation in the query process is greatly reduced in combination with the double-layer index design of physical block storage, dictionary coding and block-level metadata index and in-block bitmap index; the data positioning and statistical calculation efficiency is obviously improved; meanwhile, on the basis of an asynchronous running dynamic recombination optimization mechanism, self-adaptive adjustment of data storage layout on query requirements is achieved by collecting historical query loads, mining frequent query modes, simulating and verifying candidate strategies and dynamically updating physical partitioning strategies, and the problem that a traditional static strategy is difficult to adapt to the pain point of the dynamic query loads is effectively solved; according to the method, high performance, intelligence and self-evolution of economic information statistical analysis are realized, the result readability and the use convenience are improved, and the method is adaptive to precise analysis requirements of large-scale and multi-source heterogeneous economic data.

Description

Technical Field

[0001] This invention relates to the field of information analysis technology, specifically to an automated statistical analysis method and system for economic information. Background Technology

[0002] As an interdisciplinary field that integrates economics, statistics, and computer science, automated statistical analysis of economic information is gaining increasing importance in the industry. It represents a strategic transformation from the traditional extensive economic analysis model that relies on manual collection, report summarization, and experience-based judgment to a new paradigm that is precise and intelligent, characterized by data-driven approaches, intelligent algorithms, and real-time feedback.

[0003] In the field of economic information statistical analysis, traditional systems typically rely on static data storage and query optimization strategies. When processing large-scale, multi-source economic data, these systems often employ fixed data partitioning or indexing methods, which cannot adapt to dynamically changing query loads. For example, when query patterns change, the original data layout may lead to the scanning of a large amount of irrelevant data, resulting in high I / O load and query latency. Furthermore, existing systems lack adaptive optimization mechanisms, making it difficult to automatically adjust the physical data structure based on historical query behavior, thus failing to consistently guarantee high query performance. Although some systems have introduced indexing techniques, they typically cannot perform dynamic optimization at the physical data partitioning level, causing query efficiency to significantly decrease with increasing data volume and query complexity. Therefore, this paper proposes an automated statistical analysis method and system for economic information. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an automated statistical analysis method and system for economic information. This method significantly reduces invalid I / O operations during the query process, substantially improves data location and statistical calculation efficiency, and ensures continuous and stable query service response by asynchronously executing data reorganization in the background. Combined with the visualized output of statistical results, this invention achieves high performance, intelligence, and self-evolution in economic information statistical analysis, while also improving the readability and ease of use of the results, thus adapting to the precise analysis needs of large-scale, multi-source, heterogeneous economic data.

[0005] The technical solution adopted in this invention is as follows: An automated statistical analysis method for economic information, comprising the following steps: Step S100: Collecting and processing economic information data from multiple data sources to obtain structured economic data; Step S200: According to the currently effective physical block division strategy, the structured economic data is divided into multiple data blocks at the physical storage level; Step S300: Encode the column data within each data block to generate encoded column data, and construct an intra-block bitmap index for at least one dimension field; Step S400: Create block-level metadata for each data block and build a block-level metadata index for fast retrieval of data blocks; Step S500: Receive a statistical query request, use the block-level metadata index to filter out a set of candidate data blocks, then locate and load the data through the intra-block bitmap index, and generate and output the statistical query results after calculation; Step S600: Asynchronous with the query processing in step S500, execute a dynamic reorganization loop, the dynamic reorganization loop including: Step S610: Collect historical query requests and their execution performance data to obtain query load samples; Step S620: Analyze the query load sample to generate one or more candidate physical block strategies; Step S630: For at least one candidate physical partitioning strategy, perform performance verification based on the query load sample in a simulation environment, and calculate the expected performance improvement; Step S640: In response to the expected performance improvement exceeding a preset threshold, update the currently effective physical partitioning strategy to a new physical partitioning strategy and trigger the data to be reorganized according to the new physical partitioning strategy.

[0006] Furthermore, in step S200, the currently effective physical partitioning strategy is the default strategy during system initialization and is dynamically updated during operation by step S640.

[0007] Further, in step S620, generating candidate physical partitioning strategies includes: performing frequent pattern mining on the query load sample to identify the most frequently queried dimension combinations; and generating candidate strategies with these dimensions as primary partitioning keys based on the identified dimension combinations.

[0008] Furthermore, in step S630, performance verification is performed in a simulation environment, including: logically reorganizing the structure of the block-level metadata index according to the candidate physical partitioning strategy; simulating the query execution process on the logically reorganized metadata index using the query load sample; and statistically analyzing the degree of reduction in logical I / O load during the simulated query process as the expected performance improvement.

[0009] Furthermore, the logical I / O load is quantified by the ratio of the number of skipped data blocks to the number of loaded data blocks.

[0010] Furthermore, the data reorganization triggered in step S640 is executed asynchronously as a background task. During the reorganization process, the system continues to respond to query requests normally based on the old physical block strategy until the reorganization is completed.

[0011] Furthermore, in step S300, the column data is encoded using dictionary encoding, and the construction of the intra-block bitmap index is based on the result of dictionary encoding.

[0012] Furthermore, the output of statistical query results in step S500 includes visualizing the statistical query results in the form of charts.

[0013] An automated statistical analysis system for economic information includes: The data acquisition and processing module is used to collect economic information data from multiple data sources and process it to obtain structured economic data. An adaptive block storage module is used to divide the structured economic data into multiple data blocks at the physical storage level according to the currently effective physical block strategy, and to encode the column data in each data block and construct the bitmap index within the block. The metadata index management module is used to create block-level metadata for each data block and to build a block-level metadata index for fast retrieval of data blocks; The query execution engine module is used to receive statistical query requests, process queries using the block-level metadata index and the intra-block bitmap index, and output statistical query results. The dynamic reorganization and optimization engine module runs asynchronously with the query execution engine module, and the dynamic reorganization and optimization engine module includes: The load acquisition unit is used to collect historical query requests and their execution performance data to obtain query load samples; The strategy analysis unit is used to analyze the query load sample and generate one or more candidate physical block strategies. The simulation verification unit is used to perform performance verification of candidate physical partitioning strategies in a simulation environment and calculate the expected performance improvement. The strategy execution unit is used to update the currently effective physical block strategy and trigger data reorganization when the expected performance improvement exceeds a preset threshold.

[0014] Furthermore, when performing performance verification, the simulation verification unit estimates the degree of reduction in I / O load by logically reorganizing the block-level metadata index and simulating query execution.

[0015] The beneficial effects of this invention are as follows: This invention lays a unified data foundation through multi-source data integration and structured processing. Combined with a two-layer index design of physical block storage, dictionary encoding, and block-level metadata indexing and intra-block bitmap indexing, it significantly reduces invalid I / O operations during the query process, greatly improving data location and statistical calculation efficiency. Simultaneously, relying on an asynchronous dynamic reorganization optimization mechanism, it achieves adaptive adjustment of data storage layout to query demands by collecting historical query loads, mining frequent query patterns, simulating and verifying candidate strategies, and dynamically updating physical block strategies. This effectively solves the pain point of traditional static strategies being unable to adapt to dynamic query loads. Furthermore, data reorganization is executed asynchronously in the background, ensuring a continuous and stable response to query services. Coupled with the visualized output of statistical results, it achieves high performance, intelligence, and self-evolution in economic information statistical analysis, while also improving the readability and ease of use of the results, adapting to the precise analysis needs of large-scale, multi-source, heterogeneous economic data. Attached Figure Description

[0016] Figure 1 This is a flowchart of an automated statistical analysis method for economic information according to the present invention; Figure 2 This is a detailed flowchart of dynamic recombination optimization according to an embodiment of the present invention; Figure 3 This is a flowchart of a two-level index structure according to an embodiment of the present invention; Figure 4 This is a system architecture diagram of one embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1 like Figures 1-3 As shown in the figure, an automated statistical analysis method for economic information according to an embodiment of the present invention includes the following steps: Step S100: Collect and process economic information data from multiple data sources to obtain structured economic data; Step S200: Based on the currently effective physical block partitioning strategy, divide the structured economic data into multiple data blocks at the physical storage level; Raw data collected from multiple data sources After a unified data processing function Transformed into regularized structured economic data , This provides data processing functions, including deduplication, format normalization, and missing value imputation, which provides a consistent data interface for all subsequent operations.

[0019] Subsequently, the system uses a currently effective physical partitioning strategy. (A default strategy is used during system initialization) Through block functions Dataset At the physical storage level, it is divided into a series of data blocks. ,in This is a block distribution function that distributes data into different blocks based on the value of K. The block distribution strategy here... Essentially, physical partitioning is an ordered combination of one or more dimension fields that determines the data's distribution on disk. Good physical partitioning should ensure that frequently accessed data in subsequent queries is clustered within the fewest possible data blocks, aiming to reduce unnecessary I / O operations and improve query efficiency.

[0020] Step S300: Encode the column data within each data block to generate encoded column data, and build an intra-block bitmap index for at least one dimension field; Within each data block, the system further accelerates data location through efficient encoding and indexing techniques. For each data block... The column data uses dictionary encoding technology to encode each unique value in the column. Mapped to a compact integer encoding This encoding process not only compresses storage space, but more importantly, lays the foundation for building efficient intra-block bitmap indexes. Bitmap Index For each dimension field Each value Create a binary vector Each bit in the array indicates whether the row has that value. By performing fast bitwise operations (such as AND and OR) on multiple bitmaps, the set of rows that simultaneously meet multiple conditions can be located efficiently, with a time complexity far lower than traditional row-by-row scanning.

[0021] Step S400: Create block-level metadata for each data block and build a block-level metadata index for fast retrieval of data blocks; Step S500: Receive statistical query request, filter candidate data block set using block-level metadata index, locate and load data using intra-block bitmap index, generate and output statistical query result after calculation; To quickly filter out the small number of data blocks relevant to the query from thousands of data blocks, the system introduces an abstraction layer of block-level metadata and its index. Each data block... Each comes with a set of concise metadata. For example, the range of a numeric field it contains. All of this block-level metadata is indexed by a higher-level block-level metadata index for fast retrieval. Managed. When the query execution engine receives a query with filtering conditions... Statistical query request At that time, it first visits By utilizing the statistical information contained in the metadata, a large number of queries that absolutely do not meet the query criteria can be quickly eliminated. This allows us to obtain a highly accurate set of candidate data blocks. satisfies This process narrows the range of data requiring fine-grained scanning from the entire dataset to a few specific data blocks, achieving the first order-of-magnitude improvement in query performance.

[0022] Step S600: Asynchronous with the query processing in step S500, execute the dynamic reorganization loop, which includes: Step S610: Collect historical query requests and their execution performance data to obtain query load samples; Static, initial partitioning strategy It struggles to adapt to continuously changing query loads. Therefore, a parallel, asynchronous, dynamic reorganization loop is designed to enable continuous optimization. This loop begins by reorganizing historical query loads. Continuous data collection and analysis. The strategy analysis unit performs analysis on... Perform frequent pattern mining (e.g., using the Apriori algorithm) to find the most frequently queried dimension combinations. The mathematical basis is to find all that satisfy... Frequent itemsets , among which supp For support, This is the preset minimum support threshold. These frequent itemsets This reveals hidden association patterns within the query load, and the system uses these high-frequency dimension combinations as primary partition keys. Candidate strategies.

[0023] Step S620: Analyze the query load samples to generate one or more candidate physical block strategies; Step S630: For at least one candidate physical partitioning strategy, perform performance verification based on query load samples in a simulation environment and calculate the expected performance improvement; Step S640: In response to the expected performance improvement exceeding a preset threshold, update the currently effective physical partitioning strategy to a new physical partitioning strategy and trigger the data to be reorganized according to the new physical partitioning strategy.

[0024] When adopting any candidate strategy Previously, the system underwent rigorous performance verification in a low-cost simulation environment. The verification process did not involve actually moving physical data, but rather logically based on… Reorganize the metadata index This structure forms a simulated metadata index. Subsequently, the system will use historical query load samples. exist The entire query execution process is simulated by replaying the previous code. During this process, the system critically quantifies the expected performance improvement brought about by the new strategy, with the core metric being the reduction in logical I / O load. Logical I / O load is defined as IO... This is the ratio of the number of skipped data blocks to the number of loaded data blocks. A higher ratio indicates better filtering of the metadata index, meaning fewer data blocks need to be actually loaded and scanned. The expected performance improvement is calculated using the formula Improvement. It is calculated. Only when I ( The system only decides to adopt a new strategy when the threshold value is set (preset). Ultimately, the strategy execution unit triggers the data according to the new physical partitioning strategy in an asynchronous, background task manner. The system is restructured, and during the restructuring period, all queries still respond normally based on the old strategy, achieving a seamless optimization process for users.

[0025] Specifically, in step S200, the currently effective physical block partitioning strategy is the default strategy during system initialization and is dynamically updated by step S640 during operation.

[0026] Specifically, frequent pattern mining identifies the most frequently queried dimension combinations; based on the identified dimension combinations, candidate strategies using these dimensions as primary partitioning keys are generated.

[0027] Specifically, in step S630, performance verification is performed in a simulation environment, including: logically reorganizing the structure of the block-level metadata index according to the candidate physical partitioning strategy; simulating the query execution process on the logically reorganized metadata index using query load samples; and statistically analyzing the degree of reduction in logical I / O load during the simulated query process as an expected performance improvement.

[0028] Specifically, logical I / O load is quantified by the ratio of the number of skipped data blocks to the number of loaded data blocks.

[0029] Specifically, the data reorganization triggered in step S640 is executed asynchronously as a background task. During the reorganization process, the system continues to respond to query requests normally based on the old physical block strategy until the reorganization is completed.

[0030] Specifically, in step S300, the column data is encoded using dictionary encoding, and the construction of the intra-block bitmap index is based on the result of dictionary encoding.

[0031] Specifically, step S500 outputs the statistical query results, including visualizing the statistical query results in the form of charts.

[0032] Example 2 like Figure 4 As shown, an automated statistical analysis system for economic information includes: The data acquisition and processing module is used to collect and process economic information data from multiple data sources to obtain structured economic data. The data acquisition and processing module is the data entry point of the entire system and is responsible for the integration and standardization of multi-source heterogeneous economic data. The adaptive block storage module is used to divide structured economic data into multiple data blocks at the physical storage level according to the currently effective physical block strategy, and to encode the column data in each data block and construct the bitmap index within the block. The adaptive block storage module is the core of the system's physical storage, directly receiving structured data from upstream. And based on the currently effective physical partitioning strategy. Execute physical block function The adaptive block storage module internally maintains a policy context manager, which is responsible for storage, versioning management, and applying block strategies. Its block partitioning process is essentially based on the block key. The value of the variable is used to allocate data rows to different physical data block files through algorithms such as consistent hashing or range partitioning. In the middle. Next, the adaptive block storage module performs operations for each newly generated data block. Start a column-based encoding and index builder. The index builder first performs dictionary encoding on each data column. This generates a compact sequence of integers. Based on this encoding result, an intra-block bitmap index is constructed in parallel for the specified dimension fields. This design eliminates the need to rescan the original string data when building the bitmap index, improving construction efficiency and ensuring consistency between the encoded data and the index data.

[0033] The metadata index management module is used to create block-level metadata for each data block and build a block-level metadata index for fast retrieval of data blocks. It is the core of the system's physical storage. The metadata index management module maintains a two-level index structure, establishing an efficient retrieval abstraction layer on top of the massive data block set. The metadata index management module listens for data write events from the adaptive block storage module, responding whenever a data block... Once the construction is complete, the metadata index management module automatically triggers its metadata extractor from... Calculate its block-level metadata These metadata It is a concise summary of the data block content. The metadata index builder collects the metadata of all data blocks and builds a global block-level metadata index. This index typically uses a B+ tree structure suitable for range queries, with its key being a block key. The corresponding metadata fields (such as) The order of a B+ tree index It is a key parameter that balances the index depth and node size, directly affecting the set of candidate data blocks. The filtering speed is improved. The existence of the metadata index management module allows the query execution engine to query this lightweight data without accessing the physical data itself. This allows for the filtering of most irrelevant data blocks, which is key to achieving high-performance queries.

[0034] The query execution engine module receives statistical query requests, processes queries using block-level metadata indexes and intra-block bitmap indexes, and outputs statistical query results. It is the terminal processing unit facing user requests and adopts a filtering-validation paradigm. The dynamic reorganization and optimization engine module runs asynchronously with the query execution engine module, operating independently and asynchronously within the query execution process. It is dedicated to long-term system performance optimization and includes: The load acquisition unit is used to collect historical query requests and their execution performance data to obtain query load samples; The strategy analysis unit is used to analyze query load samples and generate one or more candidate physical block strategies. The simulation verification unit is used to perform performance verification of candidate physical partitioning strategies in a simulation environment and calculate the expected performance improvement. The strategy execution unit is used to update the currently effective physical block strategy and trigger data reorganization when the expected performance improvement exceeds a preset threshold.

[0035] Specifically, when the simulation verification unit performs performance verification, it estimates the degree of reduction in I / O load by logically reorganizing the block-level metadata index and simulating query execution.

[0036] This invention achieves high performance and self-evolution in economic statistical analysis by combining data partitioning with a two-level index and asynchronous dynamic optimization. The system first cleanses and transforms multi-source economic data into structured data. Based on the current partitioning strategy, it divides the data into physical data blocks and constructs dictionary encoding and bitmap indexes for each block to achieve rapid data location. Simultaneously, it creates block-level metadata indexes to quickly filter candidate data blocks during queries, thereby significantly reducing I / O and accelerating query response. Meanwhile, the system asynchronously runs a dynamic optimization loop in the background: continuously collecting query logs, generating new candidate partitioning strategies by analyzing high-frequency query patterns, and verifying their expected performance improvement in a simulation environment. Once the target is met, it seamlessly switches to the new strategy and triggers background data reorganization, thus enabling the physical data layout to continuously adapt to changes in query load and achieve continuous performance improvements transparent to the user.

[0037] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0038] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0039] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0040] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0041] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0042] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0043] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An automated statistical analysis method for economic information, characterized in that, Includes the following steps: Step S100: Collect and process economic information data from multiple data sources to obtain structured economic data; Step S200: According to the currently effective physical block division strategy, the structured economic data is divided into multiple data blocks at the physical storage level; Step S300: Encode the column data within each data block to generate encoded column data, and construct an intra-block bitmap index for at least one dimension field; Step S400: Create block-level metadata for each data block and build a block-level metadata index for fast retrieval of data blocks; Step S500: Receive a statistical query request, use the block-level metadata index to filter out a set of candidate data blocks, then locate and load the data through the intra-block bitmap index, and generate and output the statistical query results after calculation; Step S600: Asynchronous with the query processing in step S500, execute a dynamic reorganization loop, the dynamic reorganization loop including: Step S610: Collect historical query requests and their execution performance data to obtain query load samples; Step S620: Analyze the query load sample to generate one or more candidate physical block strategies; Step S630: For at least one candidate physical partitioning strategy, perform performance verification based on the query load sample in a simulation environment, and calculate the expected performance improvement; Step S640: In response to the expected performance improvement exceeding a preset threshold, update the currently effective physical partitioning strategy to a new physical partitioning strategy and trigger the data to be reorganized according to the new physical partitioning strategy.

2. The automated statistical analysis method for economic information according to claim 1, characterized in that, In step S200, the currently effective physical block partitioning strategy is the default strategy during system initialization and is dynamically updated by step S640 during operation.

3. The automated statistical analysis method for economic information according to claim 1, characterized in that, In step S620, generating candidate physical partitioning strategies includes: performing frequent pattern mining on the query load sample to identify the most frequently queried dimension combinations; and generating candidate strategies with these dimensions as primary partitioning keys based on the identified dimension combinations.

4. The automated statistical analysis method for economic information according to claim 1, characterized in that, In step S630, performance verification is performed in a simulation environment, including: logically reorganizing the structure of the block-level metadata index according to the candidate physical partitioning strategy; simulating the query execution process on the logically reorganized metadata index using the query load sample; and statistically analyzing the degree of reduction in logical I / O load during the simulated query process as the expected performance improvement.

5. The automated statistical analysis method for economic information according to claim 4, characterized in that, The logical I / O load is quantified by the ratio of the number of skipped data blocks to the number of loaded data blocks.

6. The automated statistical analysis method for economic information according to claim 1, characterized in that, The data reorganization triggered in step S640 is executed asynchronously as a background task. During the reorganization process, the system continues to respond to query requests normally based on the old physical block strategy until the reorganization is completed.

7. The automated statistical analysis method for economic information according to claim 1, characterized in that, In step S300, the column data is encoded using dictionary encoding, and the construction of the intra-block bitmap index is based on the result of dictionary encoding.

8. The automated statistical analysis method for economic information according to claim 1, characterized in that, The step S500 outputs statistical query results, including visualizing the statistical query results in the form of charts.

9. An automated statistical analysis system for economic information, characterized in that, include: The data acquisition and processing module is used to collect economic information data from multiple data sources and process it to obtain structured economic data. An adaptive block storage module is used to divide the structured economic data into multiple data blocks at the physical storage level according to the currently effective physical block strategy, and to encode the column data in each data block and construct the bitmap index within the block. The metadata index management module is used to create block-level metadata for each data block and to build a block-level metadata index for fast retrieval of data blocks; The query execution engine module is used to receive statistical query requests, process queries using the block-level metadata index and the intra-block bitmap index, and output statistical query results. The dynamic reorganization and optimization engine module runs asynchronously with the query execution engine module, and the dynamic reorganization and optimization engine module includes: The load acquisition unit is used to collect historical query requests and their execution performance data to obtain query load samples; The strategy analysis unit is used to analyze the query load sample and generate one or more candidate physical block strategies. The simulation verification unit is used to perform performance verification of candidate physical partitioning strategies in a simulation environment and calculate the expected performance improvement. The strategy execution unit is used to update the currently effective physical block strategy and trigger data reorganization when the expected performance improvement exceeds a preset threshold.

10. The automated statistical analysis system for economic information according to claim 9, characterized in that, When performing performance verification, the simulation verification unit estimates the degree of reduction in I / O load by logically reorganizing the block-level metadata index and simulating query execution.

Images