Dynamic database division and table division system and device based on flow prediction data

Abstract

The invention is suitable for the technical field of intelligent transportation, and provides a dynamic database and table division system and device based on flow prediction data. Historical data and real-time data of the portal are analyzed, processed and learned through the flow prediction model, and flow prediction data in a period of time in the future are obtained. Based on the flow prediction data, the system dynamically adjusts library table resources related to portal data storage in real time, and allocates idle resources from a pre-constructed annular buffer resource pool when the peak period is about to arrive, so as to realize dynamic expansion of partitions for storing portal data. And when the traffic is less, migrating the data of the low-load logic partition to other logic partitions, and then recovering the low-load logic partition to the annular buffer resource pool so as to realize the dynamic reduction of the partition for storing the portal data. Therefore, the data maintenance cost is reduced, and the data utilization efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular, to a dynamic database sharding and table partitioning system and device based on traffic prediction data. Background Art

[0002] With the rapid development of the Intelligent Transportation System (ITS), urban traffic management is gradually transforming from traditional static management to dynamic and intelligent management. Traffic flow, as a key parameter in the urban traffic system, directly affects traffic congestion and road use efficiency. Accurately predicting traffic flow is of great significance for traffic planning, road maintenance, signal control, and traffic flow optimization. However, existing traffic flow prediction methods mostly rely on historical data, lacking real-time and dynamic adjustment capabilities, and are difficult to adapt to the rapid changes in traffic flow.

[0003] In the field of database management, the cost of data storage and maintenance increases significantly with the increase in data volume. Especially in the traffic field, a large amount of traffic flow data needs to be collected, stored, and analyzed in real time. Traditional database systems usually adopt static resource allocation strategies and cannot dynamically adjust storage resources according to the actual data volume, resulting in insufficient resources during peak traffic periods and resource waste during low peak periods, thus increasing the data maintenance cost and reducing the data utilization efficiency. Summary of the Invention

[0004] In view of this, this application provides a dynamic database sharding and table partitioning system and device based on traffic prediction data to achieve real-time prediction of traffic flow changes, dynamically adjust database resources according to the prediction results, thereby reducing data maintenance costs and improving data utilization efficiency.

[0005] In the first aspect of this application, a dynamic database sharding and table partitioning system based on traffic prediction data is provided. The system includes a traffic prediction model construction module, a database initialization module, a data reception module, and a database dynamic adjustment module;

[0006] The traffic prediction model construction module is used to construct a traffic prediction model through gantry historical passing data and process the gantry real-time passing data through the traffic prediction model to obtain traffic prediction data;

[0007] The database initialization module is used to determine historical traffic data through the gantry historical passing data, divide the database into several logical partitions and a circular buffer resource pool according to the historical traffic data to determine the initial partition state and initial resource allocation of the database, where the circular buffer resource pool does not store resource data;

[0008] The data reception module is used to receive the gantry real-time passing data and store the gantry real-time passing data into the corresponding logical partition according to preset conditions;

[0009] The database dynamic adjustment module is used to allocate idle resources or release idle logical partition resources from the circular buffer resource pool according to the traffic prediction data generated by the traffic prediction model, so as to complete the dynamic expansion or dynamic reduction of the database.

[0010] Optionally, the system further includes a data consistency guarantee module;

[0011] The data consistency guarantee module is used to record log information during the dynamic expansion or dynamic reduction process, and when it is detected that the dynamic expansion or dynamic reduction fails, data rollback is performed according to the log information.

[0012] Optionally, the allocation of idle resources or the release of idle logical partition resources from the circular buffer resource pool includes:

[0013] A first logical partition load threshold for triggering dynamic expansion and a second logical partition load threshold for triggering dynamic reduction are preset, wherein the first logical partition load threshold is greater than the second logical partition load threshold;

[0014] When it is detected that the traffic prediction data is greater than the first logical partition load threshold, idle resources are allocated from the circular buffer resource pool to complete the dynamic expansion of the database;

[0015] When it is detected that the traffic prediction data is less than the second logical partition load threshold, a low-load logical partition is determined, and the data of the low-load logical partition is migrated to other logical partitions, and then the low-load logical partition is recycled to the circular buffer resource pool to complete the dynamic reduction of the database.

[0016] Optionally, the database initialization module further includes a monitoring module;

[0017] The monitoring module is used to perform real-time monitoring on each logical partition through a deployed real-time monitoring tool to determine the load condition of the logical partition.

[0018] Optionally, the processing of the gantry real-time passing data by the traffic prediction model to obtain traffic prediction data includes:

[0019] A number of basic models are pre-constructed to form the traffic prediction model, and the basic models are used to process the gantry historical passing data to obtain a number of prediction results;

[0020] The respective prediction results are fused by a dynamic weight allocation method to obtain the traffic prediction data of the gantry real-time passing data.

[0021] The second aspect of the present application provides a dynamic database and table partitioning device based on traffic prediction data, and the device includes:

[0022] A traffic prediction data acquisition unit, configured to construct a traffic prediction model through gantry historical passing data, and process the gantry real-time passing data through the traffic prediction model to obtain traffic prediction data;

[0023] A database initialization unit, configured to determine historical traffic data through the gantry historical passing data, divide the database into several logical partitions and a circular buffer resource pool according to the historical traffic data, so as to determine the initial partition state and initial resource allocation of the database, wherein the circular buffer resource pool does not store resource data;

[0024] A data receiving unit, configured to receive the gantry real-time passing data, and store the gantry real-time passing data into the corresponding logical partition according to a preset condition;

[0025] A dynamic adjustment unit, configured to allocate idle resources or release idle logical partition resources from the circular buffer resource pool according to the traffic prediction data generated by the traffic prediction model, so as to complete the dynamic expansion or dynamic reduction of the database

[0026] Optionally, the device further includes:

[0027] A data consistency guarantee unit, configured to record log information during the dynamic expansion or dynamic reduction process, and perform data rollback according to the log information when it is detected that the dynamic expansion or dynamic reduction fails.

[0028] Optionally, the allocating idle resources or releasing idle logical partition resources from the circular buffer resource pool in the dynamic adjustment unit includes:

[0029] A first logical partition load threshold for triggering dynamic expansion and a second logical partition load threshold for triggering dynamic reduction are preset in advance, wherein the first logical partition load threshold is greater than the second logical partition load threshold;

[0030] When it is detected that the traffic prediction data is greater than the first logical partition load threshold, idle resources are allocated from the circular buffer resource pool to complete the dynamic expansion of the database;

[0031] When it is detected that the traffic prediction data is less than the second logical partition load threshold, a low-load logical partition is determined, the data of the low-load logical partition is migrated to other logical partitions, and then the low-load logical partition is recycled to the circular buffer resource pool to complete the dynamic reduction of the database.

[0032] Optionally, the device further includes:

[0033] The monitoring unit is used to perform real-time monitoring on each logical partition through the deployed real-time monitoring tool to determine the load conditions of each logical partition.

[0034] Optionally, the process of obtaining traffic prediction data by the traffic prediction model for the real-time passing data of the gantry includes:

[0035] Construct a number of basic models in advance to form the traffic prediction model, and process the historical passing data of the gantry through the basic models to obtain a number of prediction results;

[0036] Fuse each prediction result through the dynamic weight allocation method to obtain the traffic prediction data of the real-time passing data of the gantry.

[0037] In the embodiment provided by the present application, the historical data and real-time data of the gantry are analyzed, processed and learned through the traffic prediction model to obtain the traffic prediction data in a future period of time. Based on this traffic prediction data, the system dynamically adjusts the table resources related to the gantry data storage in real time. When approaching the peak period, idle resources are allocated from the pre-constructed circular buffer resource pool to realize the dynamic expansion of the partition for storing gantry data. When the traffic is small, the data of the low-load logical partition is migrated to other logical partitions, and then the low-load logical partition is recycled to the circular buffer resource pool to realize the dynamic reduction of the partition for storing gantry data. Thereby reducing the data maintenance cost and improving the data utilization efficiency. Brief Description of the Drawings

[0038] Figure 1 It is the system module diagram provided by the embodiment of the present application;

[0039] Figure 2 It is the device structure diagram provided by the embodiment of the present application;

[0040] Figure 3 It is the internal structure diagram of the computer device provided by the embodiment of the present application. Detailed Embodiment

[0041] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0042] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a", "the", and "said" used in this application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0043] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to a determination".

[0044] This application provides a dynamic database sharding and table partitioning system and device based on traffic prediction data to achieve real-time prediction of traffic flow changes, dynamically adjust database resources according to the prediction results, thereby reducing data maintenance costs and improving data utilization efficiency.

[0045] The technical solutions of this application will be described in detail below with specific embodiments. These several specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0046] As Figure 1 shown, it is a system module diagram of a dynamic database sharding and table partitioning system based on traffic prediction data provided by this application. The system includes a traffic prediction model construction module, a database initialization module, a data reception module, and a database dynamic adjustment module. The functions and implementation processes of each module will be described below.

[0047] 1. Traffic prediction model construction module. This module is used to construct a traffic prediction model through the historical passing data of gantries and process the real-time passing data of gantries through the traffic prediction model to obtain traffic prediction data.

[0048] In this embodiment, the traffic prediction model construction method is as follows:

[0049] 1. Based on the historical passing data of gantries, generate preliminary historical data sets according to time granularities such as 5 minutes, 10 minutes, 1 hour, etc. This data covers monthly, quarterly, and annual dimensions to ensure the integrity of the time series of the data and provide a basis for subsequent analysis.

[0050] 2. Perform data cleaning on the above preliminary historical data sets, which may include the following steps:

[0051] Data Completion: Use interpolation or machine learning algorithms to fill in missing values and ensure data continuity.

[0052] Noise Removal: Use methods such as moving average or wavelet transform to identify and remove outliers, improving data quality and stability.

[0053] 3. Feature extraction can be performed on the above-cleaned data, which may include the following steps:

[0054] Time Feature Alignment: Align the data for each time period with the historical data for the same period according to time periods such as daily, weekly, and yearly, extract time-related features, and capture regular trends.

[0055] Statistical Feature Construction: Extract multi-dimensional features based on statistical methods such as variance, mean, and distribution pattern, and finally obtain the feature data of the gantry historical data.

[0056] 4. Model training is performed based on the above feature data. Use preset algorithms such as XGBoost, GBDT, LSTM, and GRU to construct an initial traffic prediction model. Then divide the above feature data into a training set, a test set, and a validation set. Train the initial traffic prediction model using the training set, and then optimize the model using the test set and the validation set to obtain a trained traffic prediction model.

[0057] After determining the traffic prediction model, collect the real-time traffic data of the gantry through the gantry, and input the real-time traffic data of the gantry into the traffic prediction model to obtain traffic prediction data.

[0058] In another embodiment, processing the real-time traffic data of the gantry through the traffic prediction model to obtain traffic prediction data includes:

[0059] Pre-construct a number of basic models to form the traffic prediction model, and process the historical traffic data of the gantry through the basic models to obtain a number of prediction results;

[0060] Fuse each prediction result through a dynamic weight allocation method to obtain the traffic prediction data of the real-time traffic data of the gantry.

[0061] In this embodiment, when constructing the initial traffic prediction model, several basic models can be constructed respectively for various different algorithms such as XGBoost, GBDT, LSTM, and GRU. Then, the different basic models are respectively trained and optimized through the above-mentioned training set, test set, and validation set to obtain several trained basic models. Then, these several trained basic models are respectively used to predict the real-time passing data of the gantry, and the respective prediction results are fused through a preset method, such as an average weighting strategy or a dynamic weight allocation method based on the performance of the validation set, and the fused result is determined as the final traffic prediction data.

[0062] Exemplarily, assume that the real-time traffic flow data obtained by the gantry is Q = [Q1, Q2, Q3, Q4,... Qn]. The processing result of Q by the basic model constructed by the XGBoost algorithm is Qt1, the processing result of Q by the basic model constructed by the GBDT algorithm is Qt2, the processing result of Q by the basic model constructed by the LSTM algorithm is Qt3, and the processing result of Q by the basic model constructed by the GRU algorithm is Qt4. Then, according to the corresponding weight values α1, α2, α3, α4 of Qt1, Qt2, Qt3, Qt4, the final traffic prediction data can be determined through the formula Qt1*α1 + Qt2*α2 + Qt3*α3 + Qt4*α4.

[0063] If the average weighting strategy is used to determine the weight values, it is preset that the weight values α1, α2, α3, α4 are all the same. If the dynamic weight allocation method is used to determine the weight values, the respective weight values can be set according to the actual scenario or historical calculation results, and the weight values can be continuously optimized during use.

[0064] In this embodiment, the real-time passing data of the gantry is predicted respectively by the basic models of multiple different algorithms, which can optimize the prediction accuracy and improve the robustness of the model.

[0065] 2. Database initialization module. This module is used to determine the historical traffic data through the historical passing data of the gantry, divide the database into several logical partitions and a circular buffer resource pool according to the historical traffic data, so as to determine the initial partition state and initial resource allocation of the database.

[0066] In this embodiment, a unified database resource table can be constructed first to record the status of all database partitions, such as the storage capacity, TPS, and resource allocation of each partition, and store it in a distributed coordination service such as ZooKeeper. Then, according to the statistics of the historical passing data of the gantry (such as hourly and daily traffic), several logical partitions are pre-allocated, and the partition capacity is initialized. Finally, the routing rules of each initial partition are set to store the real-time passing data of the received gantry. The consistent Hash algorithm can be used to ensure uniform data distribution.

[0067] In another embodiment, the database initialization module further includes a monitoring module. This module is used to perform real-time monitoring on each logical partition through the deployed real-time monitoring tool to determine the load condition of the logical partition.

[0068] In this embodiment, the real-time monitoring tool includes Prometheus + Grafana, and the load condition can be determined through metrics such as storage utilization rate, TPS / QPS, and latency. Storage utilization rate helps us evaluate the usage of storage resources, TPS / QPS measures the processing capacity of the system, and latency directly reflects the user experience. By monitoring and analyzing these metrics, we can determine the load limit, fault tolerance, and number of concurrent users of the system, and evaluate the performance and stability of the system under high load conditions.

[0069] 3. The data receiving module. It is used to receive the real-time passing data of the gantry and store the real-time passing data of the gantry into the corresponding logical partition according to preset conditions.

[0070] In this embodiment, the preset conditions can be routed to the corresponding partition through the consistent Hashing algorithm according to its key fields (such as gantry code, timestamp).

[0071] 4. The database dynamic adjustment module. This module is used to allocate idle resources or release idle logical partition resources from the circular buffer resource pool according to the traffic prediction data generated by the traffic prediction model to complete the dynamic expansion or dynamic reduction of the database.

[0072] In this embodiment, the process of dynamically expanding the database is as follows:

[0073] 1. Perform expansion trigger condition detection. For example, when the load of a single partition reaches the set threshold (such as 80% storage capacity or TPS / QPS), or it is detected that the data traffic exceeds the set peak threshold for three consecutive minutes, it is determined that the database needs to be dynamically expanded.

[0074] 2. Allocate idle resources from the circular buffer pool, create a new logical partition, and update the consistent Hashing routing rule to allocate the newly written data to the new partition.

[0075] 3. Background asynchronous data migration: Migrate the data with low access frequency first according to the data access frequency, and execute it asynchronously in batches to avoid the impact of migration on the performance of online services.

[0076] 4. After the data migration is completed, update the database resource unified table, mark the expansion as completed, and at the same time enable real-time monitoring for the new partition.

[0077] The process of dynamically reducing the database is as follows:

[0078] 1. Perform reduction trigger condition detection. For example, the partition load with continuous low traffic is lower than 30%, or the resource utilization rate drops to the set low threshold.

[0079] 2. Data merging and resource recycling. Gradually migrate the data of low-load partitions to other active partitions, and release the resources of the idle partitions (such as deleting the partition table or recycling storage nodes) after the migration is completed.

[0080] 3. Update the routing rules, recalculate the consistent Hash, and remove the routing information of the merged partitions to ensure the correct subsequent data writing logic.

[0081] 4. Update the unified database resource table, release the relevant metadata records, and return the recycled resources to the buffer pool at the same time.

[0082] In another embodiment, the system further includes a data consistency guarantee module. This module is used to record log information during the dynamic expansion or dynamic reduction process, and perform data rollback according to the log information when it is detected that the dynamic expansion or dynamic reduction fails.

[0083] Furthermore, this module can also perform idempotency design, making data operations based on a unique identifier (such as a primary key or an operation ID), to ensure that retry operations during the adjustment process will not cause data duplication or loss.

[0084] Furthermore, this module can also implement a double-buffer mechanism, allowing the old and new partitions to run online simultaneously during the data migration phase, ensuring consistency through read-write separation, and reducing the impact on the business during the switch.

[0085] Thus far, the Figure 1 shown process is completed.

[0086] In the embodiments of this application, the historical data and real-time data of the gantry are analyzed, processed, and learned through a traffic prediction model to obtain traffic prediction data for a period of time in the future. Based on this traffic prediction data, this system dynamically and real-time adjusts the table resources related to gantry data storage. When approaching the peak period, allocate idle resources from the pre-constructed circular buffer resource pool to achieve dynamic expansion of the partitions storing gantry data. When the traffic is low, migrate the data of the low-load logical partition to other logical partitions, and then recycle the low-load logical partition to the circular buffer resource pool to achieve dynamic reduction of the partitions storing gantry data. Thereby reducing the data maintenance cost and improving the data utilization efficiency.

[0087] As Figure 2 shown, this application also provides a dynamic database and table partitioning device based on traffic prediction data. The device includes:

[0088] The traffic prediction data acquisition unit 201 is configured to construct a traffic prediction model based on the historical passing data of the gantry, and process the real-time passing data of the gantry through the traffic prediction model to obtain traffic prediction data;

[0089] The database initialization unit 202 is configured to determine historical traffic data based on the historical passing data of the gantry, divide the database into several logical partitions and a circular buffer resource pool according to the historical traffic data, so as to determine the initial partition state and initial resource allocation of the database, wherein the circular buffer resource pool does not store resource data;

[0090] The data receiving unit 203 is configured to receive the real-time passing data of the gantry and store the real-time passing data of the gantry into the corresponding logical partition according to a preset condition;

[0091] The dynamic adjustment unit 204 is configured to allocate idle resources or release idle logical partition resources from the circular buffer resource pool according to the traffic prediction data generated by the traffic prediction model, so as to complete the dynamic expansion or dynamic reduction of the database.

[0092] In another embodiment, the device further includes:

[0093] The data consistency guarantee unit 205 is configured to record log information during the dynamic expansion or dynamic reduction process, and perform data rollback according to the log information when it is detected that the dynamic expansion or dynamic reduction fails.

[0094] In another embodiment, allocating idle resources or releasing idle logical partition resources from the circular buffer resource pool in the dynamic adjustment unit includes:

[0095] A first logical partition load threshold for triggering dynamic expansion and a second logical partition load threshold for triggering dynamic reduction are preset, wherein the first logical partition load threshold is greater than the second logical partition load threshold;

[0096] When it is detected that the traffic prediction data is greater than the first logical partition load threshold, idle resources are allocated from the circular buffer resource pool to complete the dynamic expansion of the database;

[0097] When it is detected that the traffic prediction data is less than the second logical partition load threshold, a low-load logical partition is determined, the data of the low-load logical partition is migrated to other logical partitions, and then the low-load logical partition is recycled to the circular buffer resource pool to complete the dynamic reduction of the database.

[0098] In another embodiment, the device further includes:

[0099] The monitoring unit is used to monitor each logical partition in real time through the deployed real-time monitoring tool to determine the load conditions of each logical partition.

[0100] In another embodiment, the process of obtaining traffic prediction data by the traffic prediction data acquisition unit through the traffic prediction model for the gantry real-time passing data includes:

[0101] Construct a number of basic models in advance to form the traffic prediction model, and process the gantry historical passing data through the basic models to obtain a number of prediction results;

[0102] Fuse each prediction result through the dynamic weight allocation method to obtain the traffic prediction data of the gantry real-time passing data.

[0103] In the above embodiments of the present invention, a dynamic database sharding and table partitioning system based on traffic prediction data is provided, and based on this system, a dynamic database sharding and table partitioning device based on traffic prediction data is provided. Through the above system and device, the real-time prediction of the change of vehicle flow is realized, the database resources are dynamically adjusted according to the prediction results, thereby reducing the data maintenance cost and improving the data utilization efficiency.

[0104] This embodiment also discloses a computer device, as Figure 3 shown, the computer device includes a processor and a memory, and at least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned dynamic database sharding and table partitioning system based on traffic prediction data.

[0105] In addition, in the implementation manner of the above-mentioned dynamic database sharding and table partitioning device based on traffic prediction data, the logical division of each program module is only for illustration. In actual applications, according to needs, for example, considering the configuration requirements of the corresponding hardware or the convenience of software implementation, the above functions can be assigned to different program modules to complete, that is, the internal structure of the dynamic database sharding and table partitioning device based on traffic prediction data is divided into different program modules to complete all or part of the functions described above.

[0106] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the scope of protection of the present application.

Claims

1. A dynamic database and table sub-system based on traffic prediction data, characterized in that: The system includes a traffic prediction model building module, a database initialization module, a data receiving module, and a database dynamic adjustment module; The traffic prediction model building module is used to build a traffic prediction model through the historical traffic data of the gantry, and process the real-time traffic data of the gantry through the traffic prediction model to obtain traffic prediction data; The database initialization module is used to determine the historical traffic data through the historical traffic data of the gantry, and divide the database into a plurality of logical partitions and a ring buffer resource pool according to the historical traffic data to determine the initial partition state and initial resource allocation of the database, wherein the ring buffer resource pool does not store resource data; The data receiving module is used to receive the real-time passage data of the gantry, and store the real-time passage data of the gantry into the corresponding logical partition according to preset conditions; The database dynamic adjustment module is used to allocate idle resources or release idle logical partition resources from the ring buffer resource pool according to the traffic prediction data generated by the traffic prediction model, so as to achieve dynamic expansion or dynamic reduction of the database.

2. The system according to claim 1, characterized in that The system also includes a data consistency assurance module; The data consistency guarantee module is used to record log information during the dynamic expansion or dynamic reduction process, and when a dynamic expansion or dynamic reduction failure is detected, data is rolled back according to the log information.

3. The system according to claim 1, characterized in that The allocating idle resources or releasing idle logical partition resources from the ring buffer resource pool comprises: Presetting a first logical partition load threshold for triggering dynamic expansion and a second logical partition load threshold for triggering dynamic reduction, wherein the first logical partition load threshold is greater than the second logical partition load threshold; When it is detected that the traffic prediction data is greater than the first logical partition load threshold, allocating idle resources from the ring buffer resource pool to complete the dynamic expansion of the database; When it is detected that the traffic prediction data is less than the second logical partition load threshold, a low-load logical partition is determined, and the data of the low-load logical partition is migrated to other logical partitions, and then the low-load logical partition is recovered to the ring buffer resource pool to complete the dynamic reduction of the database.

4. The system according to claim 1, characterized in that The database initialization module also includes a monitoring module; The monitoring module is used to monitor each logical partition in real time through a deployed real-time monitoring tool to determine the load status of the logical partition.

5. The system according to claim 1, characterized in that The processing of the real-time gantry traffic data by the traffic prediction model to obtain traffic prediction data includes: Pre-constructing a plurality of basic models to form the traffic prediction model, and processing the historical traffic data of the gantry through the basic models to obtain a plurality of prediction results; The various prediction results are fused through a dynamic weight allocation method to obtain the flow prediction data of the real-time traffic data of the gantry.

6. A dynamic library and table sub-device based on traffic prediction data, characterized in that: The device comprises: A flow prediction data acquisition unit is used to construct a flow prediction model through the historical traffic data of the gantry, and process the real-time traffic data of the gantry through the flow prediction model to obtain the flow prediction data; A database initialization unit, used to determine historical traffic data through the portal historical traffic data, and divide the database into a plurality of logical partitions and a ring buffer resource pool according to the historical traffic data to determine the initial partition state and initial resource allocation of the database, wherein the ring buffer resource pool does not store resource data; A data receiving unit, used for receiving the real-time passage data of the gantry, and storing the real-time passage data of the gantry into a corresponding logical partition according to preset conditions; A dynamic adjustment unit is used to allocate idle resources or release idle logical partition resources from the ring buffer resource pool according to the traffic prediction data generated by the traffic prediction model to complete the dynamic expansion or dynamic reduction of the database.

7. The device according to claim 6, characterized in that The device also includes: The data consistency guarantee unit is used to record log information during the dynamic expansion or dynamic reduction process, and when a dynamic expansion or dynamic reduction failure is detected, data is rolled back according to the log information.

8. The device according to claim 6, characterized in that Allocating idle resources from the ring buffer resource pool or releasing idle logical partition resources in the dynamic adjustment unit includes: Presetting a first logical partition load threshold for triggering dynamic expansion and a second logical partition load threshold for triggering dynamic reduction, wherein the first logical partition load threshold is greater than the second logical partition load threshold; When it is detected that the traffic prediction data is greater than the first logical partition load threshold, allocating idle resources from the ring buffer resource pool to complete the dynamic expansion of the database; When it is detected that the traffic prediction data is less than the second logical partition load threshold, a low-load logical partition is determined, and the data of the low-load logical partition is migrated to other logical partitions, and then the low-load logical partition is recovered to the ring buffer resource pool to complete the dynamic reduction of the database.

9. The device according to claim 6, characterized in that The device also includes: The monitoring unit is used to monitor each logical partition in real time through a deployed real-time monitoring tool to determine the load status of each logical partition.

10. The device according to claim 6, characterized in that The flow prediction data acquisition unit processes the real-time gantry traffic data by the flow prediction model to obtain the flow prediction data, including: Pre-constructing a plurality of basic models to form the traffic prediction model, and processing the historical traffic data of the gantry through the basic models to obtain a plurality of prediction results; The various prediction results are fused through a dynamic weight allocation method to obtain the flow prediction data of the real-time traffic data of the gantry.

Images