A database calling method based on the ocean forecast and disaster reduction model

Abstract

The present invention provides a database calling method based on a large model of ocean forecasting and disaster reduction, belonging to the technical field of ocean forecasting. The present invention constructs a double-layer time granularity index structure, divides data into high-frequency and low-frequency calling units according to the calling frequency, constructs a GPU display memory acceleration matrix and a CPU memory storage matrix respectively, designs a heterogeneous parallel scheduling algorithm to realize data migration based on multi-dimensional indicator optimization, and applies a spatiotemporal index self-optimization mechanism to dynamically adjust the time granularity division, and predicts the data calling mode through a multi-head attention-driven ocean spatiotemporal feature predictor, combines a gated weight function to dynamically balance the accuracy and efficiency requirements, and finally achieves a significant improvement in the data calling efficiency of the large model of ocean forecasting and disaster reduction, so that the system can efficiently respond to the data requirements of high-precision, large-scale, and multi-time forecast scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of ocean forecasting, and in particular relates to a database calling method based on an ocean forecast disaster reduction large model. Background Art

[0002] The Ocean Forecast and Disaster Reduction Large Model is a complex computing system that utilizes massive amounts of multi-source, heterogeneous data for marine environmental forecasting and disaster warning. Traditional ocean forecasting systems typically employ a single storage architecture, storing massive amounts of forecast data uniformly on disk or in memory. This data is organized using a simple spatiotemporal indexing scheme and accessed on demand during model execution. This approach can meet basic requirements for small-scale forecasting scenarios, but as forecast accuracy and coverage continue to improve, the data size and computational complexity are rapidly increasing. Existing methods for accessing ocean forecast databases suffer from numerous drawbacks: First, a single storage hierarchy cannot accommodate data access demands of varying frequencies, resulting in repeated access of high-frequency data and increased system load; second, a lack of optimized indexing structures tailored to the spatiotemporal characteristics of ocean data results in inefficient data retrieval; third, the ineffective utilization of heterogeneous GPU and CPU computing resources results in wasted computing power; and finally, static data management strategies fail to dynamically adjust data organization based on the forecast model, making them difficult to adapt to complex and changing forecast scenarios. Faced with the growing demand for ocean forecasting, existing technologies are unable to solve the technical problem of low data call efficiency in large-scale ocean forecast and disaster reduction models. Especially in high-precision, large-scale, and multi-time forecast scenarios, data call has become a bottleneck restricting the overall performance of the system. There is an urgent need to design an efficient database call method. Summary of the Invention

[0003] In view of this, the present invention provides a database calling method based on the ocean forecast disaster reduction large model, which can solve the technical problem of low efficiency in calling data of the ocean forecast disaster reduction large model in the prior art.

[0004] The present invention is implemented as follows: The present invention provides a database calling method based on a large model of ocean forecasting and disaster reduction, comprising: constructing a double-layer time granularity index structure, organizing ocean forecast data according to a first time granularity and a second time granularity to form a spatiotemporal four-dimensional index matrix; dividing the data into high-frequency calling units and low-frequency calling units according to the spatiotemporal characteristics and calling frequency of the data required for calling by the ocean forecast model; constructing a GPU video memory data acceleration matrix for the high-frequency calling units; constructing a CPU memory data storage matrix for the low-frequency calling units; designing a heterogeneous parallel scheduling algorithm, realizing data adaptive migration between GPU video memory and CPU memory based on a data migration optimization function; constructing a spatiotemporal index self-optimization mechanism, dynamically adjusting the division ratio of the first time granularity and the second time granularity according to the model calling mode; applying an ocean spatiotemporal feature predictor to predict the data calling mode of the forecast model, combining distributed computing resource pooling management, and optimizing the computing resource allocation strategy according to the data demand characteristics under different forecast scenarios.

[0005] The first time granularity refers to the basic index unit of ocean forecast data in the time dimension, and the second time granularity refers to a time unit larger than the first time granularity that is aggregated on the basis of the first time granularity.

[0006] The calling frequency refers to the number of times the ocean forecast and disaster reduction model accesses data in a unit of time, and the data is divided into high-frequency data and low-frequency data according to the calling frequency.

[0007] The calling unit refers to a data set with similar spatiotemporal characteristics and access patterns, which serves as the basic unit for data scheduling and storage optimization.

[0008] The GPU memory data acceleration matrix includes a read-only data sparse matrix and a read-write data compression matrix.

[0009] The read-only data sparse matrix refers to ocean static parameters and historical data stored in the GPU memory that do not need to be modified, and is represented by a sparse matrix to reduce storage space occupancy.

[0010] The read-write data compression matrix refers to the ocean dynamic parameters and forecast intermediate results that need to be frequently updated and stored in the GPU memory, and a real-time compression algorithm is used to reduce data transmission overhead.

[0011] The heterogeneous parallel scheduling algorithm refers to an algorithm that simultaneously utilizes the computing characteristics of GPU processors and CPU processors to perform parallel processing on ocean data with different characteristics, including a task decomposition module, a load balancing module and a resource management module.

[0012] The input of the data migration optimization function includes a data call frequency index, a data timeliness index, a storage capacity pressure index, a computing load balancing index, and a data dependency index.

[0013] Among them, the ocean spatiotemporal feature predictor is a multi-head attention-driven ocean data call pattern prediction model. The number of attention heads in the ocean spatiotemporal feature predictor needs to be dynamically determined according to the first time granularity, the second time granularity and the number of high-frequency call units.

[0014] The database includes structured and unstructured ocean observation and forecast data from Xinchuang Database, SQL, Oracl, Mango Database, etc. It also includes the step of combining and calling multiple different databases including structured and unstructured ones.

[0015] The present invention comprehensively improves the data call efficiency of the large model of ocean forecasting and disaster reduction by constructing innovative technologies such as a double-layer time granularity index structure, hierarchical storage of data frequency, heterogeneous parallel scheduling, and an ocean spatiotemporal feature predictor. This method solves the key defects in traditional technologies: the double-layer time granularity index structure realizes the efficient organization and retrieval of data at different time scales; the hierarchical storage architecture in which high-frequency data is stored in the GPU video memory acceleration matrix and low-frequency data is stored in the CPU memory matrix effectively utilizes heterogeneous computing resources; the heterogeneous parallel scheduling algorithm and data migration optimization function realize the intelligent scheduling of data between different storage levels, avoiding data call bottlenecks; the ocean spatiotemporal feature predictor can predict future data call patterns, realize data preloading and optimize resource allocation. Through the above-mentioned innovative design, the present invention effectively solves the technical problem of low data call efficiency of the large model of ocean forecasting and disaster reduction, and significantly improves system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the method of the present invention.

[0017] Figure 2 This is a structural diagram of the ocean spatiotemporal characteristics predictor involved in the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0019] like Figure 1 FIG. 1 is a flow chart of a method for calling a database based on a large ocean forecast and disaster reduction model provided by the present invention. The method includes the following steps:

[0020] S01. Construct a two-layer time granularity index structure, organize the ocean forecast data according to the first time granularity and the second time granularity, and form a spatiotemporal four-dimensional index matrix;

[0021] S02. Dividing the data into high-frequency call units and low-frequency call units according to the spatiotemporal characteristics and call frequency of the data required by the ocean forecast model;

[0022] S03. Constructing a GPU memory data acceleration matrix for high-frequency call units, including a read-only data sparse matrix and a read-write data compression matrix;

[0023] S04. Build a CPU memory data storage matrix for low-frequency call units and manage them using a hierarchical cache structure;

[0024] S05. Design a heterogeneous parallel scheduling algorithm to implement adaptive data migration between GPU memory and CPU memory based on a data migration optimization function, wherein the input of the data migration optimization function includes a data call frequency index, a data timeliness index, a storage capacity pressure index, a computing load balancing index, and a data dependency index;

[0025] S06. Build a spatiotemporal index self-optimization mechanism to dynamically adjust the division ratio of the first time granularity and the second time granularity according to the model call mode;

[0026] S07. Use the ocean spatiotemporal feature predictor to predict the data call pattern of the forecast model, combine it with the distributed computing resource pool management, and optimize the computing resource allocation strategy based on the data demand characteristics under different forecast scenarios.

[0027] Among them, the first time granularity refers to the basic index unit of ocean forecast data in the time dimension, usually at the minute level, which is used to support the data call requirements of high-frequency ocean dynamics models.

[0028] Among them, the second time granularity refers to a larger time unit aggregated on the basis of the first time granularity, usually at the hourly level, which is used to support the data call needs of medium and low frequency marine environmental element forecasts.

[0029] The calling frequency refers to the number of times the ocean forecast and disaster reduction model accesses data in a unit of time. The data is divided into high-frequency data and low-frequency data according to the calling frequency.

[0030] Among them, the call unit refers to a data set with similar spatiotemporal characteristics and access patterns, which serves as the basic unit of data scheduling and storage optimization.

[0031] Among them, the data acceleration matrix refers to the high-frequency call data structure organized in the GPU video memory, which uses sparse storage and compression algorithms to optimize storage space utilization.

[0032] Among them, the read-only data sparse matrix refers to the ocean static parameters and historical data stored in the GPU memory that do not need to be modified, and is represented by a sparse matrix to reduce storage space.

[0033] Among them, the read and write data compression matrix refers to the ocean dynamic parameters and forecast intermediate results that need to be frequently updated and stored in the GPU video memory. A real-time compression algorithm is used to reduce data transmission overhead.

[0034] Among them, the heterogeneous parallel scheduling algorithm refers to an algorithm that simultaneously utilizes the computing characteristics of GPU processors and CPU processors to perform parallel processing on ocean data with different characteristics. It includes a task decomposition module, a load balancing module, and a resource management module, and maximizes the utilization of computing resources by dynamically adjusting the allocation ratio of computing tasks.

[0035] Among them, the data migration optimization function is used to evaluate the necessity and optimal timing of data migration between GPU video memory and CPU memory. The data call frequency index is obtained from step S02, the data timeliness index is determined by the update cycle of ocean forecast data, the storage capacity pressure index is calculated by the remaining space of GPU video memory and CPU memory, the computational load balancing index is provided by the load balancing module in the heterogeneous parallel scheduling algorithm, and the data dependency index is obtained by analyzing the dependency graph between data call units. The output of the data migration optimization function includes the data migration priority score and the migration timing judgment result, which are used for the data adaptive migration decision between GPU video memory and CPU memory in step S05.

[0036] The ocean spatiotemporal feature predictor is a multi-head attention-driven ocean data call pattern prediction model. The number of attention heads in the ocean spatiotemporal feature predictor needs to be dynamically determined based on the first time granularity, the second time granularity and the number of high-frequency call units.

[0037] The specific structure of the ocean spatiotemporal feature predictor is a bidirectional architecture including an encoder and a decoder. The encoder part uses a three-dimensional spatiotemporal convolutional network to extract the spatiotemporal features of ocean forecast data, and the decoder part uses a multi-head attention mechanism to fuse spatiotemporal features of different scales. The number of attention heads is determined by the number of high-frequency call units. Each attention head is responsible for capturing the spatiotemporal correlation of a class of call patterns. The core of the ocean spatiotemporal feature predictor includes three parts: a spatiotemporal feature extraction module, a call pattern recognition module, and a prediction optimization module. The ocean spatiotemporal feature predictor also introduces a skip connection mechanism to ensure the effective transmission of spatiotemporal information in the deep network. The output layer of the ocean spatiotemporal feature predictor uses a probability distribution form to represent the access probability of each data call unit in the future time window.

[0038] The steps of establishing the training data set of the ocean spatiotemporal feature predictor specifically include collecting data call logs from historical ocean forecast tasks, extracting the call time series and spatial distribution characteristics of each data unit in each forecast process, generating a call frequency heat map and a time series change curve, marking high-frequency call areas and low-frequency call areas, and constructing paired samples containing input features and labels. The input features are data call patterns in historical time windows, and the labels are data call probability distributions in future time windows. Training samples are constructed through a sliding window method, and data enhancement processing is performed to improve the generalization ability of the model.

[0039] The steps of training the ocean spatiotemporal feature predictor specifically include first pre-training the ocean spatiotemporal feature predictor, using a self-supervised learning method to allow the ocean spatiotemporal feature predictor to learn the basic spatiotemporal pattern of ocean data calls, then using labeled data for supervised fine-tuning, using a cross-entropy loss function to evaluate the difference between the predicted results and the actual call pattern, using an adaptive learning rate optimizer to update parameters, introducing an early stopping mechanism to prevent overfitting, setting a validation set for different forecast scenarios to evaluate the performance of the ocean spatiotemporal feature predictor, and finally saving the model parameters that perform best on the validation set.

[0040] The gated weight function is used to adjust the weight distribution of the multi-head attention mechanism in the ocean spatiotemporal feature predictor. The gated weight function is calculated based on four data: ocean forecast accuracy requirements, computing resource constraints, data call real-time requirements, and model reasoning efficiency, to obtain a multi-objective balance value. When the multi-objective balance value belongs to the high-precision priority interval, the accuracy-enhanced weight adjustment function is used to increase the weight of historical data patterns and enhance the ocean spatiotemporal feature predictor's ability to perceive changes in the ocean environment; when the multi-objective balance value belongs to the efficiency priority interval, the computational acceleration weight adjustment function is used to increase the weight of local spatiotemporal features and reduce the processing of global information to improve the reasoning speed; when the multi-objective balance value belongs to the balanced interval, the adaptive balanced weight adjustment function is used to dynamically adjust the importance distribution of each attention head to achieve a dynamic balance between accuracy and efficiency.

[0041] The ocean forecast accuracy requirement is determined by the application scenario of the ocean forecast and disaster reduction large model, the computing resource constraint is jointly limited by the GPU video memory capacity and the CPU memory capacity, the real-time data call requirement is determined by the forecast timeliness requirement of the ocean forecast and disaster reduction large model, and the model reasoning efficiency is determined by the computational complexity of the ocean spatiotemporal feature predictor.

[0042] The precision-enhanced weight adjustment function allocates more attention weights to attention heads related to historical data patterns, the computational acceleration weight adjustment function allocates more attention weights to attention heads related to local spatiotemporal features, and the adaptive balancing weight adjustment function dynamically adjusts the weight ratio of each attention head according to real-time forecasting requirements.

[0043] The specific implementation of the above steps is described in detail below.

[0044] The specific implementation of step S01 is to construct a two-layer time granularity index structure. First, the first time granularity is determined, usually 1 to 5 minutes, as the basic time unit for accessing high-frequency data of the ocean dynamics model. Then, the second time granularity is determined, usually 30 to 120 minutes, as the time unit for environmental factor forecasts. The ocean forecast data is then organized according to these two granularities in the time dimension, forming a two-layer index in the time dimension. Then, an octree structure is used to divide the three-dimensional ocean space in the spatial dimension, and a spatial index is constructed for the ocean space according to the three dimensions of longitude, latitude, and depth. Finally, the two-layer time index is combined with the three-dimensional spatial index to construct a four-dimensional spatiotemporal index matrix, in which each element corresponds to a spatiotemporal data block. This step uses multi-resolution spatiotemporal indexing technology to achieve efficient organization and rapid retrieval of data at different time and spatial scales, thereby improving the efficiency of data access for the large-scale ocean forecast and disaster reduction model.

[0045] The specific implementation of step S02 is to divide the data based on the spatiotemporal characteristics and call frequency of the data required by the ocean forecast model. First, statistical analysis is performed on the historical data call log to calculate the average access frequency of each spatiotemporal data block. Then, a call frequency threshold is set, usually 10 calls per second, and data blocks above this threshold are classified as high-frequency call units. Next, data blocks below the call frequency threshold are classified as low-frequency call units. The spatial distribution characteristics of each call unit are then analyzed, and data blocks with adjacent spatial locations and similar call patterns are merged to form larger call units. Finally, based on the spatiotemporal characteristics and data size of each call unit, its initial allocation strategy in GPU video memory or CPU memory is determined. This step uses data access pattern analysis technology to identify the usage characteristics of different data, provide a basis for subsequent heterogeneous storage optimization, and improve the overall data access efficiency of the system.

[0046] The specific implementation method of step S03 is to construct a GPU memory data acceleration matrix for high-frequency call units. First, the read and write characteristics of the data in the high-frequency call units are analyzed and divided into two categories: read-only data and read-write data. Then, the sparsity analysis of the read-only data is performed, and a collaborative filtering algorithm is used to identify redundancy and sparse patterns in the data. Then, a sparse matrix of read-only data is constructed, and the sparse data is compressed using a coordinate format storage method, which can usually reduce storage space by 60% to 80%. After that, the read and write data are analyzed for characteristics and a compression algorithm suitable for the characteristics of marine data is selected, such as wavelet transform or principal component analysis. Finally, a read and write data compression matrix is ​​constructed to achieve high compression ratio storage while ensuring data accuracy, reducing data transmission overhead. This step uses data sparse representation and compressed storage technology to maximize the use of limited GPU memory space, store more high-frequency access data, reduce the number of data transmissions between the CPU and GPU, and improve computing efficiency.

[0047] The specific implementation of step S04 is to construct a CPU memory data storage matrix for low-frequency call units. First, the low-frequency call units are further subdivided into medium-frequency data and low-frequency data according to the access frequency. The access threshold of medium-frequency data is usually 1 to 10 times per minute. Then, a multi-level cache structure is designed, including L1 cache, L2 cache and main storage area. The L1 cache is usually configured to be 5% to 10% of the total memory, and the L2 cache is configured to be 15% to 25% of the total memory. Then, a cache replacement strategy based on the least recently used algorithm is established, which prioritizes the placement of medium-frequency data in the L1 cache and the placement of low-frequency data in the L2 cache or main storage area. Then, a cache prefetch mechanism is constructed. Based on the principle of spatiotemporal locality of data access, data blocks that may be accessed are predicted and loaded in advance. Finally, a cache consistency maintenance mechanism is established to ensure that data copies in caches at all levels are synchronized in a timely manner when data is updated. This step adopts hierarchical cache management technology to optimize the storage and access efficiency of low-frequency data, reduce memory access latency, and improve system throughput.

[0048] The specific implementation of step S05 is to design a heterogeneous parallel scheduling algorithm to achieve adaptive data migration between the GPU and CPU. First, a data migration optimization function is constructed, which comprehensively considers multiple indicators: data call frequency indicator, data timeliness indicator, storage capacity pressure indicator, computational load balancing indicator, and data dependency indicator. Then, a data migration priority score is calculated based on these five indicators. Using a weighted summation method, the weights of each indicator are dynamically adjusted according to the specific application scenario. Next, a migration trigger threshold is set. Generally, when the priority score is greater than 0.75, migration to the GPU is triggered, and when the score is less than 0.25, migration to the CPU is triggered. Then, a batch data migration mechanism is implemented to merge multiple small data blocks that need to be migrated into a large data block for transmission, reducing migration overhead. Finally, a migration logging system is established to track data migration history and optimize future migration decisions. This step adopts multi-objective optimization and heterogeneous computing technologies to achieve efficient utilization of computing resources, dynamically adjust data storage locations according to real-time needs, and improve the overall operational efficiency of the large-scale marine disaster forecasting and mitigation model.

[0049] The specific implementation of step S06 is to build a spatiotemporal index self-optimization mechanism. First, a model call pattern monitoring system is established to record the access of the marine disaster forecasting and mitigation model to each spatiotemporal data block in real time. Then, the temporal characteristics of the call pattern are analyzed, and the statistical characteristics of data access at different time granularities, such as average access frequency, standard deviation, and peak distribution, are calculated. Next, the efficiency indicators of the current time granularity division are evaluated, including index hit rate, data block access balance, and storage space utilization. The division ratio of the first and second time granularities is then dynamically adjusted based on the efficiency indicators. This adjustment is usually triggered when the index hit rate falls below 80%. Finally, a progressive index reconstruction algorithm is applied to complete the update of the spatiotemporal index structure without interrupting system operation. This step uses adaptive index optimization technology to enable the index structure to adapt to changing data access patterns and improve the data access efficiency of the marine disaster forecasting and mitigation model in different application scenarios.

[0050] The specific implementation of step S07 is to use the ocean spatiotemporal feature predictor to optimize the computing resource allocation strategy. First, the trained ocean spatiotemporal feature predictor is activated and the current ocean forecast scenario parameters and historical data call patterns are input. The ocean spatiotemporal feature predictor is then used to predict the access probability distribution of each data call unit within a future time window, which is typically set to 10 to 30 minutes. Next, a resource allocation optimization model is constructed, using the prediction results as input, taking into account the performance characteristics of computing nodes and the network topology. Distributed computing resource pooling management technology is then applied to classify computing nodes according to computing power and storage capacity to form resource pools. Finally, computing task allocation and data storage strategies are dynamically adjusted based on the data demand characteristics and resource pool status under different forecast scenarios. This step uses prediction-driven resource optimization technology. The goal is to make resource allocation decisions in advance by predicting future data access patterns, reducing resource competition and waiting time, and improving the computational efficiency and forecast accuracy of the large-scale ocean forecast and disaster reduction model.

[0051] The detailed structure of the ocean spatiotemporal feature predictor includes three core modules: spatiotemporal feature extraction module, call pattern recognition module and prediction optimization module. The spatiotemporal feature extraction module adopts a three-dimensional spatiotemporal convolutional network with a specific structure of four layers of three-dimensional convolution layers. The number of convolution kernels in each layer is 32, 64, 128 and 256 respectively. The convolution kernel size is , with a stride of 1 and padding of 1. Each convolution layer is followed by a batch normalization layer and a rectified linear unit activation function. Finally, an adaptive average pooling layer is used to reduce the feature map to a fixed size. The call pattern recognition module is based on a multi-head attention mechanism. The number of heads is determined by the number of high-frequency call units, typically 4 to 16. Each attention head consists of a query matrix, a key matrix, and a value matrix with matrix dimensions of 64 or 128. Attention weights are calculated via scaled dot products, and the attention scores are normalized using the softmax function. The prediction optimization module consists of two fully connected layers. The first layer has 1024 neurons and uses the rectified linear unit activation function. The second layer has a number of neurons equal to the total number of data call units in the prediction time window. The sigmoid activation function is used to output the access probability of each call unit. To ensure the effective transmission of spatiotemporal information in the deep network, residual connections are introduced between convolutional layers. Every two convolution layers form a residual block, and identity mapping is used to achieve direct information transfer. The final output layer uses probability distribution to represent the access probability of each data call unit in the future time window. The probability threshold is usually set to 0.6. Call units above this threshold will be given priority in resource allocation.

[0052] The detailed steps of establishing the training data set of the ocean spatiotemporal feature predictor are as follows: first, data call logs are collected from historical ocean forecasting tasks. The logs contain information such as data block identification, call timestamp, call process identification and access type. Usually, log data from the last three months are collected; then, the call time series of each data unit in each forecast process is extracted, and the number of accesses to each data unit in each time window is counted. The time window length is usually 5 minutes; then, the spatial distribution characteristics of the data units are analyzed, the call correlation of spatially adjacent data units is calculated, and a spatial association network is constructed; then, a call frequency heat map is generated, and the access frequency of different areas is represented by color depth, and a time series change curve is drawn to show the evolution of the access pattern over time; then, according to the set A frequency threshold is set to label high-frequency and low-frequency call areas. High-frequency areas are generally defined as areas with an access frequency twice as high as the average. Paired samples are then constructed, consisting of input features and labels. The input features are the data call patterns within a historical time window, typically 30 minutes long, and the labels are the probability distribution of data calls within a future time window, typically 10 minutes long. Training samples are then constructed using a sliding window method with a 5-minute sliding step, extracting a large number of sample pairs from historical logs. Finally, data augmentation processing is performed, including adding Gaussian noise, randomly masking certain areas, and timescale transformations, to increase sample diversity and improve model generalization. The original sample size is typically expanded by 3 to 5 times. Once the training samples are constructed, the dataset is divided into training, validation, and test sets in a ratio of 7:2:1 for training and evaluation of the ocean spatiotemporal feature predictor.

[0053] It should be noted that the first key technical idea of ​​the present invention is a two-layer time granularity index structure, which organizes ocean forecast data according to a first time granularity of minutes and a second time granularity of hours to form a spatiotemporal four-dimensional index matrix. Unlike the single time scale index used in traditional ocean forecast systems, this two-layer structure can more accurately match the calling characteristics of data at different time scales in ocean forecast models. In traditional methods, all data are indexed according to a unified time scale, resulting in low efficiency in high-frequency data retrieval or low-frequency data occupying too many storage resources. The present invention uses differentiated time granularity indexes to organize and retrieve high-frequency dynamic model data and low-frequency environmental factor forecast data at the most suitable time scale, thereby greatly improving data retrieval efficiency and reducing system resource consumption.

[0054] The second key technical idea is a hierarchical storage mechanism based on call frequency, which divides data into high-frequency call units and low-frequency call units, and constructs a GPU memory data acceleration matrix and a CPU memory data storage matrix respectively. Traditional ocean forecasting systems usually store all data in the same type of storage medium, failing to fully utilize the characteristics of heterogeneous computing resources. The present invention fully considers the characteristics of GPUs with high computing throughput but limited storage capacity and CPUs with large storage capacity but relatively low computing efficiency. By placing high-frequency access data in GPU memory and low-frequency access data in CPU memory, the optimal configuration of storage resources is achieved. This hierarchical storage strategy not only improves the access speed of high-frequency data, but also significantly reduces system latency by reducing the data transmission overhead between the GPU and the CPU.

[0055] The third key technical idea is the ocean spatiotemporal feature predictor, which uses a bidirectional architecture driven by multi-head attention to predict data call patterns. Data scheduling in traditional systems is often based on historical statistics or simple rules, lacking the ability to predict future data access patterns. The ocean spatiotemporal feature predictor of the present invention extracts the spatiotemporal features of ocean data through a three-dimensional spatiotemporal convolutional network, and uses a multi-head attention mechanism to capture the spatiotemporal correlation of different call patterns. It can accurately predict the access probability distribution of each data call unit in the future time window. This predictive capability enables the system to perform data scheduling and resource allocation in advance, avoiding the performance fluctuations and resource waste caused by traditional passive response scheduling.

[0056] The synergistic effect of these three key technical ideas has produced significant technical effects and advantages. The two-layer time granularity index structure provides the basis for efficient organization and retrieval of data, the hierarchical storage mechanism realizes the optimal utilization of heterogeneous computing resources, and the ocean spatiotemporal feature predictor injects forward-looking intelligence into the entire system. The combination of the three forms a closed-loop optimization system: the predictor predicts future data call patterns, guides the two-layer time granularity index structure to dynamically adjust the time granularity division ratio, and guides the hierarchical storage mechanism to preload and migrate data. This collaborative mechanism enables the system to adaptively adjust data organization and storage strategies according to changes in forecast scenarios, achieving optimal allocation of computing resources and maximizing data call efficiency. Compared with the static data management and passive scheduling mechanism of traditional systems, the present invention realizes full-chain optimization of the data call process, fundamentally solving the technical problem of low data call efficiency of large-scale models for marine forecasting and disaster reduction.

[0057] Specifically, the principle of the present invention is: the technical solution of the present invention improves the efficiency of calling data of the large-scale marine forecast and disaster reduction model based on the following core principles:

[0058] First, a two-layer time-granularity index structure, based on the spatiotemporal characteristics and call characteristics of ocean forecast data, organizes data according to a primary time granularity (minutes) and a secondary time granularity (hours), forming a spatiotemporal four-dimensional index matrix. This structure fully accounts for the differences in call frequencies of data at different time scales in ocean forecast models, enabling differentiated indexing and management of high- and low-frequency data, thereby improving data retrieval efficiency. The primary time granularity primarily addresses the call requirements for high-frequency data in ocean dynamics models, while the secondary time granularity targets the call requirements for medium- and low-frequency forecast data of marine environmental factors. This hierarchical index structure is highly compatible with the data call characteristics of ocean forecast models.

[0059] Secondly, a data frequency-tiered storage mechanism divides data into high-frequency and low-frequency units based on call frequency. A corresponding GPU memory data acceleration matrix and a CPU memory data storage matrix are designed. High-frequency data is organized in GPU memory using sparse and compressed matrices, while low-frequency data is managed in CPU memory using a hierarchical cache structure. This tiered storage strategy leverages the complementary advantages of the GPU's high computational efficiency and the CPU's large storage capacity, effectively resolving the conflict between massive data storage and efficient access.

[0060] Third, the heterogeneous parallel scheduling algorithm and data migration optimization function achieve efficient utilization of computing resources and intelligent data scheduling. The data migration optimization function comprehensively considers multiple metrics, including data call frequency, timeliness, storage pressure, computing load, and data dependencies, to calculate the priority and optimal timing of data migration. This guides the adaptive migration of data between GPU memory and CPU memory, reducing unnecessary data transfer and call latency. The heterogeneous parallel scheduling algorithm, through task decomposition, load balancing, and resource management modules, enables collaborative computing between the GPU and CPU.

[0061] Finally, the ocean spatiotemporal feature predictor, using a bidirectional architecture driven by multi-head attention, predicts the data call patterns of the forecast model, providing forward-looking guidance for data organization and scheduling. The predictor's three-dimensional spatiotemporal convolutional network extracts the spatiotemporal characteristics of ocean data, while the multi-head attention mechanism captures the spatiotemporal correlations between different call patterns and outputs the access probability distribution of each data call unit within the future time window. Based on these predictions, the system can proactively adjust data storage locations and index structures, enabling data preloading and resource pre-allocation, further improving data call efficiency.

[0062] A specific embodiment 1 of the present invention is provided below. The specific implementation of each step in this embodiment 1 is described in detail as follows.

[0063] The specific implementation of step S01 is to construct a two-layer time granularity index structure. First, the first time granularity is determined, usually 1 minute to 5 minutes, as the basic time unit for accessing high-frequency data of the ocean dynamics model; then the second time granularity is determined, usually 30 minutes to 120 minutes, as the time unit for environmental factor forecasts; then the ocean forecast data is organized according to these two granularities in the time dimension to form a two-layer index in the time dimension; then, the octree structure is used to divide the three-dimensional ocean space in the spatial dimension, and the spatial index is constructed according to the three dimensions of longitude, latitude and depth; finally, the time double-layer index is combined with the spatial three-dimensional index to construct a spatiotemporal four-dimensional index matrix, in which each element corresponds to a spatiotemporal data block. The spatiotemporal four-dimensional index matrix can be expressed as:

[0064] ;

[0065] Where, is the spatiotemporal four-dimensional index matrix; The matrix is ​​located at The first time granularity, The second time granularity, spatial position, Data elements of depth layers; The range is to , The number of first time granularity divisions; The range is to , Divide the quantity into the second time granularity; The range is to , Divide the quantity into spatial locations; The range is to , The number of divisions for the depth layer.

[0066] The mathematical expression of the spatial index using the octree structure is:

[0067] ;

[0068] Where, Represents an octree structure; Indicates the current node; Represents the root node; Indicates the child nodes, The range is to .

[0069] This step uses multi-resolution spatiotemporal indexing technology to achieve efficient organization and rapid retrieval of data at different time and spatial scales, and to improve the efficiency of data access by the large-scale marine forecasting and disaster reduction model.

[0070] The specific implementation of step S02 is to divide the data according to the spatiotemporal characteristics and call frequency of the call data required by the ocean forecast model. First, statistical analysis is performed on the historical data call log to calculate the average access frequency of each spatiotemporal data block; then a call frequency threshold is set, usually 10 calls per second, and data blocks above the threshold are classified as high-frequency call units; then data blocks below the call frequency threshold are classified as low-frequency call units; then the spatial distribution characteristics of each call unit are analyzed, and data blocks with adjacent spatial positions and similar call patterns are merged to form a larger call unit; finally, based on the spatiotemporal characteristics and data size of each call unit, its initial allocation strategy in the GPU video memory or CPU memory is determined. The average access frequency calculation formula is:

[0071] ;

[0072] Where, Represents a data block Average visit frequency; Indicates the time period Internal data block Number of visits; Indicates the total number of time periods for statistics.

[0073] The similarity calculation formula for calling unit merging is:

[0074] ;

[0075] Where, Indicates the calling unit and The total similarity of Indicates spatial location similarity, usually using the inverse of the Euclidean distance; Indicates the similarity of call patterns, usually using the Pearson correlation coefficient; and is the weight coefficient, and ,generally The value is 0.4, The value is 0.6.

[0076] This step uses data access pattern analysis technology to identify the usage characteristics of different data, provide a basis for subsequent heterogeneous storage optimization, and improve the overall data access efficiency of the system.

[0077] The specific implementation method of step S03 is to construct a GPU memory data acceleration matrix for high-frequency call units. First, the read and write characteristics of the data in the high-frequency call units are analyzed and divided into two categories: read-only data and read-write data; then the sparsity analysis of the read-only data is performed, and the collaborative filtering algorithm is used to identify redundancy and sparse patterns in the data; then a sparse matrix of read-only data is constructed, and the sparse data is compressed using a coordinate format storage method, which can usually reduce storage space by 60% to 80%; then the feature analysis of the read and write data is performed, and a compression algorithm suitable for the characteristics of marine data is selected, such as wavelet transform or principal component analysis; finally, a read and write data compression matrix is ​​constructed to achieve high compression ratio storage while ensuring data accuracy, reducing data transmission overhead. Among them, the coordinate format of the read-only data sparse matrix is ​​expressed as:

[0078] ;

[0079] Where, represents the coordinates and values ​​of non-zero elements, Indicates the first time granularity index, represents the second time granularity index, Represents the spatial position index, represents the depth layer index, Indicates the data value at the corresponding position.

[0080] The mathematical expression of wavelet transform for reading and writing data compression is:

[0081] ;

[0082] Where, Indicates signal Wavelet transform result of represents the wavelet function; represents the conjugate of the wavelet function; represents the scale parameter; Represents the translation parameter.

[0083] This step uses data sparse representation and compressed storage technology to maximize the use of limited GPU memory space, store more frequently accessed data, reduce the number of data transfers between the CPU and GPU, and improve computing efficiency.

[0084] The specific implementation method of step S04 is to construct a CPU memory data storage matrix for the low-frequency call unit. First, the low-frequency call unit is further subdivided into medium-frequency data and low-frequency data according to the access frequency. Usually, the access threshold of medium-frequency data is 1 to 10 times per minute; then a multi-level cache structure is designed, including L1 cache, L2 cache and main storage area. The L1 cache is usually configured as 5% to 10% of the total memory, and the L2 cache is configured as 15% to 25% of the total memory; then a cache replacement strategy based on the least recently used algorithm is established, and medium-frequency data is placed in the L1 cache first, and low-frequency data is placed in the L2 cache or main storage area; then a cache prefetch mechanism is constructed, based on the principle of spatiotemporal locality of data access, to predict and preload data blocks that may be accessed; finally, a cache consistency maintenance mechanism is established to ensure that data copies in caches at all levels are synchronized in time when data is updated. The mathematical expression of the least recently used algorithm is:

[0085] ;

[0086] Where, Represents a data block LRU score; Indicates the current system time; Represents a data block The last visit time of the .

[0087] The prediction model of the cache prefetching mechanism can be expressed as:

[0088] ;

[0089] Where, Represents a data block in the current context Probability of being visited; Represents historical access patterns; Represents temporal locality characteristics; Represents spatial locality characteristics; Represents the prediction function, which is usually implemented using a Bayesian network or Markov model.

[0090] This step uses hierarchical cache management technology to optimize the storage and access efficiency of low-frequency data, reduce memory access latency, and improve system throughput.

[0091] The specific implementation method of step S05 is to design a heterogeneous parallel scheduling algorithm to realize the adaptive migration of data between GPU and CPU. First, a data migration optimization function is constructed, which comprehensively considers multiple indicators: data call frequency indicator, data timeliness indicator, storage capacity pressure indicator, computational load balancing indicator and data dependency indicator; then, the data migration priority score is calculated based on these five indicators, and the weighted summation method is used. The weight of each indicator is dynamically adjusted according to the specific application scenario; then, the migration trigger threshold is set. Usually, when the priority score is greater than 0.75, migration to the GPU is triggered, and when the score is less than 0.25, migration to the CPU is triggered; then, a batch data migration mechanism is implemented to merge multiple small data blocks that need to be migrated into large data blocks for transmission, reducing migration overhead; finally, a migration log recording system is established to track data migration history for optimizing future migration decisions. Among them, the mathematical expression of the data migration optimization function is:

[0092] ;

[0093] Where, Indicates the calling unit Migration priority score; Indicates the data call frequency index, with a value range of [0, 1]; Indicates the data timeliness index, with a value range of [0, 1]; Indicates the storage capacity pressure index, with a value range of [0, 1]; Indicates the calculation load balancing index, the value range is [0, 1]; Represents the data dependency index, with a value range of [0, 1]; to is the weight coefficient, and .

[0094] The calculation methods of each indicator are as follows:

[0095] ;

[0096] Where, Indicates the calling unit Average visit frequency; Indicates the minimum access frequency of all call units; Indicates the maximum access frequency of all call units.

[0097] ;

[0098] Where, Indicates the current system time; Indicates the calling unit Last updated time; Indicates the calling unit update cycle.

[0099] ;

[0100] Where, Indicates the remaining space in the current storage system; Indicates the total capacity of the storage system.

[0101] ;

[0102] Where, Indicates the current load level of the computing system; Indicates the minimum load level allowed by the system; Indicates the maximum load level allowed for the system.

[0103] ;

[0104] Where, Representation and Calling Unit A set of call units with dependencies; Indicates the calling unit The weight of Indicates the calling unit The position indicator, when It is 1 when located on the GPU and 0 when located on the CPU; Indicates the size of the dependency set.

[0105] This step uses multi-objective optimization and heterogeneous computing technology to achieve efficient use of computing resources, dynamically adjust data storage locations according to real-time needs, and improve the overall operating efficiency of the ocean forecasting and disaster reduction model.

[0106] The specific implementation method of step S06 is to build a spatiotemporal index self-optimization mechanism. First, a model call mode monitoring system is established to record the access of the ocean forecast and disaster reduction model to each spatiotemporal data block in real time; then the time characteristics of the call mode are analyzed, and the statistical characteristics of data access at different time granularities are calculated, such as the average access frequency, standard deviation and peak distribution; then the efficiency index of the current time granularity division is evaluated, including the index hit rate, data block access balance and storage space utilization; then the division ratio of the first time granularity and the second time granularity is dynamically adjusted according to the efficiency index, usually when the index hit rate is lower than 80% The adjustment is triggered; finally, a progressive index reconstruction algorithm is applied to complete the update of the spatiotemporal index structure without interrupting the system operation. Among them, the calculation formula of the index hit rate is:

[0107] ;

[0108] Where, Indicates the index hit rate; Indicates the number of index hits; Indicates the total number of visits.

[0109] The calculation formula for data block access balance is:

[0110] ;

[0111] Where, Indicates access balance; represents the standard deviation of the access frequency of all data blocks; Indicates the average access frequency of all data blocks.

[0112] The adjustment formula for the time granularity division ratio is:

[0113] ;

[0114] Where, Indicates the adjusted time granularity division ratio; Indicates the time granularity division ratio before adjustment; It represents the adjustment coefficient, which is usually between 0.1 and 0.3; Indicates the change in efficiency, which is determined by the index hit rate, access balance, and storage space utilization.

[0115] This step uses adaptive index optimization technology to enable the index structure to adapt to changing data access patterns and improve the data access efficiency of the ocean forecast and disaster reduction model in different application scenarios.

[0116] The specific implementation of step S07 is to apply the ocean spatiotemporal feature predictor to optimize the computing resource allocation strategy. First, the trained ocean spatiotemporal feature predictor is activated, and the current ocean forecast scenario parameters and historical data call mode are input; then the ocean spatiotemporal feature predictor is used to predict the access probability distribution of each data call unit in the future time window, which is usually set to 10 minutes to 30 minutes; then a resource allocation optimization model is constructed, using the prediction results as input, and comprehensively considering the performance characteristics of the computing nodes and the network topology; then, distributed computing resource pooling management technology is applied to classify the computing nodes according to computing power and storage capacity to form a resource pool; finally, the computing task allocation and data storage strategy are dynamically adjusted according to the data demand characteristics and resource pool status under different forecast scenarios. Among them, the core structure of the ocean spatiotemporal feature predictor can be expressed as:

[0117] ;

[0118] Where, Represents the prediction result, that is, the access probability distribution of each data call unit in the future time window; Represents input features, including current ocean forecast scenario parameters and historical data call mode; represents the encoder function, which is implemented using a 3D spatiotemporal convolutional network; Represents the decoder function, which is implemented using a multi-head attention mechanism.

[0119] The encoder’s 3D spatiotemporal convolution operation can be expressed as:

[0120] ;

[0121] Where, Indicates the Feature maps of layers; Indicates the Tier Feature maps of channels; Indicates the Tier The convolution kernel of channels; Indicates the The bias term of the layer; Represents a three-dimensional convolution operation; represents the activation function, usually the rectified linear unit function; Indicates the The number of channels of the layer.

[0122] The multi-head attention mechanism in the decoder can be expressed as:

[0123] ;

[0124] Where, represents the multi-head attention function; represents the query matrix; represents the bond matrix; represents the value matrix; Indicates the The output of an attention head; Represents the output projection matrix; Represents a matrix concatenation operation.

[0125] The calculation formula for each attention head is:

[0126] ;

[0127] Where, 、 、 Respectively represent The query, key, and value projection matrices of each attention head.

[0128] The calculation formula of the attention function is:

[0129] ;

[0130] Where, represents the dimension of the key vector; Represents the softmax normalization function.

[0131] The objective function of the resource allocation optimization model can be expressed as:

[0132] ;

[0133] ;

[0134] Where, Represents the collection of all data call units; Indicates the calling unit The probability of access; Indicates allocation to the calling unit the amount of resources; Indicates the calling unit In terms of resources Execution efficiency under Indicates the total available resources.

[0135] This step uses prediction-driven resource optimization technology, with the aim of making resource allocation decisions in advance by predicting future data access patterns, reducing resource competition and waiting time, and improving the computational efficiency and forecast accuracy of the large-scale ocean forecasting and disaster reduction model.

[0136] The detailed structure of the ocean spatiotemporal feature predictor includes three core modules: spatiotemporal feature extraction module, call pattern recognition module and prediction optimization module. The spatiotemporal feature extraction module adopts a three-dimensional spatiotemporal convolutional network with a specific structure of four layers of three-dimensional convolution layers. The number of convolution kernels in each layer is 32, 64, 128 and 256 respectively. The convolution kernel size is , with a stride of 1 and a padding of 1. Each convolution layer is followed by a batch normalization layer and a rectified linear unit activation function. Finally, the feature map is reduced to a fixed size through an adaptive average pooling layer. The call pattern recognition module is based on a multi-head attention mechanism. The number of heads is determined by the number of high-frequency call units, usually 4 to 16. Each attention head consists of a query matrix, a key matrix, and a value matrix. The matrix dimension is 64 or 128. The attention weight is calculated by scaling the dot product, and the attention score is normalized using the softmax function. The prediction optimization module contains two fully connected layers. The number of neurons in the first layer is 1024, and the rectified linear unit activation function is used. The number of neurons in the second layer is equal to the total number of data call units in the prediction time window. The sigmoid activation function is used to output the access probability of each call unit.

[0137] The mathematical expression of the gating weight function in the ocean spatiotemporal feature predictor is:

[0138] ;

[0139] Where, Indicates the The gating weights of the attention heads; Indicates the The original weights of the attention heads; Represents the multi-objective balance value; Represents the total number of attention heads.

[0140] The calculation formula of the multi-objective balance value is:

[0141] ;

[0142] Where, Indicates the ocean forecast accuracy requirement, with a value range of [0, 1]; Indicates the degree of computing resource constraints, with a value range of [0, 1]; Indicates the real-time requirement of data call, and the value range is [0, 1]; Indicates the computational complexity of model reasoning, with a value range of [0, 1]; 、 、 and is the weight coefficient, and .

[0143] Optional, residual connection / skip connection mechanism formula in the ocean spatiotemporal feature predictor:

[0144] ;

[0145] Where, Indicates the Feature maps of layers; Indicates the Feature maps of layers; Represents a convolution operation, which includes a combination of convolution, batch normalization, and activation functions. This formula describes the basic structure of the residual block, which enables the efficient transfer of spatiotemporal information in deep networks through identity mapping.

[0146] Optional, precision-enhanced weight adjustment function in the gated weight function:

[0147] ;

[0148] Where, Indicates the first Attention head weights; represents the original weight; It represents the enhancement factor, which is usually between 0.2 and 0.5; Indicates the An indicator of the degree to which each attention head focuses on historical data patterns, with a value range of [0, 1]. This function is used when the multi-objective balance value falls within the high-precision priority range. Its purpose is to increase the weight of attention to historical data patterns and enhance the ocean spatiotemporal feature predictor's ability to perceive changes in the ocean environment.

[0149] Optional, computationally accelerated weight adjustment function in the gated weight function:

[0150] ;

[0151] Where, Indicates the calculation after acceleration Attention head weights; represents the original weight; It represents the acceleration factor, which is usually between 0.2 and 0.5; Indicates the The attention head's focus on local spatiotemporal features, with a value range of [0, 1]. This function is used when the multi-objective balance value is in the efficiency priority range. The purpose is to increase the weight of focusing on local spatiotemporal features and reduce the processing of global information to improve inference speed.

[0152] Optional, adaptive balancing weight adjustment function in the gated weight function:

[0153] ;

[0154] Where, Represents the first Attention head weights; represents the original weight; Indicates the equalization coefficient, usually ranging from 0.1 to 0.3; Indicates the The current accuracy requirement of the attention head and real-time requirements The balance index under , with a value range of [-1, 1]. This function is used when the multi-objective balance value belongs to the balance interval. The purpose is to dynamically adjust the importance distribution of each attention head to achieve a dynamic balance between accuracy and efficiency.

[0155] in, The calculation formula is:

[0156] ;

[0157] Where, is the balance factor, with a value range of [0, 1], which is dynamically adjusted according to the current system state; when When the value is higher, Will tend to increase the weight of historical data; when When the value is high, It tends to increase the weight of focusing on local features.

[0158] Optional, interval judgment conditions for multi-objective balance values:

[0159] when When , it belongs to the high-precision priority interval;

[0160] when When , it belongs to the efficiency priority range;

[0161] when When , it belongs to the equilibrium range;

[0162] Where, Indicates the threshold of the high-precision priority interval, usually set to 0.7; Indicates the threshold of the efficiency priority interval, usually set to 0.3; Represents the multi-objective balance value.

[0163] The following provides a general and simple embodiment 2 for implementing the numerical control call of the present invention, which is used to better understand the database call of the present invention, especially the steps of performing combined calls on multiple different databases including structured and unstructured ones. The databases in this embodiment include structured and unstructured ocean observation and forecast data in databases such as Xinchuang Database, SQL, Oracl, and Mango Database. The present invention realizes data interaction and communication between the large model and the early warning database through API requests. The database call capability is encapsulated into a service using the API data interface, and the business process of question input, query element extraction, request sending, data reading, response processing and information output is constructed to form an early warning data query tool that can be flexibly embedded in the intelligent body workflow as a data query node.

[0164] The following are the specific implementation steps of the early warning data query tool:

[0165] First, API interface encapsulation

[0166] This invention enables data interaction and communication between the large model and the early warning database through API requests. By encapsulating database call capabilities into services using the API data interface, the system constructs a business process for question input, query element extraction, request sending, data reading, response processing, and information output, forming an early warning data query tool that can be flexibly embedded into the intelligent agent workflow as a data query node.

[0167] Specific implementation steps of the early warning data query tool:

[0168] Step 1: Build a pre-alarm API data interface, clarify the API endpoint, request method, request parameters, and return value, and implement the data query business logic based on the database table structure information.

[0169] Step 2: Identify keywords in the conversation, including query elements such as forecast area, forecast time, and forecast elements, and construct API request parameters;

[0170] Step 3: The data interface is executed, a query request is sent, and the background program calls the early warning database and returns the query results;

[0171] Step 4: Process the response, parse the returned data, and extract the early warning information.

[0172] Second, obtain database query conditions

[0173] Based on the input sentence, a large model is used to infer and extract the query conditions, and the query conditions are organized into JSON format for easy interface query.

[0174] Step 1: Natural language understanding and intent recognition. Use a large language model based on the Transformer architecture (such as ERNIE and GPT) to perform semantic analysis on user input sentences, identify key entities, actions, and modifiers through the attention mechanism, and use the intent classification model to determine the query type (such as forecast, consultation, etc.).

[0175] Step 2: Query condition extraction and structuring. Apply named entity recognition technology to extract key parameters. For fuzzy queries, use query expansion technology to supplement implicit conditions and establish domain-specific condition mapping rules (such as time → timestamp, location → geocode).

[0176] Step 3: JSON standardization and dynamic construction of JSON schema:

[0177] {

[0178] "intent": "search|consult|generate",

[0179] "entities": [

[0180] {

[0181] "type": "object|time|location",

[0182] "value": "concrete_value",

[0183] "confidence": 0.95

[0184] }

[0185] ],

[0186] "constraints": {

[0187] "temporal": {"start": "2025-01-01", "end": "2025-05-19"},

[0188] "spatial": {"radius": 5000, "unit": "m"}

[0189] }

[0190] }

[0191] Third, get query data

[0192] Use Python code to modify the format and organize the input parameters of the interface file. Use HTTP requests to query the required data through the interface. Initiate HTTP requests (GET / POST methods) to call the target API interface, parse the interface documentation file using the JSON module, normalize input parameters using a dictionary data structure (including required field verification, parameter type conversion, and default value filling), then construct the request header and body in accordance with RESTful specifications, and finally process the response data (status code check, JSON deserialization, and exception handling).

[0193] import json

[0194] def main(arg1: str) -> dict:

[0195] # Parse the JSON string into a dictionary

[0196] # Parse the JSON string into a dictionary

[0197] parsed = json.loads(arg1)

[0198] return {

[0199] "area": ​​parsed.get("area"),

[0200] "beginDate": parsed.get("beginDate"),

[0201] "element": parsed.get("element"),

[0202] }

[0203] Fourth, parse the query results

[0204] The acquired data is parsed by time, location, and parameters. When performing structured parsing on the acquired raw data, the timestamp field is converted to a standard datetime object (supporting UTC time zone conversion). A spatial index is then constructed for the geographic location information based on the administrative region code table. Finally, type verification and normalization are performed on multidimensional parameters (such as sea temperature, waves, and tides), ultimately outputting a structured dataset that meets the analysis requirements.

[0205] Fifth, algorithm encapsulation

[0206] Encapsulate database query algorithms within large models, establish an efficient, secure, and semantically comprehensible interaction layer, and focus on enabling natural language to database queries. Implement OpenAI-compatible protocol interfaces, standardize input and output formats to adapt to third-party systems, and use gRPC service-oriented encapsulation to support highly concurrent remote calls.

[0207] from langchain.tools import tool

[0208] def market_analysis(keyword: str) -> dic # Call the data API and process the results

[0209] return processed_data # Return structured data: ml-citation{ref="5,8" data="citationList"}

[0210] from langchain.agents import initialize_agent

[0211] tools = [market_analysis, data_visualization] # Load custom toolset

[0212] agent = initialize_agent(tools, llm, agent_type="structured-chat-react") # :ml-citation{ref="5,8" data="citationList"}

[0213] from fastapi import FastAPI

[0214] app = FastAPI()

[0215] async def openai_proxy(request: OpenAIRequest):

[0216] return await agent_execute(request) # Native compatibility

[0217] After the above processing, a complete database analysis result can be obtained and conveniently encapsulated, making it easy to unify data records from multiple databases into a single data set for unified call. This solves the problem of large models understanding and calling ocean data from different databases, such as structured and unstructured ones.

[0218] To better understand and implement the present invention, Example 3 of a specific application scenario of the present invention is provided below: A database call method based on a large-scale ocean forecast and disaster reduction model was implemented in a storm surge forecast scenario at a certain marine agency. This agency is responsible for monitoring and forecasting storm surges in a certain sea area and needs to process large amounts of ocean observation data and perform high-precision forecast calculations. Researchers used this method to build an ocean forecast and disaster reduction system, which was used to conduct a six-month storm surge forecast experiment. The system is based on a 32-node heterogeneous computing cluster, each equipped with two NVIDIA A100 GPUs (80GB of video memory) and 512GB of CPU memory. The system manages a total of 240TB of raw ocean data, including ocean hydrological data, meteorological observation data, and historical storm surge path data.

[0219] First, according to step S01, the researchers constructed a two-layer temporal granularity index structure. The first temporal granularity was set to 2 minutes for accessing high-frequency dynamics model data; the second temporal granularity was set to 60 minutes for environmental factor forecasting. Spatially, an octree structure was used, dividing the responsible area into 8 main regions, each of which was further subdivided into 64 subregions. The depth-wise index structure was divided into 12 layers. The specific configuration of the spatiotemporal index is shown in Table 1.

[0220] Table 1 Spatiotemporal index configuration table

[0221]

[0222] Next, according to step S02, researchers analyzed historical data call logs and counted the access frequency of each data block. Setting the call frequency threshold to 8 times per second, the data was divided into high-frequency call units and low-frequency call units. High-frequency call units primarily focus on surface and subsurface data in the storm surge-affected area, accounting for approximately 15% of the total data volume. The results of this call unit division are shown in Table 2.

[0223] Table 2 Data call unit division table

[0224]

[0225] For high-frequency call units, a GPU memory data acceleration matrix was constructed according to step S03. Analysis using a collaborative filtering algorithm revealed that approximately 62% of the data in these high-frequency call units was sparse. A coordinate format storage method was used to construct a sparse matrix for read-only data, achieving a compression rate of 78%. For read-write data, a wavelet transform compression algorithm was used, achieving an average compression rate of 65%. This configuration allows all high-frequency data to be stored entirely in GPU memory, avoiding frequent CPU-GPU data transfers.

[0226] For the low-frequency and medium-frequency call units, a CPU memory data storage matrix was constructed according to step S04. The L1 cache was configured to account for 8% of the total memory, storing medium-frequency data accessed at least three times per minute. The L2 cache was configured to account for 22% of the total memory, storing medium-frequency data accessed between one and three times per minute. The remaining low-frequency data was stored in the main memory area. The configuration and performance data for each cache level are shown in Table 3.

[0227] Table 3 Multi-level cache configuration and performance

[0228]

[0229] Based on step S05, the researchers designed a heterogeneous parallel scheduling algorithm to achieve adaptive data migration between GPU memory and CPU memory. The weights of the five metrics in the data migration optimization function were set to: call frequency 0.35, data timeliness 0.25, storage capacity pressure 0.15, computational load balancing 0.15, and data dependency 0.10. Experiments determined that the trigger thresholds for migration to the GPU were 0.72, and the threshold for migration to the CPU was 0.28. During the experiment, the system performed an average of 12.3 data migration operations per minute, with an average data size of 256MB per migration.

[0230] To adapt to changing data access patterns, researchers constructed a spatiotemporal index self-optimization mechanism in step S06. Initially, the index hit rate was 78.6%. By dynamically adjusting the time granularity, the index hit rate increased to 92.4% by the end of the experiment. Data block access balance improved from 0.73 to 0.88, and storage space utilization increased from 82% to 94%.

[0231] Finally, according to step S07, the researchers trained and applied an ocean spatiotemporal feature predictor, which uses 12 attention heads corresponding to 12 major data call patterns. A training set was constructed by analyzing 82,000 data call logs from historical forecasting tasks. The predictor achieved an accuracy of 89.7% on the validation set. The data call probability distribution output by the predictor guides computing resource allocation, significantly reducing resource contention and waiting time.

[0232] Through a six-month practical application comparison, this method has improved in many performance indicators compared with the traditional ocean data calling method based on fixed cache levels, as shown in Table 4.

[0233] Table 4 Performance comparison table

[0234]

[0235] Traditional ocean forecast data calling methods usually use a fixed cache hierarchy and a predefined data allocation strategy, which cannot be dynamically adjusted according to changes in data access patterns, resulting in low resource utilization, high data access latency, and limited computing efficiency. The present invention, by constructing a two-layer time-granularity index structure, a high-frequency data GPU acceleration matrix, a heterogeneous parallel scheduling algorithm, and a resource optimization technology based on an ocean spatiotemporal feature predictor, achieves adaptive optimization of data storage and access, thereby improving the overall performance of the system. Compared with traditional methods, the present invention has significantly improved in data access latency, computing throughput, storage space utilization, and forecast accuracy, with an average improvement of approximately 17.9%, providing accurate data for storm surge warning and disaster prevention and mitigation.

[0236] It should be noted that the variables involved in the present invention are explained in detail as shown in Tables 5, 6 and 7 below.

[0237] Table 5 Variable Explanation Table (Part I)

[0238]

[0239] Table 6 Variable Explanation Table (Part II)

[0240]

[0241] Table 7 Variable Explanation Table (Part 3)

[0242]

[0243] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. A database calling method based on a large ocean forecast and disaster reduction model, characterized in that: include: Construct a two-layer time granularity index structure, organize the ocean forecast data according to the first time granularity and the second time granularity, and form a spatiotemporal four-dimensional index matrix; According to the spatiotemporal characteristics and call frequency of the data required by the large-scale marine forecast and disaster reduction model, the data is divided into high-frequency call units and low-frequency call units; Build a GPU memory data acceleration matrix for high-frequency call units; build a CPU memory data storage matrix for low-frequency call units; design a heterogeneous parallel scheduling algorithm and implement adaptive data migration between GPU memory and CPU memory based on data migration optimization functions; Construct a spatiotemporal index self-optimization mechanism to dynamically adjust the division ratio of the first time granularity and the second time granularity according to the model call pattern; apply the ocean spatiotemporal feature predictor to predict the data call pattern of the ocean forecast and disaster reduction large model, and combine it with distributed computing resource pooling management to optimize the computing resource allocation strategy based on the data demand characteristics under different forecast scenarios.

2. The method for calling a database based on a large ocean forecast and disaster reduction model according to claim 1, characterized in that: The first time granularity refers to the basic index unit of ocean forecast data in the time dimension, and the second time granularity refers to a time unit larger than the first time granularity that is aggregated on the basis of the first time granularity.

3. The method for calling a database based on a large ocean forecast and disaster reduction model according to claim 2, characterized in that: The calling frequency refers to the number of times the ocean forecast and disaster reduction model accesses data in a unit of time. The data is divided into high-frequency data and low-frequency data according to the calling frequency.

4. The method for calling a database based on a large ocean forecast and disaster reduction model according to claim 3, characterized in that: The calling unit refers to a data set with similar spatiotemporal characteristics and access patterns, which serves as the basic unit for data scheduling and storage optimization.

5. The method for calling a database based on a large ocean forecast and disaster reduction model according to claim 4, characterized in that: The GPU display memory data acceleration matrix includes a read-only data sparse matrix and a read-write data compression matrix.

6. The method for calling a database based on a large ocean forecast and disaster reduction model according to claim 5, characterized in that: The read-only data sparse matrix refers to the ocean static parameters and historical data stored in the GPU video memory that do not need to be modified, and is represented by a sparse matrix to reduce storage space occupancy; the read-write data compression matrix refers to the ocean dynamic parameters and forecast intermediate results stored in the GPU video memory that need to be frequently updated, and a real-time compression algorithm is used to reduce data transmission overhead.

7. The method for calling a database based on a large ocean forecast and disaster reduction model according to claim 6, characterized in that: The heterogeneous parallel scheduling algorithm refers to an algorithm that uses the computing characteristics of GPU processors and CPU processors to perform parallel processing on ocean data with different characteristics, including a task decomposition module, a load balancing module and a resource management module.

8. The method for calling a database based on a large ocean forecast and disaster reduction model according to claim 7, characterized in that: The input of the data migration optimization function includes a data call frequency index, a data timeliness index, a storage capacity pressure index, a computing load balancing index, and a data dependency index.

9. The method for calling a database based on a large ocean forecast and disaster reduction model according to claim 8, characterized in that: The ocean spatiotemporal feature predictor is a multi-head attention-driven ocean data call pattern prediction model. The number of attention heads in the ocean spatiotemporal feature predictor needs to be dynamically determined based on the first time granularity, the second time granularity and the number of high-frequency call units.

10. The method for calling a database based on a large ocean forecast and disaster reduction model according to claim 9, characterized in that: The method also includes the step of making combined calls to a plurality of different databases including structured and unstructured ones.

Images