Database calling method based on ocean forecast disaster reduction large model

Abstract

The invention provides a database calling method based on a large ocean forecast disaster reduction model, and belongs to the technical field of ocean forecast. A double-layer time granularity index structure is constructed, data is divided into a high-frequency calling unit and a low-frequency calling unit according to calling frequency, and a GPU video memory acceleration matrix and a CPU memory storage matrix are constructed respectively; a heterogeneous parallel scheduling algorithm is designed to realize data migration based on multi-dimensional index optimization, time granularity division is dynamically adjusted by applying a spatio-temporal index self-optimization mechanism, a data calling mode is predicted through a multi-head attention-driven marine spatio-temporal feature predictor, and the accuracy and efficiency requirements are dynamically balanced in combination with a gating weighting function. And finally, the data calling efficiency of the ocean forecast disaster reduction large model is remarkably improved, so that the system can efficiently meet the data requirements of high-precision, large-range and multi-aging forecast scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of ocean forecasting, and more particularly, relates to a database calling method based on a large ocean forecasting and disaster reduction model. Background Art

[0002] The large ocean forecasting and disaster reduction model is a complex computing system that uses massive multi-source heterogeneous data for ocean environment forecasting and disaster warning. Traditional ocean forecasting systems usually adopt a single storage architecture, storing massive forecasting data uniformly on disks or in memory, and organizing the data through a simple spatio-temporal indexing method, and calling the data as needed during the model operation. This method can meet the basic needs in small-scale forecasting scenarios, but with the continuous improvement of forecasting accuracy and coverage, the data scale and computational complexity increase sharply. There are many defects in the existing ocean forecasting database calling methods: First, the single storage hierarchy architecture cannot adapt to the data calling requirements of different frequencies, resulting in an increase in the system load due to repeated calls of high-frequency data; Second, the lack of an optimized index structure for the spatio-temporal characteristics of ocean data makes the data retrieval efficiency low; Third, the heterogeneous computing resources of GPUs and CPUs are not effectively utilized, resulting in a waste of computing power; Finally, the static data management strategy cannot dynamically adjust the data organization method according to the forecasting model, making it difficult to adapt to complex and changing forecasting scenarios. Facing the increasing ocean forecasting requirements, the existing technologies are difficult to solve the technical problem of low data calling efficiency in the large ocean forecasting and disaster reduction model. Especially in high-precision, large-scale, and multi-temporal forecasting scenarios, data calling has become a bottleneck restricting the overall performance of the system, and there is an urgent need to design an efficient database calling method. Summary of the Invention

[0003] In view of this, the present invention provides a database calling method based on a large ocean forecasting and disaster reduction model, which can solve the technical problem of low data calling efficiency in the large ocean forecasting and disaster reduction model existing in the prior art.

[0004] The present invention is implemented as follows: The present invention provides a database calling method based on a large ocean forecasting and disaster reduction model, including: constructing a two-layer time granularity index structure, organizing ocean forecasting data according to a first time granularity and a second time granularity to form a spatio-temporal four-dimensional index matrix; dividing the data into high-frequency calling units and low-frequency calling units according to the spatio-temporal characteristics and calling frequencies of the data required by the ocean forecasting model; constructing a GPU video memory data acceleration matrix for the high-frequency calling units; constructing a CPU memory data storage matrix for the low-frequency calling units; designing a heterogeneous parallel scheduling algorithm to achieve adaptive data migration between the GPU video memory and the CPU memory based on a data migration optimization function; constructing a spatio-temporal index self-optimization mechanism to dynamically adjust the division ratio of the first time granularity and the second time granularity according to the model calling pattern; applying an ocean spatio-temporal feature predictor to predict the data calling pattern of the forecasting model, and combining with distributed computing resource pooling management to optimize the computing resource allocation strategy for the data demand characteristics under different forecasting scenarios.

[0005] Among them, the first time granularity refers to the basic index unit of ocean forecasting data in the time dimension, and the second time granularity refers to a time unit larger than the first time granularity aggregated on the basis of the first time granularity.

[0006] Among them, the calling frequency refers to the number of times the large ocean forecasting and disaster reduction model accesses data per unit time, and the data is divided into high-frequency data and low-frequency data according to the calling frequency.

[0007] Among them, the calling unit refers to a data set with similar spatio-temporal characteristics and access patterns, which is used as the basic unit for data scheduling and storage optimization.

[0008] Among them, the GPU video memory data acceleration matrix includes a read-only data sparse matrix and a read-write data compression matrix.

[0009] Among them, the read-only data sparse matrix refers to the ocean static parameters and historical data stored in the GPU video memory that do not need to be modified, and is represented by a sparse matrix to reduce the storage space occupation.

[0010] Among them, the read-write data compression matrix refers to the ocean dynamic parameters and forecast intermediate results stored in the GPU video memory that need to be updated frequently, and a real-time compression algorithm is used to reduce the data transmission overhead.

[0011] Among them, the heterogeneous parallel scheduling algorithm refers to an algorithm that simultaneously utilizes the computing characteristics of the GPU processor and the CPU processor to perform parallel processing on ocean data with different characteristics, including a task decomposition module, a load balancing module, and a resource management module.

[0012] Among them, the inputs of the data migration optimization function include data call frequency metrics, data timeliness metrics, storage capacity pressure metrics, computing load balancing metrics, and data dependency relationship metrics.

[0013] Among them, the ocean spatio-temporal feature predictor is a prediction model for ocean data call patterns driven by multi-head attention. The number of attention heads in the ocean spatio-temporal feature predictor needs to be dynamically determined according to the first time granularity, the second time granularity, and the number of high-frequency call units.

[0014] Among them, the database includes structured and unstructured ocean observation and forecasting data in databases such as domestic replacement databases, SQL, Oracl, Mango Database, etc. It also includes the step of making combined calls to multiple different databases including structured and unstructured ones.

[0015] Through innovative technologies such as constructing a two-layer time granularity index structure, hierarchical storage of data frequencies, heterogeneous parallel scheduling, and an ocean spatio-temporal feature predictor, the present invention comprehensively improves the data call efficiency of the ocean forecasting and disaster reduction large model. This method solves the key defects in traditional technologies: the two-layer time granularity index structure realizes the efficient organization and retrieval of data at different time scales; the hierarchical storage architecture with high-frequency data stored in the GPU video memory acceleration matrix and low-frequency data stored in the CPU memory matrix effectively utilizes heterogeneous computing resources; the heterogeneous parallel scheduling algorithm and the data migration optimization function realize the intelligent scheduling of data between different storage levels, avoiding data call bottlenecks; the ocean spatio-temporal feature predictor can predict future data call patterns, realizing data preloading and optimized resource allocation. Through the above innovative design, the present invention effectively solves the technical problem of low data call efficiency of the ocean forecasting and disaster reduction large model and significantly improves the system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flowchart of the method of the present invention.

[0017] Figure 2 is a structural diagram of the ocean spatio-temporal feature predictor involved in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0019] As Figure 1 shown, it is a flowchart of a database call method based on an ocean forecasting and disaster reduction large model provided by the present invention. This method includes the following steps: S01. Construct a two - layer time - granularity index structure, organize the ocean forecast data according to the first time granularity and the second time granularity, and form a four - dimensional spatio - temporal index matrix; S02. According to the spatio - temporal characteristics and call frequencies of the data required by the ocean forecast model, divide the data into high - frequency call units and low - frequency call units; S03. Construct a GPU video - memory data acceleration matrix for high - frequency call units, including a read - only data sparse matrix and a read - write data compression matrix; S04. Construct a CPU memory data storage matrix for low - frequency call units and manage it using a hierarchical cache structure; S05. Design a heterogeneous parallel scheduling algorithm, and realize the adaptive data migration between the GPU video - memory and the CPU memory based on a data migration optimization function. The inputs of the data migration optimization function include data call frequency metrics, data timeliness metrics, storage capacity pressure metrics, computational load balancing metrics, and data dependency relationship metrics; S06. Construct a spatio - temporal index self - optimization mechanism to dynamically adjust the division ratio of the first time granularity and the second time granularity according to the model call pattern; S07. Apply an ocean spatio - temporal feature predictor to predict the data call pattern of the forecast model, and combine with the distributed computing resource pooling management to optimize the computing resource allocation strategy according to the data demand characteristics under different forecast scenarios.

[0020] Among them, the first time granularity refers to the basic index unit of the ocean forecast data in the time dimension, usually at the minute level, and is used to support the data call requirements of high - frequency ocean dynamics models.

[0021] Among them, the second time granularity refers to a larger time unit aggregated on the basis of the first time granularity, usually at the hour level, and is used to support the data call requirements of medium - and low - frequency ocean environmental element forecasts.

[0022] Among them, the call frequency refers to the number of times the ocean forecast disaster reduction large model accesses data within a unit time. The data is divided into high - frequency data and low - frequency data according to the call frequency.

[0023] Among them, the call unit refers to a data set with similar spatio - temporal characteristics and access patterns, and serves as the basic unit for data scheduling and storage optimization.

[0024] Among them, the data acceleration matrix refers to a high - frequency call data structure organized in the GPU video - memory, and uses sparse storage and compression algorithms to optimize the storage space utilization rate.

[0025] Among them, the read - only data sparse matrix refers to the ocean static parameters and historical data stored in the GPU video - memory that do not need to be modified, and is represented by a sparse matrix to reduce the storage space occupation.

[0026] Among them, the read-write data compression matrix refers to the ocean dynamic parameters and forecast intermediate results stored in the GPU video memory that need to be frequently updated, and a real-time compression algorithm is adopted to reduce the data transmission overhead.

[0027] Among them, the heterogeneous parallel scheduling algorithm refers to an algorithm that simultaneously utilizes the computing characteristics of the GPU processor and the CPU processor to perform parallel processing on ocean data with different characteristics, including a task decomposition module, a load balancing module, and a resource management module, and maximizes the utilization of computing resources by dynamically adjusting the allocation ratio of computing tasks.

[0028] Among them, the data migration optimization function is used to evaluate the necessity and optimal timing of data migration between the GPU video memory and the CPU memory. The data call frequency index is obtained from step S02, the data timeliness index is determined by the update cycle of the ocean forecast data, the storage capacity pressure index is calculated from the remaining space of the GPU video memory and the CPU memory, the computing load balancing index is provided by the load balancing module in the heterogeneous parallel scheduling algorithm, the data dependency relationship index is obtained by analyzing the dependency graph between data call units, and the output of the data migration optimization function includes the data migration priority score and the migration timing judgment result, which are used for the data adaptive migration decision between the GPU video memory and the CPU memory in step S05.

[0029] The ocean spatio-temporal feature predictor is a multi-head attention-driven ocean data call pattern prediction model, and the number of attention heads in the ocean spatio-temporal feature predictor needs to be dynamically determined according to the first time granularity, the second time granularity, and the number of high-frequency call units.

[0030] The specific structure of the ocean spatio-temporal feature predictor is a bidirectional architecture including an encoder and a decoder. The encoder part uses a three-dimensional spatio-temporal convolutional network to extract the spatio-temporal features of the ocean forecast data, and the decoder part uses a multi-head attention mechanism to fuse the spatio-temporal features of different scales. The number of attention heads is determined by the number of high-frequency call units, and each attention head is responsible for capturing the spatio-temporal correlation of a class of call patterns. The core of the ocean spatio-temporal feature predictor includes three parts: a spatio-temporal feature extraction module, a call pattern recognition module, and a prediction optimization module. The ocean spatio-temporal feature predictor also introduces a skip connection mechanism to ensure the effective transmission of spatio-temporal information in the deep network. The output layer of the ocean spatio-temporal feature predictor represents the access probability of each data call unit within the future time window in the form of a probability distribution.

[0031] The steps for establishing the training dataset of the ocean spatio-temporal feature predictor specifically include collecting data call logs from historical ocean forecasting tasks, extracting the call time series and spatial distribution features of each data unit during each forecasting process, generating a call frequency heat map and a time series change curve, annotating high-frequency call areas and low-frequency call areas, constructing paired samples containing input features and labels, where the input features are the data call patterns within the historical time window, and the labels are the data call probability distributions within the future time window, constructing training samples through a sliding window method, and performing data augmentation processing to improve the generalization ability of the model.

[0032] The steps for training the ocean spatio-temporal feature predictor specifically include first pre-training the ocean spatio-temporal feature predictor, using self-supervised learning methods to let the ocean spatio-temporal feature predictor learn the basic spatio-temporal patterns of ocean data calls, and then performing supervised fine-tuning using labeled data, using the cross-entropy loss function to evaluate the difference between the prediction result and the true call pattern, using an adaptive learning rate optimizer for parameter update, introducing an early stopping mechanism to prevent overfitting, setting a validation set for different forecasting scenarios to evaluate the performance of the ocean spatio-temporal feature predictor, and finally saving the model parameters that perform best on the validation set.

[0033] The gating weight function is used to adjust the weight allocation of the multi-head attention mechanism in the ocean spatio-temporal feature predictor. The gating weight function is calculated based on four data: the ocean forecasting accuracy requirement, the computing resource constraint, the real-time requirement of data calls, and the model inference efficiency, to obtain a multi-objective balance value. When the multi-objective balance value belongs to the high-precision priority interval, an accuracy-enhanced weight adjustment function is used to increase the weight of focusing on historical data patterns, strengthening the ocean spatio-temporal feature predictor's perception ability of ocean environmental changes; when the multi-objective balance value belongs to the efficiency priority interval, a computing acceleration type weight adjustment function is used to increase the weight of focusing on local spatio-temporal features, reducing the processing of global information to improve the inference speed; when the multi-objective balance value belongs to the balance interval, an adaptive equilibrium type weight adjustment function is used to dynamically adjust the importance distribution of each attention head to achieve a dynamic balance between accuracy and efficiency.

[0034] The ocean forecasting accuracy requirement is determined by the application scenario of the ocean forecasting and disaster reduction large model. The computing resource constraint is jointly restricted by the GPU video memory capacity and the CPU memory capacity. The real-time requirement of data calls is determined by the forecasting timeliness requirement of the ocean forecasting and disaster reduction large model. The model inference efficiency is determined by the computational complexity of the ocean spatio-temporal feature predictor.

[0035] The precision-enhanced weight adjustment function assigns more attention weights to the attention heads related to historical data patterns. The computation-accelerated weight adjustment function assigns more attention weights to the attention heads related to local spatio-temporal features. The adaptive balance weight adjustment function dynamically adjusts the weight ratios of each attention head according to real-time forecasting requirements.

[0036] The following describes the specific implementation manners of the above steps in detail.

[0037] The specific implementation manner of step S01 is to construct a two-layer time granularity index structure. First, determine the size of the first time granularity, usually from 1 minute to 5 minutes, as the basic time unit for accessing high-frequency data of the ocean dynamics model. Then, determine the size of the second time granularity, usually from 30 minutes to 120 minutes, as the time unit for environmental element forecasting. Next, organize the ocean forecasting data in the time dimension according to these two granularities to form a two-layer index in the time dimension. After that, use an octree structure to divide the three-dimensional ocean space in the space dimension, and construct a space index for the ocean space according to the three dimensions of longitude, latitude, and depth. Finally, combine the two-layer time index with the three-dimensional space index to construct a four-dimensional spatio-temporal index matrix, where each element in the matrix corresponds to a spatio-temporal data block. This step adopts the multi-resolution spatio-temporal indexing technology, aiming to achieve the efficient organization and rapid retrieval of data at different time scales and space scales, and improve the data access efficiency of the ocean forecasting and disaster reduction large model.

[0038] The specific implementation manner of step S02 is to perform data partitioning according to the spatio-temporal characteristics and call frequencies of the data required to be called by the ocean forecasting model. First, perform statistical analysis on the historical data call logs to calculate the average access frequency of each spatio-temporal data block. Then, set a call frequency threshold, usually 10 calls per second, and classify the data blocks with a call frequency higher than this threshold as high-frequency call units. Next, classify the data blocks with a call frequency lower than the call frequency threshold as low-frequency call units. After that, analyze the spatial distribution characteristics of each call unit, and merge the data blocks with adjacent spatial positions and similar call patterns to form larger call units. Finally, determine the initial allocation strategy of each call unit in the GPU video memory or CPU memory according to its spatio-temporal characteristics and data volume. This step adopts the data access pattern analysis technology, aiming to identify the usage characteristics of different data, provide a basis for subsequent heterogeneous storage optimization, and improve the overall data access efficiency of the system.

[0039] The specific implementation of step S03 is to construct a GPU video memory data acceleration matrix for high-frequency call units. First, analyze the read and write characteristics of the data in the high-frequency call units and classify them into two categories: read-only data and read-write data. Then, conduct a sparsity analysis on the read-only data and use a collaborative filtering algorithm to identify redundant and sparse patterns in the data. Next, construct a read-only data sparse matrix and compress the sparse data using the coordinate format storage method, which can usually reduce the storage space by 60% to 80%. After that, conduct a feature analysis on the read-write data and select a compression algorithm suitable for the characteristics of ocean data, such as wavelet transform or principal component analysis. Finally, construct a read-write data compression matrix to achieve high-compression ratio storage while ensuring data accuracy and reduce the data transmission overhead. This step uses data sparse representation and compressed storage technologies with the aim of maximizing the utilization of the limited GPU video memory space, storing more frequently accessed data, reducing the number of data transmissions between the CPU and the GPU, and improving the computing efficiency.

[0040] The specific implementation of step S04 is to construct a CPU memory data storage matrix for low-frequency call units. First, further divide the low-frequency call units into medium-frequency data and low-frequency data according to the access frequency. Usually, the access threshold for medium-frequency data is 1 to 10 times per minute. Then, design a multi-level cache structure, including L1 cache, L2 cache, and main storage area. The L1 cache is usually configured to be 5% to 10% of the total memory, and the L2 cache is configured to be 15% to 25% of the total memory. Next, establish a cache replacement policy based on the least recently used algorithm, give priority to putting medium-frequency data into the L1 cache, and put low-frequency data into the L2 cache or the main storage area. After that, construct a cache prefetch mechanism, based on the principle of temporal and spatial locality of data access, to predict and preload data blocks that may be accessed in advance. Finally, establish a cache coherence maintenance mechanism to ensure that the data copies in each level of cache are synchronized in a timely manner when the data is updated. This step uses hierarchical cache management technology with the aim of optimizing the storage and access efficiency of low-frequency data, reducing the memory access latency, and improving the system throughput.

[0041] The specific implementation of step S05 is to design a heterogeneous parallel scheduling algorithm to achieve adaptive data migration between the GPU and the CPU. First, a data migration optimization function is constructed, which comprehensively considers multiple metrics: data call frequency metric, data timeliness metric, storage capacity pressure metric, computing load balance metric, and data dependency relationship metric. Then, based on these five metrics, the data migration priority score is calculated using the weighted summation method, and the weights of each metric are dynamically adjusted according to the specific application scenario. Next, a migration trigger threshold is set. Usually, when the priority score is greater than 0.75, the migration to the GPU is triggered, and when the score is less than 0.25, the migration to the CPU is triggered. After that, a batch data migration mechanism is implemented to merge multiple small data blocks that need to be migrated into large data blocks for transmission, reducing the migration overhead. Finally, a migration log recording system is established to track the data migration history, which is used to optimize future migration decisions. This step adopts multi-objective optimization and heterogeneous computing technologies, aiming to achieve the efficient utilization of computing resources, dynamically adjust the data storage location according to real-time requirements, and improve the overall operation efficiency of the large ocean forecasting and disaster reduction model.

[0042] The specific implementation of step S06 is to construct a spatio-temporal index self-optimization mechanism. First, a model call pattern monitoring system is established to record the access situation of the large ocean forecasting and disaster reduction model to each spatio-temporal data block in real time. Then, the time characteristics of the call pattern are analyzed, and the statistical characteristics of data access at different time granularities are calculated, such as average access frequency, standard deviation, and peak distribution. Next, the efficiency metrics of the current time granularity division are evaluated, including index hit rate, data block access balance, and storage space utilization. After that, the division ratio of the first time granularity and the second time granularity is dynamically adjusted according to the efficiency metrics. Usually, when the index hit rate is lower than 80%, the adjustment is triggered. Finally, the progressive index reconstruction algorithm is applied to complete the update of the spatio-temporal index structure without interrupting the system operation. This step adopts adaptive index optimization technology, aiming to enable the index structure to adapt to the changing data access pattern and improve the data access efficiency of the large ocean forecasting and disaster reduction model in different application scenarios.

[0043] The specific implementation of step S07 is to optimize the computing resource allocation strategy using the ocean spatio-temporal feature predictor. First, activate the trained ocean spatio-temporal feature predictor and input the current ocean forecast scenario parameters and historical data call patterns. Then, use the ocean spatio-temporal feature predictor to predict the access probability distribution of each data call unit within the future time window, which is usually set from 10 minutes to 30 minutes. Next, construct a resource allocation optimization model, taking the prediction results as input and comprehensively considering the computing node performance characteristics and network topology. After that, apply the distributed computing resource pooling management technology to classify the computing nodes according to their computing power and storage capacity to form a resource pool. Finally, dynamically adjust the computing task allocation and data storage strategy according to the data demand characteristics and resource pool status under different forecast scenarios. This step adopts a prediction-driven resource optimization technology, aiming to make resource allocation decisions in advance through the prediction of future data access patterns, reduce resource competition and waiting time, and improve the computing efficiency and forecast accuracy of the ocean forecast and disaster reduction large model.

[0044] The detailed structure of the ocean spatio-temporal feature predictor includes three core modules: the spatio-temporal feature extraction module, the call pattern recognition module, and the prediction optimization module. The spatio-temporal feature extraction module uses a three-dimensional spatio-temporal convolutional network, and its specific structure is 4 layers of three-dimensional convolutional layers. The number of convolutional kernels in each layer is 32, 64, 128, and 256 respectively, the convolutional kernel size is , the stride is 1, the padding is 1. After each convolution, a batch normalization layer and a rectified linear unit activation function are connected. Finally, the feature map is reduced to a fixed size through an adaptive average pooling layer. The call pattern recognition module is based on the multi-head attention mechanism. The number of heads is determined according to the number of high-frequency call units, usually 4 to 16. Each attention head consists of a query matrix, a key matrix, and a value matrix. The matrix dimension is 64 or 128. The attention weights are calculated through scaled dot product, and the attention scores are normalized using the softmax function. The prediction optimization module contains two layers of fully connected layers. The number of neurons in the first layer is 1024, using the rectified linear unit activation function. The number of neurons in the second layer is equal to the total number of data call units within the prediction time window, using the sigmoid activation function to output the access probability of each call unit. To ensure the effective transmission of spatio-temporal information in the deep network, residual connections are introduced between the convolutional layers. Every two convolutional layers form a residual block, and identity mapping is used to directly transmit information. The final output layer represents the access probability of each data call unit within the future time window in the form of a probability distribution. The probability threshold is usually set to 0.6, and the call units with probabilities higher than this threshold will be given priority in resource allocation.

[0045] The detailed steps for establishing the training dataset of the ocean spatio-temporal feature predictor are as follows: First, collect the data call logs from historical ocean forecasting tasks. The logs contain information such as data block identifiers, call timestamps, call process identifiers, and access types. Usually, collect the log data for the most recent 3 months. Then, extract the call time series of each data unit during each forecasting process, and count the access times of each data unit within each time window. The length of the time window is usually 5 minutes. Next, analyze the spatial distribution characteristics of the data units, calculate the call correlation of spatially adjacent data units, and construct a spatial association network. After that, generate a call frequency heat map, use the shade of color to represent the access frequencies in different regions, and plot a time series change curve to show the evolution of the access pattern over time. Then, according to the set frequency threshold, label the high-frequency call regions and low-frequency call regions. The high-frequency regions are usually defined as the regions where the access frequency is higher than twice the average value. Next, construct paired samples containing input features and labels. The input features are the data call patterns within the historical time window. The length of the historical window is usually 30 minutes, and the label is the probability distribution of data calls within the future time window. The length of the future window is usually 10 minutes. After that, construct training samples through the sliding window method. The window sliding step size is 5 minutes, and a large number of sample pairs are extracted from the historical logs. Finally, perform data augmentation processing, including adding Gaussian noise, randomly masking some regions, and time scale transformation, etc., to enhance the sample diversity and improve the generalization ability of the model. Usually, expand the original sample quantity by 3 to 5 times. After the training samples are constructed, divide the dataset into a training set, a validation set, and a test set according to the ratio of 7:2:1 for the training and evaluation of the ocean spatio-temporal feature predictor.

[0046] It should be noted that the first key technical idea of the present invention is a two-layer time granularity index structure. This structure organizes ocean forecasting data according to the first time granularity at the minute level and the second time granularity at the hour level, forming a spatio-temporal four-dimensional index matrix. Different from the single time scale index adopted by traditional ocean forecasting systems, this two-layer structure can more accurately match the call characteristics of data at different time scales in the ocean forecasting model. In the traditional method, all data is indexed according to a unified time scale, resulting in problems such as low retrieval efficiency for high-frequency data or excessive storage resources occupied by low-frequency data. However, through the differential time granularity index of the present invention, both high-frequency dynamic model data and low-frequency environmental element forecasting data can be organized and retrieved at the most suitable time scale, thereby greatly improving the data retrieval efficiency and reducing the system resource consumption.

[0047] The second key technical idea is a hierarchical storage mechanism based on call frequency. This mechanism divides data into high-frequency call units and low-frequency call units, and constructs a GPU video memory data acceleration matrix and a CPU memory data storage matrix respectively. Traditional ocean forecasting systems usually store all data uniformly in the same type of storage medium, failing to make full use of the characteristics of heterogeneous computing resources. The present invention fully considers the characteristics of high computing throughput but limited storage capacity of the GPU and large-capacity storage but relatively low computing efficiency of the CPU. By placing high-frequency access data in the GPU video memory and storing low-frequency access data in the CPU memory, the optimal allocation of storage resources is achieved. This hierarchical storage strategy not only improves the access speed of high-frequency data, but also significantly reduces the system latency by reducing the data transmission overhead between the GPU and the CPU.

[0048] The third key technical idea is the ocean spatio-temporal feature predictor, which uses a multi-head attention-driven bidirectional architecture to predict data call patterns. Data scheduling in traditional systems often relies on historical statistics or simple rules and lacks the ability to prospectively predict future data access patterns. The ocean spatio-temporal feature predictor of the present invention extracts the spatio-temporal features of ocean data through a three-dimensional spatio-temporal convolutional network and uses the multi-head attention mechanism to capture the spatio-temporal correlations of different call patterns, and can accurately predict the access probability distribution of each data call unit within a future time window. This prediction ability enables the system to perform data scheduling and resource allocation in advance, avoiding the performance fluctuations and resource waste caused by traditional passive reactive scheduling.

[0049] The synergistic effect of these three key technical ideas has produced significant technical effects and advantages. The double-layer time granularity index structure provides a basis for the efficient organization and retrieval of data. The hierarchical storage mechanism realizes the optimal utilization of heterogeneous computing resources, while the ocean spatio-temporal feature predictor injects prospective intelligence into the entire system. The combination of the three forms a closed-loop optimization system: the predictor predicts future data call patterns, guiding the double-layer time granularity index structure to dynamically adjust the time granularity division ratio, and at the same time guiding the hierarchical storage mechanism for data preloading and migration. This synergistic mechanism enables the system to adaptively adjust the data organization and storage strategy according to the changes in the forecasting scenario, achieving the optimal allocation of computing resources and the maximization of data call efficiency. Compared with the static data management and passive scheduling mechanism of traditional systems, the present invention realizes the full-chain optimization of the data call process, fundamentally solving the technical problem of low data call efficiency of the ocean forecasting and disaster reduction large model.

[0050] Specifically, the principle of the present invention is: The technical solution of the present invention is based on the following core principle to improve the data call efficiency of the ocean forecasting and disaster reduction large model: First, based on the spatio-temporal characteristics and call characteristics of ocean forecast data, the double-layer time granularity index structure organizes the data according to the first time granularity (minute level) and the second time granularity (hour level) to form a spatio-temporal four-dimensional index matrix. This structure fully considers the differences in the call frequencies of data at different time scales in the ocean forecast model, realizes the differential indexing and management of high-frequency and low-frequency data, and improves the data retrieval efficiency. The first time granularity mainly aims at the high-frequency data call requirements in the ocean dynamics model, while the second time granularity is for the call of medium- and low-frequency ocean environmental element forecast data. This hierarchical indexing structure highly matches the data call characteristics of the ocean forecast model.

[0051] Secondly, the data frequency hierarchical storage mechanism divides the data into high-frequency call units and low-frequency call units according to the call frequency, and correspondingly designs a GPU video memory data acceleration matrix and a CPU memory data storage matrix. High-frequency data is organized in the GPU video memory through sparse matrices and compression matrices, while low-frequency data is managed in the CPU memory through a hierarchical cache structure. This hierarchical storage strategy takes advantage of the complementary advantages of the high computing efficiency of the GPU and the large-capacity storage of the CPU, and effectively solves the contradiction between the storage of massive data and efficient access.

[0052] Thirdly, the heterogeneous parallel scheduling algorithm and the data migration optimization function achieve the efficient utilization of computing resources and the intelligent scheduling of data. The data migration optimization function comprehensively considers multi-dimensional indicators such as data call frequency, timeliness, storage pressure, computing load, and data dependency relationships, calculates the priority and optimal timing of data migration, and guides the adaptive migration of data between the GPU video memory and the CPU memory, reducing unnecessary data transmission and call latency. The heterogeneous parallel scheduling algorithm realizes the collaborative computing of the GPU and the CPU through task decomposition, load balancing, and resource management modules.

[0053] Finally, the ocean spatio-temporal feature predictor predicts the data call pattern of the forecast model through a bidirectional architecture driven by multi-head attention, providing forward-looking guidance for data organization and scheduling. The three-dimensional spatio-temporal convolutional network of the predictor extracts the spatio-temporal features of ocean data, and the multi-head attention mechanism captures the spatio-temporal correlations of different call patterns, outputting the access probability distribution of each data call unit within the future time window. Based on these prediction results, the system can adjust the data storage location and index structure in advance, realize data preloading and resource preallocation, and further improve the data call efficiency.

[0054] The following provides a specific Embodiment 1 of the present invention, and the specific implementation manners of each step in this Embodiment 1 are described in detail as follows.

[0055] The specific implementation of step S01 is to construct a two-layer time granularity index structure. First, determine the first time granularity size, usually from 1 minute to 5 minutes, as the basic time unit for accessing high-frequency data of the ocean dynamics model; then determine the second time granularity size, usually from 30 minutes to 120 minutes, as the time unit for environmental element forecasting; then organize the ocean forecast data in the time dimension according to these two granularities to form a two-layer index in the time dimension; then use the octree structure to divide the three-dimensional ocean space in the space dimension, and construct a space index for the ocean space according to the three dimensions of longitude, latitude, and depth; finally, combine the two-layer time index with the three-dimensional space index to construct a four-dimensional spatio-temporal index matrix, and each element in this matrix corresponds to a spatio-temporal data block. Among them, the four-dimensional spatio-temporal index matrix can be expressed as: ; In the formula, is the four-dimensional spatio-temporal index matrix; is the data element in the th first time granularity, the th second time granularity, the th space position, and the th depth layer of this matrix; ranges from to , is the number of divisions of the first time granularity; ranges from to , is the number of divisions of the second time granularity; ranges from to , is the number of divisions of the space position; ranges from to , is the number of divisions of the depth layer.

[0056] The mathematical expression for the space index using the octree structure is: ; In the formula, represents the octree structure; represents the current node; represents the root node; represents the th child node, ranges from to .

[0057] This step adopts a multi-resolution spatio-temporal indexing technique, aiming to achieve efficient organization and rapid retrieval of data at different time scales and spatial scales, and improve the data access efficiency of the large marine forecasting and disaster reduction model.

[0058] The specific implementation of step S02 is to divide the data according to the spatio-temporal characteristics and call frequencies of the data required by the marine forecasting model. First, statistical analysis is performed on the historical data call logs to calculate the average access frequency of each spatio-temporal data block; then a call frequency threshold is set, usually 10 calls per second, and the data blocks with a frequency higher than this threshold are classified as high-frequency call units; then the data blocks with a frequency lower than the call frequency threshold are classified as low-frequency call units; then the spatial distribution characteristics of each call unit are analyzed, and the data blocks with adjacent spatial positions and similar call patterns are merged to form larger call units; finally, according to the spatio-temporal characteristics and data volume of each call unit, its initial allocation strategy in the GPU video memory or CPU memory is determined. Among them, the formula for calculating the average access frequency is: ; In the formula, represents the average access frequency of the data block ; represents the number of accesses to the data block within the time period ; represents the total number of time periods for statistics.

[0059] The similarity formula for call unit merging is: ; In the formula, represents the total similarity between call units and ; represents the spatial position similarity, usually the reciprocal of the Euclidean distance; represents the call pattern similarity, usually the Pearson correlation coefficient; and are weight coefficients, and , usually takes the value of 0.4, takes the value of 0.6.

[0060] This step adopts a data access pattern analysis technique, aiming to identify the usage characteristics of different data, provide a basis for subsequent heterogeneous storage optimization, and improve the overall data access efficiency of the system.

[0061] The specific implementation of step S03 is to construct a GPU video memory data acceleration matrix for high-frequency call units. First, analyze the read and write characteristics of the data in the high-frequency call units, and classify them into two categories: read-only data and read-write data. Then, perform sparsity analysis on the read-only data, and use collaborative filtering algorithms to identify redundant and sparse patterns in the data. Next, construct a sparse matrix for the read-only data, and use the coordinate format storage method to compress the sparse data, which can usually reduce the storage space by 60% to 80%. After that, perform feature analysis on the read-write data, and select a compression algorithm suitable for the characteristics of ocean data, such as wavelet transform or principal component analysis. Finally, construct a compression matrix for the read-write data, and achieve high compression ratio storage while ensuring data accuracy, reducing data transmission overhead. Among them, the coordinate format of the sparse matrix of read-only data is expressed as: ; In the formula, represents the coordinates and values of non-zero elements, represents the first time granularity index, represents the second time granularity index, represents the spatial position index, represents the depth layer index, represents the data value at the corresponding position.

[0062] The mathematical expression for using wavelet transform for read-write data compression is: ; In the formula, represents the wavelet transform result of the signal ; represents the wavelet function; represents the conjugate of the wavelet function; represents the scale parameter; represents the translation parameter.

[0063] This step adopts data sparse representation and compressed storage technology, aiming to maximize the use of limited GPU video memory space, store more high-frequency access data, reduce the number of data transmissions between the CPU and the GPU, and improve the computing efficiency.

[0064] The specific implementation of step S04 is to construct a CPU memory data storage matrix for low-frequency call units. First, the low-frequency call units are further divided into medium-frequency data and low-frequency data according to the access frequency. Usually, the access threshold for medium-frequency data is 1 to 10 times per minute. Then, a multi-level cache structure is designed, including L1 cache, L2 cache, and main storage area. The L1 cache is usually configured as 5% to 10% of the total memory, and the L2 cache is configured as 15% to 25% of the total memory. Next, a cache replacement policy based on the least recently used (LRU) algorithm is established. Medium-frequency data is preferentially placed in the L1 cache, and low-frequency data is placed in the L2 cache or main storage area. After that, a cache prefetch mechanism is constructed. Based on the principle of temporal and spatial locality of data access, it predicts and preloads data blocks that may be accessed in advance. Finally, a cache coherence maintenance mechanism is established to ensure that data copies in each level of cache are synchronized in a timely manner when the data is updated. Among them, the mathematical expression of the least recently used algorithm is: ; In the formula, represents the LRU score of data block . represents the current system time; represents the last access time of data block .

[0065] The prediction model of the cache prefetch mechanism can be expressed as: ; In the formula, represents the probability that data block is accessed in the current context; represents the historical access pattern; represents the temporal locality feature; represents the spatial locality feature; represents the prediction function, usually implemented using a Bayesian network or Markov model.

[0066] This step adopts a hierarchical cache management technology, aiming to optimize the storage and access efficiency of low-frequency data, reduce memory access latency, and improve system throughput.

[0067] The specific implementation of step S05 is to design a heterogeneous parallel scheduling algorithm to achieve adaptive data migration between the GPU and the CPU. First, a data migration optimization function is constructed, which comprehensively considers multiple metrics: data call frequency metric, data timeliness metric, storage capacity pressure metric, computing load balance metric, and data dependency relationship metric. Then, based on these five metrics, the data migration priority score is calculated using the weighted summation method, and the weights of each metric are dynamically adjusted according to the specific application scenario. Next, a migration trigger threshold is set. Usually, when the priority score is greater than 0.75, the migration to the GPU is triggered, and when the score is less than 0.25, the migration to the CPU is triggered. After that, a batch data migration mechanism is implemented to merge multiple small data blocks to be migrated into large data blocks for transmission, reducing the migration overhead. Finally, a migration log recording system is established to track the data migration history for optimizing future migration decisions. Among them, the mathematical expression of the data migration optimization function is: ; In the formula, represents the migration priority score of the calling unit ; represents the data call frequency metric, and its value range is [0, 1]; represents the data timeliness metric, and its value range is [0, 1]; represents the storage capacity pressure metric, and its value range is [0, 1]; represents the computing load balance metric, and its value range is [0, 1]; represents the data dependency relationship metric, and its value range is [0, 1]; to are weight coefficients, and .

[0068] The calculation methods of each metric are as follows: ; In the formula, represents the average access frequency of the calling unit ; represents the minimum access frequency of all calling units; represents the maximum access frequency of all calling units.

[0069] ; In the formula, represents the current system time; represents the last update time of the calling unit ; represents the update period of the calling unit .

[0070] ; In the formula, represents the remaining space of the current storage system; represents the total capacity of the storage system.

[0071] ; In the formula, represents the load level of the current computing system; represents the minimum load level allowed by the system; represents the maximum load level allowed by the system.

[0072] ; In the formula, represents the set of calling units that have a dependency relationship with the calling unit ; represents the weight of the calling unit ; represents the position index of the calling unit , which is 1 when is located on the GPU and 0 when located on the CPU; represents the size of the dependency set.

[0073] This step adopts multi-objective optimization and heterogeneous computing technologies with the aim of achieving efficient utilization of computing resources, dynamically adjusting the data storage location according to real-time requirements, and improving the overall operation efficiency of the ocean forecasting and disaster reduction large model.

[0074] The specific implementation method of step S06 is to construct a spatio-temporal index self-optimization mechanism. First, a model call pattern monitoring system is established to record the access situation of the ocean forecasting and disaster reduction large model to each spatio-temporal data block in real time; then, the time characteristics of the call pattern are analyzed, and the statistical characteristics of data access at different time granularities are calculated, such as average access frequency, standard deviation, and peak distribution; next, the efficiency indicators of the current time granularity division are evaluated, including index hit rate, data block access balance degree, and storage space utilization rate; then, the division ratio of the first time granularity and the second time granularity is dynamically adjusted according to the efficiency indicators, usually triggered when the index hit rate is lower than 80%; finally, the progressive index reconstruction algorithm is applied to complete the update of the spatio-temporal index structure without interrupting the system operation. Among them, the calculation formula for the index hit rate is: ; In the formula, represents the index hit rate; represents the number of index hits; represents the total number of accesses.

[0075] The calculation formula for the data block access balance degree is: ; In the formula, represents the access balance degree; represents the standard deviation of the access frequencies of all data blocks; represents the average value of the access frequencies of all data blocks.

[0076] The adjustment formula for the time granularity division ratio is: ; In the formula, represents the adjusted time granularity division ratio; represents the time granularity division ratio before adjustment; represents the adjustment coefficient, usually taking values from 0.1 to 0.3; represents the efficiency change amount, which is jointly determined by the index hit rate, access balance degree, and storage space utilization rate.

[0077] This step adopts the adaptive index optimization technology, aiming to enable the index structure to adapt to the changing data access patterns and improve the data access efficiency of the ocean forecasting and disaster reduction large model in different application scenarios.

[0078] The specific implementation method of step S07 is to apply the ocean spatio-temporal feature predictor to optimize the computing resource allocation strategy. First, activate the trained ocean spatio-temporal feature predictor and input the current ocean forecasting scenario parameters and historical data call patterns; then use the ocean spatio-temporal feature predictor to predict the access probability distribution of each data call unit within the future time window, and the time window is usually set to 10 minutes to 30 minutes; then construct the resource allocation optimization model, take the prediction result as the input, and comprehensively consider the computing node performance characteristics and network topology structure; then apply the distributed computing resource pooling management technology to classify the computing nodes according to the computing power and storage capacity to form a resource pool; finally, dynamically adjust the computing task allocation and data storage strategy according to the data demand characteristics and resource pool status under different forecasting scenarios. Among them, the core structure of the ocean spatio-temporal feature predictor can be expressed as: ; In the formula, represents the prediction result, that is, the access probability distribution of each data call unit within the future time window; represents the input features, including the current ocean forecasting scenario parameters and historical data call patterns; represents the encoder function, which is implemented by a three-dimensional spatio-temporal convolutional network; represents the decoder function, which is implemented by a multi-head attention mechanism.

[0079] The three-dimensional spatio-temporal convolutional operation of the encoder can be expressed as: ; In the formula, Represents the feature map of the th layer; Represents the feature map of the th layer and the th channel; Represents the convolutional kernel of the th layer and the th channel; Represents the bias term of the th layer; Represents a three-dimensional convolution operation; Represents an activation function, usually the rectified linear unit function; Represents the number of channels of the th layer.

[0080] The multi-head attention mechanism in the decoder can be expressed as: ; In the formula, Represents the multi-head attention function; Represents the query matrix; Represents the key matrix; Represents the value matrix; Represents the output of the th attention head; Represents the output projection matrix; Represents a matrix concatenation operation.

[0081] The calculation formula for each attention head is: ; In the formula, , , respectively represent the query, key, and value projection matrices of the th attention head.

[0082] The calculation formula for the attention function is: ; In the formula, Represents the dimension of the key vector; Represents the softmax normalization function.

[0083] The objective function of the resource allocation optimization model can be expressed as: ; ; In the formula, Represents the set of all data call units; Represents the access probability of the call unit . Indicates the amount of resources allocated to the calling unit ; Indicates the calling unit Under the amount of resources The execution efficiency under; Indicates the total available amount of resources.

[0084] This step adopts a prediction-driven resource optimization technique, aiming to make resource allocation decisions in advance through the prediction of future data access patterns, reduce resource competition and waiting time, and improve the computing efficiency and prediction accuracy of the marine forecast and disaster reduction large model.

[0085] The detailed structure of the marine spatio-temporal feature predictor includes three core modules: a spatio-temporal feature extraction module, a call pattern recognition module, and a prediction optimization module. The spatio-temporal feature extraction module uses a three-dimensional spatio-temporal convolutional network, and the specific structure is 4 layers of three-dimensional convolutional layers. The number of convolutional kernels in each layer is 32, 64, 128, and 256 respectively. The size of the convolutional kernel is , the stride is 1, the padding is 1. After each layer of convolution, a batch normalization layer and a rectified linear unit activation function are connected. Finally, the feature map is reduced to a fixed size through an adaptive average pooling layer. The call pattern recognition module is based on the multi-head attention mechanism. The number of heads is determined according to the number of high-frequency calling units, usually 4 to 16. Each attention head consists of a query matrix, a key matrix, and a value matrix. The matrix dimension is 64 or 128. The attention weights are calculated through scaled dot product, and the attention scores are normalized using the softmax function. The prediction optimization module contains two layers of fully connected layers. The number of neurons in the first layer is 1024, and the rectified linear unit activation function is used. The number of neurons in the second layer is equal to the total number of data calling units within the prediction time window, and the sigmoid activation function is used to output the access probability of each calling unit.

[0086] The mathematical expression of the gating weight function in the marine spatio-temporal feature predictor is: ; In the formula, Indicates the gating weight of the th attention head; Indicates the original weight of the th attention head; Indicates the multi-objective balance value; Indicates the total number of attention heads.

[0087] The calculation formula of the multi-objective balance value is: ; In the formula, Indicates the marine forecast accuracy requirement, and the value range is [0, 1]; Indicates the degree of computing resource constraint, and the value range is [0, 1]; Indicates the real-time requirement for data invocation, with a value range of [0, 1]; Indicates the computational complexity of model inference, with a value range of [0, 1]; , , and are weight coefficients, and .

[0088] Optionally, the formula for the residual connection / jump connection mechanism in the ocean spatio-temporal feature predictor: ; In the formula, represents the feature map of the th layer; represents the feature map of the th layer; represents the convolution operation, including the combination of convolution, batch normalization, and activation function. This formula describes the basic structure of the residual block and enables the effective transmission of spatio-temporal information in the deep network through the identity mapping.

[0089] Optionally, the precision-enhanced weight adjustment function in the gating weight function: ; In the formula, represents the weight of the th attention head after precision enhancement; represents the original weight; represents the enhancement coefficient, usually with a value range of 0.2 to 0.5; represents the attention degree index of the th attention head to the historical data pattern, with a value range of [0, 1]. This function is used when the multi-objective balance value belongs to the high-precision priority interval, aiming to increase the weight of the attention to the historical data pattern and strengthen the perception ability of the ocean spatio-temporal feature predictor to the changes in the ocean environment.

[0090] Optionally, the calculation acceleration-based weight adjustment function in the gating weight function: ; In the formula, represents the weight of the th attention head after calculation acceleration; represents the original weight; represents the acceleration coefficient, usually with a value range of 0.2 to 0.5; represents the The attention index of a single attention head for local spatio-temporal features ranges from [0, 1]. This function is used when the multi-objective balance value belongs to the efficiency-priority interval, aiming to increase the weight of attention to local spatio-temporal features and reduce the processing of global information to improve the inference speed.

[0091] Optionally, the adaptive equilibrium weight adjustment function in the gating weight function: ; In the formula, represents the weight of the th attention head after adaptive equilibrium; represents the original weight; represents the equilibrium coefficient, usually taking values from 0.1 to 0.3; represents the th attention head's balance index under the current accuracy requirement and real-time requirement ranging from [-1, 1]. This function is used when the multi-objective balance value belongs to the balance interval, aiming to dynamically adjust the importance distribution of each attention head and achieve a dynamic balance between accuracy and efficiency.

[0092] Among them, The calculation formula of ; In the formula, is the balance factor, ranging from [0, 1], and is dynamically adjusted according to the current system state; when has a higher value, will tend to enhance the weight of attention to historical data; when has a higher value, will tend to enhance the weight of attention to local features.

[0093] Optionally, the interval judgment condition of the multi-objective balance value: When , it belongs to the high-precision priority interval; When , it belongs to the efficiency priority interval; When , it belongs to the balance interval; In the formula, represents the threshold of the high-precision priority interval, usually taking the value of 0.7; represents the threshold of the efficiency priority interval, usually taking the value of 0.3; represents the multi-objective balance value.

[0094] The following provides a general and simple Embodiment 2 for implementing the numerical control call of the present invention, which is used to better understand the database call of the present invention, especially the steps of combined call of multiple different databases including structured and unstructured databases. Among them, the databases in this embodiment include structured and unstructured ocean observation and forecasting data in Xinchuang databases, SQL, Oracl, Mango Database, etc.; the present invention realizes data interaction and communication between the large model and the early warning database through the API request method. The database call capability is encapsulated into a service using the API data interface, and business processes such as problem input, query element extraction, request sending, data reading, response processing, and information output are constructed to form an early warning data query tool, which can be flexibly embedded into the intelligent agent workflow as a data query node.

[0095] The following are the specific implementation steps of the early warning data query tool: First, API interface encapsulation

[0096] The present invention realizes data interaction and communication between the large model and the early warning database through the API request method. The database call capability is encapsulated into a service using the API data interface, and business processes such as problem input, query element extraction, request sending, data reading, response processing, and information output are constructed to form an early warning data query tool, which can be flexibly embedded into the intelligent agent workflow as a data query node.

[0097] The specific implementation steps of the early warning data query tool: Step 1: Construct an early warning API data interface, clarify the endpoints, request methods, request parameters, and return values of the API, and combine the database table structure information to implement the data query business logic.

[0098] Step 2: Identify the keywords in the dialogue, including query elements such as forecast area, forecast time limit, and forecast elements, and construct API request parameters; Step 3: Execute the data interface, send a query request, and the background program calls the early warning database to return the query result; Step 4: Process the response, parse the returned data, and extract the early warning information.

[0099] Second, obtain the database query conditions

[0100] According to the input statement, use the large model to infer and extract the query conditions, and organize the query conditions into a json format for easy interface query.

[0101] Step 1: Natural Language Understanding and Intent Recognition. Use large language models based on the Transformer architecture (such as ERNIE, GPT, etc.) to perform semantic parsing on the user input statement. Identify key entities, actions, and modifiers through the attention mechanism, and use the intent classification model to determine the query type (such as forecast type, consultation type, etc.).

[0102] Step 2: Query Condition Extraction and Structuring. Apply named entity recognition technology to extract key parameters. For fuzzy queries, use query expansion technology to supplement implicit conditions, and establish domain-specific condition mapping rules (such as time → timestamp, location → geocode).

[0103] Step 3, JSON Standardization Processing. Dynamically construct a JSON schema: { "intent": "search|consult|generate", "entities": { "type": "object|time|location", "value": "concrete_value", "confidence": 0.95 } , "constraints": { "temporal": {"start": "2025-01-01", "end": "2025-05-19"}, "spatial": {"radius": 5000, "unit": "m"} } } Third, obtain query data Use Python code to modify the format and organize the input parameters of the interface file. Use an HTTP request to query the required data through the interface. Initiate an HTTP request (GET / POST method) to call the target API interface, use the json module to parse the interface document file, standardize the input parameters through the dictionary data structure (including mandatory field verification, parameter type conversion, and default value filling), then construct a request header and request body that conform to the RESTful specification, and finally process the response data (status code check, JSON deserialization, and exception capture).

[0104] import json def main(arg1: str) -> dict: # Parse the JSON string into a dictionary # Parse the JSON string into a dictionary parsed = json.loads(arg1) return { "area": parsed.get("area"), "beginDate": parsed.get("beginDate"), "element": parsed.get("element"), } Fourth, parse the query results Parse the obtained data according to time, location, and parameters. When performing structured parsing on the obtained raw data, convert the timestamp field to a standard datetime object (supporting UTC time zone conversion), then construct a spatial index for geographical location information based on the administrative region coding table, and finally perform type checking and normalization processing on multi-dimensional parameters (such as sea temperature, sea waves, tides, etc.). Finally, output a structured data set that meets the analysis requirements.

[0105] Fifth, algorithm encapsulation Encapsulate the database query algorithm in the large model, establish an efficient, secure, and semantically understanding interactive layer, and focus on implementing the query from natural language to the database. Implement the OpenAI compatible protocol interface, standardize the input and output formats to adapt to third-party systems, and use gRPC service encapsulation to support high-concurrency remote calls.

[0106] from langchain.tools import tool def market_analysis(keyword: str) -> dic # Call the data API and process the results return processed_data # Return structured data:ml-citation{ref="5,8" data="citationList"} from langchain.agents import initialize_agent tools = [market_analysis, data_visualization] # Load the custom tool set agent = initialize_agent(tools, llm, agent_type="structured-chat-react") # :ml-citation{ref="5,8" data="citationList"} from fastapi import FastAPI app = FastAPI() async def openai_proxy(request: OpenAIRequest): return await agent_execute(request) # Native compatibility After the above processing, a complete database parsing result can be obtained, and it is conveniently encapsulated to unify the data records in multiple databases into a single integrated data set for unified invocation. This solves the problem of the large model's understanding and invocation of ocean data in structured and unstructured databases and the like.

[0107] To better understand and implement the present invention, the following provides Example 3 of a specific application scenario of the present invention: In the storm surge forecasting scenario of a certain ocean institution, a database calling method based on an ocean forecasting and disaster reduction large model was implemented. This institution is responsible for storm surge monitoring and forecasting in a certain sea area, and needs to process a large amount of ocean observation data and perform high-precision forecasting calculations. Researchers used this method to construct an ocean forecasting and disaster reduction system and conducted a 6-month storm surge forecasting experiment using this system. The system is based on a 32-node heterogeneous computing cluster, and each node is equipped with 2 NVIDIA A100 GPUs (80GB video memory) and 512GB CPU memory. The total amount of original ocean data managed by the system is 240TB, and the data types include ocean hydrological data, meteorological observation data, historical storm surge path data, etc.

[0108] First, the researchers constructed a two-layer time granularity index structure according to step S01. The first time granularity is set to 2 minutes for high-frequency kinetic model data access; the second time granularity is set to 60 minutes for environmental element forecasting. Spatially, an octree structure is adopted, and the responsible area is divided into 8 main regions, each main region is further divided into 64 sub-regions, and the depth direction is divided into 12 layers. The specific configuration of the spatio-temporal index is shown in Table 1.

[0109] Table 1 Spatio-temporal index configuration table

[0110] Next, according to step S02, the researchers analyzed the historical data call logs and counted the access frequencies of each data block. The call frequency threshold was set at 8 times per second, and the data was divided into high-frequency call units and low-frequency call units. The high-frequency call units mainly concentrated on the surface and subsurface data in the storm surge affected areas, accounting for about 15% of the total data volume. The division results of the call units are shown in Table 2.

[0111] Table 2 Division Table of Data Call Units

[0112] For the high-frequency call units, a GPU video memory data acceleration matrix was constructed according to step S03. Through collaborative filtering algorithm analysis, it was found that about 62% of the data in the high-frequency call units had sparse characteristics. A read-only data sparse matrix was constructed using the coordinate format storage method, and the compression ratio reached 78%. For read-write data, the wavelet transform compression algorithm was used, and the average compression ratio reached 65%. This configuration enabled all high-frequency data to be fully stored in the GPU video memory, avoiding frequent CPU-GPU data transfers.

[0113] For the low-frequency and medium-frequency call units, a CPU memory data storage matrix was constructed according to step S04. The L1 cache was configured as 8% of the total memory and was used to store medium-frequency data with an access frequency of more than 3 times per minute; the L2 cache was configured as 22% of the total memory and was used to store medium-frequency data with an access frequency of 1-3 times per minute; the remaining low-frequency data was stored in the main storage area. The configurations and performance data of each level of cache are shown in Table 3.

[0114] Table 3 Multilevel Cache Configuration and Performance Table

[0115] The researchers designed a heterogeneous parallel scheduling algorithm according to step S05 to achieve adaptive data migration between the GPU video memory and the CPU memory. The weight settings of the five indicators of the data migration optimization function are: call frequency indicator 0.35, data timeliness indicator 0.25, storage capacity pressure indicator 0.15, computing load balancing indicator 0.15, and data dependency relationship indicator 0.10. Through experiments, the trigger threshold for migrating to the GPU was determined to be 0.72, and the trigger threshold for migrating to the CPU was determined to be 0.28. During the experiment, the system executed an average of 12.3 data migration operations per minute, and the average data volume per migration was 256MB.

[0116] To adapt to the changing data access patterns, the researchers constructed a spatio-temporal index self-optimization mechanism according to step S06. At the initial stage of the experiment, the index hit rate was 78.6%. By dynamically adjusting the time granularity division ratio, the index hit rate increased to 92.4% at the end of the experiment. The data block access balance increased from 0.73 to 0.88, and the storage space utilization rate increased from 82% to 94%.

[0117] Finally, according to step S07, the researchers trained and applied an ocean spatio-temporal feature predictor, which adopted 12 attention heads corresponding to 12 main data call patterns. By analyzing 82,000 data call logs in historical forecast tasks to construct a training set, the prediction accuracy of the predictor on the validation set reached 89.7%. The data call probability distribution output by the predictor was used to guide the calculation resource allocation, significantly reducing resource competition and waiting time.

[0118] Through a 6-month actual application comparison, this method has improvements in multiple performance metrics compared with the traditional ocean data call method based on a fixed cache hierarchy, as shown in Table 4.

[0119] Table 4 Performance comparison table

[0120] Traditional ocean forecast data call methods usually adopt a fixed cache hierarchy structure and predefined data allocation strategies, which cannot be dynamically adjusted according to changes in data access patterns, resulting in low resource utilization, high data access latency, and limited computing efficiency. While the present invention realizes the adaptive optimization of data storage and access through constructing a two-layer time granularity index structure, a high-frequency data GPU acceleration matrix, a heterogeneous parallel scheduling algorithm, and a resource optimization technology based on an ocean spatio-temporal feature predictor, improving the overall performance of the system. Compared with the traditional method, the present invention has significant improvements in aspects such as data access latency, computing throughput, storage space utilization rate, and forecast accuracy, with an average improvement of about 17.9%, providing accurate data for storm surge early warning and disaster prevention and mitigation.

[0121] It should be noted that the detailed explanations of the variables involved in the present invention are shown in Tables 5, 6, and 7 below.

[0122] Table 5 Variable explanation table (the first part)

[0123] Table 6 Variable explanation table (the second part)

[0124] Table 7 Variable explanation table (the third part)

[0125] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, and all should be covered within the protection scope of the present invention.

Claims

1. A database call method based on a large ocean forecasting and disaster reduction model, characterized in that, Including: Construct a two - layer time - granularity index structure, organize ocean forecast data according to the first time granularity and the second time granularity, and form a four - dimensional spatio - temporal index matrix; According to the spatio - temporal characteristics and call frequencies of the data required by the ocean forecast model, divide the data into high - frequency call units and low - frequency call units; Construct a GPU video - memory data acceleration matrix for high - frequency call units; construct a CPU memory data storage matrix for low - frequency call units; design a heterogeneous parallel scheduling algorithm, and realize the adaptive data migration between GPU video - memory and CPU memory based on the data migration optimization function; Construct a spatio - temporal index self - optimization mechanism, and dynamically adjust the division ratio of the first time granularity and the second time granularity according to the model call pattern; apply an ocean spatio - temporal feature predictor to predict the data call pattern of the forecast model, and combine with the distributed computing resource pooling management to optimize the computing resource allocation strategy according to the data demand characteristics under different forecast scenarios.

2. The database call method based on the large marine forecasting and disaster reduction model according to claim 1, wherein The first time granularity refers to the basic index unit of ocean forecast data in the time dimension, and the second time granularity refers to the time unit larger than the first time granularity aggregated on the basis of the first time granularity.

3. The database calling method based on the large marine forecasting and disaster reduction model according to claim 2, characterized in that, The call frequency refers to the number of times the ocean forecast disaster reduction large model accesses data per unit time. The data is divided into high - frequency data and low - frequency data according to the call frequency.

4. The database call method based on the large ocean forecasting and disaster reduction model according to claim 3, characterized in that The call unit refers to a data set with similar spatio - temporal characteristics and access patterns, and serves as the basic unit for data scheduling and storage optimization.

5. The database calling method based on the large ocean forecasting and disaster reduction model according to claim 4, characterized in that, The GPU video - memory data acceleration matrix includes a read - only data sparse matrix and a read - write data compression matrix.

6. The database calling method based on the large ocean forecasting and disaster reduction model according to claim 5, wherein, The read - only data sparse matrix refers to the ocean static parameters and historical data stored in the GPU video - memory that do not need to be modified, and is represented by a sparse matrix to reduce the storage space occupation; the read - write data compression matrix refers to the ocean dynamic parameters and forecast intermediate results stored in the GPU video - memory that need to be frequently updated, and a real - time compression algorithm is used to reduce the data transmission overhead.

7. The database call method based on the ocean forecasting and disaster reduction large model according to claim 6, wherein The heterogeneous parallel scheduling algorithm refers to an algorithm that simultaneously utilizes the computing characteristics of GPU processors and CPU processors to perform parallel processing on ocean data with different characteristics, including a task decomposition module, a load balancing module, and a resource management module.

8. The database call method based on the large ocean prediction and disaster reduction model according to claim 7, characterized in that, The inputs of the data migration optimization function include data call frequency metrics, data timeliness metrics, storage capacity pressure metrics, computing load balancing metrics, and data dependency relationship metrics.

9. The database call method based on the large ocean forecasting and disaster reduction model according to claim 8, wherein The ocean spatio - temporal feature predictor is a multi - head attention - driven ocean data call pattern prediction model. The number of attention heads in the ocean spatio - temporal feature predictor needs to be dynamically determined according to the first time granularity, the second time granularity, and the number of high - frequency call units.

10. The database calling method based on the large ocean forecasting and disaster reduction model according to claim 9, characterized in that, It also includes the step of combinatorial call of multiple different databases including structured and unstructured databases.

Images