Digital twin water conservancy large language model inference method based on tensor parallel processing

Abstract

The present invention provides a digital twin water conservancy large language model inference method based on tensor parallel processing. The method comprises the following steps in sequence: S1. Gradient calculation optimization; S2. Self-attention tensor dimensionality reduction and blocking; S3. Low-dimensional layer normalization parallel; S4. Inter-layer overall weak expansion resource scheduling processing for the low-dimensional layer after the normalization processing of the middle layer in step S3; S5. Use the decoder intermediate tensor result to convert it into a specific prediction value. The present invention significantly improves the inference efficiency of large-scale models by dividing the parameters and input data of the large language model into tensors and distributing them to multiple computing nodes for parallel processing. The use of the present invention can greatly improve the computational efficiency and real-time performance of the large water conservancy language model. By parallelizing the inference tasks of the large-scale model, each computing node can work simultaneously, significantly reducing the computational burden of a single node. It can not only process larger-scale water conservancy data, but also maintain efficient and accurate risk prediction capabilities.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and digital twin technology in water conservancy engineering, and in particular to a digital twin water conservancy large language model inference architecture and method based on tensor parallel processing. TECHNICAL BACKGROUND

[0002] As an innovative engineering management technology, digital twin provides a new perspective for water conservancy engineering by virtualizing the physical environment. In the monitoring of water conservancy facilities such as dams, digital twin technology can collect and simulate various physical parameters (such as water level, flow, pressure, etc.) in real time, creating a high-precision virtual model. By combining a large language model, this virtual environment can achieve detailed analysis and prediction of actual facilities. The introduction of digital twin not only provides an intuitive reference for facility operation and maintenance, but also plays an important role in preventive maintenance and disaster warning, improving the response capability to extreme weather events and other potential risks.

[0003] In the digital twin of water conservancy engineering, the application of tensor parallel technology greatly improves the efficiency of large language models in data analysis. Tensor parallelism breaks down complex computation tasks into multiple subtasks that can be processed simultaneously, maximizing the utilization of computing resources and speeding up the data processing process. In complex water conservancy facilities such as dams, real-time processing of massive sensor data is crucial to ensure facility safety. With tensor parallelism, reaction time can be significantly shortened while ensuring calculation accuracy, and this fast computing capability is crucial for timely prediction and handling of potential risks.

[0004] Although large language models have strong capabilities in data analysis, they still face challenges in computing efficiency. In particular, in dam safety risk prediction, the model needs to process massive, extensive and multi-source data. In this environment, traditional computing architectures are prone to bottlenecks, leading to waste of resources and processing delays. This delay may lead to inaccurate predictions in the face of emergencies. To overcome this problem, the application of tensor parallel architecture, with its unique high-dimensional data processing capabilities, can optimize devices and algorithms, effectively improving the inference efficiency of large language models. Through the use of distributed architecture, tensor parallelism not only significantly improves the speed of large-scale real-time computing, but also improves the efficiency of resource allocation and use. Ultimately, this improvement provides a more accurate and efficient solution for the safety management of dams and other water conservancy facilities in real-time monitoring and early warning capabilities. SUMMARY

[0005] This paper addresses the high computational complexity and slow inference response speed of large language models in water conservancy digital twin systems and proposes a digital twin water conservancy large language model inference method based on tensor parallel processing. This method applies a tensor parallel architecture, leveraging its unique high-dimensional data processing capabilities compared to other technologies, to optimize equipment and algorithms, effectively improving the inference efficiency of large language models. By adopting a distributed architecture, tensor parallelism can not only significantly increase the speed of large-scale real-time computing, but also improve the allocation and utilization efficiency of resources. Ultimately, this improvement in real-time monitoring and early warning capabilities provides a more accurate and efficient solution for the safety management of dams and other water conservancy facilities.

[0006] To achieve the above technical objectives, the present invention provides a digital twin water conservancy large language model inference method based on tensor parallel processing, which specifically includes the following steps:

[0007] S1. Gradient calculation optimization: Real-time water conservancy data is collected as input to the digital twin water conservancy language model. The gradient calculation process is optimized using the SUMMA algorithm, and the matrix is ​​processed in a distributed manner, allowing each computing node to independently complete its own computing task. The optimized gradient information obtained realizes parallel processing of the real-time collected water conservancy data.

[0008] S2. Self-attention tensor dimensionality reduction and partitioning: Using the gradient information optimized in step S1 as the basis for the self-attention mechanism, the self-attention mechanism is implemented using the Transformer model. Long sequence data is processed in parallel through row and column partitioning to achieve the purpose of dimensionality reduction and partitioning of the gradient information optimized in step S1.

[0009] S3. Parallel normalization of low-dimensional layers: Normalize the low-dimensional layers of the dimensionality reduction blocks in step S2 using vector parallelization;

[0010] S4. Perform inter-layer weakly extended resource scheduling for the low-dimensional layers after the layer normalization in step S3. After the layer normalization calculation in step S3 stabilizes, distribute the computing load to different computing nodes, so that each computing node is responsible for the calculation of a specific layer. Each node completes the calculation of the specific layer and generates the corresponding inference intermediate tensor result for that layer.

[0011] S5. Use the decoder based on the fully connected layer and the Softmax layer to convert the intermediate tensor results of the model into specific prediction values.

[0012] A further technical solution of the present invention: the water conservancy data in step S1 is collected in real time by a variety of sensors arranged in water conservancy facilities, specifically including water level, flow velocity, flow, seepage pressure, and surface movement data.

[0013] The further technical scheme of the present application: the two-dimensional matrix multiplication scheme of the scalable general matrix multiplication algorithm is used for optimizing the gradient calculation process in the S1 step through the SUMMA algorithm, in the context of deep learning, the system carries out matrix multiplication C=A*B of the matrix output by the target function, the differential, that is, the gradient, is calculated by using the chain rule, a plurality of sub-matrices are formed, then the sub-matrices are distributed to different calculation nodes, each node independently calculates the gradient of the sub-matrix responsible for by the node, and the calculated gradient serves as an important basis for subsequent model updating; the specific calculation process is as follows:

[0014] Let C' be the gradient of the target function with respect to C, A' and B' be the gradients with respect to the input matrices A and B respectively, then:

[0015]

[0016] Where C' is the gradient of the target function with respect to C;

[0017] A' and B' are the gradients with respect to the input matrices A and B respectively, denotes the partial derivative.

[0018] The further technical scheme of the present application: in the S2 step, the dimensionality reduction is firstly performed through linear projection, the input hidden vector is mapped into query (Q), key (K) and value (V), the attention weight is calculated through matrix multiplication, and is multiplied with the value V to obtain the output of the self-attention module; wherein the original hidden vector X is firstly projected into query Q, key K and value V through three independent linear layers respectively:

[0019] Q=XW Q +b Q

[0020] K=XW K +b K

[0021] V=XW V +b V

[0022] Where W Q , W K and W V are learnable weight matrices, b Q , b K and b V are biases;

[0023] After using the linear transformation, the query and the key are dot multiplied to obtain the attention score, and then the attention weight A is obtained through scaling and softmax transformation:

[0024]

[0025] Where A is the attention weight matrix calculated; Q is the query matrix, representing the input query information;

[0026] K is the key matrix, representing the input key information; K T is the transpose of the key matrix, which is the basis for calculating the similarity measure;

[0027] d k is the dimension of the key, and the scaling operation helps to avoid extreme values in the softmax;

[0028] softmax function, which converts the input into a probability distribution, the sum of which is 1. The attention weight A obtained in this way is multiplied by the value V to calculate the weighted sum, and thus the output of the self-attention module is obtained:

[0029] Output=AV

[0030] Finally, the obtained output is rearranged back to the hidden size and processed through another linear layer to produce the final output of the self-attention module:

[0031] Final Output=Output W O +b O

[0032] Where W O is the weight matrix of the second linear layer; b O is its bias.

[0033] Further technical solutions of the application: after the self-attention module is calculated, the S2 step is normalized, each processor independently calculates the mean and variance of the input data, and performs normalization operation; The calculation of the layer normalization can be represented as:

[0034]

[0035] Where, represents the output after normalization processing; Y represents the input data tensor; μ represents the mean E[Y];

[0036] σ 2 represents the variance Var[Y]; δ represents a small constant to prevent division by zero error; usually used to prevent division by zero error and maintain numerical stability;

[0037] The calculation formula of the mean μ and variance σ 2 is:

[0038]

[0039] σ 2 =E[Y 2]-μ 2

[0040] Wherein, m is the number of samples;

[0041] E[Y 2 ] is the expected value of the square of input data, indicating the average value of the square of all elements in the input data vector Y;

[0042] Y j is the jth element in the input sample.

[0043] The further technical scheme of the present application: for the calculation of mean and variance, the processor aggregates the calculation results, and uses all-reduce function to summarize the data of all computing nodes to obtain accurate global mean and variance information, and for the gradient calculation of Y, the function description is as follows:

[0044]

[0045] Wherein, Indicates the partial derivative.

[0046] The further technical scheme of the present application: the calculation load in the S4 step is distributed to different computing nodes, so that each node is responsible for the calculation of a specific layer, which is specifically as follows:

[0047] Assuming that the total number of layers is L, if it is distributed to N GPUs, the number of layers L i Can be expressed as:

[0048]

[0049] When processing time series data, the input of multiple time steps is divided into small batches and distributed to different GPUs for independent calculation; assuming that the number of time steps is T, the time steps T j Processed by each GPU can be expressed as:

[0050]

[0051] A further technical solution of the present application: in the S5 step, a high-speed communication mechanism is used to integrate the intermediate tensor results of all computing nodes to form a complete global tensor, ensuring that the intermediate results of all nodes are synchronized and consistent; the global tensor contains complete water conservancy feature data; the full connection layer maps the input global tensor to the output space through linear transformation, extracts the basic information related to safety value prediction, and outputs the feature vector related to safety value prediction; the Softmax layer probabilistically processes the feature vector output by the full connection layer to generate the probability distribution of each safety level; based on the probability distribution generated by the Softmax layer, combined with the following classification rules, the safety level is divided, and finally the prediction result of the risk level is generated; wherein, the safety level classification rule of the risk level probability P is as follows: P>0.6, predicted as low risk; 0.3≤P≤0.6, predicted as medium risk; P<0.3, predicted as high risk.

[0052] The present application divides the computing task of the large language model into multiple tensor computing tasks through the tensor parallel processing architecture, and distributes them to multiple computing nodes for parallel processing. First, the system splits the parameters and input data of the large language model into small-scale tensors and distributes them to different computing nodes. Each node independently completes its own inference calculation, and then ensures the consistency and synchronization of the intermediate results through high-speed communication between nodes. Finally, the system aggregates, corrects and post-processes the inference results of all nodes to output the final dam safety risk prediction result. During the entire inference process, the system adjusts the computing load in real time through dynamic load balancing and node monitoring to ensure the efficient and stable operation of the system. Overall weak expansion resource scheduling between layers is an important strategy to improve the training and inference efficiency of large-scale deep learning models. Through reasonable resource monitoring, dynamic adjustment, mixed precision calculation and efficient communication, etc., the utilization rate of computing resources and the overall performance of the model can be greatly improved.

[0053] Through this tensor parallel processing method, the system realizes efficient parallel inference of large-scale water conservancy large language models, significantly improving the computing speed and real-time performance of model inference. Finally, the system can process massive monitoring data while outputting dam safety risk prediction results in a timely manner, ensuring rapid response and accurate assessment of dam safety hazards. Compared with the traditional serial inference method, the present application reduces the computing burden of a single node through multi-node parallel computing, improves the overall computing efficiency of the system, and provides a more economical and efficient solution for digital twin technology in water conservancy projects.

[0054] The beneficial effects of the present application are:

[0055] (1) The present application realizes significant improvement in the inference efficiency of large-scale water conservancy models by dividing the computing tasks of large language models into multiple tensor parallel processing. The system works in parallel through multiple computing nodes, effectively reducing the computing burden of single nodes, and through high-speed communication and synchronization mechanism, realizes the rapid prediction of dam safety risk. This method ensures the balance between real-time and accuracy, significantly improves the inference response speed and system stability.

[0056] (2) The present application effectively improves the ability of large language model to process large-scale data through multi-node parallel computing, and continuously optimizes the utilization rate of computing resources of each node by using dynamic load balancing mechanism, ensuring the efficient and stable operation of the system. Compared with the prior art, the present application not only can significantly improve the inference speed, but also can reduce the consumption of computing resources, improve the real-time response ability, and provide a more economical and efficient solution for large-scale water conservancy data processing and dam safety risk management.

[0057] (3) The present application specifically adopts tensor parallel technology to split model parameters and input data into small-scale tensors for distributed parallel inference, thereby realizing efficient inference response of water conservancy large language model. This method not only realizes efficient parallel processing of model inference, but also ensures the efficiency and stability of the system through dynamic load balancing and inter-node communication optimization, effectively solving the problems of slow inference speed and high resource occupation in the prior art. DETAILED DESCRIPTION

[0058] The present application will be further described below in conjunction with the embodiments. The technical solutions shown in the embodiments below are specific solutions of the present application, and are not intended to limit the scope of the claimed application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0059] The digital twin water conservancy large language model inference method based on tensor parallel processing provided by the embodiment can significantly improve the inference efficiency of large language model in water conservancy digital twin system, realize real-time processing and efficient response of large-scale water conservancy data. The specific steps are as follows:

[0060] S1. Gradient calculation optimization; real-time acquisition of water conservancy data as input of digital twin water conservancy large language model, optimization of gradient calculation process through SUMMA algorithm, and distributed processing of matrix, so that each node independently completes its own computing task, and the obtained optimized gradient information realizes parallel processing of real-time acquired water conservancy data.

[0061] In the implementation, 64 GPUs are used, each equipped with 32 GB of video memory, and connected by a 10 Gbps high-speed network. The real-time collected water data includes sensors measuring water level, flow rate, flow, permeation pressure, ground movement, etc. The data volume reaches GB level, the sequence length is set to 512, and the batch size is 128. These real-time collected physical data are used as the input of the large language model. The data stream is not only large, but also requires extremely high real-time performance. To cope with the efficient processing demand of large-scale sensor data stream, the system adopts gradient calculation optimization technology. Specifically, the SUMMA algorithm is used to optimize the gradient calculation process, making it more suitable for distributed environment and achieving efficient and real-time computing capability. To cope with the efficient processing demand of large-scale sensor data stream, the system adopts gradient calculation optimization technology. The sensor data is divided into small batches and input into the large language model. Each GPU calculates the average distribution of the sub-matrix gradient, for example, GPU1 is responsible for the first 16 columns of the gradient matrix; GPU2 is responsible for the 17th-32nd column of the gradient matrix, and so on. The optimized gradient information is used to update the model parameters to ensure that the model can capture the latest dynamic information. This method involves a two-dimensional matrix multiplication scheme of a scalable general matrix multiplication algorithm, especially suitable for large-scale parallel computing. In the context of deep learning, the matrix multiplication C=AxB output by the objective function is calculated using the chain rule. Multiple sub-matrices are formed, and then these sub-matrices are distributed to different computing nodes. Each node independently calculates the gradient of the sub-matrix it is responsible for, reducing the global computing bottleneck. Let C' be the gradient of the objective function with respect to C, A' and B' be the gradients of the input matrices A and B, respectively, then

[0062]

[0063] where C' is the gradient of the objective function with respect to C;

[0064] A' and B' are the gradients of the input matrices A and B, respectively; denotes the partial derivative.

[0065] In the context of water infrastructure monitoring, gradient computation is used to optimize the prediction of the safety status of dams or embankments by the model. Specifically, the system partitions the matrix output by the objective function (e.g., C = A x B) and, through the chain rule, the computed gradients serve as an important basis for subsequent model updates. This parallel processing reduces the global computational bottleneck, allowing each processor to focus only on the calculation of the sub-matrix it manages. After the calculation is complete, each processor will aggregate the local calculation results to a centralized location, or combine all partial results through repeated all-reduce processes to obtain the global gradient information. In this way, the resulting gradient matrix distribution is consistent with the partial division of the original matrices A and B. By adopting the SUMMA algorithm, the complexity of gradient computation is significantly reduced, allowing large-scale parallel processing and efficient use of computing resources, thereby significantly improving the speed and efficiency of model training. The gradient information optimized by the S1 step will provide the basis for the self-attention mechanism in step S2, ensuring that the model can perform efficient inference with the latest parameter state when performing further calculations.

[0066] Table 1 Model partition parameter settings

[0067] Parameter Value Description Tensor split number 4 Split the model's tensor into 4 parts Number of compute nodes 4 Each compute node is responsible for one tensor part Network bandwidth 10 Gbps Ensure high-speed communication between nodes Memory per node 32 GB Available memory per compute node Partitioning strategy Uniform partitioning Uniformly partition the tensor to achieve load balancing Data locality High Optimize the locality of data to reduce latency

[0068] S2. Self-attention tensor dimensionality reduction blocking; after completing the gradient calculation, the system enters the implementation phase of the self-attention mechanism, which improves the computational efficiency of the model through dimensionality reduction blocking. Dimensionality reduction first maps the input hidden vector to query (Q), key (K), and value (V) through linear projection. The attention weight is calculated through matrix multiplication and multiplied by the value V. Each GPU calculates the matrix portion assigned to it, for example, GPU1 is responsible for the first 128 time steps; GPU2 is responsible for the 129-256th time step, thereby obtaining the output of the self-attention module. The original hidden vector X is first projected into query Q, key K, and value V through three independent linear layers:

[0069] Q = XW Q + b Q

[0070] K = XW K + b K

[0071] V = XW V + b V

[0072] where W Q , W K , and W V are learnable weight matrices; b Q , b K , and b V are biases.

[0073] After using linear transformation, the query and key are dot producted to get attention scores, and then the attention weights A are obtained by scaling and softmax transformation:

[0074]

[0075] Where A is the calculated attention weight matrix; Q is the query matrix, representing the input query information;

[0076] K is the key matrix, representing the input key information; K T is the transpose of the key matrix, which is the basis for calculating the similarity measure;

[0077] d k is the dimension of the key, and the scaling operation helps to avoid extreme values in the softmax;

[0078] softmax function converts the input into a probability distribution, whose sum is 1.

[0079] The attention weight A obtained in this way is multiplied by the value V to calculate the weighted sum, thereby obtaining the output of the self-attention module:

[0080] Output=AV

[0081] Finally, the obtained output is rearranged back to the hidden size and processed through another linear layer to produce the final output of the self-attention module:

[0082] Final Output=Output W O +b O

[0083] Where W O is the weight matrix of the second linear layer; b O is its bias.

[0084] The output of the self-attention module obtained by the above steps can improve the processing capacity of long sequence data by splitting the tensor into small-scale sub-tensors and processing in parallel. The self-attention module and the MLP each have two linear layers, which facilitates tensor partitioning and greatly improves computational efficiency and parallel processing capacity. The model not only efficiently processes long sequence data, but also fully utilizes the advantages of parallel computing on a multi-GPU system, thereby improving the overall performance and efficiency of the model. This is the dimensionality reduction and block method. The self-attention mechanism relies on the optimized gradient calculated in step S1, allowing the model to update the attention weight based on the latest gradient information. In addition, the low-dimensional layer normalization in step S3 is performed after the output of the self-attention module to ensure the stability of the model during processing. This process focuses on differentiated resource scheduling.

[0085] Table 2 Parallel inference parameter settings

[0086]

[0087] S3. Low-dimensional layer normalization in parallel; the low-dimensional layer of the dimensionality reduction and block in step S2 is normalized by vector parallelization calculation; after the self-attention module is calculated, the redundant low-dimensional layer of S2 is normalized to further improve the stability and convergence speed of the model. Each processor independently calculates the mean and variance of the input data and performs normalization operations to avoid the delay of global communication. Through vector parallel technology, each node can effectively utilize local results to ensure the efficiency of the calculation; for the layer normalization calculation, it can be represented as:

[0088]

[0089] wherein, represents the output after normalization processing; Y represents the input data tensor;

[0090] μ represents the mean E[Y]; σ 2 represents the variance Var[Y];

[0091] δ represents a small constant to prevent division by zero errors, which is usually used to prevent division by zero errors and maintain numerical stability.

[0092] wherein, the calculation formula of the mean μ and the variance σ 2 is:

[0093]

[0094] σ 2 = E[Y 2 ]- μ 2

[0095] wherein m is the number of samples;

[0096] E[Y 2 ] is the expected value of the square of the input data, representing the average value of the square of all elements in the input data vector Y;

[0097] Y j is the jth element in the input sample.

[0098] Each processor respectively calculates the mean and variance of the input X and X 2 , so as to avoid unnecessary global communication; in each residual connection, the bias addition operation is broadcasted, and the matrix is expanded to each column, so that it can be smoothly executed in the forward process. When backpropagation, the gradient is restored to the processor of the 0th row. The designed aggregation function aggregates the local mean and variance results of each computing node, and in backpropagation, the aggregation function propagates the gradient of Y to the global to obtain accurate global mean and variance information, ensuring that each node can obtain complete global gradient. The designed normalization output function for the gradient calculation of Y is described as follows:

[0099]

[0100] wherein, represents the partial derivative.

[0101] Through such processing, layer normalization can maintain stability during training, and achieve more uniform and rapid convergence for each batch of data. The use of vector parallel technology enables each node to effectively utilize local results in large-scale parallel computing, reducing the delay of global communication. The system finally outputs the prediction result of the safety risk of the dam. This process ensures the real-time and accuracy of the prediction, and effectively improves the inference efficiency of the large-scale language model. The parallel method of the feature tensor after the dimension reduction of the self-attention tensor is shown by the above steps, and the low-dimensional layer normalization ensures that the output of the self-attention module in S2 can be stabilized in the subsequent steps. At the same time, this process also lays the foundation for the weak expansion resource scheduling in step S4, ensuring the efficient use of computing resources.

[0102] S4. Layer inter-total weak expansion resource scheduling processing is performed on the low-dimensional layer after the layer normalization processing in step S3; after the layer normalization calculation is stable in step S3, the calculation load is distributed to different computing nodes, so that each node is responsible for the calculation of a specific layer. After each node completes the calculation of a specific layer, the corresponding inference intermediate result of the layer is generated; the decoder converts the intermediate tensor result of the model into specific prediction values based on the fully connected layer and the Softmax layer.

[0103] In this step, after the layer normalization is stabilized, the system allocates the computing tasks to different GPUs by layer, for example: GPU1 is responsible for the calculation of layers 1-4; GPU2 is responsible for layers 5-8, and so on, so that each node is responsible for the calculation of a specific layer. The data is divided into small batches in time series and allocated to different GPUs for independent calculation. This batch processing method can significantly speed up the overall processing speed. Weak expansion refers to the case where the number of computing nodes is increased, and the load of each node is not uniform, while the overall workload and structure remain unchanged. By dividing the layers, each GPU is only responsible for the calculation of a few layers. Assuming the total number of layers is L, if it is allocated to N GPUs, each GPU will bear the number of layers L i which can be expressed as:

[0104]

[0105] This can effectively reduce the memory pressure of each node and improve the computing efficiency. When processing time series data, the input of multiple time steps can be divided into small batches and allocated to different GPUs for independent calculation. Assuming the number of time steps is T, then each GPU processes T j which can be expressed as:

[0106]

[0107] The division allows multiple GPUs to process time series in parallel, significantly speeding up the overall computing process, and the weak expansion resource scheduling ensures that the data can be efficiently processed to achieve real-time prediction. Finally, after multiple steps of processing, the system can quickly and accurately output the dam safety risk prediction results, providing timely decision support for water conservancy project managers. The parameter comparison table for the three different network structures is shown in Table 3. For the three different network structures, the corresponding hidden layer sizes are set, and the performance is evaluated by the following indicators: throughput is defined as the ratio of batch size to the sum of forward and backward propagation time in each iteration, and inference efficiency is the ratio of batch size to the sum of forward propagation time. In the weak expansion running condition, each GPU uses the same memory configuration, but due to the difference in communication requirements between GPUs, experiments using fewer GPUs usually perform better in terms of throughput and inference efficiency. In the weak expansion setting, by optimizing resource scheduling and configuration, the computing efficiency and inference ability of the model are significantly improved, effectively promoting the application of deep learning models in large-scale distributed systems. Based on the resource normalization of tensor layers, the multi-extreme parallelization realizes the block parallel computing total scheduling, further optimizing the computing performance. Performance comparison experiments were conducted on an NVIDIA A100 GPU cluster to verify the improvement effect of the optimization strategy on large-scale distributed deep learning tasks.

[0108] With 64 GPUs, the throughput of Tesseract four-dimensional tensor parallelism reaches:

[0109]

[0110] Compared with Optimus optimal parallelism:

[0111]

[0112] For inference, Tesseract four-dimensional tensor parallelism reaches:

[0113]

[0114] Compared with Optimus optimal parallelism:

[0115]

[0116] This use effect supports the ability of Tesseract to better utilize resources on GPU servers than other 1-D and 2-D tensor parallelism. In the weak expansion setting, optimized resource scheduling and configuration can significantly improve the computing efficiency and inference ability of the model, thus better promoting the application of deep learning models in large-scale distributed systems. Through the aforementioned multi-pole parallelization of resource normalization at the tensor level, it is the total scheduling method of block parallel computation.

[0117] Table 3 Comparison of different GPU models

[0118] Parallelization GPUs GPU form Hidden capacity Throughput Inference amount Tesseract 4 [2,2,1] 512 0.720 2.971 16 [4,4,1] 1024 0.502 2.136 64 [4,4,4] 1024 0.510 2.218 Optimus 4 [2,2] 512 0.783 3.041 16 [4,4] 1024 0.325 1.217 64 [8,8] 2048 0.302 1.142 Megatron-LM 4 [2,2,1] 512 0.765 3.114 16 [4,4,1] 1024 0.343 1.142 64 [4,4,4] 2048 0.143 0.524

[0119] The intermediate tensor results of each GPU calculation are summarized through high-speed communication, and the summarized tensor is generated after the last decoding layer to generate the output. The decoding result is the real-time safety risk prediction value of the dam, and the normalized feature tensor obtained from the inter-layer resource scheduling, and then the input low-dimensional feature tensor is mapped to the output space through linear transformation to obtain the basic information required for prediction.

[0120] Specifically, a distributed computing framework is utilized to process the assigned feature tensors independently by each computing node through a tensor parallel mechanism, to achieve parallelized computation. A high-speed communication mechanism (such as all-reduce) is used to integrate the intermediate tensor results of all computing nodes to form a complete global tensor, ensuring that the intermediate results of all nodes are synchronized and consistent. The intermediate tensor results contain the multi-dimensional feature representation of the processed (such as gradient optimization, self-attention mechanism processing, and normalization) water conservancy data. The integrated global tensor contains complete water conservancy feature data, covering core dimensions such as future water level prediction and flow rate monitoring, providing a complete and reliable feature basis for subsequent risk prediction.

[0121] The decoder module includes two parts: a fully connected layer and a Softmax layer. Based on the fully connected layer and the Softmax layer, key features are extracted from the integrated global tensor, and safety level prediction results are generated, represented as specific probability distribution or classification results. The fully connected layer maps the input global tensor through linear transformation to the output space, extracting the basic information related to safety value prediction; dimensionality reduction is performed on the high-dimensional tensor to retain key features to reduce computational complexity, and the generated feature representation contains the basic features of each risk level, providing input for the probabilistic processing of the Softmax layer. The Softmax layer performs probabilistic processing on the feature vector output by the fully connected layer, generating a probability distribution for each safety level. The Softmax function is used to convert the linear output to a probability value ranging from 0 to 1, ensuring that the sum of the probabilities of all categories is 1, facilitating subsequent classification and analysis. Based on the probability distribution generated by the Softmax layer, combined with the following classification rules, the safety level is divided into low safety risk: P > 0.6, medium safety risk: 0.3 ≤ P ≤ 0.6, and high safety risk: P < 0.3. By setting thresholds, the model can accurately classify samples according to different probability distributions, and the output classification results can be used for risk warning and management in actual water conservancy projects.

[0122] Based on the input time series, the future trends of key parameters of water conservancy facilities are predicted, including water level: the water level change curve in the next 24 hours. Flow rate: flow rate change trend in different river basins. Seepage pressure: pressure anomaly detection for potential leakage areas.

[0123] The application proposes a large language model inference architecture based on tensor parallel processing. By tensor splitting the parameters and input data of the large language model and distributing them to multiple computing nodes for parallel processing, the inference efficiency of large-scale models is significantly improved. This tensor parallel architecture effectively solves the problems of high computational complexity and slow response speed in existing single-node inference architectures, and is particularly suitable for large-scale water conservancy data and dam safety risk prediction scenarios. Using the application can greatly improve the computing efficiency and real-time performance of water conservancy large language models. By parallelizing the inference tasks of large-scale models, each computing node can work simultaneously, significantly reducing the computational burden of a single node.

[0124] The application is applied to water conservancy, which not only can handle larger-scale water conservancy data, but also can ensure that the system maintains high-efficiency and accurate risk prediction ability while responding quickly, and is particularly suitable for dam safety management applications that require real-time monitoring and high response speed. The system can cope with the challenges of large-scale water conservancy data processing, realize real-time analysis and prediction of dam monitoring data, and provide efficient and accurate risk assessment; multi-node parallel processing ensures the efficiency and stability of the system, avoids single-point computing bottlenecks, and makes the entire system more flexible and economical. The application can make the digital twin system of water conservancy projects better adapt to complex water conservancy environments, and provide strong technical support for dam safety management.

[0125] The above is only one embodiment of the application, which is described in more detail and in more detail, but it cannot be understood as a limitation on the scope of the application. It should be noted that for those skilled in the art, without departing from the concept of the application, a number of modifications and improvements can be made, which are within the scope of the application. Therefore, the protection scope of the application should be subject to the appended claims.

Claims

1. A digital twin water conservancy large language model reasoning method based on tensor parallel processing, characterized by: The specific steps include: S1. Gradient calculation optimization: Real-time water conservancy data is collected as input to the digital twin water conservancy language model. The gradient calculation process is optimized using the SUMMA algorithm, and the matrix is ​​processed in a distributed manner, allowing each computing node to independently complete its own computing task. The optimized gradient information obtained realizes parallel processing of the real-time collected water conservancy data. The gradient calculation process is optimized by the SUMMA algorithm. A two-dimensional matrix multiplication scheme with a scalable general matrix multiplication algorithm is adopted. In the context of deep learning, the system multiplies the matrix output of the objective function. = × , its differential, i.e., gradient, is calculated using the chain method to form multiple sub-matrices, which are then assigned to different computing nodes. Each node independently calculates the gradient of the sub-matrix it is responsible for. The calculated gradient will serve as an important basis for subsequent model updates. The specific calculation process is as follows: Assume The objective function is about The gradient, and The input matrices are and The gradient of , then: ； ； in, The objective function is about gradient; and The input matrices are and The gradient, represents partial derivative; S2. Self-attention tensor dimensionality reduction and partitioning: Using the gradient information optimized in step S1 as the basis for the self-attention mechanism, the self-attention mechanism is implemented using the Transformer model. Long sequence data is processed in parallel through row and column partitioning to achieve the purpose of dimensionality reduction and partitioning of the gradient information optimized in step S1. S3. Parallel normalization of low-dimensional layers: Normalize the low-dimensional layers of the dimensionality reduction blocks in step S2 using vector parallelization; S4. Perform inter-layer weak expansion resource scheduling for the low-dimensional layers after the layer normalization processing in step S3. After the layer normalization calculation in step S3 stabilizes, the computing load is distributed to different computing nodes, so that each computing node is responsible for the calculation of a specific layer and generates the inference intermediate tensor results corresponding to the specific layer. S5. A decoder is used to convert the intermediate tensor results of the model into specific prediction values ​​based on the fully connected layer and the Softmax layer; in the S5 step, a high-speed communication mechanism is used to integrate the intermediate tensor results of all computing nodes to form a complete global tensor, ensuring that the intermediate results of all nodes are synchronized and consistent; the global tensor contains complete water conservancy feature data; the fully connected layer maps the input global tensor to the output space through a linear transformation, extracts basic information related to the safety value prediction, and outputs the feature vector related to the safety value prediction. The Softmax layer probabilistically processes the feature vector output by the fully connected layer to generate a probability distribution for each safety level; based on the probability distribution generated by the Softmax layer, the safety level is divided in combination with the following classification rules, and finally a risk level prediction result is generated; wherein, the safety level classification rules for the probability P of the risk level are as follows: P > 0.6, predicted as low risk; 0.3 ≤ P ≤ 0.6, predicted as medium risk; P < 0.3, predicted as high risk.

2. The digital twin water conservancy large language model inference method based on tensor parallel processing according to claim 1 is characterized by: The water conservancy data in step S1 is collected in real time through a variety of sensors arranged in water conservancy facilities, and specifically includes water level, flow velocity, flow, seepage pressure, and surface movement data.

3. The digital twin water conservancy large language model inference method based on tensor parallel processing according to claim 1 or 2 is characterized by: In the S2 step, the dimensionality reduction is first performed by linear projection, mapping the input hidden vector to the query ( ),key( ) and value ( ), calculate the attention weight by matrix multiplication and add it to the value Multiply them together to get the output of the self-attention module; the original hidden vector X is first projected into query vectors through three independent linear layers. ,key Sum : ； ； ； in, 、 and is a learnable weight matrix, 、 and is bias; After applying linear transformation, the query and key are dot-producted to get the attention score, which is then Transform to obtain attention weights : ； in, is the calculated attention weight matrix; is the query matrix, which represents the input query information; is the key matrix, which represents the input key information; is the transpose of the bond matrix, which serves as the basis for calculating the similarity measure; is the dimension of the key, scaling helps avoid The median value is too extreme; Function, which converts the input into a probability distribution so that its sum is 1; the attention weight obtained in this way and value Multiply and calculate the weighted sum to obtain the output of the self-attention module: ； Finally, the resulting output is resized back to the hidden size and processed through another linear layer to produce the final output of the self-attention module: ； in, is the weight matrix of the second linear layer; is its bias.

4. A digital twin water conservancy large language model inference method based on tensor parallel processing according to claim 1 or 2, characterized in that: After the self-attention module is calculated, step S3 normalizes the redundant low-dimensional layer of step S2. Each processor independently calculates the mean and variance of the input data and performs normalization. The calculation of the layer normalization can be expressed as: ； in, represents the normalized output; Represents the input data tensor; represents the mean E[Y]; represents the variance Var[Y]; Represents a small constant to prevent division by zero errors; usually used to prevent division by zero errors and maintain numerical stability; The mean and variance The calculation formula is: ； ； in, is the number of samples; is the expected value of the square of the input data, indicating that the input data vector The average of the squares of all elements in ; is the first elements.

5. The digital twin water conservancy large language model reasoning method based on tensor parallel processing according to claim 4 is characterized by: For the calculation of mean and variance, the processor aggregates the calculation results and uses the all-reduce function to summarize the data of all computing nodes to obtain accurate global mean and variance information. For the gradient calculation of Y, its function description is as follows: ； in, represents the partial derivative.

6. A digital twin water conservancy large language model inference method based on tensor parallel processing according to claim 1 or 2, characterized in that: In step S4, the computation load is distributed to different computing nodes, so that each node is responsible for the computation of a specific layer, as follows: Assume that the total number of layers is , if it is assigned to GPUs, the number of layers each GPU handles It can be expressed as: ； When processing time series data, the input of multiple time steps is divided into small batches and assigned to different GPUs for independent calculation; the number of time steps is set to , then the time step processed by each GPU is It can be expressed as: 。

Images