A soil and rock dam compaction quality multi-index prediction method based on visual language model and expert network co-evolution

Abstract

The application discloses to the technical field of dam monitoring of water conservancy and hydropower engineering, and particularly relates to a soil and rock dam compaction quality multi-index prediction method based on visual language model and expert network co-evolution, which comprises the following steps: collecting soil and rock dam compaction construction and material source parameters, generating a water content-dry density relationship curve, generating a semantic description thereof, and constructing a numerical-image-text ternary multi-modal data set; constructing a soil and rock dam numerical-image-text ternary multi-modal fusion model based on a visual language model; based on multi-modal fusion features output by the model, constructing a dry density and permeability coefficient double-drive collaborative mapping model based on a multi-expert network; constructing a co-evolution training framework of the visual language model multi-modal base and the double-drive collaborative mapping model for the double-control target of soil and rock dam compaction quality, and performing stage-by-stage training to obtain a prediction result. The application solves the problem of low soil and rock dam compaction quality prediction accuracy and improves the cross-project generalization ability of the model.

Description

Technical Field

[0001] This invention belongs to the field of dam monitoring technology for water conservancy and hydropower projects, and specifically relates to a multi-index prediction method for the compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks. Background Technology

[0002] Compaction is a crucial step in earth-rock dam construction, and its quality significantly impacts dam settlement and deformation. As a critical component of the project, compaction is characterized by long construction periods, stringent schedule requirements, massive workload and resource consumption, complex processes, high technical difficulty, and stringent quality control. Earth-rock dams are often built in high mountain and canyon areas, making quality and schedule control extremely challenging. Uncontrolled dam construction quality can lead to anything from minor issues like deformation coordination problems to serious consequences such as piping, cracking, and even dam failure, resulting in enormous loss of life, property damage, and environmental destruction. Therefore, effectively controlling the compaction quality of earth-rock dams is of paramount importance in the entire project.

[0003] During dam construction, factors affecting the compaction quality of earth-rock dams include construction parameters (compaction speed, excitation force, number of compaction passes, compaction thickness, etc.) and material source parameters (including moisture content, P5 content, uniformity coefficient, etc.). The dry density and permeability coefficient of the compacted soil are the most direct evaluation indicators for the compaction quality of the dam. Compaction quality control mainly relies on post-assessment methods, that is, after the overall construction of a certain structural layer is completed, the soil density and moisture content need to be measured at designated locations. The most commonly used method is the sand cone method, which determines the dry density through volume replacement and then obtains the moisture content through subsequent laboratory tests. However, the sand cone method is a destructive test, which can affect the integrity of the dam. This method also has disadvantages such as high cost and low efficiency, and the number of test points in the post-assessment report is often insufficient.

[0004] Traditional analytical methods mainly include statistical regression and the analytic hierarchy process (AHP). Statistical regression usually requires constructing multiple regression equations for different objectives. When too many variables are introduced, the model can easily become complex, and its generalization ability will decrease significantly. On the other hand, the AHP relies heavily on the subjective judgment of experts and has high requirements for sample size and sampling standardization, making it difficult to adapt to large-scale and dynamically changing engineering scenarios.

[0005] With the increasing maturity and widespread application of advanced technologies such as artificial intelligence, cloud computing, and big data, dam construction is gradually evolving from digitalization to intelligentization. Existing technologies commonly use data-driven modeling methods for compaction quality analysis, such as backpropagation neural networks, Elman neural networks, and extreme learning machines. These methods are suitable for regression prediction problems involving high-dimensional nonlinear data and have gradually become the mainstream approach for compaction quality assessment in recent years. However, these methods typically rely on large-scale, high-quality training data, and their performance may be limited when data is insufficient or unevenly distributed. Furthermore, while some machine learning models, such as random forests, are robust to noise and irrelevant features, they have limitations in predictions beyond the scope of the training dataset, often leading to overfitting during regression prediction training.

[0006] Existing methods for analyzing the compaction quality of earth-rock dam faces primarily use compaction parameters and some material source parameters as model inputs. They fail to effectively utilize the empirical relationship curves and engineering principles contained in textual materials such as experimental reports, construction logs, and professional literature accumulated on-site, making it difficult to comprehensively reflect the intrinsic relationship between construction parameters and compaction quality. Furthermore, existing research often relies on a limited number of field test samples, making it difficult to obtain sufficient training data for small- to medium-sized projects or those with time constraints. This results in insufficient generalization ability and stability of compaction quality prediction models based on small sample conditions.

[0007] Therefore, there is an urgent need for a multi-index prediction method for the compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks, in order to solve the problem of low prediction accuracy of compaction quality of earth-rock dams and improve the model's cross-engineering generalization ability. Summary of the Invention

[0008] Since accurately predicting the compaction quality of earth-rock dams is crucial for ensuring their safety and preventing engineering failures, this invention addresses the problems of traditional monitoring methods, such as high cost, low efficiency, potential damage to dams, small data collection volume, and low generalization of machine learning methods. It proposes a multi-index prediction method for earth-rock dam compaction quality based on the co-evolution of a visual language model and an expert network. This method rapidly predicts the compaction quality of earth-rock dams under small sample conditions, providing data support and theoretical guidance for subsequent evaluation of engineering quality and optimization of construction plans.

[0009] The purpose of this invention is to provide a multi-index prediction method for the compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks, comprising the following steps:

[0010] Collect the compaction and material source parameters of the earth-rock dam, generate the moisture content-dry density relationship curve, and generate a semantic description of the moisture content-dry density relationship curve to construct a numerical-image-text ternary multimodal dataset.

[0011] Based on the numerical-image-text ternary multimodal dataset, a numerical-image-text ternary multimodal fusion model for earth-rock dams based on a visual language model is constructed.

[0012] Based on the multimodal fusion features output by the numerical-image-text ternary multimodal fusion model of the earth-rock dam, a dual-drive collaborative mapping model of dry density and permeability coefficient based on a multi-expert network is constructed.

[0013] A co-evolutionary training architecture for a visual language model multimodal base and a dual-drive co-mapping model for controlling the compaction quality of earth-rock dams is constructed. The numerical-image-text ternary multimodal fusion model of earth-rock dams and the dual-drive co-mapping model of dry density and permeability coefficient based on a multi-expert network are trained in stages to obtain the final prediction result of the compaction quality of earth-rock dams.

[0014] The process of collecting the compaction and material source parameters of the earth-rock dam, generating the moisture content-dry density relationship curve, generating a semantic description of the moisture content-dry density relationship curve, and constructing a numerical-image-text ternary multimodal dataset specifically includes:

[0015] The compaction process parameters during the earth-rock dam compaction construction process are obtained by using on-site engineering monitoring equipment. These compaction process parameters include: number of static rolling passes, number of vibratory rolling passes, compaction thickness, and compaction speed.

[0016] Soil samples are obtained and experiments are conducted on the soil samples to obtain material source characteristic parameters, including: moisture content, P5 content, non-uniformity coefficient and curvature coefficient. At the same time, the dry density and permeability coefficient, which are the characteristics of compaction quality targets, are obtained through the experiments.

[0017] The compaction process parameters and the material source characteristic parameters are combined to form structured numerical data for characterizing the construction conditions. The structured numerical data constitutes the compaction construction and material source parameters of the earth-rock dam.

[0018] After drying the soil sample, soil samples with different moisture contents were re-prepared and compaction tests were conducted. A moisture content-dry density relationship curve was plotted, and an engineering semantic text description of the moisture content-dry density relationship curve was generated, so that the relationship curve image and the engineering semantic text constitute a semantically aligned image-text pair.

[0019] The compaction process parameters, material source characteristic parameters, and target features are standardized, and each set of structured numerical data is combined with the corresponding semantically aligned image-text pairs to construct a numerical-image-text ternary multimodal dataset containing structured numerical vectors, moisture content-dry density relationship curve images, and their engineering semantic text.

[0020] The standardization process for the compaction process parameters, material source characteristic parameters, and target characteristics includes:

[0021] The obtained compaction process parameters, material source characteristic parameters, and target features are reversibly standardized, and the expression formula is as follows:

[0022] (1)

[0023] In the formula, z represents the standardized result, x represents the original value, μ represents the mean of the original data, and σ represents the variance of the original data.

[0024] The construction of a visual language model-based numerical-image-text ternary multimodal fusion model for earth-rock dams, based on the numerical-image-text ternary multimodal dataset, includes:

[0025] The empirical relationship curve images and corresponding engineering semantic text in the numerical-image-text ternary multimodal dataset are embedded and encoded using a pre-trained visual language model to obtain image embedding features and text embedding features respectively.

[0026] The image embedding features and the text embedding features are fused at the feature level to generate joint multimodal features that characterize the empirical laws of earth-rock dam compaction engineering.

[0027] Based on the earth-rock dam compaction construction and material source parameters, the joint multimodal features are subjected to directional weighting processing to match the focus of the multimodal features with the current construction conditions;

[0028] When fusing the joint multimodal features with the structured numerical data, the contribution weights of different feature sources are adaptively adjusted according to the construction conditions to obtain stable multi-source fused features.

[0029] By further integrating and processing the multi-source fusion features, a numerical-image-text ternary multimodal fusion model output for earth-rock dams is generated for subsequent multi-index prediction of compaction quality.

[0030] The process of embedding and encoding the empirical relationship curve images and corresponding engineering semantic text in the numerical-image-text ternary multimodal dataset using a pre-trained visual language model to obtain image embedding features and text embedding features respectively is specifically described as follows:

[0031] (2)

[0032] In the formula, Fig represents the moisture content-dry density relationship curve corresponding to the input features, which is a 3-channel 224-pixel × 224-pixel image after VLM model preprocessing, and VLM(·) represents converting the preprocessed image into (1, d VLMThe embedding tensor VisionEmb of Fig; Text represents the corresponding text description of Fig, and VLM(·) represents converting the text into (1, d) tensor. VLM The embedded tensor TextEmb;

[0033] The step of fusing the image embedding features and the text embedding features at the feature level to generate joint multimodal features representing the empirical laws of earth-rock dam compaction engineering includes:

[0034] An image and text embedding method is used to concatenate them along the feature dimension, and a multimodal enhancement model is used to fuse visual and text features into a unified representation.

[0035] (3)

[0036] In the formula, Concatenate(·) represents the concatenation of visual features and text features, and the Enhancement(·) function can be understood as... The mapping process is implemented by a two-layer multilayer perceptron, MD. i This represents the fused visual text features;

[0037] Based on the earth-rock dam compaction construction and material source parameters, the joint multimodal features are subjected to directional weighting processing to match the focus of the multimodal features with the current construction conditions;

[0038] Targeted refinement of multimodal features based on a compaction parameter-driven cross-modal attention mechanism:

[0039] The structured numerical vector composed of the compaction process parameters and material source characteristic parameters is mapped to a high-dimensional space through a learnable linear projection layer, aligned with the mixed modality dimension in the joint multimodal features. The high-dimensional vector of the compaction process parameters and material source characteristic parameters is used as the query vector, and the mixed modality is used as the key vector and value vector, specifically including:

[0040] (4)

[0041] In the formula, For query vector, For key vectors, For value vectors, This means mapping the input structured numerical data to the same high-dimensional space as the visual text features. The combined visual text features. The learnable matrix to be multiplied by the query vector. The learnable matrix that is multiplied by the key vector. It is a learnable matrix that can be multiplied by a value vector;

[0042] The query vector, key vector, and value vector are divided into four subspaces along the feature dimension, allowing each head to focus on different aspects of information, specifically including:

[0043] (5)

[0044] In the formula, d k =d model / 4;d model The hidden layer dimension of the model;

[0045] The specific process of performing attention calculations is described below:

[0046] (6)

[0047] In the formula, W O The weights represent learnable weights, and MF represents the fused features. This is the normalized similarity weight matrix. This represents the result of concatenating multi-head attention calculations. For value vectors, Let be the dimension of each subspace in multi-head attention. The weight matrix is ​​a learnable matrix. This is the transpose of the key vector;

[0048] The step of adaptively adjusting the contribution weights of different feature sources according to the construction conditions when fusing the joint multimodal features and the structured numerical data to obtain stable multi-source fused features includes:

[0049] Adaptive dynamic weighted fusion of multi-source features based on gating mechanism;

[0050] The joint multimodal features are concatenated with the structured numerical data, and gating calculations are performed. The calculation steps are as follows:

[0051] (7)

[0052] In the formula, G=[g m , g u ], and g m +g u =1, g m Controlling the contribution weights of multimodal refined features, g u Control the preservation strength of original numerical features; For Hadamard product, we have element-wise multiplication; FF represents the features fused after gating; C is the concatenated result; X is the input structured numerical data; b g The bias is used in the linear transformation; MF is the result of the multi-head attention calculation.

[0053] The step of further integrating and processing the multi-source fusion features to generate a numerical-image-text ternary multimodal fusion model output for subsequent multi-index prediction of compaction quality of earth-rock dams specifically includes:

[0054] Information flow is enhanced through residual connections to generate a deeply fused multimodal hybrid output. The FF (Fluid Filter) is then concatenated with the structured numerical data, followed by a linear transformation to reduce the number of channels to half the concatenated number. This concatenated FF is then re-concatenated with the structured numerical data to obtain the multimodal hybrid model output. The specific calculation steps are as follows:

[0055] (8)

[0056] In the formula, MM is the output of the multimodal hybrid model, MLP represents linear transformation, FF is the fused feature after gating, and X is the input structured numerical data.

[0057] The construction of a dual-drive collaborative mapping model for dry density and permeability coefficient based on the multimodal fusion features output by the numerical-image-text ternary multimodal fusion model of the earth-rock dam includes:

[0058] Two independent dedicated task projection layers are designed for the dry density prediction task and the permeability coefficient prediction task, respectively.

[0059] For the dry density prediction task, a three-layer multilayer perceptron is used to map the shared features to the output space of the dry density prediction task.

[0060] For the permeability prediction task, the same projection structure is used, and the shared features are mapped to the output space of the permeability prediction task using parameters independent of those used for the dry density prediction task.

[0061] The specific steps are as follows:

[0062] (9)

[0063] In the formula, Let represent the final predicted results for dry density and permeability coefficient, respectively. W and b represent the specific weights and biases for different prediction targets, respectively. GELU(·) represents the activation function GELU. This indicates transpose.

[0064] The co-evolutionary training architecture for constructing a visual language model multimodal base and a dual-drive collaborative mapping model for dual control targets of earth-rock dam compaction quality specifically includes the following phased training of the earth-rock dam numerical-image-text ternary multimodal fusion model and the dual-drive collaborative mapping model based on multi-expert networks for dry density and permeability coefficient:

[0065] Step S401: Based on the multimodal foundation pre-training with dual-index joint supervision, drive the model to learn the general representation of the compressive mass.

[0066] The intermediate result MM obtained from the numerical-image-text ternary multimodal fusion model of the earth-rock dam is linearly transformed, and the mean square error between the transformed result and the true value is calculated as the loss. The specific calculation process is as follows:

[0067] (10)

[0068] (11)

[0069] In the formula, Y represents the predicted value, Y=[y1, y2] represents the actual value, and MSE(·) represents the calculated MSE.

[0070] The linear projection module described in formula (10) is only used to provide auxiliary supervision signals during the pre-training stage and will be removed in the subsequent prediction stage. Its parameters do not participate in the final prediction.

[0071] Step S402: Based on task difficulty awareness, dual expert prediction head training is carried out to achieve collaborative optimization of the dual control targets of dry density and permeability coefficient.

[0072] After completing step S401, fix all network parameters of the model trained in step S401, keep the learned compaction quality sensitive representation unchanged, and only train the weights of the dual-drive collaborative mapping model of dry density and permeability coefficient based on multi-expert network. Calculate the loss for each output separately. The calculation process is as follows:

[0073] (12)

[0074] In the formula, This represents the final prediction result, where y1 and y2 represent the actual values.

[0075] The final loss is a weighted sum of the losses from the dry density prediction task and the permeability coefficient prediction task, calculated as follows:

[0076] (13)

[0077] In the formula, α is 0.6 and β is 0.4.

[0078] Another object of the present invention is to provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the multi-index prediction method for earth-rock dam compaction quality based on the co-evolution of visual language model and expert network according to the present invention.

[0079] Another object of the present invention is to provide a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the processor performs the multi-index prediction method for earth-rock dam compaction quality based on the co-evolution of visual language models and expert networks according to the present invention.

[0080] The beneficial effects of this invention are as follows:

[0081] Compared with existing technologies, this invention utilizes a visual language model to introduce prior knowledge into the prediction of compaction quality for earth-rock dams. The pre-trained visual language model avoids retraining a large number of parameters, enhancing the model's ability to handle small sample tasks. Furthermore, the phased training strategy designed in this invention allows the model to focus on different stages, unaffected by upstream or downstream stages. Finally, the dual-drive collaborative mapping model for dry density and permeability coefficient based on a multi-expert network (referred to as the multi-output prediction model) designed in this invention achieves full utilization of the same feature by different tasks without mutual interference between tasks. Compared with traditional machine learning and deep learning models, this invention improves prediction accuracy, overcomes the difficulties of small sample prediction, and constructs a deep learning model based on a limited number of field compaction test data before construction to accurately predict the compaction quality during the subsequent full-scale construction of earth-rock dams, providing guidance for decision-making. Simultaneously, the model proposed in this invention also provides new ideas for predicting other engineering parameters, showing promising prospects for engineering applications. Attached Figure Description

[0082] Figure 1 This is the flowchart of a multi-index prediction method for compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks, as proposed in this invention.

[0083] Figure 2 A schematic diagram of the structure and training process of a multimodal hybrid model and a multi-output prediction model for a multi-index prediction method of earth-rock dam compaction quality based on the co-evolution of a visual language model and an expert network, provided in an embodiment of the present invention;

[0084] Figure 3 A comparison chart of prediction indicators for a multi-index prediction method for compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks, provided in an embodiment of the present invention. Detailed Implementation

[0085] Accurate prediction of compaction quality in earth-rock dams is crucial for ensuring dam safety and preventing engineering failures. Addressing the shortcomings of traditional monitoring methods—high cost, low efficiency, potential damage to dams, small data collection volumes, and low generalization of machine learning methods—this invention provides a multi-index prediction method for earth-rock dam compaction quality based on the co-evolution of a visual language model and an expert network. This method rapidly predicts earth-rock dam compaction quality under small sample conditions, providing data support and theoretical guidance for subsequent project quality assessment and construction scheme optimization. The specific embodiments of this invention are described in detail below with reference to the accompanying drawings.

[0086] like Figure 1 The embodiment of the present invention shown discloses a multi-index prediction method for the compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks, including the following steps:

[0087] Collect the compaction and material source parameters of the earth-rock dam, generate the moisture content-dry density relationship curve, and generate a semantic description of the moisture content-dry density relationship curve to construct a numerical-image-text ternary multimodal dataset.

[0088] Based on the numerical-image-text ternary multimodal dataset, a numerical-image-text ternary multimodal fusion model for earth-rock dams based on a visual language model is constructed.

[0089] Based on the multimodal fusion features output by the numerical-image-text ternary multimodal fusion model of the earth-rock dam, a dual-drive collaborative mapping model of dry density and permeability coefficient based on a multi-expert network is constructed.

[0090] A co-evolutionary training architecture for a visual language model multimodal base and a dual-drive co-mapping model for controlling the compaction quality of earth-rock dams is constructed. The numerical-image-text ternary multimodal fusion model of earth-rock dams and the dual-drive co-mapping model of dry density and permeability coefficient based on a multi-expert network are trained in stages to obtain the final prediction result of the compaction quality of earth-rock dams.

[0091] This invention discloses a method for predicting the multi-index compaction quality of earth-rock dams based on the co-evolution of a visual language model and an expert network. This method introduces multimodal embedding based on a pre-trained visual language model, cross-modal feature enhancement methods, and multi-task learning into the task of predicting the compaction quality of earth-rock dams. By introducing prior knowledge, this method addresses the challenge of learning with few samples, solves the problem of low accuracy in predicting the compaction quality of earth-rock dams, and improves the model's cross-engineering generalization ability.

[0092] In a specific embodiment, the multi-index prediction method for compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks specifically includes the following steps:

[0093] Step 1: Collect the compaction and material source parameters of the earth-rock dam, generate the moisture content-dry density relationship curve and its semantic description, and construct a numerical-image-text ternary multimodal dataset;

[0094] Step 2: Construct a numerical-image-text ternary multimodal fusion model for earth-rock dams based on a visual language model (VLM);

[0095] Step 3: Construct a dual-drive collaborative mapping model of dry density and permeability coefficient based on a multi-expert network;

[0096] Step 4: Construct a VLM multimodal base and dual-drive mapping model co-evolution training architecture for dual control objectives of earth-rock dam compaction quality. Train the earth-rock dam numerical-image-text ternary multimodal fusion model constructed in Step 2 and the dry density and permeability coefficient dual-drive co-mapping model constructed in Step 3 in stages to obtain the final prediction results of earth-rock dam compaction quality.

[0097] Furthermore, step 1 includes the following sub-steps:

[0098] Step S101: Obtain compaction process parameters through on-site monitoring equipment, including: number of static rolling passes, number of vibratory rolling passes, compaction thickness, and compaction speed;

[0099] Step S102: Obtain soil samples, conduct experiments, and obtain material source characteristic parameters, including moisture content, P5 content, non-uniformity coefficient, and curvature coefficient. At the same time, obtain the target characteristics: dry density and permeability coefficient through experiments. Combine the compaction process parameters and material source characteristic parameters into a set of numerical data.

[0100] Step S103: After drying the soil sample, prepare soil samples with different moisture contents and conduct compaction tests to draw the moisture content-dry density relationship curve; generate structured descriptive text based on the structural characteristics of VLM, and the text and curve image constitute a semantically aligned image-text pair for subsequent VLM embedding training.

[0101] Step S104: The obtained compaction process parameters, material source characteristic parameters, and target features are reversibly standardized, expressed by the following formula:

[0102] (1)

[0103] In the formula, z represents the standardized result, x represents the original value, μ represents the mean of the original data, and σ represents the variance of the original data;

[0104] Step S105 involves designing a structured dataset for cross-modal fusion. Each set of numerical values ​​and image-text pairs is combined into a ternary aligned sample, containing a structured numerical vector, a solid curve image, and its corresponding engineering semantic text, forming a structured multimodal dataset specifically for training visual language models.

[0105] In this embodiment, the structured descriptive text generated based on the structural characteristics of the Vision-Language Model (VLM) describes the relationship between dry density and moisture content, which first increases and then decreases. This text can be used as independent knowledge input and also to describe image information.

[0106] The construction of a visual language model-based numerical-image-text ternary multimodal fusion model for earth-rock dams, based on the numerical-image-text ternary multimodal dataset, includes:

[0107] The empirical relationship curve images and corresponding engineering semantic text in the numerical-image-text ternary multimodal dataset are embedded and encoded using a pre-trained visual language model to obtain image embedding features and text embedding features respectively.

[0108] The image embedding features and the text embedding features are fused at the feature level to generate joint multimodal features that characterize the empirical laws of earth-rock dam compaction engineering.

[0109] Based on the earth-rock dam compaction construction and material source parameters, the joint multimodal features are subjected to directional weighting processing to match the focus of the multimodal features with the current construction conditions;

[0110] When fusing the joint multimodal features with the structured numerical data, the contribution weights of different feature sources are adaptively adjusted according to the construction conditions to obtain stable multi-source fused features.

[0111] By further integrating and processing the multi-source fusion features, a numerical-image-text ternary multimodal fusion model output for earth-rock dams is generated for subsequent multi-index prediction of compaction quality.

[0112] In one specific embodiment, step 2 includes the following sub-steps:

[0113] Step S201: Based on the pre-trained visual language model, semantic alignment embedding encoding of the moisture content-dry density curve image and the engineering semantic text is achieved.

[0114] The process of embedding and encoding the empirical relationship curve images and corresponding engineering semantic text in the numerical-image-text ternary multimodal dataset using a pre-trained visual language model to obtain image embedding features and text embedding features respectively is specifically described as follows:

[0115] The image and its corresponding text are input into the VLM to form the corresponding image and text embedding. The embedding process is described as follows:

[0116] (2)

[0117] In the formula, Fig represents the moisture content-dry density relationship curve corresponding to the input features, which is a 3-channel 224-pixel × 224-pixel image after VLM model preprocessing, and VLM(·) represents converting the preprocessed image into (1, d VLM The embedding tensor VisionEmb of Fig; Text represents the corresponding text description of Fig, and VLM(·) represents converting the text into (1, d) tensor. VLM The embedded tensor TextEmb;

[0118] Step S202: Based on the multilayer perceptron, perform semantic enhancement fusion on the visual-text embedding to generate a joint multimodal representation for compaction quality modeling.

[0119] The step of fusing the image embedding features and the text embedding features at the feature level to generate joint multimodal features representing the empirical laws of earth-rock dam compaction engineering includes:

[0120] First, an image and text embedding method is used to concatenate them along the feature dimension, and then a multimodal enhancement model is used to fuse the visual and text features into a unified representation.

[0121] (3)

[0122] In the formula, Concatenate(·) represents the concatenation of visual features and text features, and the Enhancement(·) function can be understood as... The mapping process is implemented by a two-layer multilayer perceptron, MD. i This represents the fused visual text features;

[0123] Based on the earth-rock dam compaction construction and material source parameters, the joint multimodal features are subjected to directional weighting processing to match the focus of the multimodal features with the current construction conditions;

[0124] Step S203: Targeted refinement of multimodal features based on a compaction parameter-driven cross-modal attention mechanism.

[0125] The structured numerical vector composed of compaction process parameters and material source characteristic parameters obtained in step 1 is mapped to a high-dimensional space through a learnable linear projection layer, and aligned with the mixed modality dimension of visual and textual features obtained in step S202. The high-dimensional vector of compaction process parameters and material source characteristic parameters is used as the query vector, and the mixed modality is used as the key vector and value vector. This process is described as follows:

[0126] (4)

[0127] In the formula, For query vector, For key vectors, For value vectors, This means mapping the input structured numerical data to the same high-dimensional space as the visual text features. The combined visual text features. The learnable matrix to be multiplied by the query vector. The learnable matrix that is multiplied by the key vector. It is a learnable matrix that can be multiplied by a value vector;

[0128] Then, the query vector, key vector, and value vector are partitioned into four subspaces along the feature dimension, allowing each head to focus on different aspects of information. This process is described below:

[0129] (5)

[0130] In the formula, d k =d model / 4;d model The hidden layer dimension of the model;

[0131] Then, attention calculation is performed. The process is described below:

[0132] (6)

[0133] In the formula, W O The weights represent learnable weights, and MF represents the fused features. This is the normalized similarity weight matrix. This represents the result of concatenating multi-head attention calculations. For value vectors, Let be the dimension of each subspace in multi-head attention. The weight matrix is ​​a learnable matrix. This is the transpose of the key vector;

[0134] The step of adaptively adjusting the contribution weights of different feature sources according to the construction conditions when fusing the joint multimodal features and the structured numerical data to obtain stable multi-source fused features includes:

[0135] Step S204: Adaptive dynamic weighted fusion of multi-source features based on gating mechanism.

[0136] The output of step S203 is concatenated with the data obtained in step 1 to perform gating calculation. The calculation steps are as follows:

[0137] (7)

[0138] Where G=[g m , g u ], and g m +g u =1, g m Controlling the contribution weights of multimodal refined features, g u Control the preservation strength of original numerical features; For Hadamard product, it means element-wise multiplication; FF represents the features fused after gating.

[0139] Step S205: Enhance the information flow through residual connections to generate a deeply fused multimodal hybrid output.

[0140] Finally, the FF data is concatenated with the data obtained in step 1 and then subjected to a linear transformation to reduce the number of channels to half of the concatenated data. This concatenated data is then used to obtain the multimodal hybrid model output.

[0141] Information flow is enhanced through residual connections to generate a deeply fused multimodal hybrid output. The FF (Fluid Filter) is then concatenated with the structured numerical data, followed by a linear transformation to reduce the number of channels to half the concatenated number. This concatenated FF is then re-concatenated with the structured numerical data to obtain the multimodal hybrid model output. The specific calculation steps are as follows:

[0142] (8)

[0143] In the formula, MM is the output of the multimodal hybrid model, MLP represents linear transformation, FF is the fused feature after gating, and X is the input structured numerical data.

[0144] The construction of a dual-drive collaborative mapping model for dry density and permeability coefficient based on the multimodal fusion features output by the numerical-image-text ternary multimodal fusion model of the earth-rock dam includes:

[0145] Two independent dedicated task projection layers are designed for the dry density prediction task and the permeability coefficient prediction task, respectively.

[0146] For the dry density prediction task, a three-layer multilayer perceptron is used to map the shared features to the output space of the dry density prediction task.

[0147] For the permeability prediction task, the same projection structure is used, and the shared features are mapped to the output space of the permeability prediction task using parameters independent of those used for the dry density prediction task.

[0148] In this embodiment, step 3 is as follows:

[0149] Two independent task-specific projection layers were designed for the two tasks. In this embodiment, the task-specific projection layer refers to a three-layer multilayer perceptron. These two three-layer multilayer perceptrons have the same structure, differing only in their model parameters during training.

[0150] For the dry density prediction task (Task 1), a single fully connected layer maps the shared features to the output space of the dry density prediction task. The permeability coefficient prediction task (Task 2) uses the same projection structure, namely: a task-specific projection layer, which maps the shared features to the output space of the permeability coefficient prediction task using parameters independent of those in Task 1. The specific steps are as follows:

[0151] (9)

[0152] In the formula, Let represent the final predicted results for dry density and permeability coefficient, respectively. W and b represent the specific weights and biases for different prediction targets, respectively. GELU(·) represents the activation function GELU. This indicates transpose.

[0153] In this embodiment, the structure and training process of the multimodal hybrid model and multi-output prediction model of the multi-index prediction method for earth-rock dam compaction quality based on the co-evolution of visual language model and expert network are as follows: Figure 2 As shown.

[0154] A co-evolutionary training architecture for a visual language model multimodal base and a dual-drive collaborative mapping model for the dual control target of compaction quality of earth-rock dams is constructed. The numerical-image-text ternary multimodal fusion model of earth-rock dams and the dual-drive collaborative mapping model of dry density and permeability coefficient based on multi-expert network are trained in stages.

[0155] In this embodiment, step 4 is as follows:

[0156] Step S401: Train the multimodal hybrid model constructed in step 2.

[0157] The intermediate result MM obtained from the multimodal mixture model constructed in step 2 is linearly transformed, and the mean squared error (MSE) of the transformed result and the true value is calculated as the loss. The calculation process is as follows:

[0158] (10)

[0159] (11)

[0160] In the formula, Y represents the predicted value, Y=[y1, y2] represents the actual value, and MSE(·) represents the calculated MSE.

[0161] The linear projection module described in formula (10) is only used to provide auxiliary supervision signals during the pre-training stage and will be removed in the subsequent prediction stage. Its parameters will not participate in the final prediction.

[0162] Step S402: Based on task difficulty awareness, dual expert prediction head training is carried out to achieve collaborative optimization of the dual control targets of dry density and permeability coefficient.

[0163] After completing step S401, fix all network parameters of the model trained in step S401, keep the learned compaction quality sensitive representation unchanged, and only train the weights of the dual-drive collaborative mapping model of dry density and permeability coefficient based on multi-expert network. Calculate the loss for each output separately. The calculation process is as follows:

[0164] (12)

[0165] In the formula, This represents the final prediction result, where y1 and y2 represent the actual values.

[0166] The final loss is a weighted sum of the losses from the dry density prediction task and the permeability coefficient prediction task, calculated as follows:

[0167] (13)

[0168] In the formula, α is 0.6 and β is 0.4.

[0169] To verify the effectiveness of the multi-index prediction method for earth-rock dam compaction quality based on the co-evolution of visual language models and expert networks disclosed in this invention, the prediction indices of the multi-index prediction method for earth-rock dam compaction quality based on the co-evolution of visual language models and expert networks provided in an embodiment of this invention are compared. The comparison results are as follows: Figure 3 As shown, Figure 3 The middle section shows a comparison of the predicted results for dry density and permeability coefficient. From... Figure 3 As can be seen, the present invention has better predictive performance, and the specific reasons are analyzed as follows:

[0170] Current quality analysis of dam face compaction construction mainly relies on compaction parameters and some material source parameters as model inputs, failing to fully integrate the rich prior knowledge contained in various textual materials accumulated at the construction site, such as experimental reports, construction logs, and professional research papers in the field of compaction. These textual materials, in particular, contain empirical relationship curves and patterns that are crucial for guiding construction quality—such as the relationship between moisture content and dry density. These curves and patterns are essentially the condensation of long-term engineering practice and system experiments, profoundly revealing the intrinsic connections and optimal matching ranges between parameters. Introducing this structured or semi-structured domain knowledge into the analysis model will not only overcome the limitations of purely data-driven approaches but also significantly improve the accuracy, interpretability, and reliability of quality judgments, achieving intelligent analysis and control through deep integration of data and industry knowledge.

[0171] Regarding training data size, previous studies on earth-rock dam compaction quality typically used sample sizes ranging from 300 to 900 samples. Not all projects can provide sufficient and reliable training data during the field testing phase, especially small to medium-sized projects or those with tight deadlines where sufficient data may be difficult to obtain. Therefore, many researchers have attempted to reduce the training data size to achieve more efficient compaction quality assessment. The application of small datasets in compaction quality assessment is receiving increasing attention. However, few-shot learning itself presents significant challenges because the incomplete training space makes it difficult to develop a large, general-purpose model.

[0172] This invention discloses a multi-index prediction method for the compaction quality of earth-rock dams based on the co-evolution of a visual language model and an expert network. The invention utilizes a visual language model to introduce prior knowledge into the task of predicting the compaction quality of earth-rock dams, and the pre-trained visual language model avoids the retraining of a large number of parameters, enhancing the model's ability to handle small sample tasks. Furthermore, the phased training strategy designed in this invention allows the model to focus on different stages, unaffected by upstream or downstream stages. Finally, the multi-output prediction model designed in this invention achieves full utilization of the same feature by different tasks without mutual interference between tasks. Compared with traditional machine learning and deep learning models, this invention improves prediction accuracy, overcomes the difficulties of small sample prediction, and constructs a deep learning model based on a limited number of field compaction test data before construction to accurately predict the compaction quality during the subsequent full-scale construction of earth-rock dams, providing guidance for decision-making. At the same time, the model proposed in this invention also provides new ideas for the prediction of other engineering parameters and has good prospects for engineering applications.

[0173] Another embodiment of the present invention provides a computer device including a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the multi-index prediction method for earth-rock dam compaction quality based on the co-evolution of visual language model and expert network according to the present invention.

[0174] Another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor performs the multi-index prediction method for earth-rock dam compaction quality based on the co-evolution of visual language models and expert networks according to the present invention.

[0175] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multi-index prediction method for compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks, characterized in that, Includes the following steps: Collect the compaction and material source parameters of the earth-rock dam, generate the moisture content-dry density relationship curve, and generate a semantic description of the moisture content-dry density relationship curve to construct a numerical-image-text ternary multimodal dataset. Based on the numerical-image-text ternary multimodal dataset, a numerical-image-text ternary multimodal fusion model for earth-rock dams based on a visual language model is constructed. Based on the multimodal fusion features output by the numerical-image-text ternary multimodal fusion model of earth-rock dam, a dual-drive collaborative mapping model of dry density and permeability coefficient based on a multi-expert network is constructed. A co-evolutionary training architecture for a visual language model multimodal base and a dual-drive co-mapping model for controlling the compaction quality of earth-rock dams is constructed. The numerical-image-text ternary multimodal fusion model of earth-rock dams and the dual-drive co-mapping model of dry density and permeability coefficient based on a multi-expert network are trained in stages to obtain the final prediction result of the compaction quality of earth-rock dams.

2. The method for predicting the multi-index compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks according to claim 1, characterized in that, The process of collecting the compaction and material source parameters of the earth-rock dam, generating the moisture content-dry density relationship curve, generating a semantic description of the moisture content-dry density relationship curve, and constructing a numerical-image-text ternary multimodal dataset specifically includes: The compaction process parameters during the earth-rock dam compaction construction process are obtained by using on-site engineering monitoring equipment. These compaction process parameters include: number of static rolling passes, number of vibratory rolling passes, compaction thickness, and compaction speed. Soil samples are obtained and experiments are conducted on the soil samples to obtain material source characteristic parameters, including: moisture content, P5 content, non-uniformity coefficient and curvature coefficient. At the same time, the dry density and permeability coefficient, which are the characteristics of compaction quality targets, are obtained through the experiments. The compaction process parameters and the material source characteristic parameters are combined to form structured numerical data for characterizing the construction conditions. The structured numerical data constitutes the compaction construction and material source parameters of the earth-rock dam. After drying the soil sample, soil samples with different moisture contents were re-prepared and compaction tests were conducted. A moisture content-dry density relationship curve was plotted, and an engineering semantic text description of the moisture content-dry density relationship curve was generated, so that the relationship curve image and the engineering semantic text constitute a semantically aligned image-text pair. The compaction process parameters, material source characteristic parameters, and target features are standardized, and each set of structured numerical data is combined with the corresponding semantically aligned image-text pairs to construct a numerical-image-text ternary multimodal dataset containing structured numerical vectors, moisture content-dry density relationship curve images, and their engineering semantic text.

3. The method for predicting the multi-index compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks according to claim 2, is characterized in that... The standardization process for the compaction process parameters, material source characteristic parameters, and target characteristics includes: The obtained compaction process parameters, material source characteristic parameters, and target features are reversibly standardized, and the expression formula is as follows: （1） In the formula, z represents the standardized result, x represents the original value, μ represents the mean of the original data, and σ represents the variance of the original data.

4. The method for predicting the multi-index compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks according to claim 1, characterized in that, The construction of a visual language model-based numerical-image-text ternary multimodal fusion model for earth-rock dams, based on the numerical-image-text ternary multimodal dataset, includes: The empirical relationship curve images and corresponding engineering semantic text in the numerical-image-text ternary multimodal dataset are embedded and encoded using a pre-trained visual language model to obtain image embedding features and text embedding features respectively. The image embedding features and the text embedding features are fused at the feature level to generate joint multimodal features that characterize the empirical laws of earth-rock dam compaction engineering. Based on the earth-rock dam compaction construction and material source parameters, the joint multimodal features are subjected to directional weighting processing to match the focus of the multimodal features with the current construction conditions; When fusing the joint multimodal features with the structured numerical data, the contribution weights of different feature sources are adaptively adjusted according to the construction conditions to obtain stable multi-source fused features. By further integrating and processing the multi-source fusion features, a numerical-image-text ternary multimodal fusion model output for earth-rock dams is generated for subsequent multi-index prediction of compaction quality.

5. The method for predicting the multi-index compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks according to claim 1, characterized in that, The process of embedding and encoding the empirical relationship curve images and corresponding engineering semantic text in the numerical-image-text ternary multimodal dataset using a pre-trained visual language model to obtain image embedding features and text embedding features respectively is specifically described as follows: （2） In the formula, Fig represents the moisture content-dry density relationship curve corresponding to the input features, which is a 3-channel 224-pixel × 224-pixel image after VLM model preprocessing, and VLM(·) represents converting the preprocessed image into (1, d VLM The embedding tensor VisionEmb of Fig; Text represents the corresponding text description of Fig, and VLM(·) represents converting the text into (1, d) tensor. VLM The embedded tensor TextEmb; The step of fusing the image embedding features and the text embedding features at the feature level to generate joint multimodal features representing the empirical laws of earth-rock dam compaction engineering includes: An image and text embedding method is used to concatenate them along the feature dimension, and a multimodal enhancement model is used to fuse visual and text features into a unified representation. （3） In the formula, Concatenate(·) represents the concatenation of visual features and text features, and the Enhancement(·) function can be understood as... The mapping process is implemented by a two-layer multilayer perceptron, MD. i This represents the fused visual text features; Based on the earth-rock dam compaction construction and material source parameters, the joint multimodal features are subjected to directional weighting processing to match the focus of the multimodal features with the current construction conditions; Targeted refinement of multimodal features based on a compaction parameter-driven cross-modal attention mechanism: The structured numerical vector composed of the compaction process parameters and material source characteristic parameters is mapped to a high-dimensional space through a learnable linear projection layer, aligned with the mixed modality dimension in the joint multimodal features. The high-dimensional vector of the compaction process parameters and material source characteristic parameters is used as the query vector, and the mixed modality is used as the key vector and value vector, specifically including: （4） In the formula, For query vector, For key vectors, For value vectors, This means mapping the input structured numerical data to the same high-dimensional space as the visual text features. The combined visual text features. The learnable matrix to be multiplied by the query vector. The learnable matrix that is multiplied by the key vector. It is a learnable matrix that can be multiplied by a value vector; The query vector, key vector, and value vector are divided into four subspaces along the feature dimension, allowing each head to focus on different aspects of information, specifically including: （5） In the formula, d k =d model / 4;d model The hidden layer dimension of the model; The specific process of performing attention calculations is described below: （6） In the formula, W O The weights represent learnable weights, and MF represents the fused features. This is the normalized similarity weight matrix. This represents the result of concatenating multi-head attention calculations. For value vectors, Let be the dimension of each subspace in multi-head attention. The weight matrix is ​​a learnable matrix. This is the transpose of the key vector; The step of adaptively adjusting the contribution weights of different feature sources according to the construction conditions when fusing the joint multimodal features and the structured numerical data to obtain stable multi-source fused features includes: Adaptive dynamic weighted fusion of multi-source features based on gating mechanism; The joint multimodal features are concatenated with the structured numerical data, and gating calculations are performed. The calculation steps are as follows: （7） In the formula, G=[g m , g u ], and g m +g u =1, g m Controlling the contribution weights of multimodal refined features, g u Control the preservation strength of original numerical features; For Hadamard product, we have element-wise multiplication; FF represents the features fused after gating; C is the concatenated result; X is the input structured numerical data; b g The bias is used in the linear transformation; MF is the result of the multi-head attention calculation. The step of further integrating and processing the multi-source fusion features to generate a numerical-image-text ternary multimodal fusion model output for subsequent multi-index prediction of compaction quality of earth-rock dams specifically includes: Information flow is enhanced through residual connections to generate a deeply fused multimodal hybrid output. The FF (Fluid Filter) is then concatenated with the structured numerical data, followed by a linear transformation to reduce the number of channels to half the concatenated number. This concatenated FF is then re-concatenated with the structured numerical data to obtain the multimodal hybrid model output. The specific calculation steps are as follows: （8） In the formula, MM is the output of the multimodal hybrid model, MLP represents linear transformation, FF is the fused feature after gating, and X is the input structured numerical data.

6. The method for predicting the multi-index compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks according to claim 1, characterized in that, The construction of a dual-drive collaborative mapping model for dry density and permeability coefficient based on the multimodal fusion features output by the numerical-image-text ternary multimodal fusion model of the earth-rock dam includes: Two independent dedicated task projection layers are designed for the dry density prediction task and the permeability coefficient prediction task, respectively. For the dry density prediction task, a three-layer multilayer perceptron is used to map shared features to the output space of the dry density prediction task. For the permeability prediction task, the same projection structure is used, and the shared features are mapped to the output space of the permeability prediction task using parameters independent of those used for the dry density prediction task. The specific steps are as follows: （9） In the formula, Let represent the final predicted results for dry density and permeability coefficient, respectively. W and b represent the specific weights and biases for different prediction targets, respectively. GELU(·) represents the activation function GELU. This indicates transpose.

7. The method for predicting the multi-index compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks according to claim 1, characterized in that, The co-evolutionary training architecture for constructing a visual language model multimodal base and a dual-drive collaborative mapping model for dual control targets of earth-rock dam compaction quality specifically includes the following phased training of the earth-rock dam numerical-image-text ternary multimodal fusion model and the dual-drive collaborative mapping model based on multi-expert networks for dry density and permeability coefficient: Step S401: Based on the multimodal foundation pre-training with dual-index joint supervision, drive the model to learn the general representation of the compressive mass. The intermediate result MM obtained from the numerical-image-text ternary multimodal fusion model of the earth-rock dam is linearly transformed, and the mean square error between the transformed result and the true value is calculated as the loss. The specific calculation process is as follows: （10） （11） In the formula, Y represents the predicted value, Y=[y1, y2] represents the actual value, and MSE(·) represents the calculated MSE. The linear projection module described in formula (10) is only used to provide auxiliary supervision signals during the pre-training stage and will be removed in the subsequent prediction stage. Its parameters do not participate in the final prediction. Step S402: Based on task difficulty awareness, dual expert prediction head training is carried out to achieve collaborative optimization of the dual control targets of dry density and permeability coefficient. After completing step S401, fix all network parameters of the model trained in step S401, keep the learned compaction quality sensitive representation unchanged, and only train the weights of the dual-drive collaborative mapping model of dry density and permeability coefficient based on multi-expert network. Calculate the loss for each output separately. The calculation process is as follows: （12） In the formula, This represents the final prediction result, where y1 and y2 represent the actual values. The final loss is a weighted sum of the losses from the dry density prediction task and the permeability coefficient prediction task, calculated as follows: （13） In the formula, α is 0.6 and β is 0.

4.

8. A computer device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the multi-index prediction method for earth-rock dam compaction quality based on the co-evolution of visual language model and expert network according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the processor executes the multi-index prediction method for compaction quality of earth-rock dams based on the co-evolution of visual language models and expert networks according to any one of claims 1 to 7.

Images