Metal power-on weightlessness prediction method and system based on multi-modal fusion

Abstract

The invention relates to the technical field of electrochemistry, in particular to a metal power-on weightlessness prediction method and system based on multi-modal fusion, and the method comprises the steps: obtaining weightlessness experiment data of a metal sheet under different current densities and different power-on time and a corresponding surface topography image sequence, and forming a multi-modal data set; wherein the numerical time sequence data uses a first neural network to extract numerical time sequence features representing a local dependency relationship; the image sequence uses a second neural network to extract image time sequence features representing global space-time evolution; and carrying out feature interaction and fusion on the numerical time sequence features and the image time sequence features by utilizing a cross attention fusion mechanism, and outputting a metal sheet weightlessness prediction value at a specific moment in the future. A cross-modal cross attention mechanism is introduced, and a numerical rule and visual form change information are dynamically fused to form complementary and enhanced joint feature representation, so that the precision of a weightlessness prediction value is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of material electrochemistry, in particular to a metal weight loss prediction method and system based on multi-modal fusion. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] In the field of material science and electrochemistry industry, accurately predicting the corrosion rate and weight loss of metal materials under specific environments (such as power electrolysis) is crucial for evaluating material life, ensuring equipment safety, and optimizing production processes. The oxidation and dissolution of aluminum sheets under the action of electric current is a typical electrochemical corrosion process, and the evolution law of its mass (weight loss) with time is the core of the study.

[0004] Currently, the research methods for this problem mainly exist in the following two categories: Traditional experiments and empirical formula fitting methods: This method relies on a large number of repetitive experiments, by measuring the actual weight loss data of aluminum sheets under different current densities and power-on times, and using mathematical statistical methods (such as linear regression, polynomial fitting) to establish an empirical model to realize weight loss prediction. Although this method is intuitive, it has obvious limitations: first, it requires a large amount of manpower, material resources and time cost for experiments; second, the established model has poor generalization ability and is difficult to cope with complex and variable external conditions, and once the current conditions or environmental factors change, the model prediction accuracy will decrease sharply.

[0005] Single-mode time series prediction method: In recent years, with the development of deep learning, time series prediction models such as recurrent neural network (RNN), long short-term memory network (LSTM) and time convolution network (TCN) have been tried to be applied in the field of corrosion prediction. These methods usually only use numerical time series data (such as current, time series) for learning, which improves the prediction efficiency to a certain extent. However, the corrosion weight loss of metal is a typical physical and chemical process, its essence is not only related to the power-on parameters (current, time), but also closely related to the dynamic evolution of the micro-topography of the material surface (such as the generation of corrosion pits, the expansion of cracks, the change of surface roughness). The existing single-mode method cannot effectively capture and fuse the visual information of the surface topography change, leading to the model's inability to deeply understand the internal mechanism of corrosion, and the prediction ability is insufficient for the weight loss acceleration caused by the sudden change of surface morphology (such as pitting initiation), and the prediction accuracy reaches a bottleneck. SUMMARY

[0006] In order to solve the technical problems in the background art, the application provides a metal current loss weight prediction method and system based on multi-modal fusion, which realizes accurate and efficient prediction of the weight loss by constructing a multi-modal deep learning model, deeply fusing the numerical sequence of current and the image sequence of aluminum sheet surface morphology, and ultimately achieving the goal of reducing experimental dependence, saving research and development cost, and accelerating the research process of material corrosion.

[0007] In order to achieve the above-mentioned purpose, the application adopts the following technical solutions: The first aspect of the application provides a metal current loss weight prediction method based on multi-modal fusion, comprising the following steps: Obtain the weight loss experiment data of the metal sheet under different current densities and different current times and the corresponding surface morphology image sequence, form a multi-modal data set, and pre-process the data, the pre-processed data is divided into numerical time series data and image sequence; Among them, the numerical time series data uses a first neural network to extract numerical time series features representing local dependence; the image sequence uses a second neural network to extract image time series features representing global spatio-temporal evolution; The obtained numerical time series features and image time series features use a dynamic weight function and a cross-attention fusion mechanism for dynamic feature interaction and fusion to obtain fusion features; The obtained fusion features are based on a prediction module to output the metal sheet weight loss prediction value at a specific future time.

[0008] As a further technical solution, the first neural network is TCN, and the second neural network is Mamba.

[0009] As a further technical solution, the pre-processing includes: The numerical time series data in the weight loss data is denoised and normalized; The surface morphology image sequence is size standardized and denoised; The processed data is sliced according to the time window to construct a sample pair for model training.

[0010] As a further technical solution, the first neural network uses causal one-dimensional dilated convolution to extract numerical time series features, as shown in the following formula: ; ; Among them, is the dilated factor, is the convolution kernel, is the time, is the index position on the convolution kernel from 0 to , is the size of the convolution kernel, The output at time t, For input, The output is a numerical time series feature. For numerical feature dimensions.

[0011] As a further technical solution, the image sequence utilizes a second neural network to extract image temporal features representing global spatiotemporal evolution, including the following steps: The preprocessed surface morphology image is used to detect corrosion regions through a saliency detection network to obtain an updated sequence of corrosion region images. The updated sequence of eroded region images is input into a state-space model-based Mamba encoder. Through its input-dependent selective mechanism, the encoder models the dynamic changes of image features over time, outputting temporal image features. .

[0012] As a further technical solution, corrosion region detection is performed on the preprocessed surface morphology image to obtain a corrosion significance heatmap, as shown in the following formula: ; in, For the input image, Indicates position The saliency weight of the erosion at each pixel.

[0013] As a further technical solution, based on the obtained corrosion significance heatmap, a set of corrosion candidate regions is extracted using threshold segmentation and connected component analysis methods, as shown in the following formula: ; in, This indicates operations such as threshold segmentation and connected component analysis. Indicates time The extracted first Each eroded region is encoded using a shared encoder to obtain its region features, as shown in the following formula: ; Then, the features of each region are subjected to max pooling to obtain the time step. t The region aggregation features are used to form an updated sequence of eroded region images, as shown in the following formula: .

[0014] As a further technical solution, the updated erosion region image sequence is input into a Mamba encoder based on a state-space model. This encoder models the dynamic changes of image features over time through its input-dependent selective mechanism, outputting the temporal features of the image. Specifically, for each input frame image... Firstly, spatial features are extracted by a shallow convolutional network, as shown in the following formula: ; wherein, represents a convolution and nonlinear activation combination operation, is the output of the eroded image feature.

[0015] As a further technical solution, the obtained eroded image feature is flattened or embedded to obtain a serialized representation, and a Mamba encoder based on a state space model outputs an image time sequence feature , as shown in the following formula: ; ; ; wherein, represents an input image feature, represents a hidden state, represents an output feature, and the matrix is a learnable parameter, is the number of image frames, is the image feature dimension.

[0016] As a further technical solution, the obtained numerical time sequence feature and image time sequence feature utilize a cross-attention fusion mechanism and a phased dynamic weight mechanism to interact and fuse the features to obtain a fused feature, specifically: taking the numerical time sequence feature as a query vector , taking the image time sequence feature as a key vector and a value vector , the query, key and value vectors are obtained by linear transformation, and a dynamic weight function is introduced to realize feature fusion by calculating cross-attention weight, as shown in the following formula: ; ; ; ;

[0017] wherein, is the query vector, is the key vector, is the value vector, is a learnable parameter matrix, is the image time sequence feature, is the numerical time sequence feature, is the feature dimension, and are dynamic weight functions for controlling numerical modal features and image modal features, respectively, satisfying , This indicates that the change in current precedes the change in the image. Each time step This indicates a feature concatenation operation. These are the features after fusion.

[0018] As a further technical solution, during cross-attention fusion, a multi-head attention mechanism is adopted to compute multiple attention heads in parallel and concatenate their outputs, as shown in the following equation: ; Each of them , For query vectors, For key vectors, For value vectors, This is the output mapping matrix.

[0019] As a further technical solution, the prediction module is a multilayer perceptron (MLP), which maps the fused features to weightlessness prediction values ​​through nonlinear transformation, as shown in the following equation: ; in, , This is the weight matrix. For bias terms, As the activation function, the prediction result It indicates the remaining mass or percentage of weight loss of the metal sheet at a future point in time.

[0020] As a further technical solution, during the training phase, the weightlessness data of the metal sheet during the experiment was used. As a monitoring signal, the mean squared error (MSE) is used as the loss function, which is shown in the following equation: ; in, The total number of samples, This indicates the remaining mass or percentage of weight loss of the metal sheet at a future point in time. This refers to the weight loss data of the metal sheet during the experiment.

[0021] A second aspect of the present invention provides a metal weight loss prediction system based on multimodal fusion, comprising: The data acquisition module is configured to acquire the weightlessness experimental data of the metal sheet under different current densities and different energizing times, as well as the corresponding surface morphology image sequences, to form a multimodal dataset, and to perform preprocessing. The preprocessed data is divided into numerical time-series data and image sequences. Among them, numerical time-series data utilizes a first neural network to extract numerical time-series features representing local dependencies; image sequences utilize a second neural network to detect eroded regions and extract image time-series features representing global spatiotemporal evolution. The multimodal fusion module is configured to: use a dynamic weighting function and a cross-attention fusion mechanism to perform feature interaction and fusion of the obtained numerical time-series features and image time-series features to obtain fused features; The prediction output module is configured to output the predicted weight loss value of the metal sheet at a specific future time based on the obtained fused features.

[0022] A third aspect of the present invention provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the above-described method for predicting metal weight loss based on multimodal fusion.

[0023] A fourth aspect of the present invention provides an electronic device including at least one processor and a memory connected to the processor, the memory being used to store a computer program; the processor is used to execute the computer program, enabling the electronic device to implement the above-described method for predicting metal weight loss based on multimodal fusion.

[0024] Compared with existing technologies, one or more of the above technical solutions have the following beneficial effects: 1. By integrating numerical time-series data (current, time) with image time-series data (surface morphology), the limitations of single-modal data are overcome. It can simultaneously capture the driving factors of electrochemical reactions (current) and their physical manifestations (surface corrosion morphology evolution), enabling the model to deeply understand the intrinsic mechanism of the corrosion process. This allows for more accurate prediction of metal sheet weight loss, especially for complex corrosion stages.

[0025] 2. The first neural network is TCN, which is sensitive to short-term local fluctuations such as sudden current changes. The second neural network is Mamba, which is good at capturing long-term corrosion trends. The combination of the two types of networks enables the model to effectively cope with various experimental conditions such as different current densities and different energizing durations, and significantly improves the model's generalization ability and robustness under unknown working conditions.

[0026] 3. Traditional prediction models only provide results, while this approach, through a cross-attention mechanism, dynamically displays the correlation weights between numerical and image features. This provides researchers with an interpretable perspective, revealing which surface morphology features (such as corrosion pits in a specific area) play a crucial role in weight loss prediction at a particular current or time point, thus aiding in a deeper understanding of the corrosion mechanism. Simultaneously, addressing the core characteristic of electrochemical corrosion—"current change precedes morphological response"—a staged dynamic weight fusion mechanism models the time-delay relationship between numerical and image signals through a dynamic weight function. This elevates the fusion process from simple feature superposition to causal inference following physical laws. This improvement ensures the model associates the correct information at the correct time, significantly enhancing the physical rationality of the predictions and their accuracy under complex dynamic conditions.

[0027] 4. Once the model training is complete, it can quickly predict weightlessness results under new parameters without requiring numerous repetitive physical experiments. This significantly reduces material costs, labor input, and time costs, accelerates the efficiency of new material research and development, process optimization, and equipment life assessment, and provides reliable technical support for intelligent manufacturing and materials informatics.

[0028] 5. Both the TCN and Mamba architectures employed are highly efficient in sequence modeling. TCN's convolutional operations are easily parallelized, resulting in stable training; Mamba maintains powerful capabilities for long sequence modeling while having lower computational complexity than the traditional Transformer architecture. The combination of these two architectures enables this invention to achieve both high predictive performance and high computational efficiency, facilitating practical deployment and application. Attached Figure Description

[0029] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0030] Figure 1 This is a schematic diagram of a metal weight loss prediction process based on multimodal fusion provided by one or more embodiments of the present invention; Figure 2 This is a schematic diagram of a network architecture provided by one or more embodiments of the present invention; Figure 3 These are corrosion images under different working conditions provided by one or more embodiments of the present invention; Figure 4 This is a schematic diagram comparing the performance of different methods provided by one or more embodiments of the present invention on a test set. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0032] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0033] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0034] Terminology Explanation: Weight loss prediction is crucial in materials science and electrochemical industry. Metals undergo electrochemical reactions such as oxidation and dissolution under different currents, resulting in a gradual decrease in mass over time – a phenomenon known as weight loss. Predicting the corrosion rate and weight loss of metallic materials under specific environments (such as electrolysis) can help assess material lifespan, ensure equipment safety, and optimize production processes.

[0035] As described in the background section, electrochemical corrosion tests on aluminum sheets suffer from insufficient accuracy and poor adaptability in predicting weight loss. Traditional methods either rely solely on numerical data (such as current and time) and fit them using empirical formulas or simple time-series models, or analyze images at only a single time point. These methods cannot simultaneously and dynamically utilize both types of information. However, the weight loss of aluminum sheets is the result of both electrochemical reactions (driven by current) and physical morphological changes (surface corrosion). Relying solely on numerical data, models cannot grasp the evolution of surface morphologies such as corrosion pits and cracks, which significantly influence subsequent corrosion rates. Conversely, relying solely on image data makes it difficult for models to accurately quantify the cumulative effects of current and time. This "information fragmentation" prevents models from understanding the complete corrosion mechanism, thus limiting prediction accuracy.

[0036] Because the traditional model cannot capture the complex coupling relationships mentioned above, the prediction error will increase significantly when it encounters new or unseen current conditions or when corrosion enters different stages (such as from uniform corrosion to pitting corrosion).

[0037] Therefore, this proposal presents a method and system for predicting weight loss in metals under electric current based on multimodal fusion. It utilizes a TCN network to accurately extract local dependency features of numerical time series (current, time), while simultaneously employing a Mamba network to effectively capture the global spatiotemporal evolution features of corrosion image sequences. Building upon this, a cross-modal cross-attention mechanism is introduced, dynamically fusing numerical patterns and visual morphological changes based on a weight function to form a complementary and enhanced joint feature representation. Finally, a fully connected network outputs a high-precision weight loss prediction value.

[0038] TCN (Transient Convolutional Neural Network) is a convolutional neural network architecture specifically designed for time-series data modeling. Its core principle is to ensure that predictions rely solely on historical information through causal convolution, preventing future data leakage; and to exponentially expand the receptive field through dilated convolutions, enabling it to efficiently capture long-term dependencies. TCNs typically employ residual connection structures to alleviate the training challenges of deep networks. While maintaining the high parallelism and stable training advantages of convolutional networks, they possess sequence modeling capabilities comparable to or even superior to RNNs / LSTMs, and are particularly adept at extracting local patterns and short-term dynamics from sequences.

[0039] Mamba networks are a next-generation sequence modeling architecture based on the state-space model (SSM). They achieve efficient modeling of long sequences by dynamically summarizing and passing historical information through a latent state variable.

[0040] Example 1: like Figure 1 As shown, the method for predicting metal weight loss under electric current based on multimodal fusion includes the following steps: We acquired experimental data and related image information on the weight loss of metal sheets under different current densities and energizing times, and performed noise reduction, standardization, and time series slicing to form test and training samples. Numerical time-series data is input into the TCN network, and short-term local features and long-term dependencies are extracted through causal convolution and multi-dilation rate dilated convolution, and numerical feature representations are output. The temporal data of the image is input into the spatial attention module to extract the eroded region, and then input into the Mamba network. The state space modeling structure is used to capture the spatial features of the surface corrosion morphology and the dynamic law of its evolution over time, so as to obtain the image feature representation. By fusing the numerical features output by TCN with the image features output by Mamba through a cross-attention mechanism and dynamically assigning weights, deep interaction of multimodal features in the spatiotemporal dimension is achieved.

[0041] The fused features are input into a multilayer perceptron, which outputs the predicted weight loss of the aluminum sheet at future time points. The model is trained and optimized using real experimental data as a supervision signal and the mean square error loss function.

[0042] This embodiment uses the weight loss experimental data and related image information of aluminum sheets as an example to explain the process of predicting metal weight loss based on multimodal fusion.

[0043] Step 1: Obtain experimental data and related image information of the weight loss of aluminum sheets under different current densities and energizing times, and perform noise reduction, standardization, and time series slicing to form test and training samples.

[0044] Step 2: Input the preprocessed time series data into the TCN network, and extract local temporal dependency features through convolution and residual connections.

[0045] Step 3: The input image sequence is processed by a spatial attention module to extract eroded regions, and then input into a Mamba encoder. The global dependencies are modeled through a self-attention mechanism to obtain global feature representations.

[0046] Step 4: Using the temporal dependency features and global features obtained in Step 2 and Step 3 as input, not only is a cross-attention mechanism constructed and attention weights calculated, but a phased weight allocation mechanism is also introduced to achieve the fusion of spatiotemporal features.

[0047] Step 5: Input the fused features obtained in Step 4 into the fully connected prediction layer to output the weight loss prediction values ​​of the aluminum sheet under different current and time conditions.

[0048] As a further implementation method, in step 1, under multiple current density ranges, such as 0.5A, 1A, 2A, 5A, etc., an energizing experiment is conducted on the aluminum sheet, and the mass change of the aluminum sheet under different energizing times is recorded to obtain a triplet dataset of current, time, and weight loss, while acquiring the corresponding surface morphology image; then, moving average and low-pass filter are used to remove experimental noise from the original data, and the denoised data is normalized by Min-Max.

[0049] As a further implementation method, such as Figure 2 As shown, in step 2, the data obtained in step 1 is input into the temporal feature extraction module (TCN network) for feature extraction. Unlike traditional recurrent neural networks, TCN uses one-dimensional convolution operations to process sequential data. Its core includes causal convolution and dilated convolution, which can ensure that prediction does not rely on future information while expanding the receptive field to capture the cumulative effect of time. The input is... The convolution kernel is f Then the definition of dilated convolution is: ; in As the expansion factor, For convolution kernel, For time, The index positions on the convolution kernel range from 0 to , The size of the convolution kernel. The output at time t, For input, The output is a numerical time series feature. For numerical feature dimensions, by adjusting The model can expand the receptive field while keeping the convolution kernel size fixed, thus enabling modeling of long-term dependencies.

[0050] The residual connections in TCNs ensure the trainability of deep networks and avoid the vanishing or exploding gradient problems. The final numerical time-series features can be represented as: ; in This is the numerical feature dimension. This feature vector effectively captures the dynamic law of current change over time and its potential correlation with the weightlessness trend.

[0051] As a further implementation, in step 3, during the image sequence modeling stage, this embodiment first introduces an erosion region detection mechanism, using a saliency detection network to analyze the input image. The image was processed to obtain a heatmap showing the salience of corrosion: ; in Indicates position The saliency weight of the erosion at each pixel.

[0052] Furthermore, a set of candidate erosion regions is extracted using threshold segmentation and connected component analysis methods: ; in This indicates operations such as threshold segmentation and connected component analysis. Indicates time The extracted first Each eroded region is encoded using a shared encoder to obtain its region features: .

[0053] Subsequently, the features of each region are fused (e.g., by weighted averaging or max pooling) to obtain the aggregated regional features at time t: ; After obtaining the new image sequence, a Mamba network is used to process the image sequence of the aluminum sheet surface at different time points. For each input frame image... First, spatial features are extracted using a shallow convolutional network: ; in This represents a combination of convolution and non-linear activation operations, outputting... This process can capture basic texture information of surface morphology, such as pits, cracks, and surface roughness.

[0054] Subsequently, these features are flattened or embedded to obtain a serialized representation: .

[0055] To model the dynamic changes of images over time, this embodiment introduces the Mamba network. Mamba is based on state-space modeling (SSM), and its computation process can be formally represented as follows: ; in Represents the features of the input image. This indicates a hidden state. Represents the output features, matrix These are learnable parameters.

[0056] Compared to the Transformer architecture, Mamba has lower computational complexity in long sequence modeling while possessing excellent temporal capture capabilities. After processing through multiple layers of Mamba encoders, image feature representations containing spatiotemporal dependencies are obtained: ; in For the number of image frames, This is the image feature dimension. This feature vector preserves both the surface spatial texture information and incorporates the corrosion trend that evolves over time.

[0057] As a further implementation, in step 4, numerical features and image features belong to two different modalities, and direct concatenation makes it difficult to reflect the dependency relationship between them. Therefore, this embodiment introduces a fusion module based on a cross-attention mechanism and a phased dynamic weighting mechanism. First, a linear transformation is used to obtain the query vector, key vector, and value vector as follows: ; ; ; in For learnable parameter matrix, This is the final output numerical time series feature.

[0058] The formula for calculating cross-attention is: ; in, For query vectors, For key vectors, It is a value vector.

[0059] This mechanism allows numerical features to selectively focus on the image regions most relevant to their changes, thereby achieving effective coupling between current-time patterns and surface morphology evolution. During the electrochemical corrosion of aluminum sheets, current changes occur first, followed by image changes; current features are prominent in the early stages of corrosion, while image features become more prominent in the later stages. This indicates that the change in current precedes the change in the image. At each time step, during the fusion phase, this embodiment further introduces a dynamic weighting function. and ,satisfy: .

[0060] The fused features are represented as follows: ; in This indicates a feature concatenation operation. This indicates that the change in current precedes the change in the image. Each time step This indicates a feature concatenation operation. Controlling numerical modal characteristics, To control the modal characteristics of the image, based on the physical laws of electrochemical corrosion, the current signal is more sensitive to the reaction in the early stages of corrosion; therefore, the following settings are used. In the later stages of corrosion, the macroscopic corrosion morphology gradually becomes apparent, thus gradually increasing... The proportion of this feature, which includes both numerical patterns and image evolution information, significantly enhances the model's expressive power.

[0061] Furthermore, this embodiment introduces a multi-head mechanism in the cross-attention module, that is, it computes multiple attention weights in parallel and performs weighted fusion, thereby enhancing the model's learning ability. Formalized as: ; Each of them This is the output mapping matrix.

[0062] As a further implementation, in step 5, features are fused. The input is fed into the prediction module, which consists of a multilayer perceptron (MLP) that uses nonlinear mapping to transform high-dimensional features into weightlessness predictions for future timeframes. Let the fused features be... The predicted output is: ; in , This is the weight matrix. For bias terms, As the activation function, the prediction result It can represent the remaining mass or percentage of weight loss of the aluminum sheet at a future point in time.

[0063] During model training, real experimental data are used. As a monitoring signal, mean squared error (MSE) is used as the loss function: ; in The total number of samples.

[0064] To train and evaluate the proposed aluminum sheet energized weight loss prediction method based on Mamba-TCN fusion, this embodiment uses corrosion image data captured in an independent experiment. The dataset contains 24 sets of samples, each consisting of a corrosion image, energizing time, current, and the corresponding actual weight loss.

[0065] The first 20 sets of samples were used for model training, and the remaining 4 sets were used for model testing. During training, the model was trained iteratively for 200 epochs. The AdamW optimization algorithm was used, combined with a learning rate adjustment strategy to ensure the stability and convergence of the training process. To prevent overfitting, this embodiment introduced Dropout and L2 regularization during the training phase. All training was completed on an NVIDIA GeForce 4090 graphics card.

[0066] Compared to existing technologies, this paper proposes a method for predicting weight loss in aluminum sheets using causal convolution and multi-diffraction convolution within a Temporal Convolutional Network (TCN). This method accurately captures the short-term local weight loss fluctuations of aluminum sheets under different energizing currents, and is particularly sensitive to rapid weight loss changes at the initial stage of energization or during sudden current changes, significantly improving prediction accuracy in such scenarios. Simultaneously, it introduces the next-generation, efficient sequence modeling structure Mamba, leveraging its state-space modeling characteristics to model long-term dependencies while maintaining low computational complexity. This allows for a clear characterization of the weight loss of aluminum sheets over time. This approach identifies the overall weight loss trend of the material, avoiding the trend prediction bias caused by neglecting long-term correlations in traditional methods. Furthermore, it introduces a cross-attention fusion module, achieving deep interaction between local TCN features and global Mamba temporal features through a dynamic weight allocation mechanism. This fully integrates spatiotemporal information, significantly enhancing the model's ability to express complex weight loss patterns. This solution can not only be directly applied to the electrochemical weight loss prediction of aluminum sheets but can also be extended to the corrosion resistance research of other metallic materials. Simultaneously, it provides technical support for scenarios such as service life prediction and maintenance cycle optimization of metal components in intelligent manufacturing, demonstrating a very broad application prospect.

[0067] To verify the effectiveness of the technical framework in this embodiment under small sample scenarios, an industrial electrolytic aluminum sheet (75mm×10mm×2mm) was used as the experimental object and placed in a 3.5% NaCl solution. Multiple groups covering various current densities (I: 0-4A / mm) were selected. 2 Experimental samples with energizing time t: 0-960h, with one sample per operating condition as the basic data. The samples include RGB corrosion images (e.g., Figure 3 As shown), operating parameters (I and t) and actual weight loss (calculated using an electronic analytical balance with an accuracy of 0.01 mg), for example Figure 1 As shown, from left to right, the diagrams depict the states of an aluminum sheet with a current of 2A and increasing energizing time. Additionally, 20% of the intermediate operating condition samples that were not used in the training were selected as the test set.

[0068] To address the coexistence of short-term rapid fluctuations and long-term trend dependence in the weight loss process of aluminum sheets, the number of dilated convolutional layers in the Temporal Convolutional Network (TCN) is set to 4 in the numerical modal modeling, with a receptive field covering 200 time steps to ensure sensitivity to short-term weight loss abrupt changes under different current conditions. In the image modal modeling, a Mamba state-space network with a hidden dimension d=256 is introduced to efficiently capture the long-term sequence patterns of corrosion morphology evolution over time. In the cross-modal feature fusion stage, the cross-attention module adopts an 8-head mechanism (Multi-Head). The first stage performs global feature alignment with a learning rate of 0.001, and the second stage performs fine-grained feature enhancement on weakly corroded areas with a learning rate of 0.0005. During training, mean squared error (MSE) is used as the main loss function, with a monotonic constraint coefficient λ=0.2 to penalize cases where the predicted curve does not conform to physical laws (e.g., reverse weight loss growth), thereby avoiding logical inconsistencies in prediction.

[0069] The verification results are as follows Figure 4 As shown, after 200 rounds of training iterations, the model's predicted curve on the test set closely matches the actual weightlessness curve, with the mean absolute error (MAE) remaining at 1.25 g / cm³. 2 Within, the coefficient of determination (R) 2 The MAE reached 0.86, significantly outperforming traditional methods. Comparing the three schemes, the MAE on the test set using only numerical modal analysis (single TCN) was approximately 4.92 g / cm³. 2 R 2 The MAE was 0.48; the MAE for "using only the image modality (single Mamba)" decreased to 2.71 g / cm³. 2 R 2 The MAE was increased to 0.72; while the MAE of "cross-attention fusion of numerical mode and image mode (method in this embodiment)" was further reduced to approximately 1.25 g / cm³. 2 R 2The value was increased to 0.86, which fully demonstrates the advantages of this embodiment in simultaneously capturing short-term fluctuations and long-term trends.

[0070] The core of this solution lies in constructing a multimodal, end-to-end time-series prediction model. Its features and advantages do not stem from a simple accumulation of single technologies, but rather from the organic synergy and complementarity between different components. It fundamentally changes the traditional single-modal prediction paradigm, placing numerical sequences (current / time) and visual sequences (surface morphology) on an equal footing for the first time. This not only increases the data dimension but also provides a more complete digital mapping of the electrochemical corrosion physical process.

[0071] Traditional methods, whether relying solely on numerical fitting or static image analysis, fail to establish a dynamic relationship between cause (current) and effect (morphological change). This approach, through data-level fusion, models the process mechanism. TCN's causal convolution ensures the model strictly adheres to temporal order, predicting only based on historical information, thus conforming to physical laws. By adjusting the dilation factor, the model can simultaneously capture the rapid initial response and the long-term cumulative effect, exhibiting high sensitivity to instantaneous events such as sudden current changes. Compared to RNN / LSTM, TCN offers better parallelism and gradient stability, resulting in faster and more reliable training.

[0072] Mamba is based on a state-space model (SSM), whose computational complexity increases linearly with sequence length. This overcomes the quadratic complexity bottleneck faced by Transformers with ultra-long image sequences, making it more efficient for handling long-term experimental data. Mamba's core advantage lies in its input-dependent selectivity mechanism, which dynamically decides which historical information to remember and which to ignore based on the current input image content. For example, when an image first shows an erosion pit, the model pays close attention to and remembers this key change, while efficiently filtering out frames with no significant changes. This allows it to more intelligently capture key turning points in the erosion process.

[0073] Instead of simple feature concatenation or addition, this approach uses a more advanced cross-attention mechanism. This mechanism allows one modality (numerical data) to "query" the most relevant information from another modality (image).

[0074] For example, when the model processes the numerical feature of "sudden increase in current," it can automatically focus attention weights on areas in the image sequence where "new corrosion pits are generated" or "existing cracks expand" subsequently. This dynamic, content-aware feature fusion method greatly enhances the model's ability to understand causal relationships and is key to achieving high-precision predictions.

[0075] Because the model simultaneously perceives both "driving factors" and "morphological outcomes," its predictions are based on a deep approximation of physicochemical processes, rather than simple data fitting. Therefore, it achieves significantly higher accuracy than traditional methods when predicting complex nonlinear processes, such as the sudden acceleration of pitting corrosion.

[0076] The model learns the intrinsic relationship between current, time, and morphological changes, rather than a mapping to a specific experimental setup. Therefore, it maintains good predictive ability when applied to new current densities or longer time scales not covered by the training data, overcoming the fatal weakness of poor adaptability of traditional empirical formulas.

[0077] Once the model is fully trained, it can be used as a "digital twin" tool to quickly simulate corrosion results under different parameters on a computer, greatly reducing expensive repetitive experiments and saving material, time and labor costs.

[0078] In the development of new materials or the optimization of processes, this model can be used to conduct a large number of "virtual experiments" to quickly screen out promising solutions, and then verify them through a small number of physical experiments, thereby greatly shortening the research and development cycle.

[0079] Cross-attention weight matrices can serve as an interpretable tool, allowing researchers to retrospectively analyze which time points of current data and which regions of images the model prioritized when making specific predictions. This has the potential to help uncover previously under-understood corrosion mechanisms, providing not only predictive results but also scientific insights.

[0080] Furthermore, the proposed "TCN+Mamba+Cross-Attention" multimodal time-series prediction framework has universality. It can be extended to corrosion prediction of other metallic materials (such as steel, copper, and magnesium alloys), and even to other similar industrial process prediction scenarios that simultaneously involve parameter time-series and visual morphological changes, such as battery state of health (SOH) monitoring, material fatigue crack growth prediction, and chemical deposition process monitoring, showing broad application prospects.

[0081] Example 2: A multimodal fusion-based system for predicting weight loss in metals under electric current includes: The data acquisition module is configured to acquire the weightlessness experimental data of the metal sheet under different current densities and different energizing times, as well as the corresponding surface morphology image sequences, to form a multimodal dataset, and to perform preprocessing. The preprocessed data is divided into numerical time-series data and image sequences. Among them, numerical time-series data utilizes a first neural network to extract numerical time-series features representing local dependencies; image sequences utilize a second neural network to extract image time-series features representing global spatiotemporal evolution. The multimodal fusion module is configured to: use a dynamic weighting function and a cross-attention fusion mechanism to perform feature interaction and fusion of the obtained numerical time-series features and image time-series features to obtain fused features; The prediction output module is configured to output the predicted weight loss value of the metal sheet at a specific future time based on the obtained fused features.

[0082] By constructing a multimodal deep learning model and deeply fusing numerical sequences of energized current / time with image sequences of aluminum sheet surface morphology, accurate and efficient prediction of weightlessness can be achieved. Ultimately, this aims to reduce reliance on experiments, save R&D costs, and accelerate the research process on material corrosion.

[0083] Example 3: A computer program product includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the aforementioned method for predicting metal weight loss upon electrical conduction based on multimodal fusion.

[0084] Example 4: An electronic device includes at least one processor and a memory connected to the processor, the memory being used to store a computer program; the processor is used to execute the computer program, enabling the electronic device to implement the above-described method for predicting metal weight loss based on multimodal fusion.

[0085] Example 5: A computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the aforementioned method for predicting metal weight loss based on multimodal fusion.

[0086] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for predicting metal weight loss under electric current based on multimodal fusion, characterized in that, Includes the following steps: We acquired weightlessness experimental data and corresponding surface morphology image sequences of metal sheets under different current densities and different energizing times to form a multimodal dataset. The preprocessed data was divided into numerical time-series data and image sequences. Among them, numerical time-series data utilizes a first neural network to extract numerical time-series features representing local dependencies; image sequences utilize a second neural network to extract image time-series features representing global spatiotemporal evolution. The obtained numerical time-series features and image time-series features are used to perform feature interaction and fusion using a dynamic weighting function and a cross-attention fusion mechanism to obtain fused features; The obtained fusion features are based on the prediction module, which outputs the predicted weight loss value of the metal sheet at a specific future time.

2. The method for predicting metal weight loss based on multimodal fusion as described in claim 1, characterized in that, The first neural network is TCN, and the second neural network is Mamba.

3. The method for predicting metal weight loss based on multimodal fusion as described in claim 1, characterized in that, The metal sheet is an aluminum sheet.

4. The method for predicting metal weight loss based on multimodal fusion as described in claim 1, characterized in that, Preprocessing includes: Denoising and normalization are performed on the numerical time-series data in the weightlessness data; The surface topography image sequence was subjected to size normalization and noise reduction. The processed data is sliced ​​according to time windows to construct sample pairs for model training.

5. The method for predicting metal weight loss based on multimodal fusion as described in claim 1, characterized in that, The first neural network uses causal one-dimensional dilated convolution to extract numerical time-series features, as shown in the following equation: ； ； Among them, among them, As the expansion factor, f For convolution kernel, For time, The index positions on the convolution kernel range from 0 to , The size of the convolution kernel. The output at time t, For input, The output is a numerical time series feature. For numerical feature dimensions.

6. The method for predicting metal weight loss based on multimodal fusion as described in claim 1, characterized in that, The image sequence utilizes a second neural network to extract temporal features representing global spatiotemporal evolution, including the following steps: The preprocessed surface morphology image is used to detect corrosion regions through a saliency detection network to obtain an updated sequence of corrosion region images. The updated image feature sequence is input into a state-space model-based Mamba encoder, which models the dynamic changes of image features in the time dimension through its input-dependent selectivity mechanism, and outputs image temporal features. .

7. The method for predicting metal weight loss based on multimodal fusion as described in claim 6, characterized in that, The preprocessed surface morphology image is subjected to corrosion region detection through a saliency detection network to obtain an updated corrosion region image sequence. Specifically, corrosion region detection is performed on the preprocessed surface morphology image. Based on the obtained corrosion saliency heatmap, a set of corrosion candidate regions is extracted using threshold segmentation and connected component analysis methods. Each region is encoded using a shared encoder to obtain region features. Max pooling is then performed on each region feature to obtain the region aggregation features at time t, forming an updated corrosion region image sequence.

8. The method for predicting metal weight loss based on multimodal fusion as described in claim 6, characterized in that, The updated image feature sequence is input into the Mamba encoder for spatial feature extraction, resulting in the eroded image features, as shown in the following equation: ； in, For the input image, This represents a combination of convolution and nonlinear activation operations. The output erosion image features.

9. The method for predicting metal weight loss based on multimodal fusion as described in claim 8, characterized in that, The obtained erosion image features are flattened or embedded to obtain a serialized representation. A Mamba encoder based on a state-space model outputs the temporal features of the image. As shown in the following formula: ； ； ； in, Represents the features of the input image. This indicates a hidden state. Represents the output features, matrix For learnable parameters, For the number of image frames, Dimensions represent image features.

10. The method for predicting metal weight loss based on multimodal fusion as described in claim 1, characterized in that, The obtained numerical time-series features and image time-series features are fused using a cross-attention fusion mechanism and a staged dynamic weighting mechanism to obtain fused features. Specifically, the numerical time-series features are used as the query vector. Using image temporal features as key vectors Sum value vector The query, key, and value vectors are obtained through linear transformation. A dynamic weight function is introduced to achieve feature fusion by calculating the cross-attention weights, as shown in the following formula: ； ； ； ； ； in, For query vectors, For key vectors, For value vectors, For learnable parameter matrix, For image temporal features, It is a numerical time series feature. For feature dimension, This indicates that the change in current precedes the change in the image. Each time step This indicates a feature concatenation operation. Controlling numerical modal characteristics, Controlling image modal features, This indicates a feature concatenation operation. These are the features after fusion.

11. The method for predicting metal weight loss based on multimodal fusion as described in claim 1, characterized in that, During cross-attention fusion, a multi-head attention mechanism is employed, which computes multiple attention heads in parallel and concatenates their outputs, as shown in the following equation: ； Each of them , For query vectors, For key vectors, For value vectors, This is the output mapping matrix.

12. The method for predicting metal weight loss based on multimodal fusion as described in claim 1, characterized in that, The prediction module is a multilayer perceptron (MLP), which maps the fused features to weightlessness prediction values ​​through nonlinear transformation, as shown in the following equation: ； in, , This is the weight matrix. For bias terms, The activation function is used to predict the result. It indicates the remaining mass or percentage of weight loss of the metal sheet at a future point in time.

13. A metal electrostatic weight loss prediction system implementing the method of any one of claims 1-12, characterized in that, include: The data acquisition module is configured to: acquire the weightlessness experimental data of the metal sheet under different current densities and different energizing times and the corresponding surface morphology image sequence, form a multimodal dataset, and perform preprocessing. The preprocessed data is divided into numerical time series data and image sequences. Among them, numerical time-series data utilizes a first neural network to extract numerical time-series features representing local dependencies; image sequences utilize a second neural network to extract image time-series features representing global spatiotemporal evolution. The multimodal fusion module is configured to: use a dynamic weighting function and a cross-attention fusion mechanism to perform feature interaction and fusion of the obtained numerical time-series features and image time-series features to obtain fused features; The prediction output module is configured to output the predicted weight loss value of the metal sheet at a specific future time based on the obtained fused features.

14. A computer program product, characterized in that, Includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to perform the steps in the multimodal fusion-based metal electrostatic weight loss prediction method as described in any one of claims 1-12.

15. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, the memory being used to store a computer program; the processor is used to execute the computer program, enabling the electronic device to perform the steps in the metal electrostatic weight loss prediction method based on multimodal fusion as described in any one of claims 1-12.

Images