Telescopic device deformation prediction and control system based on multi-modal fusion deep learning

Abstract

The invention discloses a telescopic device deformation prediction and control system based on multi-modal fusion deep learning, and relates to the technical field of mechanical engineering. The system comprises a data acquisition module, a preprocessing module, a data fusion module, a deep learning prediction module, a control decision module and a feedback optimization module. The data acquisition module acquires multi-modal data by using a distributed sensor array; the preprocessing module processes the data to ensure the quality; the data fusion module fuses the data to generate comprehensive data; the deep learning prediction module outputs a predicted deformation value by means of an LSTM and CNN fusion architecture; the control decision module performs hierarchical control according to a comparison result of a predicted value and a safety threshold and comprises a fuzzy control unit optimization instruction; and the feedback optimization module optimizes model parameters and control strategies online by using a reinforcement learning algorithm, so that high-precision prediction and self-adaptive control of deformation of the telescopic device are realized, and the structural safety and the system intelligence level are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mechanical engineering, in particular to a telescopic device deformation prediction and control system based on multi-modal fusion deep learning. BACKGROUND

[0002] With the development of China's economy, infrastructure construction has developed rapidly, especially large-scale transportation facilities such as tunnels, bridges, and highways have greatly improved in quantity and scale. However, these large-scale transportation facilities often need to withstand huge loads and complex external environmental conditions, so they may have varying degrees of deformation and damage during use, and even cause structural failure or safety accidents. In order to ensure the safe operation and service life of transportation facilities, it is necessary to monitor and predict their deformation behavior in order to timely detect potential problems and take effective preventive measures.

[0003] Currently, deformation monitoring of large-scale transportation facilities mainly relies on traditional measurement techniques and manual observation, which has many shortcomings: (1) Traditional measurement methods can only provide information for local areas and cannot fully reflect the deformation of the entire structure; (2) Manual observation is easily affected by subjective factors and is difficult to achieve continuous, accurate recording and analysis of large amounts of data; (3) Existing monitoring equipment and techniques have high costs and are difficult to maintain, making them difficult to promote.

[0004] Therefore, there is an urgent need to develop a new deformation monitoring and prediction scheme that can overcome the shortcomings of existing technology. SUMMARY

[0005] To solve the above problems, the present application provides a telescopic device deformation prediction and control system based on multi-modal fusion deep learning, which aims to achieve full-cycle high-precision prediction and adaptive active control of telescopic device deformation, and improve the structural safety and system intelligence level.

[0006] The telescopic device deformation prediction and control system based on multi-modal fusion deep learning provided by the present application includes the following modules: A data acquisition module for acquiring multi-modal data of the telescopic device, including temperature data, strain data, displacement data, pressure data, and working environment data; A preprocessing module for cleaning, filtering, standardizing, missing value filling, and spatiotemporal alignment processing of the multi-modal data; A data fusion module for weighted linear fusion and nonlinear feature extraction of the preprocessed multi-modal data to generate integrated data after fusion; The deep learning prediction module adopts a fusion architecture based on a long short-term memory network and a convolutional neural network, takes the comprehensive data as input, and outputs a predicted deformation value; The deformation value correction module is configured to fuse the predicted deformation value with a theoretical deformation value, and output a corrected high-precision predicted deformation value. The control decision module is configured to compare the high-precision predicted deformation value with a preset deformation safety threshold, and generate a control instruction according to a comparison result. The feedback optimization module is configured to collect actual deformation data of the telescopic device, calculate an error between the actual deformation data and the high-precision predicted deformation value, and perform online optimization on model parameters of the deep learning prediction module and a control strategy of the control decision module based on the error by using a reinforcement learning algorithm.

[0007] Further, the data collection module specifically includes: A distributed sensor array composed of temperature sensors, strain gauges, laser displacement sensors, pressure sensors, and environmental sensors arranged at key measurement points of the telescopic device, for synchronously collecting the multi-modal data. A dynamic sampling unit configured to receive instructions from the control decision module or the feedback optimization module, and dynamically adjust sampling frequencies of the sensors according to real-time running states or predicted requirements of the telescopic device. A data buffering unit configured to locally buffer, time-stamp, and packet-frame the collected raw data at the sensor nodes, and send the data to the preprocessing module through a unified communication protocol.

[0008] By using the distributed sensor array and the dynamic sampling mechanism, high-synchronism and high-integrity collection of the multi-modal data is achieved, and the sampling frequencies are adaptively adjusted according to the system states, thereby significantly reducing system energy consumption and data transmission load while ensuring data quality.

[0009] Further, the preprocessing module specifically includes: A data cleaning unit configured to remove outliers in the multi-modal data by using a 3σ criterion. A data filtering unit configured to remove impulse noise in the data by using median filtering, and to smooth the filtered data by using Kalman filtering. A missing value filling unit configured to fill missing values by using linear interpolation or K-nearest neighbor algorithm. A standardization processing unit configured to normalize all the multi-modal data by using a Z-score standardization formula. A space-time alignment unit configured to align time stamps of temperature, strain, pressure, and working environment data to sampling time points of displacement data by linear interpolation, with the time stamps of the displacement data as a reference.

[0010] Through the multi-stage preprocessing process, the noise and outliers in the original data are effectively eliminated, the missing and spatio-temporal inconsistency problems of multi-source heterogeneous data are solved, and a high-quality and high-consistency data basis is provided for subsequent fusion and prediction.

[0011] Further, the data fusion module specifically includes: a linear fusion unit configured to calculate the weight of each data source based on the Pearson correlation coefficient and perform weighted fusion; a nonlinear fusion unit configured to perform dimensionality reduction and feature extraction on nonlinear data based on kernel principal component analysis or autoencoder; an adaptive adjustment unit configured to dynamically adjust the weight ratio of linear and nonlinear fusion according to the Spearman correlation coefficient.

[0012] Through the adaptive fusion strategy combining linear and nonlinear fusion, the complementarity and redundancy among multi-modal data are fully tapped, the physical meaning of the data is retained, the deep features are extracted, and a more informative comprehensive data representation is generated.

[0013] Further, the deep learning prediction module specifically includes: a time series feature extraction unit configured to extract long-period time series dependency features in the comprehensive data using a multi-layer long short-term memory network; a spatial feature extraction unit configured to extract local spatial features and cross-modal correlation features in the comprehensive data using a one-dimensional convolutional neural network; a feature fusion unit configured to adaptively weight fuse the time series features output by the long short-term memory network and the spatial features output by the convolutional neural network through a gated attention mechanism to generate a joint feature vector; a prediction output unit configured to nonlinearly map the joint feature vector using a feedforward neural network to output a future multi-step deformation prediction sequence.

[0014] Through the fusion architecture of LSTM and CNN combined with the gated attention mechanism, the long-period time series regularity and local spatial features in the deformation data are simultaneously captured, and high-precision prediction of the future multi-step deformation sequence is realized.

[0015] Further, the deformation value correction module includes: a finite element simulation unit configured to construct a parameterized finite element model based on the geometry and material properties of the stretching device, and use real-time collected multi-modal data as input boundary conditions to calculate the theoretical deformation value; a synchronous driving unit configured to map the real-time data processed by the preprocessing module as input parameters of the finite element model to drive it to perform synchronous simulation; an error correction unit configured to calculate a first trust weight of the predicted deformation value and a second trust weight of the theoretical deformation value based on an uncertainty measure of the predicted deformation value and the theoretical deformation value and a KL divergence of an error distribution of the predicted deformation value and the theoretical deformation value relative to an ideal distribution, and to perform weighted fusion of the predicted deformation value and the theoretical deformation value according to the first trust weight and the second trust weight to generate a corrected high-precision predicted deformation value; a model updating unit configured to receive an error output by the feedback optimization module and to perform inverse fine-tuning of material properties or boundary condition parameters in the finite element model unit based on the error.

[0016] The module takes advantage of the strong extrapolation ability of the mechanism model under unknown working conditions, making up for the defects of the pure data-driven model that may be inaccurate in the prediction outside the training data coverage range. Through the fusion of data-driven and mechanism simulation, the physical rationality and generalization ability of the system output are significantly enhanced. The model updating unit enables the digital twin to continuously evolve and track the performance degradation of the physical entity, ensuring the long-term effectiveness of the correction effect, forming a self-optimizing prediction system.

[0017] Further, the error correction unit is specifically configured to: calculate a relative difference degree between the predicted deformation value and the theoretical deformation value; calculate an uncertainty measure of the predicted deformation value and the theoretical deformation value; use a kernel density estimation method to construct a probability distribution of the predicted error and a probability distribution of the theoretical error, respectively; define an ideal error distribution and calculate a KL divergence between the probability distribution of the predicted error and the probability distribution of the theoretical error and the ideal error distribution; calculate a first trust weight of the predicted deformation value and a second trust weight of the theoretical deformation value according to the uncertainty measure and the KL divergence; perform weighted fusion of the predicted deformation value and the theoretical deformation value according to the first trust weight and the second trust weight to generate the final corrected high-precision predicted deformation value; When it is monitored that the relative difference degree continuously exceeds a preset alarm threshold, a signal is triggered to the feedback optimization module to request reinforcement learning intervention or retraining of the deep learning prediction module.

[0018] By introducing a dynamic trust weight algorithm based on uncertainty quantification and KL divergence, the traditional fixed weight fusion strategy is replaced, and an adaptive fusion mechanism that can quantitatively evaluate the output credibility of data-driven models and mechanism models in real time is constructed. This mechanism enables the system to not only evaluate the long-term performance of the model (through KL divergence evaluation), but also combine its instantaneous output reliability (through uncertainty measurement evaluation) to dynamically adjust the fusion weight at a finer granularity. This ensures that in common scenarios where data quality is high and working conditions are familiar, the prediction accuracy of deep learning models is fully utilized, while in extreme or unseen working conditions with high data uncertainty, the dominant role of finite element models based on physical laws is automatically enhanced, thereby significantly improving the accuracy, robustness and generalization ability of the fusion results.

[0019] Further, the control decision module is specifically configured to: When the high-precision predicted deformation value is less than the preset deformation safety threshold, the operating parameters of the current drive motor speed and hydraulic system pressure of the telescopic device are kept unchanged, and the data acquisition module is triggered to continuously acquire multi-modal data at the original frequency, and the real-time data is synchronized to the preprocessing module to maintain the continuity of monitoring; When the high-precision predicted deformation value is greater than or equal to the preset deformation safety threshold, based on the real-time pressure data and displacement data output by the preprocessing module, a hierarchical control measure is taken to prevent excessive deformation: if the high-precision predicted deformation value is 1.0-1.2 times the preset deformation safety threshold, the drive motor speed of the telescopic device is adjusted, and the hydraulic system pressure is simultaneously changed; if the high-precision predicted deformation value exceeds 1.2 times the preset deformation safety threshold, in addition to the above-mentioned motor and hydraulic adjustment, an alarm signal is also triggered, and the telescopic action of the telescopic device is suspended until the actual deformation data falls below the safety threshold.

[0020] Through the hierarchical early warning and control mechanism, a differentiated control strategy is adopted according to the risk level, which not only ensures the normal operation of the system within the safety range, but also actively intervenes in time when the risk increases, effectively preventing excessive deformation and equipment damage.

[0021] Further, the control decision module further includes a fuzzy control unit, and the fuzzy control unit is configured to: When the high-precision predicted deformation value approaches the boundary of the preset deformation safety threshold, the error data output by the feedback optimization module is called to dynamically adjust the membership function of fuzzy logic reasoning; Expert experience knowledge of the telescopic device is combined to construct a fuzzy rule base; Based on the expert experience knowledge of fuzzy logic reasoning, smooth control instructions are generated, and the fuzzy control parameters are simultaneously fed back to the feedback optimization model as sample data for subsequent strategy optimization.

[0022] By introducing a fuzzy control unit, the expert experience knowledge is integrated into the control decision, the problem of too rigid threshold control in the traditional threshold control is solved, more smooth and reasonable control instructions are generated near the critical state, and the control quality and stability of the system are improved.

[0023] Further, the feedback optimization module specifically comprises: An error calculation unit is configured to calculate the sequence error between the high-precision predicted deformation value and the actual deformation value in real time. A strategy evaluation unit is configured to evaluate the performance of the current control strategy based on the sequence error using a state value function. A parameter updating unit is configured to generate a model parameter update amount of the deep learning prediction module and a control strategy update amount of the control decision module based on a policy gradient descent method or an actor-critic algorithm. A sample library management unit is configured to store excellent decision samples and corresponding system states, and periodically prioritize and update the stored samples for offline training of the reinforcement learning algorithm.

[0024] The error calculation and strategy evaluation unit continuously monitors the system performance to provide a quantitative target for optimization. The parameter updating unit can optimize the prediction model and the control strategy simultaneously, realizing the co-evolution of the core functions of the system. The sample library management unit breaks the time sequence correlation between data through experience replay and priority sorting, improves data utilization, and enables efficient offline batch learning of the reinforcement learning algorithm, thereby speeding up the optimization process and improving the stability of the strategy, and ultimately enabling the entire system to have the ability to continuously improve itself.

[0025] Compared with the prior art, the system has the beneficial effects that: the system realizes high-precision prediction, active early warning and adaptive control of the deformation of the telescopic device by constructing a complete closed loop integrating data acquisition, preprocessing, multi-modal fusion, deep learning prediction, intelligent decision and online optimization. The combination of multi-modal sensing, deep learning and reinforcement learning realizes the full-process automation and intelligentization from data to decision, significantly improving the accuracy and efficiency of monitoring and control; the use of a deep learning model that fuses temporal and spatial features realizes the advanced prediction of future deformation trends, enabling the control system to change from passive response to active intervention, fundamentally avoiding the risk of structural failure; through online learning of the feedback optimization module, the system can continuously adapt to uncertain factors such as equipment aging and environmental changes, ensuring the prediction accuracy and control stability during long-term operation, reducing maintenance costs and safety hazards; combined with threshold judgment and fuzzy reasoning, a hierarchical and smooth control strategy is realized, which ensures fast response in emergency situations and guarantees operation efficiency and control stability in normal working conditions. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required to be used in the description of the embodiments or the prior art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor based on the structures shown in the drawings.

[0027] Figure 1 The system schematic diagram of the present application; Figure 2 The device connection schematic diagram of the stretchable device deformation prediction and control equipment based on multi-modal fusion deep learning of the present application; Figure 3 The plan view of the stretchable device. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be described and explained in the following with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application. Based on the embodiments provided by the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0029] The present application provides a stretchable device deformation prediction and control system based on multi-modal fusion deep learning, as shown in Figure 1 The specific modules include the following: The data acquisition module is used to acquire multi-modal data of the stretchable device, including temperature data, strain data, displacement data, pressure data and working environment data.

[0030] Further, the data acquisition module specifically includes: The distributed sensor array is composed of temperature sensors, strain gauges, laser displacement sensors, pressure sensors and environmental sensors arranged at key measurement points of the stretchable device, and is used to synchronously acquire the multi-modal data.

[0031] Specifically, the temperature sensor is used to monitor the temperature change of the steel or concrete material part of the main structure of the bridge, tunnel and the like, and the temperature is the main factor causing thermal expansion and cold contraction and generating temperature stress, directly affecting displacement and strain; The strain gauge is directly pasted or installed on the surface of the main structure, and is used to measure the micro-strain generated by the main structure under the action of load, temperature and the like, which is a direct index reflecting the internal stress state of the main structure; Laser displacement sensor is a non-contact high-precision measuring device, which is used to monitor the width change (absolute displacement) of the expansion joint in real time. It is the most intuitive output of deformation. Pressure sensor is usually installed on the driving or damping device of the expansion device, which is used to monitor the oil pressure of the hydraulic system or the output torque of the motor. These are the main driving force or constraint force that causes deformation. Environmental sensors include humidity sensors, anemometers, etc. They are used to collect external working environment data, which helps to explain some reasons for deformation.

[0032] All sensors must collect data under the same time reference, which is the prerequisite for subsequent multi-modal spatiotemporal alignment and data fusion. If the data is not synchronized in time, the causal relationship analyzed will be distorted, and the prediction accuracy will be greatly reduced.

[0033] Dynamic sampling unit, used to receive instructions from the control decision module or feedback optimization module, dynamically adjust the sampling frequency of each sensor according to the real-time running state or predicted demand of the expansion device.

[0034] When the system runs smoothly and the predicted value is far below the safety threshold, a lower basic frequency is used for sampling, which is enough to monitor the state; when receiving instructions from the control decision module (such as the predicted value approaching the safety threshold) or the feedback optimization module (such as the model considering that the current data has high uncertainty), the sampling frequency of the related sensors will be immediately dynamically increased. This approach greatly reduces unnecessary data volume, reduces the processing burden and energy consumption of the entire system, and is suitable for long-term continuous monitoring applications.

[0035] Data buffering unit, used to preliminarily buffer, timestamp, and packet frame the collected raw data locally on the sensor node, and send the data to the preprocessing module through a unified communication protocol.

[0036] Specifically, on each sensor node or regional collector, the collected raw data is temporarily stored to form a small data buffer; each frame of data is marked with an accurate time stamp, which is the absolute reference for multi-modal data synchronization and subsequent all time series analysis; the data within a certain time window is packaged into a regular data frame or data packet, and information such as packet header, packet tail, and check code is added; the data packet is sent to the preprocessing module through a unified communication protocol (such as TCP / IP, MQTT, or other industrial bus protocols), and the unified protocol ensures that different manufacturers and different types of sensors can seamlessly access the system.

[0037] Preprocessing module, used to clean, filter, standardize, fill missing values, and spatiotemporal alignment of the multi-modal data.

[0038] Further, the preprocessing module specifically comprises: A data cleaning unit adopts a 3σ criterion to identify and eliminate abnormal values generated due to sensor transient failure, electromagnetic interference or transmission error.

[0039] A data filtering unit removes pulse noise in the data through median filtering without making the edges of the signal fuzzy; and then performs smoothing processing on the filtered data through Kalman filtering, effectively smoothing the data, removing Gaussian white noise, and preliminarily predicting future states, which is suitable for processing time series data such as displacement and speed that have continuous dynamic characteristics.

[0040] A missing value filling unit adopts a linear interpolation method or a K-nearest neighbor algorithm to fill missing values, solving the problem of data loss caused by occasional power failure of the sensor, temporary communication interruption and the like.

[0041] A standardization processing unit adopts a Z-score standardization formula to normalize all modal data, eliminating differences between different modal data due to different dimensions and orders of magnitude.

[0042] A space-time alignment unit aligns the time stamps of temperature, strain, pressure and working environment data to the sampling time of displacement data through linear interpolation based on the time stamp of the displacement data, solving the problem of time asynchronization caused by different sampling start times and sampling frequencies of different sensors.

[0043] A data fusion module is configured to perform weighted linear fusion and nonlinear feature extraction on the preprocessed multi-modal data, and generate comprehensive data after fusion.

[0044] Further, the data fusion module specifically comprises: A linear fusion unit is configured to calculate the weight of each data source based on the Pearson correlation coefficient, and perform weighted fusion.

[0045] Different data sources have different contributions to the final deformation prediction, and the linear fusion unit objectively assigns weights by calculating the linear correlation of each data source with the actual deformation, and the higher the correlation, the greater the weight.

[0046] Specifically, the Pearson correlation coefficient of a modal data and the actual deformation is calculated by the following formula: ; Wherein, represents the Pearson correlation coefficient of the i-th modal data and the actual deformation; n represents the number of samples; X is a modal data sequence; Y is an actual deformation sequence of the stretching device; X i and Yi Let these represent the i-th elements of sequences X and Y, respectively. and These are the means of sequences X and Y, respectively.

[0047] Since the Pearson correlation coefficient may be negative, it is necessary to first take the absolute value of the coefficient and then normalize it to convert it into weights. The following formula is used for calculation: ; in, This represents the normalized fusion weight of the j-th modality data; m represents the total number of multimodal data.

[0048] The final linear fusion feature is a weighted fusion of features from each data source: ; Among them, D m This is the m-th modal data matrix; All modal data are concatenated into a linear fusion matrix of a unified dimension.

[0049] Nonlinear fusion units are used to perform dimensionality reduction and feature extraction on nonlinear data based on kernel principal component analysis or autoencoders.

[0050] Kernel Principal Component Analysis (KPCA) maps the original data to a high-dimensional feature space using a kernel function, and then performs linear principal component analysis (PCA) to reduce the dimensionality in this high-dimensional space. This makes the complex nonlinear relationships in the original space linearly separable in the high-dimensional space.

[0051] An autoencoder is an unsupervised neural network model consisting of an encoder and a decoder. The encoder compresses the data into a low-dimensional, dense code, which is the extracted nonlinear feature; the decoder reconstructs the original input from this code. By minimizing the reconstruction error, the encoder learns the most important and representative features of the data, and the encoder's output is taken as the nonlinearly fused feature.

[0052] The adaptive adjustment unit is used to dynamically adjust the weight ratio of linear and nonlinear fusion based on the Spearman correlation coefficient, and is specifically calculated using the following formula: ; in, The Spearman correlation coefficient is used to quantify the degree of monotonic correlation between variables, whether linear or nonlinear. Here, it is used to determine the correlation strength between linear / nonlinear fusion features and actual deformation, so as to dynamically adjust the fusion weights. rank of the i-th sample of the modal data X, the average rank of the same value, used to convert the original data into a rank sequence, eliminating the influence of the magnitude of the value; rank of the i-th sample of the deformation sequence Y, and corresponding, used to calculate the difference in rank between the two.

[0053] By comparing the size of the Spearman correlation coefficient between the linear fusion feature and the actual deformation and the size of the Spearman correlation coefficient between the nonlinear fusion feature and the actual deformation, it is determined which fusion method has a higher correlation strength. If the former is larger, the weight of linear fusion is increased, and vice versa.

[0054] The deep learning prediction module adopts a fusion architecture based on long short-term memory network and convolutional neural network, takes the comprehensive data as input, and outputs a predicted deformation value.

[0055] Further, the deep learning prediction module specifically includes: The time sequence feature extraction unit adopts a multi-layer long short-term memory network LSTM to extract long-period time sequence dependent features in the comprehensive data.

[0056] Specifically, this unit is composed of a multi-layer LSTM network. The gating mechanism of LSTM enables it to effectively learn long-term dependencies in long sequences, such as how temperature changes hours ago affect current deformation trends. The comprehensive data is input into the LSTM network in time steps, and the LSTM hidden state output at the last time step is taken as the time sequence feature representation vector of the entire output sequence. The spatial feature extraction unit adopts a one-dimensional convolutional neural network to extract local spatial features and cross-modal correlation features in the comprehensive data.

[0057] This unit regards the comprehensive data column as an image, where the time step is the length dimension and the sensor feature channel is the width dimension. Multiple one-dimensional convolution kernels are used to slide in the time dimension, and each convolution kernel is responsible for extracting a specific local time pattern or cross-sensor modal correlation pattern. Through multiple convolution and pooling operations, the network can capture a variety of spatial-feature patterns from simple to complex. The output of the final convolution layer is globally averaged and pooled to obtain a spatial feature representation vector.

[0058] The feature fusion unit adaptively weights and fuses the time sequence features output by the long short-term memory network and the spatial features output by the convolutional neural network through the gating attention mechanism to generate a joint feature vector.

[0059] Gated attention mechanism: This unit receives the temporal feature representation vector Ft from the LSTM output and the spatial feature representation vector Fs from the CNN output. Its core is an attention gateway that computes the attention score Attention_Score between the two vectors: ; where [Ft; Fs] denotes vector concatenation; W and b are learnable parameters; σ is the sigmoid activation function.

[0060] The attention score determines how much weight the temporal features and the spatial features are given in the final joint feature. The final joint feature vector F is generated by: ; This allows the network to automatically focus on the more important feature type for the current prediction task.

[0061] A prediction output unit that performs a non-linear mapping of the joint feature vector using a feed-forward neural network, outputting a predicted deformation sequence for a plurality of future time steps.

[0062] Specifically, the joint feature vector F is input into a feed-forward neural network FFN, which serves to non-linearly map the joint feature vector F into the final prediction space. The number of neurons in the output layer of FFN is equal to the number of future time steps that need to be predicted, e.g., 5 neurons for predicting the deformation values at 5 future time points.

[0063] A deformation value correction module, which includes: A finite element simulation unit for constructing a parametric finite element model based on the geometry and material properties of the stretching device, and using real-time collected multi-modal data as input boundary conditions to calculate the theoretical deformation values.

[0064] Specifically, based on the computer-aided design drawings of the stretching device, the precise geometric model is extracted. In finite element analysis software such as ABAQUS and ANSYS, the constitutive model of the material is defined, and the physical property parameters such as elastic modulus, Poisson's ratio, density, and thermal expansion coefficient are input. When meshing, mesh refinement is performed in key stress areas (such as tooth plate connections and pressure bearing supports) to ensure calculation accuracy.

[0065] To realize fast calling and online driving of the model, parameterization processing of the model is required. The variable inputs such as environmental temperature, vehicle load, and motor driving force are set as the parameterized boundary conditions of the model. At the same time, some undetermined parameters that are difficult to measure accurately but have a significant impact on the results (such as the equivalent stiffness of the connection part and the friction coefficient) can also be set as adjustable parameters.

[0066] The unit receives the real-time multi-modal data as input, applies it to the corresponding boundary conditions of the parameterized finite element model, runs the finite element calculation, and solves the stress, strain and displacement field of each key point of the telescopic device under the current working condition. The displacement calculation result of the specific measuring point is the theoretical deformation value.

[0067] The synchronous driving unit is used to map the real-time data processed by the preprocessing module as input parameters of the finite element model, and drive it to perform synchronous simulation.

[0068] Specifically, the unit receives real-time data processed by the preprocessing module, and maintains a mapping relationship table inside, which defines the correspondence between each type of sensor data and a specific parameter in the finite element model. According to this mapping relationship, the real-time data value is automatically assigned to the corresponding parameter of the finite element model, and the finite element solver is called for calculation. To realize real-time or quasi-real-time simulation, the Reduced-Order Model (ROM) technology can be used. That is, in the offline stage of the system, by collecting the responses of the high-fidelity full model under different parameters, a proxy model with extremely fast calculation speed but extremely small precision loss is trained, and in the online stage, the proxy model is directly run, greatly improving the calculation efficiency.

[0069] The error correction unit is used to receive the predicted deformation value and the theoretical deformation value, and fuse them through a dynamic trust weight fusion algorithm based on uncertainty, and output the final corrected high-precision predicted deformation value to the control decision module.

[0070] Further, the error correction unit is specifically used for: Calculating the relative difference between the predicted deformation value and the theoretical deformation value.

[0071] Calculating the uncertainty measure Up of the predicted deformation value, which is estimated by the information entropy of the neuron activation value of the last hidden layer of the feedforward neural network in the prediction output unit The greater the information entropy indicates that the model has greater uncertainty in predicting at this time. The information entropy is calculated by the following formula: ; Where, is the probability distribution of the hidden layer output after softmax function processing; k is the number of neurons.

[0072] The uncertainty measure U f of the theoretical deformation value is calculated, which is estimated by the standard deviation of the output of the finite element model under different small perturbations of material parameters, reflecting the uncertainty of the sensitivity of the mechanism model to parameters.

[0073] Based on the data in the recent time window, the error sequence of the prediction model and the error sequence of the theoretical model are calculated respectively; Using the kernel density estimation method, the probability distribution of the prediction error and the probability distribution of the theoretical error are constructed respectively; The ideal error distribution is defined as a Gaussian distribution with a mean of zero; The KL divergence between the ideal distribution and the prediction error distribution, and the KL divergence between the ideal distribution and the theoretical error distribution are calculated, and the calculation formula is as follows: ; ; Where, is the KL divergence between the ideal distribution and the prediction error distribution; is the KL divergence between the ideal distribution and the theoretical error distribution; is the probability distribution of the ideal error; is the probability distribution of the prediction error; is the probability distribution of the theoretical error; z is the error variable.

[0074] The first trust weight and the second trust weight are calculated according to the following formula: ; ; Where, is the first trust weight; is the second trust weight; is an adjustable parameter.

[0075] According to the first trust weight and the second trust weight, the predicted deformation value and the theoretical deformation value are weighted and fused to generate the final corrected high-precision predicted deformation value; When the relative difference degree is continuously monitored to exceed the preset alarm threshold, a signal is triggered to the feedback optimization module, requesting reinforcement learning intervention or retraining of the deep learning prediction module.

[0076] A model updating unit is configured to receive the error output by the feedback optimization module, and based on the error, the material properties or boundary condition parameters in the finite element model unit are fine-tuned in reverse.

[0077] The core of this unit is an optimizer, whose goal is to minimize the error between the theoretical deformation value calculated under the new parameters and the actual deformation value by adjusting the pending parameters in the finite element model. For example, if the system finds that the theoretical value of the current model is consistently higher than the actual measured value, the optimizer may automatically and slightly increase the equivalent stiffness parameter in the model, because a stiffer structure deforms less under the same load. After several iterations of fine-tuning, the theoretical output of the digital twin model will increasingly approach the real behavior of the physical entity, thereby maintaining its long-term accuracy.

[0078] The control decision module compares the high-precision predicted deformation value with the preset deformation safety threshold and generates a control instruction based on the comparison result.

[0079] Specifically, when the high-precision predicted deformation value is less than the preset deformation safety threshold, the operating parameters of the current telescopic device drive motor speed and hydraulic system pressure are kept unchanged, and the data acquisition module is triggered to continuously collect multi-modal data at the original frequency, and the real-time data is synchronized to the preprocessing module to maintain monitoring continuity. When the high-precision predicted deformation value is greater than or equal to the preset deformation safety threshold, based on the real-time pressure data and displacement data output by the preprocessing module, hierarchical control measures are taken to prevent excessive deformation: if the high-precision predicted deformation value is 1.0-1.2 times the preset deformation safety threshold, adjust the telescopic device drive motor speed and simultaneously change the hydraulic system pressure; if the high-precision predicted deformation value exceeds 1.2 times the preset deformation safety threshold, in addition to the above-mentioned motor and hydraulic adjustment, an alarm signal is triggered, and the telescopic device extension action is suspended until the actual deformation data falls below the safety threshold.

[0080] Further, the control decision module includes a fuzzy control unit, which is configured to: When the high-precision predicted deformation value approaches the boundary of the preset deformation safety threshold, the error data output by the feedback optimization module is called to dynamically adjust the membership function of fuzzy logic reasoning.

[0081] In fuzzy logic, the membership function defines the degree to which an input value (such as the prediction error) belongs to a certain fuzzy set. By calling the historical error data output by the feedback optimization module, if the data shows that the recent prediction has been consistently large, the membership function is automatically adjusted to make the system more sensitive to positive errors, so that control measures are taken earlier and more aggressively.

[0082] Expert experience knowledge of the telescopic device is combined to construct a fuzzy rule base. The rule base is a set of rules constructed based on expert experience knowledge, which converts the operation experience of human experts into executable automated strategies.

[0083] Based on the fuzzy logic reasoning expert experience knowledge, a smooth control instruction is generated, and the fuzzy control parameters are synchronously fed back to the feedback optimization model as sample data for subsequent strategy optimization.

[0084] The fuzzy inference machine applies all relevant fuzzy rules according to the current input, performs comprehensive reasoning, and finally outputs an accurate control quantity. The control quantity is continuously variable, thereby avoiding the mutation and oscillation caused by the traditional threshold switch control. All parameters used to generate the control instruction are synchronously fed back to the feedback optimization module, and these data become high-quality samples for the reinforcement learning algorithm to optimize the control strategy, thereby accelerating and guiding the convergence process of the reinforcement learning.

[0085] The feedback optimization module is configured to collect actual deformation data of the telescopic device, calculate an error between the high-precision predicted deformation value and the actual deformation value, and perform online optimization on model parameters of the deep learning prediction module and a control strategy of the control decision module based on the error using a reinforcement learning algorithm.

[0086] Further, the feedback optimization module specifically includes: An error calculation unit is configured to calculate a sequence error between the high-precision predicted deformation value and the actual deformation value in real time.

[0087] Specifically, the high-precision predicted deformation value is accurately aligned with the actual deformation value at the corresponding time according to a time stamp, and a mean square error (MSE) between a high-precision predicted deformation value sequence and an actual deformation value sequence is calculated.

[0088] A strategy evaluation unit is configured to evaluate a performance of the current control strategy based on the sequence error using a state value function.

[0089] Specifically, the running condition of the system is defined as a state, including all sensor readings, statistical characteristics of prediction errors in a recent period of time, and a history of recently executed control instructions.

[0090] The MSE output by the error calculation unit is converted into a reward signal Reward, and the greater the error, the smaller the reward. Maximizing the cumulative reward is equivalent to minimizing the long-term prediction error.

[0091] The state value function is used to evaluate the performance, and the function is represented as: the sum of the expected future cumulative rewards obtained by following the current strategy (including the current prediction model and the control strategy) from the state.

[0092] The time difference learning method is used to estimate the state value function. If the value in a state is continuously low, it indicates that the system is in a state of poor performance, and the current strategy needs to be optimized.

[0093] The parameter updating unit is configured to generate a model parameter updating amount of the deep learning prediction module and a control policy updating amount of the control decision module based on a policy gradient descent method or an actor-critic algorithm.

[0094] Specifically, the critic calculates a TD error according to the current state of the system and the generated error, which is used as an index for measuring whether the current policy performs better or worse than expected. The TD error is used as a signal to guide the adjustment of the model parameters of the deep learning prediction model through the policy gradient descent method, so that the model parameters can be adjusted to make more accurate decisions and obtain higher rewards. Similarly, the TD error is also used to adjust the parameters of the policy network in the control decision module, such as guiding it to learn in which system state to issue which control instruction to more effectively maintain the deformation safety and obtain higher rewards.

[0095] Through continuous iteration of the above process, the policy (prediction and decision) of the actor is continuously improved under the continuous criticism and guidance of the critic, and the performance is more and more optimal.

[0096] The sample library management unit is configured to store excellent decision samples and corresponding system states, and periodically prioritize and update the stored samples for offline training of the reinforcement learning algorithm.

[0097] Further, the application also provides a telescopic device deformation prediction and control equipment based on multi-modal fusion deep learning, as shown in Figures 2-3 As shown, it comprises a telescopic device, a multi-modal system control box, a multi-modal central processor and a multi-modal data collector.

[0098] The multi-modal data collector is arranged at key measuring points of the telescopic device and is integrated with temperature sensors, strain gauges, pressure sensors, displacement sensors and the like for real-time collection of multi-modal raw data. The multi-modal data collector is directly connected to the multi-modal central processor through an industrial Ethernet or a field bus high-speed communication link and transmits the collected multi-modal data to the multi-modal central processor.

[0099] The multi-modal central processor is the core of the system and is equipped with the data fusion, deep learning prediction, control decision and feedback optimization modules. It receives data from various collectors, performs centralized processing and operation, and generates control instructions. The multi-modal central processor is connected to the multi-modal system control box through a control bus and transmits the generated control instructions to the multi-modal system control box.

[0100] The multi-modal system control box is built-in with driving circuits and execution mechanisms (such as frequency converters, servo drives and electro-hydraulic servo valves) to receive instructions from the multi-modal central processor and drive the motors or hydraulic systems of the telescopic device to complete accurate adjustment of telescopic actions.

[0101] Specifically, the strain gauge collects strain data, which reflects the direct indicators of the internal stress state of the structure. For example, vehicle load and temperature changes can cause stress changes, which in turn can cause deformation. When the load-bearing beam of the expansion device is under pressure, the strain gauge will output corresponding electrical signals, indirectly reflecting the deformation trend of the beam under stress.

[0102] The temperature sensor collects temperature data. Temperature is a key factor that causes thermal expansion and contraction, and can directly cause material deformation. For example, high temperatures in summer can cause steel components to elongate, which may cause the expansion joint to deform beyond the threshold. Temperature changes can also affect the measurement accuracy of strain gauges and pressure sensors, so they need to be collected simultaneously for subsequent data correction.

[0103] The pressure sensor collects hydraulic system pressure / motor output pressure data of the expansion device. Pressure is a direct manifestation of the driving force of active deformation. For example, the expansion and contraction of the expansion device rely on hydraulic pressure to push. If the pressure is too high, it may cause rapid deformation. If the pressure is insufficient, it may not be able to offset the deformation caused by external loads.

[0104] The displacement sensor uses a laser non-contact design and is aligned with the expansion joint or key displacement measurement point of the expansion device to collect real-time displacement data. This is the final manifestation of deformation, such as changes in expansion joint width and component displacement.

[0105] The four types of collected data are transmitted to the multi-modal central processor for processing.

[0106] After receiving the raw data, the multi-modal central processor needs to preprocess the data, including cleaning, filtering, standardizing, missing value filling, and spatiotemporal alignment.

[0107] Then, the four types of preprocessed data are fused to avoid the limitations of single sensor data: The linear correlation between the four types of data and the actual deformation is calculated. For example, the correlation between displacement data and deformation is the highest, and the weight is the largest. The correlation between temperature data is second, and the weight is second. The four types of data are weighted and summed to preserve the physical meaning of the data, such as the linear contribution of temperature to thermal deformation.

[0108] Nonlinear associations in the data are extracted, such as the nonlinear relationship between deformation under the combined action of pressure and temperature. Through dimensionality reduction, redundant information is removed, and deep features are preserved.

[0109] The weights of linear and nonlinear fusion are dynamically adjusted. In normal temperature conditions, linear relationships dominate, so linear fusion weights are increased. In extreme temperature and high pressure conditions, nonlinear relationships are significant, so nonlinear fusion weights are increased. Finally, comprehensive data is generated.

[0110] Next, the fused comprehensive data is used as input to implement future multi-time step deformation value prediction through the fusion architecture of LSTM and CNN: Temporal feature extraction: processing the correlation of time dimension, extracting long-period dependence, and outputting the temporal feature vector.

[0111] LSTM learns the temporal causal chain between temperature, strain, and displacement through the gating mechanism. Environmental or structural temperature changes indirectly trigger internal strain changes through stress conduction. When the strain accumulates to a certain extent, it eventually manifests as external displacement deformation of the expansion device. This avoids missing features due to ignoring the subsequent effects of temperature changes. For the long-period dependence of the day-night temperature cycle, the LSTM forget gate retains the correlation information between the temperature and displacement at the same time of the previous day, while forgetting the random noise. After 2 layers of LSTM operation, the hidden layer output at the last time step is taken to obtain the temporal feature vector. Each element in the vector corresponds to the key feature of the long-period correlation between temperature, strain, and displacement.

[0112] Spatial feature extraction: processing the correlation of sensor measurement point dimension, extracting cross-modal spatial features, and outputting the spatial feature vector.

[0113] A 3x1 size convolution kernel is used for sliding operation in the feature dimension of the measurement point. For example, convolution kernel 1 focuses on capturing the cooperative relationship between the strain gauge in the middle of the left bearing beam and the pressure sensor at the oil outlet of the right hydraulic drive unit. When the strain in the middle of the left bearing beam increases and the pressure at the oil outlet of the right hydraulic drive unit simultaneously rises, the convolution kernel outputs a high activation value, indicating that the local cooperative pattern of left strain and right pressure has been recognized.

[0114] The first convolution layer extracts basic spatial features, such as the correlation between a single measurement point strain and pressure. After ReLU activation function and maximum pooling dimension reduction, the second convolution layer is inputted, further extracting complex cross-modal features, such as the combination pattern of left bearing beam middle strain increase + right hydraulic drive unit oil outlet pressure rise + right bearing beam middle strain stability, corresponding to the working condition dominated by left load deformation.

[0115] The feature map output by the second convolution layer is averaged by the global average pooling layer to obtain the spatial feature vector, which directly reflects the load distribution and deformation correlation in the spatial dimension.

[0116] Gated attention fusion: dynamically allocate weights through attention mechanism, such as increasing the weight of temporal features when the current deformation is mainly dominated by temperature; increase the weight of spatial features when it is mainly dominated by local pressure, generate joint feature vector, containing temporal and spatial rules.

[0117] The joint feature vector is inputted into the feedforward neural network FFN, and the predicted deformation value of future multiple time steps is outputted through nonlinear mapping.

[0118] The predicted deformation value is fused with the theoretical deformation value to output a corrected high-precision predicted deformation value.

[0119] The high-precision predicted deformation value is compared with a preset safety threshold value, and a differentiated control instruction is generated in combination with real-time data.

[0120] A preset deformation safety threshold value is determined in three scenarios: When the high-precision predicted deformation value is less than the safety threshold value, the current state is maintained: the motor speed and the hydraulic pressure are not adjusted, and the data collector continues to monitor at the original frequency to avoid excessive control and energy consumption; When the high-precision predicted deformation value is 1.0-1.2 times the safety threshold value, mild intervention is performed: based on real-time pressure / displacement data, instructions to reduce the motor speed and the hydraulic pressure are generated to slow down the deformation trend; When the high-precision predicted deformation value is greater than 1.2 times the safety threshold value, emergency intervention is performed: in addition to adjusting the motor and the hydraulic pressure, an audible and light alarm is triggered, and the extension and retraction of the telescopic device is suspended until the actual displacement falls below the safety threshold value to avoid structural damage.

[0121] The generated control instruction is transmitted to the multi-modal system control box through the control bus by the multi-modal central processor, and the telescopic device is adjusted by the actuator in the control box to realize predictive intervention.

[0122] It should be noted that the present application is not limited to the above-mentioned embodiments. The above-mentioned embodiments are only examples, and embodiments having the same technical idea and playing the same role and effect within the scope of the technical solution of the present application are included in the technical scope of the present application. In addition, within the scope of the main idea of the present application, various modifications that can be thought of by those skilled in the art are applied to the embodiments, and other ways are constructed by combining part of the constituent elements in the embodiments.

Claims

1. A telescopic device deformation prediction and control system based on multi-modal fusion deep learning, characterized in that, The method comprises the following steps: a data acquisition module for acquiring multi-modal data of the telescopic device, including temperature data, strain data, displacement data, pressure data and working environment data; a preprocessing module for cleaning, filtering, standardizing, missing value filling and spatio-temporal alignment processing of the multi-modal data; a data fusion module for weighted linear fusion and nonlinear feature extraction of the preprocessed multi-modal data to generate integrated data after fusion; a deep learning prediction module using a fusion architecture based on long short-term memory network and convolutional neural network, taking the integrated data as input and outputting predicted deformation values; a deformation value correction module for fusing predicted deformation values with theoretical deformation values to output high-precision predicted deformation values after correction; a control decision module for comparing the high-precision predicted deformation values with the preset deformation safety threshold and generating control instructions according to the comparison results; a feedback optimization module for acquiring actual deformation data of the telescopic device, calculating the error between the actual deformation data and the high-precision predicted deformation values, and online optimizing the model parameters of the deep learning prediction module and the control strategy of the control decision module based on the error using a reinforcement learning algorithm. 2.The multi-modal fusion deep learning based stretch-forming prediction and control system of a stretch-forming device according to claim 1, wherein, The data acquisition module specifically comprises: a distributed sensor array composed of temperature sensors, strain gauges, laser displacement sensors, pressure sensors and environmental sensors arranged at key measurement points of the telescopic device for synchronous acquisition of the multi-modal data; a dynamic sampling unit for receiving instructions from the control decision module or the feedback optimization module and dynamically adjusting the sampling frequency of each sensor according to the real-time running state or predicted demand of the telescopic device; a data buffering unit for locally buffering, timestamping and packet framing the collected raw data at the sensor node, and sending the data to the preprocessing module through a unified communication protocol. 3.The multi-modal fusion deep learning based stretch-forming prediction and control system of stretch-forming apparatuses according to claim 1, wherein, The preprocessing module specifically comprises: a data cleaning unit for removing outliers in each modal data using the 3σ criterion; a data filtering unit for removing pulse noise in the data through median filtering and smoothing the filtered data through Kalman filtering; a missing value filling unit for filling missing values using linear interpolation or K-nearest neighbor algorithm; a standardization processing unit for normalizing all modal data using the Z-score standardization formula; a spatio-temporal alignment unit for aligning the timestamps of temperature, strain, pressure and working environment data to the sampling time of displacement data through linear interpolation based on the timestamp of displacement data. 4.The multi-modal fusion deep learning based stretch-forming system for deformation prediction and control of stretch-forming apparatuses according to claim 1, wherein, The data fusion module specifically comprises: a linear fusion unit for calculating the weight of each data source based on the Pearson correlation coefficient and performing weighted fusion; a nonlinear fusion unit for dimensionality reduction and feature extraction of nonlinear data based on kernel principal component analysis or autoencoder; an adaptive adjustment unit for dynamically adjusting the weight proportion of linear and nonlinear fusion according to the Spearman correlation coefficient. 5.The multi-modal fusion deep learning based stretch-forming system for deformation prediction and control of stretch-forming apparatuses according to claim 1, wherein, The deep learning prediction module specifically comprises: a time series feature extraction unit for extracting long-period time series dependence features in the integrated data using a multi-layer long short-term memory network; a convolutional neural network unit for extracting local features in the integrated data through convolution and pooling operations; The spatial feature extraction unit extracts local spatial features and cross-modal correlation features in the comprehensive data by using a one-dimensional convolutional neural network. The feature fusion unit adaptively weights and fuses the time sequence features output by the long short-term memory network and the spatial features output by the convolutional neural network through a gated attention mechanism to generate a joint feature vector. The prediction output unit performs nonlinear mapping on the joint feature vector by using a feedforward neural network to output a deformation prediction sequence at multiple future time steps. 6.The multi-modal fusion deep learning based stretch-forming system for deformation prediction and control of stretch-forming apparatuses according to claim 1, wherein, The deformation value correction module includes: The finite element simulation unit is configured to construct a parameterized finite element model based on the geometric and material properties of the telescopic device, and calculate a theoretical deformation value by taking the real-time collected multi-modal data as input boundary conditions. The synchronous driving unit is configured to map the real-time data processed by the preprocessing module as input parameters of the finite element model to drive the synchronous simulation. The error correction unit is configured to calculate a first trust weight of the predicted deformation value and a second trust weight of the theoretical deformation value based on the uncertainty measure of the predicted deformation value and the theoretical deformation value and the KL divergence of the error distribution of the predicted deformation value and the theoretical deformation value with respect to an ideal distribution, and to perform weighted fusion on the predicted deformation value and the theoretical deformation value according to the first trust weight and the second trust weight to generate a corrected high-precision predicted deformation value. The model updating unit is configured to receive the error output by the feedback optimization module and perform backward fine-tuning on the material properties or boundary condition parameters in the finite element model unit based on the error.

7. The multi-modal fusion deep learning based stretch-forming apparatus deformation prediction and control system of claim 1, wherein, The error correction unit is specifically configured to: calculate the relative difference between the predicted deformation value and the theoretical deformation value; calculate the uncertainty measure of the predicted deformation value and the theoretical deformation value; construct the probability distribution of the prediction error and the probability distribution of the theoretical error using a kernel density estimation method; define an ideal error distribution and calculate the KL divergence between the prediction error probability distribution and the theoretical error probability distribution and the ideal error distribution; calculate the first trust weight of the predicted deformation value and the second trust weight of the theoretical deformation value according to the uncertainty measure and the KL divergence. 8.The multi-modal fusion deep learning based stretch-forming system for deformation prediction and control of stretch-forming apparatuses according to claim 1, wherein, The control decision module is specifically configured to: when the high-precision predicted deformation value is less than the preset deformation safety threshold, keep the operating parameters of the drive motor speed and the hydraulic system pressure of the current telescopic device unchanged, trigger the data acquisition module to continuously collect multi-modal data at the original frequency, and synchronize the real-time data to the preprocessing module to maintain monitoring continuity; when the high-precision predicted deformation value is greater than or equal to the preset deformation safety threshold, based on the real-time pressure data and displacement data output by the preprocessing module, take hierarchical control measures to prevent excessive deformation: if the high-precision predicted deformation value is 1.0-1.2 times the preset deformation safety threshold, adjust the speed of the telescopic device drive motor and simultaneously change the hydraulic system pressure; if the high-precision predicted deformation value exceeds 1.2 times the preset deformation safety threshold, in addition to the above-mentioned motor and hydraulic adjustment, an alarm signal is triggered and the telescopic device is temporarily suspended from extension and retraction until the actual deformation data falls below the safety threshold. 9.The multi-modal fusion deep learning based stretch-forming system for deformation prediction and control of stretch-forming apparatuses as claimed in claim 1, wherein, The control decision module further comprises a fuzzy control unit, which is configured to: When the high-precision predicted deformation value approaches the boundary of the preset deformation safety threshold, error data output by a feedback optimization module is called to dynamically adjust a membership function of fuzzy logic reasoning; Expert experience knowledge of the telescopic device is combined to construct a fuzzy rule base; Based on the expert experience knowledge of fuzzy logic reasoning, smooth control instructions are generated, and fuzzy control parameters are synchronously fed back to the feedback optimization model as sample data for subsequent strategy optimization. 10.The multi-modal fusion deep learning based stretch-forming system for deformation prediction and control of stretch-forming apparatuses according to claim 1, wherein, The feedback optimization module specifically comprises: an error calculation unit configured to calculate sequence errors between the high-precision predicted deformation value and the actual deformation value in real time; a strategy evaluation unit configured to evaluate the performance of a current control strategy based on the sequence errors by using a state value function; a parameter updating unit configured to generate a model parameter updating amount of the deep learning prediction module and a control strategy updating amount of the control decision module based on a strategy gradient descent method or an actor-critic algorithm; a sample library management unit configured to store excellent decision samples and corresponding system states, and periodically prioritize and update the stored samples for offline training of a reinforcement learning algorithm.

Images