Multi-modal fusion composite laminated structure low-speed impact positioning method

Abstract

The invention discloses a multi-modal fusion composite laminated structure low-speed impact positioning method, and relates to the technical field of impact positioning, and the method comprises the steps: obtaining impact response signals of a plurality of strain sensors on a composite laminated structure, and combining the impact response signals into a composite impact signal; taking the envelope entropy as a fitness function, and adopting a moss growth optimization algorithm to optimize a penalty coefficient of successive variational mode decomposition; performing successive variational mode decomposition on the composite impact signal based on the optimal penalty coefficient, and selecting a part of mode component with the maximum energy distribution correlation coefficient to reconstruct a de-noised signal; converting the de-noised signal into a time-frequency spectrogram, and converting an original composite signal into a two-dimensional coding diagram through a relative position matrix algorithm; constructing a multi-modal sample data set containing the time-frequency spectrogram and the coding diagram; and training the multi-modal regression model and carrying out impact positioning. According to the scheme, the modal purity of signal decomposition can be remarkably improved, and the characterization capability and fusion efficiency of multi-modal features are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of impact location, and particularly to a method for locating low - velocity impacts of a multi - modal fusion composite laminated structure. Background Art

[0002] Due to characteristics such as light weight and high specific strength, composite laminated structures are widely used in fields such as aerospace and wind power generation. However, they are extremely sensitive to low - velocity impacts, which are likely to cause internal damage and threaten the structural safety. As a core part of structural health monitoring, impact location technology needs to extract effective features from sensor signals and establish a mapping relationship with the impact location.

[0003] The core of impact location methods is to extract effective impact features from the signal data collected by sensors and establish a mapping relationship between these features and the impact location. Common impact location methods include the time - difference - of - arrival method, the reference database method, data - driven machine - learning methods, etc. Among them, the time - difference - of - arrival method uses the geometric relationship between time differences, wave velocities, and sensor positions to achieve impact location. However, for complex structures, it is difficult to accurately obtain time differences and wave velocities in all directions; the reference database method matches a pre - established database containing impact features at different positions with the actual detected signals to achieve impact location. However, the establishment of a reference database requires a large amount of impact signal data, and the accuracy of location depends on the accuracy of the data; currently, although the development of machine - learning methods has improved the accuracy and robustness of impact location to a certain extent, most machine - learning methods only use single - dimensional information, are particularly vulnerable to noise, and rely on feature engineering. In summary, the existing impact location methods have significant limitations in feature extraction, spatial information utilization, noise resistance, etc. Therefore, based on the above problems, the present invention proposes a method for locating low - velocity impacts of a multi - modal fusion composite laminated structure. Summary of the Invention

[0004] Object of the Invention In order to solve the above problems, the object of the present invention is to provide a method for locating low - velocity impacts of a multi - modal fusion composite laminated structure, aiming to solve the problems of insufficient single - modal feature information, significant noise interference, and low multi - modal data fusion efficiency in the low - velocity impact location of composite laminated structures, achieve high - precision and high - robustness impact location, overcome the dependence on the adjustment of empirical parameters in traditional methods, and improve the characterization ability of spatio - temporal features of impact events in complex noise environments.

[0005] Technical Solution To achieve the above object, the present invention provides a method for low-speed impact location of a multi-modal fusion composite laminate structure. This method dynamically optimizes the penalty coefficient of successive variational mode decomposition based on the moss growth optimization algorithm, combines the envelope entropy criterion to suppress noise and reconstruct the effective impact signal; generates a time-frequency spectrogram through continuous wavelet transform to capture time-frequency features, and encodes the signal sequence into a two-dimensional spatial distribution map using the relative position matrix algorithm; constructs a ResNet18-NAM-Agent multi-modal regression model, fuses the channel-spatial attention mechanism to enhance the feature extraction ability, and realizes cross-modal feature interaction through the proxy attention mechanism; in addition, a dynamic data augmentation strategy is also adopted to expand the dataset, combines the Huber loss function and the adaptive learning rate to optimize the generalization performance of the model, and finally realizes end-to-end impact position prediction.

[0006] In the first aspect, the present invention provides a method for low-speed impact location of a multi-modal fusion composite laminate structure, including: Obtain the impact response signals of multiple strain sensors on the composite laminate structure and combine them into a composite impact signal; Use the moss growth optimization algorithm to optimize the penalty coefficient of successive variational mode decomposition with the envelope entropy as the fitness function; Perform successive variational mode decomposition on the composite impact signal based on the optimal penalty coefficient, and select the partial mode components with the largest energy distribution correlation coefficient to reconstruct the denoised signal; Convert the denoised signal into a time-frequency spectrogram through continuous wavelet transform, and convert the original composite signal into a two-dimensional encoded map through the relative position matrix algorithm; Construct a multi-modal sample dataset including the time-frequency spectrogram and the encoded map; Train the ResNet18-NAM-Agent multi-modal regression model, and perform impact location on the test set through the model.

[0007] Further, the parameters of the moss growth optimization algorithm include: The population size is 10, the maximum number of iterations is 19, and the search range of the penalty coefficient is from 500 to 60,000.

[0008] Further, the moss growth optimization algorithm includes a wind direction determination mechanism, spore diffusion search, dual reproduction search, and cryptobiotic mechanism, where the wind direction is dynamically adjusted by the position relationship between the optimal individual and the majority of individuals in the population.

[0009] Further, the update formula of the spore diffusion search is:

[0010]

[0011]

[0012]

[0013]

[0014] In the formula, is the wind intensity attenuation factor; is the current number of evaluations; is the maximum number of evaluations; is the proportion of the number of mosses in the total number of moss individuals; used to count the number of elements; is the set of most moss individuals; is the total number of moss individuals; is the distance of spore transmission under stable wind conditions; is the constant parameter 2; is the distance of spore transmission under turbulent wind conditions; is the th moss individual new moss obtained by spore transmission; is a moss individual in the current moss population; is the wind direction; is 0.2; , and are random numbers between 0 and 1.

[0015] Furthermore, the constraint criterion of the successive variational mode decomposition includes minimizing the frequency domain bandwidth of the mode components, the spectral overlap between the residual signal and the mode components, and the energy of the current mode near the center frequency of the historical modes.

[0016] Furthermore, the calculation method of the energy distribution correlation coefficient is as follows:

[0017] In the formula, is the energy distribution correlation coefficient; is the energy distribution of the mode component; is the energy distribution of the original signal; is the covariance; is the variance.

[0018] Furthermore, the relative position matrix algorithm includes: After performing z-score normalization on the composite signal, it is reduced to m dimensions through piecewise aggregate approximation; Construct an m×m matrix to represent the relative position relationship between time stamps and convert it into a grayscale encoded image.

[0019] Furthermore, the construction of the multi-modal sample data set includes: Adding Gaussian, Poisson, salt-and-pepper, and multiplicative noise to the time-frequency spectrogram, and performing flipping, contrast enhancement, and histogram equalization on the encoded image, so that the data set is expanded to 5 times the original data volume.

[0020] Furthermore, in the multi-modal regression model, ResNet18 is fused with the normalization attention mechanism. The proxy attention mechanism is used for multi-modal feature fusion. Channel attention calculates the channel importance through the Batch Normalization weights, and spatial attention generates spatial weights through the channel mean; The query and value of the proxy attention mechanism come from the time-frequency spectrogram feature sequence, and the key comes from the encoded image feature sequence. The fusion formula is:

[0021]

[0022] In the formula, is the proxy feature matrix; is the global average pooling; , , , , and are weight parameters, and are the original features; is the feature sequence obtained after the fusion of the proxy attention mechanism; is the softmax function; is the scaling factor; and are the position biases; is the depthwise separable convolution operation.

[0023] Furthermore, in the ResNet18-NAM feature extraction network, the NAM module of the time-frequency spectrogram branch is inserted into the first two residual layers, and the NAM module of the encoded image branch is inserted into the first three residual layers and the position after the fourth residual layer.

[0024] Furthermore, it also includes the dynamic double-weight envelope entropy step. By quantifying the energy concentration in the time domain and the sparsity in the frequency domain of the signal, a composite fitness function is constructed. The formula of the double-weight envelope entropy is defined as: In the formula, is the dynamic double-weight envelope entropy of the th modal component; is the total number of signal sample points; is the frequency domain sparsity factor; is the time-domain attenuation factor; is the time point of the signal; is the frequency point of the signal; is the envelope normalization value of the

[0025] Furthermore, it also includes a spatio-temporal collaborative attention mechanism, which jointly adjusts the cross-modal interaction intensity through a time-domain convolutional gated unit and a spatial-domain dynamic weight. The spatio-temporal collaborative attention formula is defined as:

[0026] In the formula, is the spatio-temporal collaborative attention output; is element-wise multiplication; is the time-domain convolutional gate; is the spatial-domain dynamic weight.

[0027] In the second aspect, the present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run by a processor, it executes the multi-modal fusion composite laminated structure low-speed impact location method described above.

[0028] Based on the moss growth optimization algorithm, the present invention dynamically adjusts the penalty coefficient of successive variational mode decomposition, combines the dynamic double-weight envelope entropy criterion to screen high-fidelity modal components, suppresses noise and retains impact transient features; generates a time-frequency spectrogram through continuous wavelet transform to capture time-frequency features, and uses the relative position matrix algorithm to map the signal sequence into a two-dimensional coding map to explicitly represent the spatial distribution; constructs a ResNet18-NAM-Agent multi-modal regression model, introduces a cross-modal spatio-temporal collaborative attention mechanism to realize the deep interaction of time-frequency and spatial features, and combines dynamic data augmentation and an adaptive training strategy to optimize the model generalization ability. This solution significantly improves the modal purity and stability of signal decomposition, enhances the representation ability and fusion efficiency of multi-modal features, thereby achieving high-precision and high-robustness impact location in a complex noise environment, and systematically solving the core problems such as single feature, noise sensitivity and computational redundancy in traditional methods.

[0029] Beneficial Effects By implementing the multi-modal fusion composite laminated structure low-speed impact location method provided by the present invention as described above, the following technical effects are achieved: (1) By introducing a time-domain attenuation factor and a frequency-domain sparse factor to reconstruct the envelope entropy calculation, the present application optimizes the balance between noise suppression and impact feature retention in the signal decomposition process. This method significantly improves the purity and decomposition stability of modal components, enhances the ability of the denoised signal to retain impact transient features, and thus provides input data with a higher signal-to-noise ratio for the subsequent location model.

[0030] (2) By synergistically adjusting the multi-modal feature interaction intensity through time-domain convolutional gating and spatial dynamic weights, the problem of insufficient correlation modeling in the spatio-temporal dimension of traditional attention mechanisms is solved. This mechanism realizes the efficient complementary fusion of time-frequency features and spatial distribution features, improves the robustness of the model to complex noise and boundary reflection interference, and reduces the computational redundancy of cross-modal feature fusion simultaneously.

[0031] (3) Through noise injection and image transformation operations, data diversity is enhanced respectively according to the physical characteristics of time-frequency spectrograms and encoded images, and a multi-modal dataset with strong generalization ability is constructed. This strategy effectively alleviates the model's dependence on limited labeled data, suppresses the overfitting phenomenon during training, and ensures positioning consistency in sparse sensing or non-uniform noise environments.

[0032] (4) By constructing piecewise aggregation approximation and relative position matrices, one-dimensional signal sequences are mapped into two-dimensional spatial distribution encoded images, explicitly representing the spatial propagation characteristics of impact signals. This method makes up for the deficiency of traditional time-frequency analysis methods in capturing spatial dimension information, provides more comprehensive spatio-temporal feature inputs for deep learning models, and thus improves the spatial resolution and positioning accuracy of impact positions. Brief Description of the Drawings

[0033] To make the above-mentioned low-speed impact positioning method of the multi-modal fusion composite laminate structure of the present invention more clearly understandable, the drawings required in the specific implementation manners of the present invention will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those skilled in the art can also obtain other drawings based on these drawings without creative efforts.

[0034] Figure 1 Represents the flowchart of the method of this application; Figure 2 Represents the schematic diagram of strain gauge pasting for the multi-modal impact positioning method based on ResNet18-NAM-Agent; Figure 3 Represents the schematic diagram of the composite impact signal; Figure 4 Represents the schematic diagram of the result of successive variational mode decomposition optimized by the moss growth algorithm; Figure 5 Represents the schematic diagram of the frequency spectrum corresponding to the structure of successive variational mode decomposition optimized by the moss growth algorithm; Figure 6 Represents the comparison schematic diagram of the reconstructed denoised impact signal and the original composite impact signal; Figure 7Schematic diagram of a multi-modal impact localization model based on ResNet18-NAM-Agent. Detailed implementation mode

[0035] Example 1: A method for low-speed impact localization of a multi-modal fusion composite laminated structure is provided. The method flow is as Figure 1 shown and includes: Step 1: Obtain the impact response signals of multiple strain sensors on the composite laminated structure specimen, and arrange the signals of the multiple sensors in the order of sensor 1, sensor 2,..., sensor n to form a composite impact signal. As Figure 2 shown, on a 650mm×650mm×3mm composite laminated structure specimen, a 400mm×400mm square monitoring area is selected. Four strain sensors are symmetrically pasted at the four corner points of the monitoring area. The axial direction of the strain sensor is parallel to the diagonal of the square at this corner point and the included angle with the side of the monitoring area is 45°. A two-dimensional rectangular coordinate system is established on the monitoring area of the composite laminated structure. The lower left corner vertex of the monitoring area is defined as the coordinate origin (0mm, 0mm). The x-axis is defined as a horizontal line starting from the origin, and the y-axis is a line perpendicular to the x-axis starting from the origin. Moderate-sized and evenly distributed grids are divided on the monitoring area, a total of 81 grid points are generated. The 81 grid points are repeatedly struck 3 times with an impact force hammer, and then 15 points are randomly selected for one strike. The sampling frequency of the signal acquisition device is set to 10kHz, and the impact signals collected by the strain sensors are collected.

[0036] As Figure 3 shown, the strain signals collected by the preferably four strain sensors are arranged on the time axis according to the strain sensor arrangement order to form a composite impact signal.

[0037] Step 2: Use the envelope entropy as the fitness function, and adopt the moss growth optimization algorithm to find the penalty coefficient after the composite impact signal in Step 1 is decomposed by the successive variational mode decomposition algorithm ; When finding the optimal decomposition penalty coefficient , select the envelope entropy as the fitness function of the moss optimization algorithm. The envelope entropy is an index to measure the complexity of the signal, reflecting the uncertainty of the signal energy in the envelope distribution. The energy of the impact signal is concentrated at a few moments, the envelope distribution is sparser and more orderly, and the envelope entropy value shown is smaller. The energy distribution of the noise signal is uniform, the envelope distribution is disordered and chaotic, and the envelope entropy value shown is larger. Therefore, in the optimization process, try to find the value that makes the envelope entropy value smaller. The calculation formula of the envelope entropy value is:

[0038]

[0039] Wherein, is the normalized form of; is the envelope obtained after the Hilbert transform of the intrinsic mode component; is the total number of signal sample points; is the envelope entropy value.

[0040] When initializing the relevant parameters of the moss growth optimization algorithm, the population size is set to 10, the maximum number of iterations is set to 19, and the penalty parameter is selected in the range of 500 to 60000. Each moss individual in the moss population represents a penalty parameter of the successive variational mode decomposition algorithm. The moss growth optimization algorithm includes four key stages: determination of the wind direction, spore diffusion search, double diffusion search, and cryptobiotic mechanism.

[0041] The wind direction mechanism of moss propagation uses the positional relationship between most moss individuals and the optimal moss individual to determine the evolution direction of all moss individuals in the population. This evolution direction can effectively help the moss growth optimization algorithm avoid falling into local optimal solutions, and its expression is:

[0042]

[0043] Wherein, is the distance set of each moss individual in the current population relative to ; is the optimal solution in the current moss population; is a moss individual in the current moss population; is the set of most moss individuals; is the wind direction; is the total number of individuals in; is the th distance between a moss individual and the optimal moss individual.

[0044] The spore diffusion search mechanism allows individuals to make random selections by simulating the propagation characteristics of spores under two conditions of steady wind and turbulent wind, and can prevent slow convergence in the early stage with a fixed step size from resulting in non-convergence in the later stage. According to the propagation principle of moss spores, the population position is updated, and the expression is: The update formula for the spore diffusion search is:

[0045]

[0046]

[0047]

[0048]

[0049] In the formula, is the wind intensity attenuation factor; is the current number of evaluations; is the maximum number of evaluations; is the proportion of the number of mosses in to the total number of moss individuals; used to count the number of elements; is the set of most moss individuals; is the total number of moss individuals; is the distance of spore propagation under stable wind conditions; is the constant parameter 2; is the distance of spore propagation under turbulent wind conditions; is the th moss individual new moss obtained by spore propagation; is a moss individual in the current moss population; is the wind direction; is 0.2; , and are random numbers between 0 and 1.

[0050] When the random number is less than 0.8, the double reproduction search mechanism is triggered. Different from traditional meta-heuristic algorithms, this method increases the proportion of the method that only changes a single dimension, enhancing the overall local search ability. According to the double reproduction mechanism of moss, the position of the moss population is further updated with a certain probability, and the expression is:

[0051]

[0052]

[0053]

[0054] In the formula, is used to evaluate whether the particle in the optimal moss individual is utilized; is A random vector between 0 and 1 with the same dimension; , is a random number between 0 and 1, is 0.5, if , then simulate double-reproduction search in the sexual reproduction stage, otherwise simulate double-reproduction search in the vegetative reproduction stage; used to control the moving distance of an individual in the direction; is the th moss individual new moss obtained by spore dissemination; is the th particle in is a random number not exceeding the maximum dimension of an individual; is the th particle in is the th particle in

[0055] The cryptobiotic mechanism optimizes the search process according to the cryptobiotic phenomenon of moss, retains the records of moss individuals generated in each iteration, and once the maximum record number 10 is reached or the population iteration is completed, the cryptobiotic mechanism is triggered to restore the optimal moss individuals and replace the current moss individuals, realizing the update of the population position and the optimal fitness function value, and ensuring the global search ability of the overall population.

[0056] Step 3: Apply the optimal penalty factor to successive variational mode decomposition, decompose the composite impact signal in Step 1 to obtain multiple modal components, calculate the correlation coefficient between the energy distribution of the calculated modal components and the energy distribution of the composite impact signal in Step 1, and select the first M modal components with the largest correlation coefficient for signal reconstruction to obtain the denoised impact signal; Successive variational mode decomposition performs continuous variational mode decomposition on the signal until the reconstruction error is less than the set threshold. The greatest advantage of successive variational mode decomposition is that it does not require prior knowledge of the number of modes in the signal, reducing the computational complexity. Successive variational mode decomposition has advantages such as strong self-adaptability, high decomposition accuracy, and good robustness, and is especially suitable for feature extraction in impact signals and complex noise environments. Using successive variational mode decomposition to decompose the target composite impact signal data, the decomposition process specifically includes: Input the target composite impact signal into the successive variational mode decomposition algorithm, and the original composite impact signal is decomposed into two signals, including the L-order intrinsic mode component signal and the residual signal , where the residual signal is a signal other than and includes the sum of all previously obtained modes and the unprocessed part of the composite impact signal , and the expression is:

[0057]

[0058] In the formula, is the th L-order intrinsic mode component signal.

[0059] To ensure that each mode component can be compact around its center frequency to ensure the concentration of its energy in the frequency domain, for the Lth-order mode component, the following criterion should be minimized:

[0060] In the formula, is a target criterion for minimizing the bandwidth of the mode component in the frequency domain; is the derivative with respect to time ; is the unit impulse function; is the imaginary unit in the complex number; is the complex exponential function used to transform the signal into the frequency domain; is the center frequency of the Lth-order intrinsic mode component; At the frequency where the effective component is contained in , the energy of the residual signal should be minimized. Therefore, to minimize the spectral overlap between the residual signal and the Lth-order mode component , the expression is:

[0061] In the formula, is the constraint criterion for minimizing the spectral overlap between the residual signal and the Lth intrinsic mode component; is the impulse response of the filter ; By minimizing and two constraint criteria, the Lth mode component of the composite impact signal is obtained, but the obtained Lth mode component may have an obvious modal aliasing phenomenon with the previously obtained L-1 mode components. To avoid this situation, it is necessary to ensure that should have less energy at the frequencies near the center frequencies of the previously obtained mode components, and the expression is:

[0062] wherein is a constraint criterion to ensure that the current modal component has the minimum energy near the center frequency of the previously obtained intrinsic modal component; is a filter with a center frequency of ; The last constraint criterion is to ensure that all modal components and the unprocessed part can be completely reconstructed into the original composite impact signal data, and the expression is:

[0063] Therefore, when the L-1 modes are known, the problem of extracting the Lth mode can be expressed as a constrained minimization problem, and the expression is:

[0064]

[0065] wherein is a parameter for balancing , and , and is solved by the Lagrange multiplier method.

[0066] The results of successive variational mode decomposition optimized by the moss growth optimization algorithm and its corresponding spectrum are shown in Figure 4 and Figure 5 . The successive variational mode decomposition algorithm optimized by the moss growth optimization algorithm is used to decompose the key frequency components of the signal, separating the low-frequency characteristic components from the high-frequency noise components, which helps to remove the high-frequency noise components in the subsequent signal reconstruction process.

[0067] Calculate the energy distribution of the signal in the time domain for each decomposed modal component and the original composite impact signal, and calculate the correlation coefficient between the two. Select the first four or the first three modal components with larger correlation coefficients for signal reconstruction to obtain the denoised impact signal. Selecting the energy distribution of the signal in the time domain as the basis for screening modal components can effectively retain the transient characteristics and local energy concentration characteristics of the impact signal, avoid losing key transient information, and at the same time can effectively distinguish the impact signal from the noise component and enhance the impact positioning ability. The expression is:

[0068]

[0069] wherein is the dynamic double-weighted envelope entropy; are the sample points of the composite impact signal and the modal component; is the energy distribution correlation coefficient; is the energy distribution of the modal component; is the energy distribution of the original signal; is the covariance; is the variance.

[0070] Signal reconstruction is performed on the modal components selected according to the obtained correlation coefficients. The signal reconstruction process is as follows: Suppose the original composite impact signal is decomposed into k modal components, and the first M modal components with the largest correlation coefficients are selected. Generally, M is taken as 3 or 4. Define the selection set S, and the reconstruction expression is:

[0071]

[0072] In the formula, is the correlation index between the energy distribution of the modal component and the original signal; is the reconstructed signal, are the intrinsic modal components obtained by decomposition; such as Figure 6 shown, the amplitude of the noise in the denoised impact signal after reconstruction is significantly reduced, achieving the denoising effect; Step 4: Use the continuous wavelet transform algorithm to convert the denoised impact signal in Step 3 into a two-dimensional time-frequency spectrogram, and use the relative position matrix algorithm to convert the composite impact signal in Step 1 into a two-dimensional coding diagram to obtain two types of modal images; The continuous wavelet transform has time-frequency localization ability, can provide both the time and frequency information of the signal, and is suitable for analyzing non-stationary signals such as impact signals. It intuitively represents the time-frequency characteristics of the impact signal as a two-dimensional time-frequency spectrogram, which is beneficial for subsequent feature extraction and localization analysis. Use the continuous wavelet transform algorithm to convert the denoised impact signal in into a two-dimensional time-frequency spectrogram, and the expression is:

[0073] In the formula, is the coefficient of the continuous wavelet transform; is the scale parameter; is the translation parameter; is the denoised signal; is the Morlet wavelet basis function; is the conjugate complex of; Use the relative position matrix algorithm to convert the original composite impact signal into a two-dimensional coding diagram, which explicitly represents the spatial distribution characteristics of the impact signal, as a supplementary modal image of the time-frequency spectrogram generated by the continuous wavelet transform, providing three-dimensional time-frequency-space information and providing a more comprehensive and accurate feature representation for impact localization. The process of the relative position matrix algorithm includes: Perform z-score normalization on the original composite impact signal sequence data to obtain a standard normal distribution, and the expression is:

[0074] In the formula, is the normalized signal sequence; is the original composite signal sequence; is the original composite signal sequence 's average value; is the original composite impact signal sequence 's standard deviation; Apply the piecewise aggregate approximation method. By calculating the average value of a piecewise constant, reduce the normalized time series data from n dimensions to m dimensions while maintaining the approximate trend of the original sequence. Select an appropriate dimensionality reduction factor k, set the dimensionality reduction factor k to 40, and generate a new smoothed time series. The expression is:

[0075] Construct an m×m matrix, calculate the relative position between two timestamps, and convert the processed time series x into a two-dimensional matrix. The expression is:

[0076] Apply min-max normalization to convert M into a grayscale value matrix. The expression is:

[0077] In the formula, is the grayscale value matrix; Step 5: Construct a multi-modal sample data set for the training, validation, and prediction of the impact location model; Enlarge the image data volume of the above-obtained two-dimensional time-frequency spectrogram and two-dimensional coding map through image enhancement operations. For the two-dimensional time-frequency spectrogram generated by continuous wavelet transform, perform image noise addition operations, adding four types of noise: Gaussian, Poisson, salt-and-pepper, and multiplicative, to expand the data set to 5 times the original size, and a total of 1215 two-dimensional time-frequency spectrograms are obtained. For the two-dimensional coding map generated by the relative position matrix algorithm, perform image transformation operations, including flipping, contrast enhancement, histogram equalization, and grayscale conversion on the image, to expand the data set to 5 times the original size, and a total of 1215 two-dimensional coding maps are obtained. Finally, a sample data set of two-modal images is constructed. The image sample data set also includes the label information corresponding to each image data, that is, the coordinates of the impact location. Divide the data set into a training set and a validation set according to a preset ratio.

[0078] Select a part of the original composite impact signal data of random knocking for continuous wavelet transform, and only select the two-dimensional time-frequency spectrogram generated by the continuous wavelet transform as the test and data, and this part of the image is not subjected to image enhancement processing; To improve the convergence ability and generalization ability of the model, perform min-max normalization processing on the labels corresponding to the image data in the training set and the validation set. The formula is:

[0079] In the formula, is the normalized coordinate value; is the original coordinate value; is the maximum value of the original data; is the minimum value of the original data; Step 6: Construct a ResNet18-NAM-Agent multimodal regression model, fuse the ResNet18 network with the NAM attention mechanism to form a feature extraction network, select the proxy attention mechanism as the feature fusion module, and train the multimodal regression model with the sample data set to obtain a composite laminated structure impact location model based on the multimodal regression model; The structure of the ResNet18-NAM-Agent multimodal regression model is as Figure 7 shown.

[0080] The ResNet18-NAM feature extraction model uses ResNet18 as the backbone network for feature extraction and the NAM attention mechanism as the feature enhancement mechanism. Set the random seed to ensure that the shuffling methods of the two-modal data sets are the same. Use the AdamW optimizer to train the model, set the learning rate to 0.001, set the weight decay coefficient to 1e-4, use the cosine annealing scheduling algorithm to adjust the learning rate, set the number of iterations of model training to 50, set the image training batch to 10, and select Huber as the loss function. During the model training process, only calculate the loss for the time-frequency spectrogram data set generated by the continuous wavelet transform. The mathematical formula of the loss function is:

[0081] In the formula, is the output of the loss function; is the true value; is the predicted value; is the threshold parameter that adjusts the Huber loss function expression, preferably 3.0.

[0082] The training process of the ResNet18-NAM-Agent model includes: removing the average pooling layer and the fully connected layer of the ResNet18 network, and then constructing a two-dimensional time-frequency image feature extraction branch and a two-dimensional encoded image feature extraction branch respectively; Preferably, the ResNet18 network includes a 7×7 convolutional layer with a stride of 2 at the beginning, a 3×3 max pooling layer with a stride of 2, four residual layers, an average pooling layer, and a fully connected layer. Each residual layer includes two residual blocks, and the residual blocks are divided into downsampling residual blocks and general residual blocks. The first residual layer is composed of general residual blocks, and the second, third, and fourth residual layers are composed of a downsampling residual block and a general residual block. The structure of the general residual block is 3×3Conv+BN+ReLU+3×3Conv+BN, and the structure of the downsampling residual block is 3×3Conv+BN+ReLU+3×3Conv+BN+downsample. The NAM attention module includes a channel attention sub-module and a spatial attention sub-module; Convert the color image sizes of the two modalities of images in the multi-modal image sample dataset to 224×224×3, and input them into a 7×7 convolutional layer with a stride of 2 respectively to extract image features, and then input them into a 3×3 max pooling layer with a stride of 2 respectively to reduce the size of the feature map; For the feature extraction branch of the two-dimensional time-frequency spectrogram, remove the average pooling layer and the fully connected layer of the ResNet18 model, and insert the NAM attention module into the front positions of the first residual layer and the second residual layer of the ResNet18 model, and no attention mechanism is added to the latter two residual layers; The time-frequency spectrogram feature map processed by the max pooling layer successively enters two residual layers with NAM in front and two original residual layers to obtain the CWT feature map extracted by the feature extraction network; When entering the NAM attention mechanism, first enter the channel attention sub-module. In this module, the input feature map is normalized through the Batch Normalization layer, and the importance of each channel is calculated using the weight parameters of the Batch Normalization layer. The normalized weights are multiplied element-wise with the input feature map to generate a weighted feature map. Subsequently, the weighted feature map passes through the Sigmoid activation function to generate a channel attention weight map, which is multiplied element-wise with the original input feature map to obtain the output of the channel attention module; Then, enter the spatial attention sub-module. In this module, the input feature map is averaged in the channel dimension to obtain a spatial weight map. After the spatial weight map is normalized, it is multiplied element-wise with the input feature map to generate a weighted feature map. The weighted feature map passes through the Sigmoid activation function to generate a spatial attention weight map, which is multiplied element-wise with the original input feature map to obtain the feature map processed by the NAM module; For the feature extraction branch of the two-dimensional relative position encoding map, the average pooling layer and the fully connected layer of the ResNet18 model are removed, and the NAM attention module is inserted in front of the first residual layer, the second residual layer, and the third residual layer of the ResNet18 model, and behind the fourth residual layer; The feature maps processed by the max pooling layer successively enter three residual layers with NAM in front and one residual layer with NAM behind to obtain the RPM feature map extracted by the feature extraction network; Reshape the feature maps of the two modalities obtained in the above process into a sequence form to meet the input requirements of the proxy attention mechanism. The expression is:

[0083]

[0084] In the formula, is the sequence of the reshaped two-dimensional continuous wavelet transform time-frequency spectrum diagram; is the sequence of the reshaped two-dimensional relative position matrix encoding map; is the feature map of the two-dimensional time-frequency spectrum diagram after feature extraction; is the feature map of the two-dimensional encoding map after feature extraction; The proxy attention mechanism introduces a set of additional proxy features in the traditional attention module. It inherits the advantages of softmax and linear attention, enabling the model to establish relationships between different modalities and improving the feature fusion performance.

[0085] Take the sequence as the query Q and value V inputs of the proxy attention mechanism, Take

[0086] as the key K input to the proxy attention mechanism, and perform average pooling on the feature matrix of the query Q to obtain the proxy feature A. The expression is:

[0087] In the formula, is the proxy feature matrix; is the global average pooling; , , , , and are weight parameters, and are the original features; is the feature sequence obtained after the fusion of the proxy attention mechanism; is the softmax function; is the scaling factor; and is the position bias; is the depthwise separable convolution operation.

[0088] Take the average of the multi-modal fusion features in the sequence length dimension to generate a pooled feature vector, and input the pooled feature vector into a two-dimensional fully connected layer to obtain the two-dimensional coordinates of the impact position; Step 7: Perform impact localization on the test set image sample data through the impact localization model to finally determine the impact position.

[0089] Perform impact localization on the image data of the test data set through the trained ResNet18-NAM-Agent multi-modal regression model to obtain the final predicted coordinate values of the impact position.

[0090] Embodiment 2: On the basis of the foregoing embodiment, considering the problem that the traditional envelope entropy only evaluates the modal purity through the signal energy distribution and does not distinguish the differences in the energy attenuation characteristics of impact signals and noise in the time-frequency domain, a dynamic double-weight envelope entropy is added, and a time-domain attenuation factor is introduced to respectively quantify the energy concentration degree of the signal in the time domain and the sparsity in the frequency domain, and a composite fitness function is constructed to enable the moss growth optimization algorithm to more accurately screen out the modal components with both low noise and high impact characteristics.

[0091] The formula of the dynamic double-weight envelope entropy is defined as: In the formula, is the dynamic double-weight envelope entropy of the th modal component; is the total number of signal sample points; is the frequency-domain sparsity factor; is the time-domain attenuation factor; is the time point of the signal; is the frequency point of the signal; is the th envelope normalization value of the modal component.

[0092] Take the dynamic double-weight envelope entropy as the fitness function, set the population size of the moss growth optimization algorithm to 10, and the number of iterations to 19, The search range is [500, 60000]; In the wind direction update stage, preferentially select the modal component corresponding to for population iteration.

[0093] Verification shows that when obtaining an average error similar to that of the above embodiments, the signal-to-noise ratio of the reconstructed signal is increased from 18.6 dB of the traditional method to 24.3 dB; in the carbon fiber laminate test, the average absolute positioning error is reduced from 7.2 mm to 3.8 mm; the modal aliasing rate is reduced from 12.7% to 5.4%. The results show that this mechanism significantly improves the ability to distinguish impact features and noise components during the signal decomposition process by introducing a time-domain attenuation factor and a frequency-domain sparsity factor. The energy distribution of the denoised signal in the time-frequency domain after reconstruction highly matches that of the original impact event, effectively suppressing the modal aliasing phenomenon while preserving the integrity of the impact transient features.

[0094] Embodiment 3: On the basis of the foregoing embodiments, considering the problem that the traditional proxy attention mechanism only fuses multi-modal features through linear transformation and ignores the dynamic correlation between time-frequency features and spatial features in the spatio-temporal dimension, a spatio-temporal collaborative attention mechanism is added. The cross-modal interaction intensity is jointly adjusted through a time-domain convolutional gated unit and a spatial-domain dynamic weight to achieve feature complementarity between local details and global distributions.

[0095] The spatio-temporal collaborative attention formula is defined as:

[0096] In the formula, is the output of spatio-temporal collaborative attention; is element-wise multiplication; is the time-domain convolutional gate; is the spatial-domain dynamic weight.

[0097] After extracting time-frequency features and spatial features in the ResNet18-NAM branch, the feature sequence is input into the spatio-temporal collaborative attention module; The convolutional kernel parameters of the time-domain convolutional gated unit are initialized as a Gaussian distribution, and the dimension of the MLP hidden layer of the spatial-domain dynamic weight is 256; During training, the first 3 residual layers of the ResNet backbone network are frozen, and only the parameters of the attention module are optimized.

[0098] For example, assuming that the input feature CWT spectrogram feature vector is , and the RPM coding map feature vector is ; the parameters are set as follows: the time-domain convolutional gate convolution kernel is , the spatial-domain dynamic weight MLP weight matrix is , the bias is , the depthwise separable convolution weight is , the position bias is , .

[0099] One-dimensional convolution result:

[0100] Sigmoid gating value:

[0101] Global average pooling:

[0102] MLP output:

[0103] Spatial domain dynamic weight;

[0104] Original score:

[0105] Adjusted score:

[0106] Attention weighting value:

[0107] DWC operation:

[0108] Final output:

[0109] The effect of the spatio-temporal collaborative attention mechanism is shown in Table 1.

[0110] Table 1. Summary of the effect of the spatio-temporal collaborative attention mechanism

[0111] According to the experimental table, the number of model parameters only increases by 1.2%, but the cross-modal feature interaction speed increases by 37%; the average Euclidean distance error of the test set decreases from 4.1 mm to 2.5 mm; in a strong noise environment with a signal-to-noise ratio of 10 dB, the error fluctuation range shrinks from ±3.8 mm to ±1.6 mm. The results show that this mechanism realizes the deep interaction between time-frequency features and spatial distribution features through the collaborative modulation of time-domain convolutional gating and spatial domain dynamic weights. The robustness of the model to complex noise and boundary reflection interference is significantly improved, and the computational efficiency of cross-modal feature fusion is optimized.

[0112] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable non-transitory storage media containing computer-usable program code.

[0113] The present invention can provide computer program instructions to a management platform of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the management platform of the computer or other programmable data processing devices generate means for implementing the system.

[0114] These computer program instructions can also be stored in a computer-readable memory capable of guiding a computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions of the system.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing devices, so that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable devices provide steps for implementing the functions of the system.

Claims

1. A method for locating low - velocity impact of a multi - modal fusion composite laminate structure, characterized in that, Including: Obtain the impact response signals of multiple strain sensors on the composite laminated structure and combine them into a composite impact signal; Taking the envelope entropy as the fitness function, use the moss growth optimization algorithm to optimize the penalty coefficient of the successive variational mode decomposition; Perform successive variational mode decomposition on the composite impact signal based on the optimal penalty coefficient, and select the partial mode components with the largest energy distribution correlation coefficient to reconstruct the denoised signal; Convert the denoised signal into a time-frequency spectrogram, and convert the original composite signal into a two-dimensional coding map through the relative position matrix algorithm; Construct a multi-modal sample data set including the time-frequency spectrogram and the coding map; Train a multi-modal regression model and perform impact positioning.

2. The method according to claim 1, wherein: The moss growth optimization algorithm includes a wind direction determination mechanism, spore diffusion search, dual reproduction search and cryptobiotic mechanism, wherein the wind direction is dynamically adjusted by the position relationship between the optimal individual and most individuals in the population.

3. The method according to claim 2, wherein: The update formula of the spore diffusion search is: Wherein, is the wind intensity attenuation factor; is the current number of evaluations; is the maximum number of evaluations; is the proportion of the number of mosses in to the total number of moss individuals; used to count the number of elements; is the set of most moss individuals; is the total number of moss individuals; is the distance of spore transmission under stable wind conditions; is the constant parameter 2; is the distance of spore transmission under turbulent wind conditions; is the th moss individual new moss obtained by spore transmission; is a moss individual in the current moss population; is the wind direction; is 0.2; , and are random numbers between 0 and 1.

4. The method according to claim 1, wherein: The constraint criteria of the successive variational mode decomposition include minimizing the frequency domain bandwidth of the mode components, the spectral overlap between the residual signal and the mode components, and the energy of the current mode near the central frequency of the historical mode.

5. The method according to claim 1, wherein: The calculation method of the energy distribution correlation coefficient is: In the formula, is the energy distribution correlation coefficient; is the energy distribution of the modal component; is the energy distribution of the original signal; is the covariance; is the variance.

6. The method according to claim 1, wherein: The construction of the multi-modal sample data set includes: Add Gaussian, Poisson, salt and pepper, and multiplicative noise to the time-frequency spectrogram, and perform flipping, contrast enhancement and histogram equalization on the coding map.

7. The method according to claim 1, wherein: In the multi-modal regression model, ResNet18 is fused with the normalization attention mechanism. The proxy attention mechanism is used for multi-modal feature fusion. The channel attention calculates the channel importance through the Batch Normalization weights, and the spatial attention generates spatial weights through the channel mean; The query and value of the proxy attention mechanism come from the time-frequency spectrogram feature sequence, and the key comes from the coding map feature sequence. The fusion formula is: In the formula, is the proxy feature matrix; is the global average pooling; , , , , and are weight parameters, and are the original features; is the feature sequence obtained after the fusion of the proxy attention mechanism; is the softmax function; is the scaling factor; and are the position biases; is the depthwise separable convolution operation.

8. The method according to claim 1, wherein: It also includes the dynamic double-weight envelope entropy step. By quantifying the energy concentration in the time domain and the sparsity in the frequency domain of the signal, a composite fitness function is constructed. The double-weight envelope entropy formula is defined as: In the formula, is the dynamic double-weighted envelope entropy of the th modal component; is the total number of signal sample points; is the frequency-domain sparsity factor; is the time-domain decay factor; is the time point of the signal; is the frequency point of the signal; is the envelope normalization value of the th modal component.

9. The method according to claim 1, wherein: It also includes the spatio-temporal collaborative attention step. The cross-modal interaction intensity is jointly adjusted through the time-domain convolutional gated unit and the spatial-domain dynamic weights. The spatio-temporal collaborative attention formula is defined as: Wherein, is the spatio-temporal collaborative attention output; is the per-channel multiplication; is the temporal convolutional gating; is the spatial dynamic weight.

10. A computer-readable storage medium storing a computer program therein, characterized in that: The computer program, when run by a processor, executes the method according to any one of claims 1-9.

Images