Laser strengthening on-line monitoring method and device fusing acousto-optic thermal signals

Abstract

The invention provides a laser strengthening online monitoring method and device fusing acousto-optic thermal signals, and relates to the technical field of laser strengthening, the method comprises the following steps: acquiring an acoustic emission signal, a spectral signal and a thermal infrared image in a laser strengthening process; respectively preprocessing the acoustic emission signal, the spectral signal and the thermal infrared image to obtain a standard acoustic signal, standard spectral data and a standard thermal infrared image; carrying out feature extraction according to the standard sound signal, the standard spectral data and the standard thermal infrared image to respectively obtain a sound signal feature, a spectral feature and a temperature feature; and a quality prediction model is constructed and trained, the sound signal features, the spectral features and the temperature features are input into the quality prediction model for laser strengthening quality prediction, a real-time quality index is obtained, and the quality prediction model is constructed based on a low-rank multi-modal fusion model. According to the method, the quality index is generated in real time based on the acousto-optic thermal signals in the laser strengthening process, and the laser strengthening machining process is accurately controlled.

Description

Technical Field

[0001] The present invention relates to the field of laser enhancement technology, and in particular to a laser enhancement online monitoring method and device integrating acoustic, optical and thermal signals. Background Art

[0002] As industrial manufacturing continues to accelerate toward high precision, high efficiency, and intelligent manufacturing, ultrafast laser enhancement technology, leveraging the cold processing properties of its ultrashort pulses, is demonstrating irreplaceable advantages in a variety of high-tech fields, including microelectronics, aerospace, and biomedicine. By utilizing extremely short laser pulses, this technology enables thermally damage-free processing of materials. This allows for the effective fabrication of micro- and nanostructures, cutting of brittle materials, and surface amorphization, significantly improving device performance and reliability.

[0003] However, in practical applications, the interaction between ultrafast lasers and materials involves complex physical and chemical processes, including plasma excitation, shock wave propagation, and phase transition behavior, accompanied by the transient evolution of multiple modal signals such as acoustic, optical, and thermal. This results in a lack of effective online monitoring methods, making it difficult to achieve precise control of ultrafast laser enhanced processing, impacting both the consistency of processing quality and the improvement of processing efficiency. Summary of the Invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method and device for online monitoring of laser enhancement that integrates acoustic, optical, and thermal signals, which effectively solves the current problem of lack of effective online monitoring means and the difficulty in achieving precise control of the ultrafast laser enhancement processing process.

[0005] In a first aspect, the present invention provides a method for online monitoring of laser enhancement by integrating acoustic, optical, and thermal signals, the method comprising: Acquire acoustic emission signals, spectral signals and thermal infrared images during laser strengthening process; Preprocessing the acoustic emission signal, the spectral signal, and the thermal infrared image respectively to obtain a standard acoustic signal, a standard spectral data, and a standard thermal infrared image; Perform feature extraction based on the standard acoustic signal, the standard spectral data, and the standard thermal infrared image to obtain acoustic signal features, spectral features, and temperature features, respectively; A quality prediction model is constructed and trained, and the acoustic signal features, the spectral features, and the temperature features are input into the quality prediction model to predict the quality of laser enhancement to obtain a real-time quality index. The quality prediction model is constructed based on a low-rank multimodal fusion model.

[0006] In an optional embodiment, the quality prediction model includes: A tensor construction unit, configured to construct a third-order tensor according to the acoustic signal feature, the spectral feature, and the temperature feature; A low-rank decomposition unit, configured to decompose the third-order tensor into a shared core tensor and a plurality of modal factor matrices; an attention weighting unit, configured to perform attention weighting on the shared core tensor to obtain a weighted core tensor; The fully connected layer unit is used to perform feature extraction on the weighted kernel tensor to obtain the real-time quality index.

[0007] In an optional embodiment, the step of training the quality prediction model includes: Extract historical acoustic signal features, historical spectral features, and historical temperature features based on historical multi-source heterogeneous data of the laser strengthening process; Constructing a historical third-order tensor according to the historical acoustic signal features, the historical spectral features, and the historical temperature features and setting corresponding feature labels to obtain a training set, a validation set, and a test set; Performing tensor decomposition on the training set using a high-order singular value decomposition algorithm to obtain a low-rank factor matrix; generating a fusion vector according to the low-rank factor matrix, inputting the fusion vector into an initial quality prediction model for prediction, and obtaining a historical quality index; Inputting the historical acoustic signal characteristics and the historical spectral characteristics into an initial quality prediction model for prediction respectively to obtain an acoustic signal index and a spectral index; constructing a loss function based on the historical quality index, the acoustic signal index, and the spectral index; Optimizing the parameters of the initial quality prediction model according to the loss function and the validation set to obtain a target quality prediction model; The target quality prediction model is evaluated according to the test set to obtain the quality prediction model.

[0008] In an optional embodiment, the acoustic signal feature, the spectral feature, and the temperature feature are input into the quality prediction model to perform quality prediction of laser enhancement to obtain a real-time quality index, including: Performing feature vector splicing according to the acoustic signal feature, the spectral feature, and the temperature feature to obtain a third-order tensor; Performing tensor decomposition on the third-order tensor to obtain the shared core tensor and three modal factor matrices; Convolving the shared core tensor to generate attention weights, and weighting the shared core tensor element by element according to the attention weights to obtain a weighted core tensor; The weighted kernel tensor is flattened into a target vector, and the target vector is input into a fully connected layer to obtain the real-time quality index.

[0009] In an optional embodiment, the acquiring of acoustic emission signals, spectral signals and thermal infrared images during the laser strengthening process includes: Acquiring the acoustic emission signal of the interaction between the material and the laser during the laser strengthening process by an acoustic emission sensor; Detecting the spectral signal of the plasma in the laser strengthening process in real time by a spectrometer; The thermal infrared image of the material during the laser strengthening process is collected by an infrared thermal imager.

[0010] In an optional embodiment, the preprocessing of the acoustic emission signal, the spectral signal and the thermal infrared image to obtain a standard acoustic signal, a standard spectral data and a standard thermal infrared image respectively includes: performing amplitude adjustment and bandpass filtering on the acoustic emission signal, and separating harmonics and shock waves to obtain the standard acoustic signal; performing invalid spectrum screening and spectrum averaging processing on the spectrum signal to obtain the standard spectrum data; Perform median filtering on the thermal infrared image to obtain the standard thermal infrared image.

[0011] In an optional embodiment, the extracting features based on the standard acoustic signal, the standard spectral data, and the standard thermal infrared image to obtain acoustic signal features, spectral features, and temperature features includes: Extracting time domain features and frequency domain features from the standard acoustic signal to obtain the acoustic signal features; Performing normalization processing on the standard spectral data to obtain the spectral characteristics; The temperature difference gradient is extracted according to the standard thermal infrared image to obtain the temperature feature.

[0012] In a second aspect, the present invention provides a laser enhancement online monitoring device integrating acoustic, optical and thermal signals, the device comprising: A data acquisition module is used to obtain acoustic emission signals, spectral signals and thermal infrared images during the laser strengthening process; a data processing module, configured to pre-process the acoustic emission signal, the spectral signal, and the thermal infrared image, respectively, to obtain a standard acoustic signal, standard spectral data, and a standard thermal infrared image; a feature extraction module, configured to extract features based on the standard acoustic signal, the standard spectral data, and the standard thermal infrared image, and obtain acoustic signal features, spectral features, and temperature features, respectively; The quality monitoring module is used to construct and train a quality prediction model, input the acoustic signal characteristics, the spectral characteristics and the temperature characteristics into the quality prediction model to predict the quality of laser enhancement, and obtain a real-time quality index. The quality prediction model is constructed based on a low-rank multimodal fusion model.

[0013] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the laser enhancement online monitoring method integrating acoustic, optical, and thermal signals as described in the first aspect of the present invention.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the laser enhancement online monitoring method integrating acoustic, optical and thermal signals as described in the first aspect of the present invention.

[0015] The online monitoring method and device for laser enhancement that integrates acoustic, optical, and thermal signals, provided by the present invention, can acquire the acoustic, optical, and thermal signals during the interaction between the laser and the material in real time. Feature extraction of these signals is then performed based on a quality prediction model, which then fuses these features. Parallel decomposition and attention weight allocation eliminate the time delay and scale differences between heterogeneous data, accurately analyze the coupling relationship between acoustic, optical, and thermal signals, and generate a quality index in real time. This real-time quality index is used to adjust the operating parameters of the laser enhancement process, enabling precise control of the laser enhancement process and improving its quality and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is a first schematic diagram of the process of the laser enhancement online monitoring method integrating acoustic, optical and thermal signals provided by an embodiment of the present invention; Figure 2 This is a second schematic diagram of the process of the laser enhancement online monitoring method integrating acoustic, optical and thermal signals provided by an embodiment of the present invention; Figure 3 This is a third schematic diagram of the process of the laser enhancement online monitoring method integrating acoustic, optical and thermal signals provided by an embodiment of the present invention; Figure 4 This is a fourth schematic diagram of the process of the laser enhancement online monitoring method integrating acoustic, optical and thermal signals provided by an embodiment of the present invention; Figure 5 This is a fifth schematic diagram of the process of the laser enhancement online monitoring method integrating acoustic, optical and thermal signals provided by an embodiment of the present invention; Figure 6 Schematic diagram of the structure of a laser enhancement online monitoring device integrating acoustic, optical and thermal signals provided by an embodiment of the present invention; Figure 7 It is a structural diagram of an electronic device provided by an embodiment of the present invention.

[0018] 200. Laser enhancement online monitoring device integrating acoustic, optical and thermal signals; 210. Data acquisition module; 220. Data processing module; 230. Feature extraction module; 240. Quality monitoring module; 300. Electronic device; 310. Processor; 320. Communication interface; 330. Memory; 340. Communication bus. DETAILED DESCRIPTION

[0019] To make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be further clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be noted that the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present invention.

[0022] Ultrafast laser enhancement technology utilizes extremely short laser pulses to achieve heat-free material processing. This allows for the effective fabrication of micro-nanostructures, cutting of brittle materials, or surface amorphization, significantly improving device performance and reliability. However, in practical applications, the interaction between ultrafast lasers and materials involves complex physical and chemical processes, including plasma excitation, shock wave propagation, and phase transition behavior of materials, accompanied by the transient evolution of multiple modal signals such as acoustic, optical, and thermal signals. This results in a lack of effective online monitoring methods, making it difficult to achieve precise control of the ultrafast laser enhancement process, impacting the consistency of processing quality and the improvement of processing efficiency.

[0023] Example 1 The embodiment of the present invention provides a laser enhancement online monitoring method that integrates acoustic, optical and thermal signals, which effectively solves the problem of the current lack of effective online monitoring means and the difficulty in achieving precise control of the ultrafast laser enhancement processing process. Figure 1 This is a first schematic diagram of the process of the laser enhancement online monitoring method integrating acoustic, optical and thermal signals provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps: S100: Acquire acoustic emission signals, spectral signals, and thermal infrared images during the laser strengthening process.

[0024] In an embodiment of the present invention, a quality monitoring study is conducted on a laser strengthening process based on multi-source heterogeneous data, where the multi-source heterogeneous data includes acoustic emission signals, spectral signals, and thermal infrared images. Figure 2 This is a second schematic diagram of the process of the laser enhancement online monitoring method integrating acoustic, optical and thermal signals provided by an embodiment of the present invention. Figure 2 As shown, data acquisition specifically includes the following steps: S110. Collecting acoustic emission signals of the interaction between the material and the laser during the laser strengthening process through an acoustic emission sensor.

[0025] During the laser strengthening process, the interaction between the material and the high-energy laser beam triggers complex microstructural changes. These structural evolutions release strain energy in the form of elastic waves, generating acoustic emission signals. In an embodiment of the present invention, an acoustic emission sensor detects the shock waves within the material in real time. The characteristics of the acoustic emission signal, such as frequency, amplitude, and duration, are closely related to the material's microstructural changes and defect formation, and can provide a real-time reflection of the physical phenomena occurring during the material surface strengthening process.

[0026] S120. Detecting the spectral signal of the plasma during the laser strengthening process in real time using a spectrometer.

[0027] In an embodiment of the present invention, the laser interacts with the material to generate plasma, and the spectrometer can monitor the spectral characteristics of the plasma in real time, measure the broadening characteristics of the current spectral line, and obtain spectral signals, thereby providing information about laser energy absorption, material removal and surface modification.

[0028] S130. Collect thermal infrared images of the material during the laser strengthening process using an infrared thermal imager.

[0029] During laser strengthening, materials are thermosensitive. When subjected to temperature changes, their resistance, voltage, or other properties also change accordingly. In an embodiment of the present invention, an infrared thermal imager captures thermal infrared images of the material during laser strengthening. This allows real-time monitoring of temperature changes, plasma volume, and plasma jet profile, reflecting the laser energy and component composition.

[0030] S200 , pre-processing the acoustic emission signal, the spectrum signal, and the thermal infrared image respectively to obtain a standard acoustic signal, a standard spectrum data, and a standard thermal infrared image.

[0031] The collected acoustic emission signals, spectral signals and thermal infrared images also need to be pre-processed such as denoising and sampling to filter out noise and enhance useful information. Figure 3 This is the third schematic diagram of the process of the laser enhancement online monitoring method integrating acoustic, optical and thermal signals provided by an embodiment of the present invention. Figure 3 As shown, the preprocessing specifically includes the following steps: S210 , performing amplitude adjustment and band-pass filtering on the acoustic emission signal, and separating harmonics and shock waves to obtain a standard acoustic signal.

[0032] Optionally, acoustic equalization technology is used to adjust the amplitude of different frequency bands of the acoustic emission signal. By accurately adjusting the amplitude of different frequency bands, the signal closely related to the surface enhancement process can be targeted and enhanced, while effectively reducing the interference of background noise, making the target acoustic emission signal more prominent. Specifically, first, a fast Fourier transform is performed on the original acoustic emission signal to obtain its spectrum information, and the spectrum is accurately divided into multiple frequency bands according to the target frequency range rule. Then, the power spectrum density estimation method Welch method is used to calculate the sum of the power spectrum density of each frequency band to obtain the frequency band energy. The weight factor of each frequency band is calculated based on the frequency band energy of each frequency band. The calculation formula of the weight factor is as follows:

[0033] In the above formula, w i Indicates the i The weighting factor for each frequency band, PSD ref ( i) means that the signal is in the first i The ideal energy distribution in each frequency band is PSD raw ( i ) represents the first i The frequency band energy of each frequency band.

[0034] To prevent excessive noise amplification, a piecewise mapping function is used to constrain the weighting factor to a target range. A sliding average is then applied to obtain a smoothly varying gain coefficient. The original spectrum is then multiplied by the gain coefficient for the corresponding frequency band to achieve frequency domain equalization. Finally, an inverse fast Fourier transform is performed on the equalized spectrum to convert the signal from the frequency domain back to the time domain, resulting in a denoised and enhanced time domain signal, completing the amplitude adjustment process.

[0035] The amplitude-adjusted signal is then filtered using bandpass filtering technology to retain only the signal components within a specific frequency range and remove high-frequency noise and low-frequency interference.

[0036] Finally, harmonic-shock wave source separation technology is used to further isolate the impact sound components associated with the surface strengthening process. This effectively removes interference from ambient noise and other irrelevant signals, significantly improving the signal-to-noise ratio (SNR) of the acoustic signal and obtaining a standard acoustic signal. Specifically, a first-order high-pass filter is first used to pre-emphasize the bandpass filtered signal, suppressing low-frequency drift and highlighting the high-frequency portion. A short-time Fourier transform is then used to perform time-frequency decomposition on the pre-emphasized signal, generating a time-frequency matrix that displays the time-frequency energy distribution. For each time frame in the time-frequency matrix, harmonic-shock detection is then performed along the frequency axis. The spectral flatness of each time frame is calculated and compared with a spectral flatness threshold to isolate the time-frequency region corresponding to the harmonic component. If the spectral flatness of a time frame is less than the spectral flatness threshold, the frame is predominantly harmonic, and the harmonic mask is marked as 1; otherwise, it is marked as 0, thus preliminarily isolating the time-frequency region corresponding to the harmonic component. The shock index is then calculated based on the characteristics of the main peak of the transient shock in the laser-induced plasma. This index measures the intensity of the shock component in each time frame. If the shock index for a time frame is greater than the shock index threshold, it is considered to have a significant shock component in that time frame and the shock mask is marked as 1; otherwise, it is marked as 0, thereby isolating the time-frequency region corresponding to the shock component. Finally, a 1-D median filter is used to process the resulting harmonic and shock masks. The harmonic and shock components are then extracted. The signal is then separated into these components in the time-frequency domain. The separated harmonic and shock components are then subjected to inverse short-time Fourier transforms and overlap-added to obtain the standard acoustic signal.

[0037] S220 , performing invalid spectrum screening and spectrum averaging processing on the spectrum signal to obtain standard spectrum data.

[0038] In this embodiment of the present invention, the collected spectral signals are analyzed to identify characteristic spectral lines, and the intensity values ​​of each characteristic spectral line in each spectrum are extracted. Based on the extracted characteristic spectral line intensities, the standard deviation is calculated and compared with a deviation threshold. If the standard deviation of a spectrum is less than the deviation threshold, the spectrum is determined to be invalid and removed from the original spectral signal, thereby obtaining a valid spectrum.

[0039] Then the effective spectrum is subjected to spectral averaging. By taking the arithmetic average or weighted average of the intensity values ​​of multiple effective spectra at the same wavelength, the influence of random noise can be effectively reduced, the spectrum can be smoother, and the spectral signal-to-noise ratio can be improved, thereby improving the quality and reliability of the spectral data and obtaining standard spectral data.

[0040] S230 , performing median filtering on the thermal infrared image to obtain a standard thermal infrared image.

[0041] In an embodiment of the present invention, a median filter is used to filter thermal infrared images. First, an appropriate window size is selected, which determines the filter's ability to smooth noise and retain edge information. The window size should be determined based on the noise level and detailed characteristics of the thermal infrared image. The selected window is then slid pixel by pixel across the thermal infrared image. The pixel values ​​within each window are sorted, and the median of the sorted values ​​is used to replace the original value of the center pixel of the window. This process is repeated until the entire thermal infrared image has been traversed, effectively smoothing noise while retaining significant temperature characteristics. The filtered thermal infrared image may contain minor imperfections or uneven areas. Smoothing and edge enhancement can be performed to further improve the quality of the thermal infrared image and obtain a standard thermal infrared image.

[0042] S300 , performing feature extraction based on the standard acoustic signal, standard spectral data, and standard thermal infrared image to obtain acoustic signal features, spectral features, and temperature features, respectively.

[0043] Figure 4 This is a fourth schematic diagram of the process of the laser enhancement online monitoring method integrating acoustic, optical and thermal signals provided by an embodiment of the present invention. Figure 4 As shown in Figure 2, feature extraction specifically includes the following steps: S310 , extracting time domain features and frequency domain features respectively according to the standard sound signal to obtain sound signal features.

[0044] In the embodiment of the present invention, the time domain characteristics include the amplitude envelope, root mean square energy and zero-crossing rate, which indirectly reflect the stress changes and defect formation during the material surface strengthening process.

[0045] The amplitude envelope is used to reflect the overall energy fluctuations of the acoustic signal in the time domain. Its extraction steps are as follows: First, perform a Hilbert transform on the original acoustic signal to obtain its orthogonal components and construct an analytical signal. Then, the modulus of the analytical signal, i.e., the instantaneous envelope, is calculated. This value represents the instantaneous energy of the signal at each sampling point. Finally, the instantaneous envelope of the entire frame is averaged to obtain the amplitude envelope value of the frame. The larger the amplitude envelope value, the more concentrated the acoustic signal energy within the frame. During the material strengthening process, sudden changes in the amplitude envelope may correspond to stress release or defect generation, providing a key basis for monitoring the laser strengthening process.

[0046] Root mean square energy is a key feature for measuring the average energy level of an acoustic signal in the time domain. The extraction steps are as follows: divide the original acoustic signal into frames according to a fixed frame length and frame shift to ensure that the length of each frame signal is consistent, obtain multiple signal sampling points, and perform root mean square calculation on all sampling points to obtain the root mean square energy. The root mean square energy is the effective value of the signal energy and is proportional to the internal stress release strength or impact energy of the material.

[0047] The zero-crossing rate is an important time-domain feature that reflects the high-frequency components and transient impulse characteristics of an acoustic signal. Its extraction steps are as follows: The original acoustic signal is divided into multiple frames according to a fixed frame length and frame shift to obtain a signal. These multiple frames are then subjected to DC removal. The DC-removed signal is then traversed, and a symbol sequence is generated based on the positive and negative signs of the sampling points. This sequence marks the polarity of the signal at each sampling point. The number of symbol changes between adjacent sampling points is then calculated to obtain the total number of zero crossings. This sign change indicates that the signal crosses the zero axis, and the number of such changes directly reflects the frequency of high-frequency oscillations. Finally, the zero-crossing rate is calculated by dividing the total number of zero crossings by the number of adjacent points. A higher zero-crossing rate indicates a higher number of high-frequency components or sharper pulses in the acoustic signal, while a lower zero-crossing rate indicates a predominance of low-frequency components.

[0048] In the embodiment of the present invention, the frequency domain features include spectral centroid, sub-band bandwidth, spectral roll-off, and bandwidth energy ratio. These features reveal the frequency distribution and energy concentration of the acoustic signal, and further reflect changes in the internal structure of the material. The spectral centroid represents the center of gravity of the frequency components of an acoustic signal, reflecting the high-frequency or low-frequency bias of the energy distribution. Specifically, each frame of the acoustic signal is windowed and subjected to a Fast Fourier Transform (FFT) to obtain an amplitude spectrum. The normalized power spectrum is then calculated. The frequency is then multiplied by the normalized power spectrum value corresponding to each frame of the acoustic signal, and the sum is calculated. The resulting sum is the spectral centroid.

[0049] Subband bandwidth is used to quantify the energy dispersion of a signal within a specific frequency band and reveal the concentration of local frequency components. Specifically, the frequency band of the acoustic signal is divided into multiple subbands, and the local spectral centroid of each subband is calculated. The subband bandwidth is calculated based on the local spectral centroid, normalized power spectrum, and frequency of each subband. The calculation formula is as follows:

[0050] In the above formula, SBW k represents the subband bandwidth, n represents the total number of subbands, represents the frequency of the sound signal, B i Indicates the i A sub-band, Indicates the i The local spectral centroid of the subbands, Indicates the i The normalized power spectrum of the subbands.

[0051] Spectral roll-off identifies the critical frequency points where signal energy is concentrated in low or high frequencies, reflecting the spectrum attenuation characteristics. Specifically, the spectrum roll-off is determined by finding the minimum frequency at which the cumulative power reaches a preset target ratio of the total power. In this embodiment of the present invention, the preset target ratio is set to 85%.

[0052] The bandwidth energy ratio quantifies the proportion of energy within a specific bandwidth to the total energy, revealing the local concentration of energy in the signal's frequency domain. Specifically, the difference between the spectral roll-off and the spectral centroid is calculated to obtain the effective bandwidth, which represents the frequency range where the energy is concentrated. The bandwidth energy ratio is then calculated by taking the ratio of the normalized power sum within the effective bandwidth to the normalized power sum across the entire frequency band to obtain the ratio, which represents the proportion of energy within the effective bandwidth to the total energy.

[0053] S320: Perform normalization processing according to the standard spectrum data to obtain spectrum characteristics.

[0054] In this embodiment of the present invention, the standard spectral data is normalized using the maximum-minimum normalization method, mapping the standard spectral data to the range [0, 1] to obtain spectral features. Through normalization, the spectral features can be unified to the same scale, facilitating subsequent model learning.

[0055] S330: Extracting temperature gradients based on the standard thermal infrared image to obtain temperature features.

[0056] In an embodiment of the present invention, a standard thermal infrared image is converted into a temperature matrix, and a Sobel operator or a Laplace operator is used to calculate a temperature difference gradient, which reflects the rate of change of the temperature field in space.

[0057] S400. Build and train a quality prediction model, input acoustic signal features, spectral features, and temperature features into the quality prediction model to predict the quality of laser enhancement, and obtain a real-time quality index. The quality prediction model is built based on a low-rank multimodal fusion model.

[0058] In an embodiment of the present invention, the quality prediction model is constructed based on a low-rank multimodal fusion model. The low-rank multimodal fusion model is a model that achieves efficient multimodal data fusion through low-rank tensor decomposition technology. Its core lies in decomposing the weight tensor into modality-specific low-rank factors, thereby reducing computational complexity and improving fusion efficiency. The quality prediction model includes a tensor construction unit, a low-rank decomposition unit, an attention weighting unit, and a fully connected layer unit, where: The tensor construction unit is used to construct a third-order tensor based on the acoustic signal features, spectral features, and temperature features. In an embodiment of the present invention, the tensor construction unit performs time-frequency analysis on the acoustic signal features to extract 9-dimensional features as acoustic emission modes, which are used to capture transient impact energy distribution. For the spectral features, 256-dimensional effective spectral line intensities are retained as spectral modes to reflect the dynamic changes in elemental composition, and for the temperature features, 64-dimensional spatial temperature distribution features are generated as temperature modes. Finally, the three features are strictly aligned in time and spliced ​​into a third-order tensor along the channel dimension to achieve efficient fusion of multimodal data.

[0059] The low-rank decomposition unit is used to decompose the third-order tensor into a shared core tensor and multiple modal factor matrices. Optionally, the low-rank decomposition unit achieves compact representation and feature fusion of multimodal data through Tucker decomposition. Using a fixed rank group (r1, r2, r3) = (4, 8, 4), the third-order tensor is decomposed into a shared core tensor, an acoustic emission modal factor matrix, a spectral modal factor matrix, and a temperature modal factor matrix. The shared core tensor captures common cross-modal interaction patterns, the acoustic emission modal factor matrix maps 9-dimensional acoustic signal features to a low-rank space, the spectral modal factor matrix compresses 256-dimensional spectral features to an 8-dimensional latent representation, and the temperature modal factor matrix reduces the dimensionality of 64-dimensional temperature gradient features. The column vectors of the factor matrix constitute a low-rank basis for each modality in the common subspace, achieving efficient fusion and redundancy removal of cross-modal information.

[0060] The attention weighting unit is used to apply attention weights to the shared core tensor to obtain a weighted core tensor. This unit enhances the shared core tensor's ability to express key features through a dynamic focusing mechanism. A 1×1×1 convolution is first applied to the shared core tensor to generate attention weights. The shared core tensor and the attention weights are then element-wise multiplied to obtain a weighted core tensor. The attention weighting unit adaptively adjusts the weights of each channel in the shared core tensor to emphasize the cross-modal interaction patterns most relevant to the current task while suppressing redundant information, thereby improving the dynamics and discriminability of feature fusion.

[0061] Fully connected layer units are used to transform the weighted core tensor to obtain a real-time quality index. These units convert the weighted core tensor into an interpretable real-time quality index through nonlinear mapping. The weighted core tensor is first flattened into a 128-dimensional feature vector. Feature compression and regression are then performed through a two-layer fully connected network: the first layer compresses from 128 dimensions to 32 dimensions, using the ReLU activation function to enhance nonlinear expression. The second layer compresses from 32 dimensions to 1 dimension, mapping the output to the [0, 1] interval using the Sigmoid function. This ultimately generates a real-time quality index that directly reflects the laser enhancement process status and product quality level represented by the current multi-source heterogeneous data.

[0062] In the embodiment of the present invention, after the initial quality prediction model is constructed, training optimization is performed. The training steps of the quality prediction model are as follows: First, historical acoustic signal features, historical spectral features, and historical temperature features are extracted from the historical multi-source heterogeneous data of the laser hardening process. A historical third-order tensor is constructed based on these historical acoustic signal features, historical spectral features, and historical temperature features, and corresponding feature labels are assigned to obtain training, validation, and test sets. In this embodiment of the present invention, 2000 sets of historical multi-source heterogeneous data from the laser hardening process are obtained for feature extraction. Each set contains 9-dimensional historical acoustic signal features, 256-dimensional spectral features, and 64-dimensional temperature features within a 10ms sliding window. Based on these features, a historical third-order tensor is constructed and corresponding hardness, residual stress, and defect area labels are assigned. The training, validation, and test sets are divided into 7:2:1 ratios.

[0063] Then, three tasks are jointly trained. The main task uses a high-order singular value decomposition algorithm to perform tensor decomposition on the training set to obtain a low-rank factor matrix. A fusion vector is generated based on the low-rank factor matrix, and the fusion vector is input into the initial quality prediction model for prediction to obtain the historical quality index. The two subtasks input the historical acoustic signal features and historical spectral features into the initial quality prediction model for prediction, respectively, to obtain the acoustic signal index and spectral index, respectively verifying the independent contributions of the acoustic emission mode and the spectral mode.

[0064] The loss function is constructed based on the historical quality index, the acoustic signal index and the spectral index. The expression of the loss function is as follows:

[0065] In the above formula, L represents the total loss, L Q Indicates the main task loss, L AE Indicates the loss of acoustic signal. L OES Indicates spectral loss.

[0066] Tune the parameters of the initial quality prediction model based on the loss function and validation set to obtain the target quality prediction model. Optionally, use the AdamW optimizer to tune the parameters of the initial quality prediction model, setting the learning rate to 0.001, the batch size to 32, and training for 100 epochs. Terminate training if the mean absolute error on the validation set does not decrease over five consecutive epochs to avoid overfitting.

[0067] The target quality prediction model is evaluated based on the test set to obtain a quality prediction model. Optionally, the target quality prediction model is evaluated using metrics such as mean absolute error, precision, and recall to ultimately obtain a trained quality prediction model.

[0068] Based on the constructed and trained quality prediction model, the real-time extracted acoustic signal features, spectral features and temperature features are input into the quality prediction model to predict the quality of laser enhancement, and a real-time quality index can be obtained. Figure 5 FIG5 is a fifth schematic diagram of the process of the laser enhancement online monitoring method integrating acoustic, optical and thermal signals provided by an embodiment of the present invention. Figure 5 As shown, predicting the real-time quality index specifically includes the following steps: S410 , performing feature vector concatenation according to the acoustic signal features, spectral features, and temperature features to obtain a third-order tensor.

[0069] In the embodiment of the present invention, the tensor construction unit splices the acoustic signal features, the spectral features, and the temperature features into a third-order tensor according to the time dimension.

[0070] S420. Perform tensor decomposition on the third-order tensor to obtain a shared core tensor and three modal factor matrices.

[0071] In an embodiment of the present invention, the low-rank decomposition unit adopts a fixed rank group (r1, r2, r3) = (4, 8, 4) to decompose the third-order tensor into a shared core tensor, an acoustic emission modal factor matrix, a spectral modal factor matrix and a temperature modal factor matrix, thereby compressing the parameter quantity.

[0072] S430. Convolve the shared core tensor to generate attention weights, and weight the shared core tensor element by element according to the attention weights to obtain a weighted core tensor.

[0073] In an embodiment of the present invention, the attention weighting unit performs a 1×1×1 convolution on the shared core tensor to generate attention weights, and then multiplies the shared core tensor and the attention weights element-by-element to obtain a weighted core tensor, thereby realizing dynamic focusing of defect-sensitive features.

[0074] S440. Flatten the weighted kernel tensor into a target vector, input the target vector into a fully connected layer, and obtain a real-time quality index.

[0075] In this embodiment of the present invention, the fully connected layer unit flattens the weighted kernel tensor into a 128-dimensional feature vector. This is then passed through a two-layer fully connected network for feature compression and regression, outputting a real-time quality index. This real-time quality index allows for real-time monitoring of laser enhancement quality. If quality is poor, millisecond-level adaptive adjustments to laser parameters such as power, pulse width, and scan path are made.

[0076] The online monitoring method for laser enhancement, which integrates acoustic, optical, and thermal signals, provided by embodiments of the present invention, can acquire the acoustic, optical, and thermal signals during the interaction between the laser and the material in real time. Feature extraction of these signals is then performed based on a quality prediction model, which then fuses these features. Parallel decomposition and attention weight allocation eliminate the time delay and scale differences between heterogeneous data, accurately analyze the coupling relationship between acoustic, optical, and thermal signals, and generate a quality index in real time. The real-time quality index is used to adjust the operating parameters of the laser enhancement process, enabling precise control of the laser enhancement process and improving its quality and efficiency.

[0077] Example 2 Based on the same technical concept as the above-mentioned embodiment 1, an embodiment of the present invention provides a laser enhancement online monitoring device that integrates acoustic, optical and thermal signals. Figure 6 FIG. 1 is a schematic structural diagram of a laser enhancement online monitoring device integrating acoustic, optical and thermal signals provided by an embodiment of the present invention. Figure 6 As shown, the laser enhancement online monitoring device 200 integrating acoustic, optical and thermal signals includes: The data acquisition module 210 is used to acquire acoustic emission signals, spectral signals and thermal infrared images during the laser strengthening process.

[0078] The data processing module 220 is used to pre-process the acoustic emission signal, the spectrum signal and the thermal infrared image respectively to obtain a standard acoustic signal, a standard spectrum data and a standard thermal infrared image.

[0079] The feature extraction module 230 is used to extract features based on the standard acoustic signal, standard spectral data and standard thermal infrared image, and obtain acoustic signal features, spectral features and temperature features respectively.

[0080] The quality monitoring module 240 is used to build and train a quality prediction model, input the acoustic signal features, spectral features and temperature features into the quality prediction model to predict the quality of laser enhancement, and obtain a real-time quality index. The quality prediction model is built based on a low-rank multimodal fusion model.

[0081] It can be understood that the implementation method of the laser enhancement online monitoring method integrating acoustic, optical and thermal signals described in the above-mentioned embodiment 1 is also applicable to this embodiment and can achieve the same technical effect, so it will not be repeated here.

[0082] Example 3 Based on the same concept, an embodiment of the present invention further provides an electronic device, Figure 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 7 As shown, the electronic device 300 may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the steps of the laser enhancement online monitoring method integrating acoustic, optical, and thermal signals as described in the above embodiments. For example, the steps include: S100, acquiring acoustic emission signals, spectral signals, and thermal infrared images during the laser strengthening process; S200, preprocessing the acoustic emission signal, the spectral signal, and the thermal infrared image respectively to obtain a standard acoustic signal, a standard spectral data, and a standard thermal infrared image; S300, performing feature extraction based on the standard acoustic signal, standard spectral data, and standard thermal infrared image to obtain acoustic signal features, spectral features, and temperature features, respectively; S400, constructing and training a quality prediction model, inputting acoustic signal features, spectral features, and temperature features into the quality prediction model to predict the quality of laser enhancement, and obtaining a real-time quality index, wherein the quality prediction model is constructed based on a low-rank multimodal fusion model.

[0083] The processor 310 may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of these chips.

[0084] Furthermore, the logic instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0085] The memory 330 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0086] Example 4 Based on the same concept, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program. The computer program includes at least one code segment that can be executed by a main control device to control the main control device to implement the steps of the laser enhancement online monitoring method integrating acoustic, optical, and thermal signals as described in the above embodiments. For example, the steps include: S100, acquiring acoustic emission signals, spectral signals, and thermal infrared images during the laser strengthening process; S200, preprocessing the acoustic emission signal, the spectral signal, and the thermal infrared image respectively to obtain a standard acoustic signal, a standard spectral data, and a standard thermal infrared image; S300, performing feature extraction based on the standard acoustic signal, standard spectral data, and standard thermal infrared image to obtain acoustic signal features, spectral features, and temperature features, respectively; S400, constructing and training a quality prediction model, inputting acoustic signal features, spectral features, and temperature features into the quality prediction model to predict the quality of laser enhancement, and obtaining a real-time quality index, wherein the quality prediction model is constructed based on a low-rank multimodal fusion model.

[0087] Based on the same technical concept, an embodiment of the present invention further provides a computer program, which, when executed by a main control device, is used to implement the above method embodiment.

[0088] The computer program may be stored in whole or in part on a computer-readable storage medium packaged with the processor, or may be stored in whole or in part on a memory not packaged with the processor.

[0089] Based on the same technical concept, an embodiment of the present invention further provides a processor for implementing the above method embodiment. The above processor may be a chip.

[0090] In summary, the method and device for online monitoring of laser enhancement that integrates acoustic, optical, and thermal signals, provided by the present invention, can acquire the acoustic, optical, and thermal signals during the interaction between the laser and the material in real time. Feature extraction of these signals is then performed based on a quality prediction model, which then fuses these features. Parallel decomposition and attention weight allocation eliminate the time delay and scale differences between heterogeneous data, accurately analyze the coupling relationship between acoustic, optical, and thermal signals, and generate a quality index in real time. This real-time quality index can be used to adjust the operating parameters of the laser enhancement process, enabling precise control of the laser enhancement process and improving its quality and efficiency.

[0091] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0092] The above-described embodiments merely illustrate several implementations of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, and all such variations and improvements fall within the scope of protection of the present invention.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A laser enhancement online monitoring method integrating acoustic, optical and thermal signals, characterized in that: The method comprises: Acquire acoustic emission signals, spectral signals and thermal infrared images during laser strengthening process; Preprocessing the acoustic emission signal, the spectral signal, and the thermal infrared image respectively to obtain a standard acoustic signal, a standard spectral data, and a standard thermal infrared image; Perform feature extraction based on the standard acoustic signal, the standard spectral data, and the standard thermal infrared image to obtain acoustic signal features, spectral features, and temperature features, respectively; A quality prediction model is constructed and trained, and the acoustic signal features, the spectral features, and the temperature features are input into the quality prediction model to predict the quality of laser enhancement to obtain a real-time quality index. The quality prediction model is constructed based on a low-rank multimodal fusion model.

2. The laser enhancement online monitoring method integrating acousto-optical and thermal signals according to claim 1 is characterized in that: The quality prediction model includes: A tensor construction unit, configured to construct a third-order tensor according to the acoustic signal feature, the spectral feature, and the temperature feature; A low-rank decomposition unit, configured to decompose the third-order tensor into a shared core tensor and a plurality of modal factor matrices; an attention weighting unit, configured to perform attention weighting on the shared core tensor to obtain a weighted core tensor; The fully connected layer unit is used to transform the weighted core tensor to obtain the real-time quality index.

3. The laser enhancement online monitoring method integrating acousto-optical and thermal signals according to claim 2 is characterized in that: The training steps of the quality prediction model include: Extract historical acoustic signal features, historical spectral features, and historical temperature features based on historical multi-source heterogeneous data of the laser strengthening process; Constructing a historical third-order tensor according to the historical acoustic signal features, the historical spectral features, and the historical temperature features and setting corresponding feature labels to obtain a training set, a validation set, and a test set; Performing tensor decomposition on the training set using a high-order singular value decomposition algorithm to obtain a low-rank factor matrix; generating a fusion vector according to the low-rank factor matrix, inputting the fusion vector into an initial quality prediction model for prediction, and obtaining a historical quality index; Inputting the historical acoustic signal characteristics and the historical spectral characteristics into an initial quality prediction model for prediction respectively to obtain an acoustic signal index and a spectral index; constructing a loss function based on the historical quality index, the acoustic signal index, and the spectral index; Optimizing the parameters of the initial quality prediction model according to the loss function and the validation set to obtain a target quality prediction model; The target quality prediction model is evaluated according to the test set to obtain the quality prediction model.

4. The laser enhancement online monitoring method integrating acousto-optical and thermal signals according to claim 2 is characterized in that: Inputting the acoustic signal feature, the spectral feature, and the temperature feature into the quality prediction model to perform quality prediction of laser strengthening to obtain a real-time quality index, including: Performing feature vector splicing according to the acoustic signal feature, the spectral feature, and the temperature feature to obtain a third-order tensor; Performing tensor decomposition on the third-order tensor to obtain the shared core tensor and three modal factor matrices; Convolving the shared core tensor to generate attention weights, and weighting the shared core tensor element by element according to the attention weights to obtain a weighted core tensor; The weighted kernel tensor is flattened into a target vector, and the target vector is input into a fully connected layer to obtain the real-time quality index.

5. The laser enhancement online monitoring method integrating acousto-optical and thermal signals according to claim 1 is characterized in that: The acquisition of acoustic emission signals, spectral signals and thermal infrared images during the laser strengthening process includes: Acquiring the acoustic emission signal of the interaction between the material and the laser during the laser strengthening process by an acoustic emission sensor; Detecting the spectral signal of the plasma in the laser strengthening process in real time by a spectrometer; The thermal infrared image of the material during the laser strengthening process is collected by an infrared thermal imager.

6. The laser enhancement online monitoring method integrating acousto-optical-thermal signals according to claim 1 is characterized in that: The preprocessing of the acoustic emission signal, the spectral signal and the thermal infrared image respectively to obtain a standard acoustic signal, a standard spectral data and a standard thermal infrared image comprises: performing amplitude adjustment and bandpass filtering on the acoustic emission signal, and separating harmonics and shock waves to obtain the standard acoustic signal; performing invalid spectrum screening and spectrum averaging processing on the spectrum signal to obtain the standard spectrum data; Perform median filtering on the thermal infrared image to obtain the standard thermal infrared image.

7. The laser enhancement online monitoring method integrating acousto-optical and thermal signals according to claim 6 is characterized in that: The feature extraction is performed based on the standard acoustic signal, the standard spectral data and the standard thermal infrared image to obtain acoustic signal features, spectral features and temperature features, including: Extracting time domain features and frequency domain features from the standard acoustic signal to obtain the acoustic signal features; Performing normalization processing on the standard spectral data to obtain the spectral characteristics; The temperature difference gradient is extracted according to the standard thermal infrared image to obtain the temperature feature.

8. A laser enhancement online monitoring device integrating acoustic, optical and thermal signals, characterized in that: The device comprises: A data acquisition module is used to obtain acoustic emission signals, spectral signals and thermal infrared images during the laser strengthening process; a data processing module, configured to pre-process the acoustic emission signal, the spectral signal, and the thermal infrared image, respectively, to obtain a standard acoustic signal, standard spectral data, and a standard thermal infrared image; a feature extraction module, configured to extract features based on the standard acoustic signal, the standard spectral data, and the standard thermal infrared image, and obtain acoustic signal features, spectral features, and temperature features, respectively; The quality monitoring module is used to construct and train a quality prediction model, input the acoustic signal characteristics, the spectral characteristics and the temperature characteristics into the quality prediction model to predict the quality of laser enhancement, and obtain a real-time quality index. The quality prediction model is constructed based on a low-rank multimodal fusion model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor executes the computer program to implement the laser enhancement online monitoring method integrating acoustic, optical and thermal signals as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the laser enhancement online monitoring method integrating acoustic, optical and thermal signals as described in any one of claims 1 to 7 is implemented.

Images