Laser cutting system based on multi-modal sensor fusion

Abstract

The application discloses a kind of laser cutting systems based on multimodal sensor fusion.The system includes multimodal acquisition module, time synchronization module, feature extraction module, feature processing module, feature fusion module and laser cutting machine.Multimodal acquisition module integrates vision sensor, infrared sensor and acoustic emission sensor, respectively acquires the image, temperature distribution and vibration acoustic signal of cutting area;Time synchronization module realizes the time synchronization of multi-source data through event alignment and clock drift compensation;Feature extraction module extracts the feature parameters such as slit width, edge roughness, temperature gradient, thermal accumulation index and abnormal vibration energy from synchronized data;Feature fusion module standardizes and weightedly fuses features, generates process stability index, and dynamically adjusts laser power, feed speed, pulse frequency and auxiliary gas pressure accordingly.Realize the multidimensional perception and intelligent regulation and control of laser cutting process, improve cutting quality and stability.

Description

Technical Field

[0001] This invention relates to laser processing technology, such as laser welding, cutting or drilling, and more particularly to a laser cutting system based on multimodal sensor fusion. Background Technology

[0002] Laser cutting systems often rely on a single sensor to monitor and control the processing.

[0003] For example, in the prior art, Chinese patent application CN104923922A discloses a laser cutting device and a laser cutting method. In this patent, the distance between the laser head and the workpiece is monitored in real time by a photoelectric sensor, and the worktable drive device and the laser cutter drive device are controlled according to the predetermined cutting surface to process complex arcs and curves.

[0004] For example, Chinese patent application CN115229329A proposes a laser cutting method and a laser cutting system. This patent determines the current laser cutting status by observing changes in the intensity of reflected laser light, and then determines whether to adjust the laser cutting parameters.

[0005] However, the laser cutting process is quite complex. It involves the coupling effects of multiple physical fields such as heat, force, and light, and a single sensor can only capture information from one aspect, making it difficult to comprehensively reflect the processing status.

[0006] Furthermore, due to the lack of multi-source data collaboration, the system has limited ability to identify and respond to sudden anomalies, which can easily lead to a decline in processing quality.

[0007] Therefore, there is a need for further improvement of the existing technology. Summary of the Invention

[0008] To address the aforementioned technical challenges, this invention proposes a laser cutting system based on multimodal sensor fusion. Through multimodal sensor fusion, multi-dimensional perception, decision-making, and dynamic control of the laser cutting process are achieved, thereby improving cutting quality and stability.

[0009] The technical solution of this invention is implemented as follows:

[0010] A laser cutting system based on multimodal sensor fusion, characterized in that it includes:

[0011] The multimodal acquisition module is used to acquire multi-source sensor data during cutting, including a vision sensor, an infrared sensor, and an acoustic emission sensor.

[0012] A time synchronization module, connected to the multimodal acquisition module, is used to synchronize the time reference of multi-source sensor data;

[0013] The feature extraction module is connected to the time synchronization module and extracts feature parameters representing the processing state from the synchronized multi-source sensor data.

[0014] Feature processing module;

[0015] A feature fusion module, connected to the feature extraction module, is used to fuse the feature parameters to generate at least one process stability index.

[0016] Laser cutting machine;

[0017] Specifically, control instructions are generated based on the process status index, and after receiving the control instructions, the working parameters of the laser cutting machine are adjusted based on the process parameter library.

[0018] Preferably, the vision sensor is used to acquire images of the cutting area and extract the kerf width and edge roughness morphology features.

[0019] Among them, the kerf width The calculation formula is as follows:

[0020]

[0021] In the formula, This represents the average width of the kerf, indicating the location of the kerf at multiple positions. The average distance between the left and right edges at a given location, where N is the number of sampling points. Indicates the location The coordinates of the left edge at that location; Indicates the location The right edge coordinates at that location

[0022] edge roughness The calculation formula is as follows:

[0023]

[0024] In the formula, The average roughness of the edge profile. This represents the coordinates of the i-th edge point. This represents the mean of the coordinates of all edge points, and calculates the average of the sum of the absolute values ​​of the deviations of the edge points from the mean.

[0025] Preferably, in the laser cutting system according to claim 1, the infrared sensor is used to collect the temperature distribution on the workpiece surface and extract the temperature gradient and heat accumulation index.

[0026] Wherein, the temperature gradient The calculation formula is:

[0027]

[0028] In the formula, This represents the difference between the extreme values ​​of the temperature gradient. It's important to note that the temperature gradient is simplified as a scalar, not a vector. This represents the set of temperature values ​​in the central region. This represents the set of temperature values ​​for the edge region.

[0029] The heat accumulation index The calculation formula is as follows:

[0030]

[0031] In the formula, Indicates the heat accumulation index, Indicates time Temperature value at time, This represents temperature data over a specific period of time.

[0032] Preferably, the acoustic emission sensor is used to collect vibration acoustic signals during the processing and extract abnormal vibration energy characteristics within the risk frequency band.

[0033] The acoustic emission sensor extracts the abnormal vibration energy E, and the calculation formula is as follows:

[0034]

[0035] In the formula, E represents the abnormal vibration energy within the risk frequency band, and s(t) represents the acoustic emission signal, which is generally the vibration signal of the workpiece under stress. This indicates that performing a Fast Fourier Transform on the signal s(t) yields the frequency domain. Indicates the risk frequency band.

[0036] Preferably, the feature fusion module includes:

[0037] The extracted feature parameters are standardized.

[0038] Based on the properties of the current processed material, weights are assigned to each feature to form a weighted feature vector;

[0039] The sum of squares of the weighted eigenvectors is calculated and used as a process stability index.

[0040] When the process stability index exceeds the threshold, the process parameter library is called according to the feature component that contributes the most to adjust the laser power, feed speed, pulse frequency or auxiliary gas pressure of the laser cutting machine.

[0041] Preferably, a function mapping model is established:

[0042]

[0043] This represents a multi-source information and material property vector collected by multiple sensors, where P represents the process parameters of the laser cutting machine.

[0044] The Pw represents laser power; V represents feed rate; f represents pulse frequency; and G represents auxiliary gas pressure.

[0045] Preferably, S1, S2, and S3 represent the feature vectors of the visual sensor, the infrared sensor, and the acoustic emission sensor, respectively.

[0046] in,

[0047] It provides workpiece geometry and surface condition information for predicting trajectory compensation;

[0048] Real-time monitoring of heat distribution on the workpiece surface during the cutting process;

[0049] Detect abnormal vibration energy within the risk frequency band and adjust the cutting state accordingly;

[0050] M represents a material property vector, including but not limited to material type, thickness, density, specific heat capacity, thermal conductivity, melting point, laser reflectivity, phase transition temperature range, and coefficient of thermal expansion. This represents the process parameter library.

[0051] Preferably, in the time synchronization module, one of the sensors is selected as the reference sensor. This indicates the characteristic timestamps of the same event collected by the reference sensors. This represents the feature timestamp of the same event collected by the i-th sensor under the reference sensor. This represents the offset of the i-th sensor relative to the reference sensor.

[0052] Time alignment compensation formula:

[0053]

[0054] in, Represents the raw timestamp of the non-reference sensor. This indicates the timestamp after correction by the non-reference sensor.

[0055] Preferably, in the time synchronization module, one of the sensors is selected as the reference sensor, and the time synchronization module captures K identical events.

[0056] This represents the characteristic timestamps of K identical events collected by the reference sensor. This represents the feature timestamps of K identical events collected by the i-th sensor under the reference sensor. This represents the offset of the i-th sensor relative to the reference sensor during the K same events.

[0057] The linear relationship between the non-reference sensor and the reference sensor is as follows:

[0058]

[0059] In the formula, Indicates the clock drift coefficient, if This indicates that the non-reference sensor operates faster than the reference sensor; if This indicates that the non-reference sensor operates slower than the reference sensor. Indicates a fixed time offset.

[0060] Preferably, it consists of K identical events. , Least squares method to fit parameters and And solve and ,

[0061] get and It corrects the original timestamp at any given time.

[0062] The laser cutting system based on multimodal sensor fusion according to the present invention has the following beneficial effects:

[0063] 1. It integrates three types of sensors: vision, infrared and acoustic emission, to collect data from three dimensions: morphology, thermal and dynamics. The time synchronization module ensures that the multi-source data are aligned on the time axis, avoiding information distortion caused by timing deviations.

[0064] 2. Based on the physical properties of different materials, dynamically adjust the weight of each feature in the fusion process.

[0065] By constructing a process stability index (PHI), the deviation between the current processing state and the ideal state is quantified. When the PHI exceeds a threshold, the main abnormal characteristics are analyzed, and parameters such as laser power, feed rate, and pulse frequency are dynamically adjusted based on the process parameter library to achieve closed-loop control.

[0066] 3. It has stronger anomaly identification and fault tolerance capabilities, avoiding defects such as excessively wide kerf, rough edges, material melt-through, and cracks, thereby improving the consistency of cutting and the yield rate.

[0067] 4. It can autonomously adjust its sensing and control strategies according to different materials and process conditions, and has good generalization ability and engineering applicability, making it suitable for high-precision laser processing needs in various industrial scenarios. Attached Figure Description

[0068] Figure 1 This is a schematic diagram of the structure of the laser cutting machine of the present invention;

[0069] Figure 2 This is a connection block diagram of the laser cutting system of the present invention;

[0070] Figure 3 This is a connection block diagram of the laser cutting system of the present invention;

[0071] Figure 4 This is a connection block diagram of the laser cutting system of the present invention;

[0072] Figure 5 This is a parameter diagram of the process parameter library of the present invention;

[0073] The attached figures are labeled as follows: multimodal acquisition module 101, time synchronization module 102, feature extraction module 103, feature processing module 104, feature fusion module 105, and laser cutting machine 100. Detailed Implementation

[0074] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0075] Reference Figures 1 to 5 As shown, this embodiment proposes a laser cutting system based on multimodal sensor fusion. Among them,

[0076] The laser cutting system includes a multimodal acquisition module 101, a time synchronization module 102, a feature extraction module 103, a feature processing module 104, a feature fusion module 105, and a laser cutting machine 100.

[0077] The multimodal acquisition module 101 is used to acquire multi-source data of the workpiece to be cut during the laser cutting process. The multimodal acquisition module 101 includes a vision sensor, an infrared sensor, and an acoustic emission sensor. The vision sensor acquires visible light or laser images of the cutting area to extract morphological features such as kerf width and edge roughness; the infrared sensor acquires the surface temperature distribution of the workpiece to be cut and calculates thermal characteristics such as temperature gradient and heat accumulation index; the acoustic emission sensor acquires vibration and acoustic signals generated during the laser cutting process to detect abnormal vibrations during processing.

[0078] Because different sensors have independent sampling clocks and data processing delays, timestamps for the same event may differ. To ensure that multi-source data is aligned on the timeline, the time synchronization module 102 unifies the time reference.

[0079] In one embodiment, the time difference between the same event sensed by different sensors is estimated. For example, the same event could be the simultaneous occurrence of image changes on the workpiece being cut by the vision sensor, the local temperature rise of the workpiece being cut by the infrared sensor, and the vibration shock wave detected by the acoustic emission sensor during laser cutting.

[0080] In the time synchronization module 102, there are i sensors, and one of them is selected as the reference sensor. This indicates the characteristic timestamps of the same event collected by the reference sensors. This represents the feature timestamp of the same event collected by the i-th sensor under the reference sensor. This represents the offset of the i-th sensor relative to the reference sensor.

[0081] Time alignment compensation formula:

[0082]

[0083] in, Represents the raw timestamp of the non-reference sensor. This indicates the timestamp after correction by the non-reference sensor.

[0084] However, each sensor has a different clock start point and operating speed. The time difference caused by the difference in operating speed varies linearly with time, i.e., clock drift. The method described above cannot eliminate clock drift and is only suitable for situations with short acquisition times.

[0085] In one embodiment, the time synchronization module 102 is equipped with i sensors, one of which is selected as a reference sensor. The time synchronization module 102 captures K identical events, which are the image changes of the workpiece to be cut collected by the vision sensor, the local temperature rise of the workpiece to be cut collected by the infrared sensor, and the vibration shock wave collected by the acoustic emission sensor during laser cutting, all occurring at the same time.

[0086] This represents the characteristic timestamps of K identical events collected by the reference sensor. This represents the feature timestamps of K identical events collected by the i-th sensor under the reference sensor. This represents the offset of the i-th sensor relative to the reference sensor during the K same events.

[0087] The linear relationship between the non-reference sensor and the reference sensor is as follows:

[0088]

[0089] In the formula, Indicates the clock drift coefficient, if This indicates that the non-reference sensor operates faster than the reference sensor; if This indicates that the non-reference sensor operates slower than the reference sensor. Indicates a fixed time offset.

[0090] From K identical events , Fit the optimal parameters and .

[0091] K identical events , As data points, they should be roughly distributed along a straight line. The least squares method is used to fit this line, and the solution is obtained. and .

[0092] get and The original timestamp at any given moment is corrected and mapped onto a reference time axis to ensure that the data collected by multiple sensor spheres are aligned on the time axis and to unify the time reference.

[0093] After multiple sensors are synchronized to a unified time base by the time synchronization module 102, they are input into the feature extraction module 103 for feature extraction. The correlation, importance, and association between all extracted features and the workpiece to be processed are fused. Finally, the weights are adjusted according to the different materials of the workpiece to be processed, and the results are input into the laser cutting machine as parameters to adjust the laser cutting machine's power, feed speed, pulse frequency, auxiliary gas pressure, etc.

[0094] In this embodiment, the visual sensor feature extraction is divided into kerf width and edge roughness.

[0095] Among them, the kerf width The calculation formula is as follows:

[0096]

[0097] In the formula, This represents the average width of the kerf, indicating the location of the kerf at multiple positions. The average distance between the left and right edges at a given location. N is the number of sampling points. Indicates the location The coordinates of the left edge at that location; Indicates the location The coordinates of the right edge at that location.

[0098] The consistency of the kerf width is quantified by calculating the average of the sum of the absolute values ​​of the differences between the left and right edges at multiple locations. For example, in laser cutting, if the kerf is too wide or too narrow, it can easily lead to cracking of the workpiece.

[0099] edge roughness The calculation formula is as follows:

[0100]

[0101] In the formula, This represents the average roughness of the edge profile. This represents the coordinates of the i-th edge point. This represents the mean of the coordinates of all edge points. The roughness of the edge is reflected by calculating the average of the sum of the absolute values ​​of the deviations of the edge points from the mean.

[0102] Furthermore, infrared sensor feature extraction is divided into temperature gradient and thermal accumulation index.

[0103] Wherein, the temperature gradient The calculation formula is:

[0104]

[0105] In the formula, This represents the difference between the extreme values ​​of the temperature gradient. It is important to note that the temperature gradient is simplified as a scalar, not a vector. This represents the set of temperature values ​​in the central region, such as multiple temperature sampling points in the central region of a workpiece. This represents a set of temperature values ​​for the edge region, such as multiple temperature sampling points on the edge of a workpiece. The difference between the extreme temperature values ​​of the central and edge regions is calculated to reflect the heat distribution on the workpiece surface.

[0106] The heat accumulation index The calculation formula is as follows:

[0107]

[0108] In the formula, Indicates the heat accumulation index, Indicates time Temperature value at time, This represents temperature data over a specific time period. By calculating the cumulative temperature over a certain time, the impact of the cumulative effect of being in a high-temperature environment on laser cutting can be evaluated.

[0109] Furthermore, the acoustic emission sensor extracts the abnormal vibration energy E, calculated using the following formula:

[0110]

[0111] In the formula, E represents the abnormal vibration energy within the risk frequency band, and s(t) represents the acoustic emission signal, which is generally the vibration signal of the workpiece under stress. This indicates that a fast Fourier transform is performed on the signal s(t) to obtain the frequency domain. The risk frequency band is defined as [5kHz, 20kHz]. Abnormal signals related to defects are captured by calculating the vibration energy within the 5-20kHz frequency band.

[0112] In this embodiment, physically meaningful features are extracted by multiple sensors and input into the feature fusion module 105. The multiple features are mapped to one or more indices. Finally, the process parameters of the laser cutting machine are dynamically adjusted according to these indices and the characteristics of the current processing material.

[0113] Establish a function mapping model:

[0114]

[0115] This means calculating the optimal laser processing parameters using multi-source information and material property vectors collected from multiple sensors.

[0116] Specifically, Pw represents laser power; V represents feed rate; f represents pulse frequency; and G represents auxiliary gas pressure.

[0117] Wherein, S1, S2, and S3 represent the feature vectors of the visual sensor, the infrared sensor, and the acoustic emission sensor, respectively. It provides workpiece geometry and surface condition information for predicting trajectory compensation. Real-time monitoring of the heat distribution on the workpiece surface during the cutting process. Detect abnormal vibration energy within the risk frequency band and adjust the cutting state accordingly.

[0118] M represents a material property vector, including but not limited to material type, thickness, density, specific heat capacity, thermal conductivity, melting point, laser reflectivity, phase transition temperature range, coefficient of thermal expansion, etc.

[0119] This refers to the process parameter library, which can be understood as including, but not limited to, thresholds, materials, parameters, and quality, and is configured based on processing experience.

[0120] Furthermore, the influence of different physical dimensions needs to be eliminated, and the most sensitive characteristics of the currently processed materials need to be highlighted. Therefore, standardization and weighting are required before fusion.

[0121] Specifically, by using the z-score model (standardization is a commonly used data standardization method that can compare and analyze values ​​between different datasets), S1, S2, and S3 are input into the z-score model to obtain the standardized feature values ​​Z. n Z n The magnitude of directly reflects the degree to which the current state deviates from the ideal state. n represents the number of input features.

[0122] Establish a material weight matrix based on the materials.

[0123]

[0124] The material weight matrix is ​​a diagonal matrix, and the values ​​on its diagonal represent the importance weights of features to the material.

[0125] Multiplying the standardized eigenvectors by the material weight matrix yields the weighted eigenvector Z. W For example, when cutting copper and aluminum, the weight of heat distribution characteristics is increased; when cutting glass and ceramics, the weight of acoustic emission characteristics is increased.

[0126] Furthermore, multiple characteristics are aggregated into a Process Stability Index (PHI) to facilitate decision-making. The formula for calculating the Process Stability Index (PHI) is: The sum of squares of all weighted standardized features represents the square of the weighted Euclidean distance from the current state to the "normal center".

[0127] In this embodiment, the process stability index (PHI) reflects the stability of the laser cutting process. A PHI value close to 0 indicates that the cutting process is relatively stable; the larger the PHI value, the further the cutting process deviates from the normal state, and the greater the risk of abnormality.

[0128] If the PHI value exceeds the alarm threshold Th1, further analysis is conducted to determine which weighted feature contributes the most. For example, if... If the contribution is the largest, it is judged as a thermal anomaly, and then the process parameter library is retrieved. It executes commands to reduce laser power or increase cutting speed, reducing energy input per unit length, suppressing heat accumulation, and preventing material from melting through or deforming.

[0129] If E contributes the most, then reduce the pulse frequency to reduce thermal shock, increase the auxiliary gas pressure, optimize the pulse heat input rhythm, enhance the slag removal capability, and reduce residual defects.

[0130] If R aThe ratio of laser power Pw to feed rate V contributes the most, thus improving the stability of the molten pool and solidification behavior, resulting in a smooth cutting surface.

[0131] In this embodiment, the adjusted process parameters are sent to the laser cutting machine, which then operates with the new parameters. Multiple sensors immediately begin collecting new data, and the entire cycle restarts until processing is complete.

[0132] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A laser cutting system based on multi-modal sensor fusion, characterized in that, Comprise: A multi-modal acquisition module for acquiring multi-source sensing data in cutting, which includes a visual sensor, an infrared sensor and an acoustic emission sensor; A time synchronization module connected with the multi-modal acquisition module for synchronizing the time base of multi-source sensing data; A feature extraction module connected with the time synchronization module for extracting feature parameters representing the processing state from the synchronized multi-source sensing data; A feature processing module; A feature fusion module connected with the feature extraction module for fusion processing of the feature parameters to generate at least one process stability index; A laser cutting machine; Wherein, according to the process stability index, a control instruction is generated, after receiving the control instruction, the working parameters of the laser cutting machine are adjusted based on the process parameter library, The visual sensor is used to acquire the image of the cutting area, and the seam width and edge roughness topography features are extracted, wherein the slit width The calculation formula is as follows: ； wherein represents the average value of the slit width, represents the average value of the left and right edge distances of the slit at a plurality of positions , N is the number of sampling points, represents the left edge coordinate value at position ; represents the right edge coordinate value at position ; Edge roughness The calculation formula is as follows: ； In the formula, denotes the average roughness of the edge profile, denotes the coordinate value of the i-th edge point, denotes the mean value of all edge point coordinates, the average of the sum of the absolute values of the deviations of the edge points from the mean value.

2. The laser cutting system of claim 1, wherein, The infrared sensor is used to acquire the temperature distribution of the workpiece surface, and the temperature gradient and heat accumulation index are extracted, The temperature gradient is calculated as follows: The calculation formula is: ； wherein denotes the difference of the extrema of the temperature gradient, it is to be noted that the temperature gradient is simplified to a scalar, not a vector, denotes the set of temperature values of the central region, denotes the set of temperature values of the edge region, The heat build-up index The calculation formula is as follows: ； In the formula, represents the heat accumulation index, represents the temperature value at time represents the temperature data at a certain time. represents the temperature data at a certain time.

3. The laser cutting system of claim 1, wherein, The acoustic emission sensor is used to acquire the vibration sound wave signal in the processing process, and the abnormal vibration energy feature in the risk frequency band is extracted, The acoustic emission sensor extracts abnormal vibration energy E, and the calculation formula is as follows: ； In the formula, E represents abnormal vibration energy in a risk frequency band, s(t) represents an acoustic emission signal, which is a vibration signal when a workpiece is stressed, represents that a fast Fourier transform is performed on the signal s(t) to obtain a frequency domain, represents a risk frequency band.

4. The laser cutting system of claim 1, wherein, The feature fusion module comprises: The extracted feature parameters are standardized; According to the properties of the current processing material, each feature is assigned a weight to form a weighted feature vector; The square sum of the weighted feature vector is calculated as the process stability index, Wherein, when the process stability index exceeds the threshold value, the process parameter library is called according to the feature component with the largest contribution, and the laser power, feed speed, pulse frequency or auxiliary gas pressure of the laser cutting machine are adjusted.

5. The laser cutting system of claim 1, wherein, Establish a function mapping model: ； Indicates the multi-source information and material attribute vector collected by multiple sensors, P represents the process parameters of the laser cutting machine, The Pw represents laser power; V represents feed speed; f represents pulse frequency; and G represents assist gas pressure.

6. The laser cutting system of claim 5, wherein, S1, S2, S3 represent the visual sensor feature vector, infrared sensor feature vector and acoustic emission sensor feature vector respectively, Wherein, , providing workpiece geometry and surface state information for trajectory compensation prediction; monitoring the thermal distribution on the workpiece surface in real time during the cutting process; detecting abnormal vibration energy in the risk frequency band, and adjusting the cutting state; M represents a material attribute vector, including material type, thickness, density, specific heat capacity, thermal conductivity, melting point, laser reflectivity, phase transition temperature interval, thermal expansion coefficient, represents a process parameter library.

7. The laser cutting system of claim 1, wherein, In the time synchronization module, one of the sensors is selected as a reference sensor, denotes a characteristic timestamp of the same event captured by the reference sensor, denotes a characteristic timestamp of the same event captured by the i-th sensor under the reference sensor, denotes the offset of the i-th sensor relative to the reference sensor, Time alignment compensation formula: ； wherein, represents the original timestamp of the non-reference sensor, represents the corrected timestamp of the non-reference sensor.

8. The laser cutting system of claim 1, wherein, In the time synchronization module, one of the sensors is selected as the reference sensor, and the time synchronization module captures K same events, represents the characteristic timestamps of K same events collected by a reference sensor, represents the characteristic timestamps of K same events collected by the i-th sensor under the reference sensor, represents the offset of the i-th sensor with respect to the reference sensor for K same events, Wherein, the linear relationship between the non-reference sensor and the reference sensor is: ； wherein represents the clock drift coefficient, if represents that the non-reference sensor runs faster than the reference sensor; if represents that the non-reference sensor runs slower than the reference sensor, represents the fixed time offset.

9. The laser cutting system of claim 8, wherein, By least squares fit of parameters , , and solving for and , and and , obtained and correcting the original time stamp for any time instant.

Images