Information acquisition system and method for robotic laser assisted roll forming

Abstract

The application discloses an information acquisition system and method for robot laser-assisted roll forming, and the information acquisition system comprises: an image acquisition mechanism fixedly arranged on one side of a plate and used for acquiring geometric information of the plate; the action range of the image acquisition mechanism covers the whole plate; a temperature acquisition mechanism fixedly arranged on the other side of the plate and used for acquiring temperature information of the plate; the action range of the temperature acquisition mechanism covers the whole plate and is at least 2 times of the action range of the image acquisition mechanism; a multi-axis mechanical sensor fixedly arranged on a robot and used for acquiring stress information of the plate; a control mechanism electrically connected with the image acquisition mechanism, the temperature acquisition mechanism and the multi-axis mechanical sensor respectively and used for acquiring the geometric information, the temperature information and the stress information; and the control mechanism comprises an online data processing unit. The information acquisition system and method can acquire multi-modal data in real time and perform online processing, and can assist in improving plate forming precision and forming quality.

Description

Technical Field

[0001] This specification relates to the field of laser-assisted robot roll forming technology, and in particular to an information acquisition system and method for robot laser-assisted roll forming. Background Technology

[0002] Laser-assisted robot roll forming is a new type of sheet metal progressive bending forming process developed on the basis of traditional roll forming. It softens the material by local laser heating and combines it with robot roll forming to achieve precision forming without molds. It has the advantages of high flexibility and low energy consumption, and is conducive to achieving local plastic forming of parts.

[0003] Currently, the acquisition of multimodal information from laser-assisted robot roll forming processes is mainly used for quality analysis and processing path prediction after part processing. The collected multimodal information is mostly extreme value information, such as the maximum springback angle of the sheet metal after processing, the maximum temperature of the sheet metal during processing, and the maximum forming force on the sheet metal. The processing method for this multimodal information is offline. By fitting functions to these extreme values ​​and summarizing physical models, the optimization of the laser-assisted robot roll forming path and process parameters can be achieved.

[0004] While the aforementioned multimodal information acquisition and offline data analysis methods can improve the forming accuracy of laser-assisted robot roll forming to some extent, this process involves extreme thermo-mechanical coupling: rapid temperature rise due to laser heating, large temperature gradient in the thickness direction, large fluctuations in dynamic contact force, oxidation and structural transformation of the surface metal structure, and vibration interference induced by robot motion. These factors can easily cause forming defects, such as uneven springback, edge wavy wrinkling, or cracks, affecting the precision and mechanical properties of the parts. Traditional methods struggle to capture multi-field coupling information, leading to deviations between the obtained optimization model and the actual processing. Therefore, it is necessary to develop a multimodal information acquisition system and online data processing methods to enable real-time synchronous acquisition of temperature, force, and geometric multimodal data, achieving process visualization and data-driven optimization. This can support closed-loop control and quality prediction, thereby promoting the improvement of forming accuracy in laser-assisted robot roll forming. Summary of the Invention

[0005] In view of the shortcomings of the prior art, one object of this specification is to provide an information acquisition system and method for robotic laser-assisted roll forming, which can acquire multimodal data in real time and perform online processing to help improve the forming accuracy and quality of sheet metal.

[0006] To achieve the above objectives, this specification provides an information acquisition system for robot laser-assisted roll forming, which uses a robot to perform laser-assisted roll forming on a sheet metal fixed to a tooling based on predetermined processing parameters; the information acquisition system includes: An image acquisition mechanism is fixedly installed on one side of the sheet metal for acquiring geometric shape information of the sheet metal; the effective range of the image acquisition mechanism covers the entire sheet metal. A temperature acquisition mechanism is fixedly installed on the other side of the sheet metal to acquire the temperature information of the sheet metal; the effective range of the temperature acquisition mechanism covers the entire sheet metal and is at least twice the effective range of the image acquisition mechanism. A multi-axis force sensor is fixedly mounted on the robot to collect the force information of the sheet metal; A control mechanism electrically connected to the image acquisition mechanism, temperature acquisition mechanism, and multi-axis mechanical sensor is used to acquire the geometric shape information, temperature information, and force information. The control mechanism includes an online data processing unit, which is used to integrate and process the geometric shape information based on a semi-global matching algorithm, and to predict the roll forming result based on the temperature information, force information, and processing parameters using a support vector regression algorithm.

[0007] In a preferred embodiment, the image acquisition mechanism includes a first industrial camera and a second industrial camera fixed at a certain angle and kept relatively stationary; the first industrial camera and the second industrial camera are set at a downward angle of 0~15° relative to the horizontal plane.

[0008] In a preferred embodiment, the shooting range of the first industrial camera is defined as the first shooting range, and the shooting range of the second industrial camera is defined as the second shooting range; the first shooting range and the second shooting range have an overlapping portion, and the overlapping portion covers a predetermined measurement interval.

[0009] In a preferred embodiment, the first shooting range and the second shooting range are shaped as a first rectangle on the sheet metal. The length of the first rectangle is greater than the length of the overlapping portion, and the width of the first rectangle is equal to the width of the overlapping portion. The length of the overlapping portion is 20mm to 40mm.

[0010] In a preferred embodiment, the sheet metal has a bending line; the lens center of the first industrial camera is defined as the first center, and the lens center of the second industrial camera is defined as the second center; the midpoint of the line connecting the first center and the second center is defined as the first midpoint, and the midpoint of the bending line located within the predetermined measurement interval is defined as the second midpoint; the line connecting the first midpoint and the second midpoint is perpendicular to the bending line.

[0011] In a preferred embodiment, the distance between the first center and the second center is 60mm to 80mm, and the vertical distance from the first center or the second center to the sheet metal is 380mm to 500mm.

[0012] In a preferred embodiment, the temperature acquisition mechanism includes a first support fixedly connected to and relatively stationary with a thermal imager, the first support being fixedly connected to the tooling.

[0013] In a preferred embodiment, the vertical distance between the thermal imager and the sheet metal is 300mm to 700mm; the measurement temperature range of the thermal imager is 625℃ to 1900℃.

[0014] In a preferred embodiment, the image acquisition mechanism further includes a second bracket for fixing the image acquisition mechanism and a checkerboard calibration plate for calibrating the image acquisition mechanism; the control mechanism includes a computer and a data cable for connecting the image acquisition mechanism, the temperature acquisition mechanism and the multi-axis force sensor to the computer; the computer is equipped with the online data processing unit.

[0015] This application also provides an information acquisition method for robot laser-assisted roll forming, wherein the information acquisition method employs the information acquisition system described in any of the above embodiments; the information acquisition method includes the following steps: Step S10: Determine and mark a predetermined measurement range on the sheet metal; the predetermined measurement range is located within the effective range of the image acquisition mechanism and the effective range of the temperature acquisition mechanism; Step S20: Open the image acquisition mechanism, temperature acquisition mechanism and multi-axis force sensor, and make the image acquisition mechanism, temperature acquisition mechanism and multi-axis force sensor electrically connected to the control mechanism respectively; adjust the position of the image acquisition mechanism and temperature acquisition mechanism so that the predetermined measurement interval is within the effective range of the image acquisition mechanism and the effective range of the temperature acquisition mechanism. Step S30: Activate the online data processing unit to collect and process the multimodal information of the sheet metal during the robot laser-assisted roll forming process; the multimodal information includes the geometric shape information, temperature information and stress information; the online processing includes integrating and processing the geometric shape information based on a semi-global matching algorithm, and predicting the roll forming result based on a support vector regression algorithm using the temperature information, stress information and processing parameters.

[0016] Beneficial effects

[0017] The information acquisition system for robotic laser-assisted roll forming provided in this embodiment, through an image acquisition mechanism fixed on one side of the sheet metal, a temperature acquisition mechanism fixed on the other side of the sheet metal, and a multi-axis mechanical sensor fixed on the robot, can collect real-time geometric morphology, real-time temperature, and real-time force coupling within a predetermined measurement range of the sheet metal during the robotic laser-assisted roll forming process. This allows for effective measurement of multimodal information during the laser-assisted forming process. Simultaneously, the data is processed by the online data processing unit of the control mechanism to form a dataset. Machine learning analysis of this dataset can improve the prediction accuracy of the sheet metal forming process. Therefore, this information acquisition system can capture multi-field coupling information, helping to improve the forming accuracy and quality of the sheet metal.

[0018] Furthermore, by acquiring information in real time and processing data online, this invention avoids the repetitive positioning accuracy errors caused by repeatedly removing and reinstalling sheet metal from tooling in traditional methods, and reduces random errors, demonstrating the feasibility of achieving closed-loop control of the processing. This invention has numerous potential applications, particularly suitable for research on high-precision forming processes for battery pack frames and other body parts in new energy vehicles.

[0019] Specific embodiments of the present invention are disclosed in detail with reference to the following description and accompanying drawings, indicating how the principles of the invention can be employed. It should be understood that the embodiments of the present invention are not limited in scope as a result.

[0020] Features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments.

[0021] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, whole, step, or component, but does not exclude the presence or addition of one or more other features, wholes, steps, or components. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the structure of an information acquisition system for robot laser-assisted roll forming provided in this embodiment; Figure 2 for Figure 1 Top view; Figure 3 for Figure 1 A schematic diagram of the structure of the sheet metal after bending; Figure 4 This is a schematic diagram illustrating the principle of laser-assisted roll forming in this embodiment; Figure 5 This is a top view showing the installation of the first and second industrial cameras in this embodiment. Figure 6 This is a schematic diagram of the sheet metal structure in this embodiment.

[0024] Figure 7 This is a schematic diagram illustrating the principle of how a 3D contour scanner processes the bending angle of a steel sheet in Comparative Example 1. Figure 8 This is a schematic diagram illustrating the principle of handling the bending angle of steel sheet material in Example 1.

[0025] Figure 9 This is a schematic diagram of the bending lines of the steel sheet after multiple re-clampings in Comparative Example 1.

[0026] Explanation of reference numerals in the attached figures: 1. Robot; 11. Roller; 2. Tooling; 3. Sheet metal; 31. Bending line; 311. Second midpoint; 32. Predetermined measurement interval; 61. First pass; 62. Second pass; 63. nth pass; 4. Image acquisition mechanism; 41. First industrial camera; 411. First center; 412. First shooting range; 42. Second industrial camera; 421. Second center; 422. Second shooting range; 401. Overlapping part; 402. First midpoint; 43. Second support; 5. Temperature acquisition mechanism; 51. First support; 52. Thermal imager; 71. First fitted line; 72. Second fitted line; 81. Initial plane; 82. Normal vector of the initial plane; 83. Effective parallax point; 84. Fitted plane; 85. Normal vector of the fitted plane; 86. Bending angle; 91. Bending line initially installed; 92. Bending line after reinstallation. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0028] It should be noted that when an element is referred to as being "set on" another element, it can be directly on the other element or may be interposed with another element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or may be interposed with another element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementations.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0030] Please see Figures 1 to 6 This application provides an information acquisition system for robot laser-assisted roll forming. Specifically, robot laser-assisted roll forming refers to using a robot 1 to perform laser-assisted roll forming on a sheet metal 3 fixed to a tooling 2 based on predetermined processing parameters. The information acquisition system includes: an image acquisition mechanism 4, a temperature acquisition mechanism 5, a multi-axis force sensor (not shown), and a control mechanism (not shown).

[0031] The image acquisition mechanism 4 is fixedly mounted on one side of the sheet metal 3 to collect its geometric shape information. The effective range of the image acquisition mechanism 4 covers the entire sheet metal 3. The temperature acquisition mechanism 5 is fixedly mounted on the other side of the sheet metal 3 to collect its temperature information. The effective range of the temperature acquisition mechanism 5 covers the entire sheet metal 3 and is at least twice the effective range of the image acquisition mechanism 4. This ensures that the temperature gradient measurement area covers areas that may affect the forming result. Besides the direct impact of the temperature gradient in the central area, the temperature gradient in the surrounding area also has a certain influence on the forming result. A multi-axis force sensor is fixedly mounted on the robot 1 to collect the force information of the sheet metal 3. The control mechanism is electrically connected to the image acquisition mechanism 4, the temperature acquisition mechanism 5, and the multi-axis force sensor to acquire geometric shape information, temperature information, and force information. The control mechanism includes an online data processing unit (not shown in the figure), which integrates and processes geometric shape information based on a semi-global matching algorithm (i.e., SGM algorithm), and uses temperature information, stress information and processing parameters based on a support vector regression algorithm (i.e., SVR algorithm) to predict the rolling forming result.

[0032] The information acquisition system for robot laser-assisted roll forming provided in this embodiment, through an image acquisition mechanism 4 fixedly mounted on one side of the sheet metal 3, a temperature acquisition mechanism 5 fixedly mounted on the other side of the sheet metal 3, and a multi-axis mechanical sensor fixedly mounted on the robot 1, can collect real-time geometric shape, real-time temperature, and real-time force coupling within a predetermined measurement range 32 of the sheet metal 3 during the robot laser-assisted roll forming process, effectively measuring multimodal information during the laser-assisted forming process. Simultaneously, the data is processed by the online data processing unit of the control mechanism to form a dataset. By performing machine learning analysis on the dataset, the prediction accuracy of the sheet metal 3 forming process can be improved.

[0033] Furthermore, by acquiring information in real time and processing data online, the invention avoids the repetitive positioning accuracy errors caused by repeatedly removing and reinstalling the sheet metal 3 from the tooling 2 in traditional methods, and reduces random errors, demonstrating the feasibility of achieving closed-loop control of the processing. This invention has numerous potential applications and is suitable for high-precision forming processes of ultra-high-strength steel, particularly for research on high-precision forming processes of new energy vehicle battery pack frames and other body parts.

[0034] In this embodiment, the image acquisition mechanism 4 includes a first industrial camera 41 and a second industrial camera 42 fixed at a certain angle and kept relatively stationary, which can extract real-time geometric shape information of the sheet metal 3 processing process with high precision. The first industrial camera 41 and the second industrial camera 42 are set at a downward angle of 0~15° relative to the horizontal plane to clearly obtain geometric shape information of the predetermined measurement range.

[0035] like Figure 5 As shown, the shooting range of the first industrial camera 41 is defined as the first shooting range 412, and the shooting range of the second industrial camera 42 is defined as the second shooting range 422. The first shooting range 412 and the second shooting range 422 have an overlapping portion 401, which covers a predetermined measurement interval 32. The predetermined measurement interval 32 is located at the bending line 31 of the sheet metal 3, and is determined and marked on the sheet metal 3 as needed.

[0036] In this embodiment, the first shooting range 412 and the second shooting range 422 are shaped as a first rectangle on the sheet metal 3. The length of the first rectangle is greater than the length of the overlapping portion 401, and the width of the first rectangle is equal to the width of the overlapping portion 401. The length of the overlapping portion 401 is 20mm to 40mm. The length and width of the first rectangle can be set as needed. In a specific embodiment, the length of the first rectangle is 118.2mm and the width is 74.9mm.

[0037] Continue to refer to Figure 5The sheet metal 3 has a bending line 31. The lens center of the first industrial camera 41 is defined as the first center 411, and the lens center of the second industrial camera 42 is defined as the second center 421. The midpoint of the line connecting the first center 411 and the second center 421 is defined as the first midpoint 402, and the midpoint of the bending line 31 located within the predetermined measurement interval 32 is defined as the second midpoint 311. Preferably, the line connecting the first midpoint 402 and the second midpoint 311 is perpendicular to the bending line 31 to ensure that the first industrial camera 41 and the second industrial camera 42 can cooperate to form binocular vision recognition, improve angular accuracy, and extract real-time geometric shape information of the sheet metal 3 processing process with high precision.

[0038] Preferably, the distance between the first center 411 and the second center 421 is 60mm to 80mm. More preferably, the distance between the first center 411 and the second center 421 is 70mm. The vertical distance from the first center 411 or the second center 421 to the sheet 3 is 380mm to 500mm.

[0039] like Figure 2 As shown, the temperature acquisition mechanism 5 includes a first support 51 fixedly connected and kept relatively stationary, and a thermal imager 52. The first support 51 is fixedly connected to the tooling 2. Preferably, the vertical distance between the thermal imager 52 and the sheet metal 3 is 300mm to 700mm. The temperature measurement range of the thermal imager 52 is 625℃ to 1900℃.

[0040] In this embodiment, the thermal imager 52 captures images of the sheet metal 3 within a second rectangle. The length and width of the second rectangle can be set as needed. In a specific embodiment, the length of the second rectangle is 240mm and the width is 150mm.

[0041] Specifically, the image acquisition mechanism 4 also includes a second bracket 43 for fixing the image acquisition mechanism 4 and a checkerboard calibration plate for calibrating the image acquisition mechanism 4. The control mechanism includes a computer and data cables for connecting the image acquisition mechanism 4, the temperature acquisition mechanism 5, and the multi-axis force sensor to the computer. The computer is equipped with the online data processing unit. The online data processing unit can be a pre-defined online data processing program that can run on the computer. The online data processing unit may include hardware acceleration modules such as GPUs (Graphics Processing Units) or FPGAs (Field-Programmable Gate Arrays) to meet the high real-time requirements of the SGM and SVR algorithms.

[0042] Preferably, the control mechanism further includes a synchronous trigger controller, which simultaneously sends trigger signals to the image acquisition mechanism, the temperature acquisition mechanism, and the multi-axis force sensor to ensure that the data acquired by the three are strictly synchronized and aligned in time, thus solving the time axis coupling problem.

[0043] In this embodiment, the SGM algorithm is selected because it has stronger robustness to changes in illumination compared to other binocular vision recognition methods, and it has a faster running speed while ensuring a certain level of accuracy and effective disparity points.

[0044] Specifically, the core principle of the SGM algorithm is to utilize binocular visual triangulation to recover scene depth by calculating the positional difference (disparity) of corresponding pixels in the left and right images. The system first uses Census transform to encode the input image using local binary mode encoding, encoding the brightness comparison result of each pixel with its neighboring pixels into a binary string. This method is robust to illumination changes, has a fast response speed, and is less prone to blurring at image edges. Then, the SGM algorithm performs multi-directional path aggregation, transforming the stereo matching problem into an energy function minimization problem, balancing matching costs and smoothing constraints to obtain a high-quality initial disparity map. Based on this, the disparity range is optimized, and sub-pixel optimization and median filtering are performed. Finally, an accurate depth map is calculated using triangulation formulas. After obtaining the depth map, the system converts the effective pixels within the region into a 3D point cloud and uses a plane fitting algorithm for plane fitting. This algorithm significantly improves the accuracy and robustness of plane fitting through adaptive thresholding, statistical filtering preprocessing, weighted least squares, and nonlinear optimization. Finally, the system calculates the plane angles based on the fitted plane normal vectors, achieving quantitative analysis of planar relationships in the scene.

[0045] Furthermore, for the SVR algorithm, the model's input features include laser power, movement speed, real-time temperature, real-time force, and the rebound angle of the previous pass, while the output target is the predicted rebound angle of the current pass. The Simulated Annealing Particle Swarm Optimization-Support Vector Regression (SAPSO-SVR) algorithm is a hybrid intelligent optimization algorithm that integrates Particle Swarm Optimization (PSO), Simulated Annealing (SA), and Support Vector Regression (SVR). This algorithm first utilizes the swarm intelligence characteristic of PSO, simulating bird flock foraging behavior by allowing multiple particles to fly and search in the parameter space (penalty factor C, insensitive loss coefficient epsilon, and kernel parameter gamma). Each particle dynamically adjusts its flight direction and speed based on its own historical best position and the global best position of the swarm, thereby quickly locating the optimal parameter combination region of the SVR model. Building upon this, the algorithm introduces a simulated annealing mechanism to enhance its global search capability. Simulated annealing mimics the cooling process of metal annealing, initially accepting poor solutions with a higher probability, helping the algorithm escape local optima. As iterations progress, the temperature gradually decreases, reducing the probability of accepting poor solutions, and the algorithm gradually converges to the global optimum. Finally, the optimized parameters (penalty factor C, insensitive loss coefficient epsilon, and kernel parameter gamma) are used to construct a support vector regression model. SVR maps the input data to a high-dimensional feature space through a kernel function, constructing an optimal hyperplane in the high-dimensional space to fit the data. This minimizes prediction error while maintaining model complexity, thereby achieving accurate prediction of the bounce angle value for the current pass.

[0046] The tooling 2 in this embodiment may include a clamp for holding and fixing the sheet metal 3. The robot 1 is equipped with a robotic arm for processing the sheet metal 3, and the end of the robotic arm is equipped with a roller 11 for direct contact with the sheet metal 3. The multi-axis force sensor may be installed on the robotic arm to sense the force between the roller 11 and the sheet metal 3.

[0047] Based on the same concept, this invention also provides an information acquisition method for robot laser-assisted roll forming, as described in the following embodiments. Since the principle behind this information acquisition method and the technical effects it achieves are similar to the information acquisition system described above, the implementation of this information acquisition method can refer to the implementation of the information acquisition system described above, and repeated details will not be elaborated further.

[0048] Specifically, the information collection method employs the information collection system described in any of the above embodiments. The information collection method includes the following steps (steps S10, S20, and S30): Step S10: Determine and mark the predetermined measurement interval 32 on the sheet 3.

[0049] The predetermined measurement interval 32 is located within the effective range of the image acquisition mechanism 4 and the effective range of the temperature acquisition mechanism 5.

[0050] Step S20: Turn on the image acquisition mechanism 4, the temperature acquisition mechanism 5 and the multi-axis force sensor, and make the image acquisition mechanism 4, the temperature acquisition mechanism 5 and the multi-axis force sensor electrically connected to the control mechanism respectively. Adjust the position of the image acquisition mechanism 4 and the temperature acquisition mechanism 5 so that the predetermined measurement interval 32 is within the effective range of the image acquisition mechanism 4 and the effective range of the temperature acquisition mechanism 5.

[0051] Specifically, step S20 includes the following steps (steps S201 and S202): Step S201: Locate the midpoint of the bend line 31 within the predetermined measurement interval 32 as the second midpoint 311. Install the first industrial camera 41 and the second industrial camera 42, ensuring the line connecting the first midpoint 402 and the second midpoint 311 is perpendicular to the bend line 31. Connect the first industrial camera 41 and the second industrial camera 42 to the computer using data cables, ensuring the overlapping portion 401 of the areas captured by the first industrial camera 41 and the second industrial camera 42 covers the predetermined measurement interval 32. Figure 5 As shown; Step S202: Install the first bracket 51, fix the thermal imager 52 on the first bracket 51, connect the thermal imager 52 to the computer using a data cable, and adjust the first bracket 51 so that the thermal imager 52 can capture a rectangular area that covers the predetermined measurement interval 32.

[0052] Step S30: Start the online data processing unit to collect and process the multimodal information of sheet 3 during the robot laser-assisted roll forming process.

[0053] The multimodal information includes geometric shape information, temperature information, and stress information. Online processing includes integrating and processing geometric shape information based on a semi-global matching algorithm, and predicting the roll forming results using temperature information, stress information, and processing parameters based on a support vector regression algorithm.

[0054] In step S30, after starting the online data processing unit, calibration processing needs to be completed first, and then the multimodal information of sheet 3 in the robot laser-assisted roll forming process is collected and processed online based on the calibration processing results, so as to make the online processing results more accurate.

[0055] Between steps S20 and S30, the following steps may also be included: calibrating and binocularly correcting the first industrial camera 41 and the second industrial camera 42 using a checkerboard calibration plate; selecting the initial position of the steel plate 3 in the selection page provided by the online data processing unit.

[0056] Preferably, step S30 may be followed by step S40: the online data processing unit adjusts the processing parameters of robot 1 in real time based on the prediction of the rolling forming result, so as to perform feedforward or feedback control on the next rolling forming process and realize closed-loop control.

[0057] Compared to traditional multimodal information acquisition platforms, the information acquisition system provided in this application has significant advantages in real-time performance and uniformity. Its binocular vision recognition design can extract real-time geometric shape information of the sheet metal 3 during processing with high precision. Combined with temperature and mechanical data obtained from the thermal imager 52 and multi-axis mechanical sensors, it can obtain the real-time force-thermal coupling situation of the sheet metal 3 during the forming process within the predetermined measurement range 32. Compared to traditional multimodal information acquisition platforms, the information acquisition system provided in this application not only improves real-time performance but also achieves process integration, adapting to rapid configuration requirements.

[0058] The information acquisition method provided in this application can be used to predict the results of robot laser-assisted roll forming. Traditional data processing methods are mostly offline, with poor real-time performance, requiring a lot of software and lacking closed-loop control potential. In contrast, the information acquisition method provided in this application can perform online data processing, centralizing multimodal data processing on a single program and generating a dataset output, thus improving data processing efficiency and adapting to real-time processing adjustment scenarios. Based on this, the information acquisition system can couple multimodal data on sheet geometry, stress conditions, and thermal conditions during processing through machine learning, thereby predicting real-time processing information and further improving the accuracy of the prediction model during part processing. This is particularly suitable for scenarios requiring real-time precision control, such as robot laser-assisted roll forming. Simultaneously, the real-time nature of online data processing avoids secondary clamping of the parts to be processed, reducing repetitive positioning accuracy errors caused by traditional data processing methods. It can support closed-loop control and quality prediction, promoting the transformation of robot laser-assisted roll forming from the laboratory to industrial applications.

[0059] The information acquisition system and method provided in this application can simultaneously acquire mechanical, temperature, and geometric morphology information of sheet metal 3 during laser-assisted roll forming by robot 1, and process this information through a program, thereby achieving real-time information acquisition and online data processing. This information acquisition system and method solve the lag problem of existing acquisition methods in the multimodal information acquisition process, realizing real-time acquisition of multimodal information of sheet metal 3 during processing. Furthermore, by combining multimodal data, this invention can predict the forming result based on multimodal information and forming process parameters during the forming process.

[0060] To better illustrate the advantages of the information acquisition system and method for robotic laser-assisted roll forming provided in this application, the following three embodiments and one comparative example are provided.

[0061] Example 1

[0062] The information acquisition system and method use martensitic steel to make plate 3. In the process of information acquisition and data processing, the first industrial camera 41 and the second industrial camera 42 are used to acquire the geometric information of plate 3, the thermal imager 52 fixed on the first bracket 51 is used to acquire temperature information, the multi-axis mechanical sensor fixed on the industrial robot 1 is used to acquire mechanical information, and the above multimodal information is processed by the online data processing unit (online data processing algorithm) running on the computer platform.

[0063] In this embodiment, the martensitic steel has a tensile strength of 1000 MPa, a room temperature elongation of not less than 5%, and a thickness of 2.0 mm.

[0064] In this embodiment, the following is adopted: Figure 1 The information acquisition system shown comprises: a first industrial camera 41 and a second industrial camera 42 fixed by a second bracket 43; a thermal imager 52 fixed by a first bracket 51 connected to a fixture 2; a steel sheet 3 clamped by the fixture 2; and rollers 11 roll-forming the sheet 3 according to the shape of the target part. During the forming process, a laser head simultaneously heats the sheet 3 locally to assist in its forming. The first industrial camera 41 and the second industrial camera 42 are placed horizontally. The distance between the first midpoint 402 and the second midpoint 311 is 380mm.

[0065] In this embodiment, robot 1 performs roll forming in each pass, with the sheet metal 3 bending at an angle of 15°, for a total of 6 passes. The bent sheet metal 3 is as follows: Figure 3 and Figure 4 As shown, Figure 4 The morphology of sheet metal 3 after the first pass 61, the second pass 62, and the nth pass 63 is shown. The maximum surface temperature of sheet metal 3 does not exceed 1900℃.

[0066] In this embodiment, the parameters of the first industrial camera 41 and the second industrial camera 42 are shown in Table 1 below: Table 1 Resolution (pixels) Frame rate (fps) Sensor size (inches) Pixel size (micrometers) Lens focal length (mm) 5472×3648 6 1 / 1.8 1.4×1.4 25 The processing parameters for this embodiment are shown in Table 2 below: Table 2 Laser power (W) Movement speed (mm / s) Spot size (mm) Angle of each bend (°) Sheet thickness (mm) Sheet length (mm) Sheet width (mm) 1200 10 2×6 15 2.0 80 80 Example 2

[0067] The information acquisition system and method use martensitic steel to make plate 3. In the process of information acquisition and data processing, the first industrial camera 41 and the second industrial camera 42 are used to acquire the geometric information of plate 3, the thermal imager 52 fixed on the first bracket 51 is used to acquire temperature information, the multi-axis mechanical sensor fixed on the industrial robot 1 is used to acquire mechanical information, and the above multimodal information is processed by the online data processing unit (online data processing algorithm) running on the computer platform.

[0068] In this embodiment, the martensitic steel has a tensile strength of 1000 MPa, a room temperature elongation of not less than 5%, and a thickness of 2.0 mm.

[0069] In this embodiment, the following is adopted: Figure 1 The information acquisition system shown comprises: a first industrial camera 41 and a second industrial camera 42 fixed by a second bracket 43; a thermal imager 52 fixed by a first bracket 51 connected to a fixture 2; a steel sheet 3 clamped by the fixture 2; and rollers 11 roll-forming the sheet 3 according to the shape of the target part. During the forming process, a laser head simultaneously heats the sheet 3 locally to assist in its forming. The first industrial camera 41 and the second industrial camera 42 are positioned at a 10° downward angle relative to the horizontal plane. The distance between the first midpoint 402 and the second midpoint 311 is 385 mm.

[0070] In this embodiment, robot 1 performs roll forming in each pass, with the sheet metal 3 bending at an angle of 15°, for a total of 6 passes. The bent sheet metal 3 is as follows: Figure 3 and Figure 4 As shown. The maximum surface temperature of sheet 3 shall not exceed 1900℃.

[0071] In this embodiment, the parameters of the first industrial camera 41 and the second industrial camera 42 are shown in Table 3 below: Table 3 Resolution (pixels) Frame rate (fps) Sensor size (inches) Pixel size (micrometers) Lens focal length (mm) 5472×3648 6 1 / 1.8 1.4×1.4 25 The processing parameters for this embodiment are shown in Table 4 below: Table 4 Laser power (W) Movement speed (mm / s) Spot size (mm) Angle of each bend (°) Sheet thickness (mm) Sheet length (mm) Sheet width (mm) 1200 15 2×6 15 2.0 80 80 Example 3

[0072] The information acquisition system and method use martensitic steel to make plate 3. In the process of information acquisition and data processing, the first industrial camera 41 and the second industrial camera 42 are used to acquire the geometric information of plate 3, the thermal imager 52 fixed on the first bracket 51 is used to acquire temperature information, the multi-axis mechanical sensor fixed on the industrial robot 1 is used to acquire mechanical information, and the above multimodal information is processed by the online data processing unit (online data processing algorithm) running on the computer platform.

[0073] In this embodiment, the martensitic steel has a tensile strength of 1300 MPa, a room temperature elongation of not less than 5%, and a thickness of 2.0 mm.

[0074] In this embodiment, the following is adopted: Figure 1 The information acquisition system shown comprises: a first industrial camera 41 and a second industrial camera 42 fixed by a second bracket 43; a thermal imager 52 fixed by a first bracket 51 connected to a fixture 2; a steel sheet 3 clamped by the fixture 2; and rollers 11 roll-forming the sheet 3 according to the shape of the target part. During the forming process, a laser head simultaneously heats the sheet 3 locally to assist in its forming. The first industrial camera 41 and the second industrial camera 42 are positioned at a 10° downward angle relative to the horizontal plane. The distance between the first midpoint 402 and the second midpoint 311 is 385 mm.

[0075] In this embodiment, robot 1 performs roll forming in each pass, with the sheet metal 3 bending at an angle of 15°, for a total of 6 passes. The bent sheet metal 3 is as follows: Figure 3 and Figure 4 As shown. The maximum surface temperature of sheet 3 shall not exceed 1900℃.

[0076] In this embodiment, the parameters of the first industrial camera 41 and the second industrial camera 42 are shown in Table 5 below: Table 5 Resolution (pixels) Frame rate (fps) Sensor size (inches) Pixel size (micrometers) Lens focal length (mm) 5472×3648 6 1 / 1.8 1.4×1.4 25 The processing parameters for this embodiment are shown in Table 6 below: Table 6 Laser power (W) Movement speed (mm / s) Spot size (mm) Angle of each bend (°) Sheet thickness (mm) Sheet length (mm) Sheet width (mm) 2000 20 2×6 15 2.0 80 80 Comparative Example 1 The difference between Comparative Example 1 and Example 1 is that Comparative Example 1 uses a 3D scanning measuring instrument to scan the contour of the steel sheet 3 after each processing pass, obtains the geometric shape information of the steel sheet 3 through 3D reconstruction, and performs offline data processing based on this. The following comparative analysis of Example 1 and Comparative Example 1 further illustrates the advantages of the present invention: 1. Analysis of the efficiency of handling the bending angle of steel sheet 3 in Comparative Example 1 and Example 1 Comparative Example 1: The principle of using a 3D contour scanner to process the bending angle of steel sheet 3 is as follows. Figure 7 As shown. The outline of steel sheet 3 is processed using a 3D scanner. This requires waiting for sheet 3 to be finished and removed from fixture 2. Then, the inner surface of steel sheet 3 is scanned multiple times from multiple angles to construct a 3D point cloud. Specific software is used for analysis, and specific straight lines are fitted (forming a shape like...). Figure 7 The first fitted line 71 and the second fitted line 72 shown increase the time required for data processing.

[0077] In Example 1, a binocular camera system (first industrial camera 41 and second industrial camera 42) is used for measurement, and its principle is as follows: Figure 8 As shown. Using a binocular camera system for measurement, only one surface of the steel sheet 3 needs to be photographed. A disparity map is obtained through disparity analysis, and a certain number (usually ≥20) of effective disparity points 83 are selected from it. A plane fitting algorithm is then used to obtain the normal vector 85 of the fitting plane 84 for the predetermined measurement interval 32 of the steel sheet 3. The angle between this plane and the normal vector 82 of the initial plane 81 is calculated to obtain the resulting bending angle 86. Compared to Comparative Example 1, this method eliminates the need for a complete 3D reconstruction of the entire sheet 3, directly extracting effective points for fitting, thus improving data processing efficiency while offering greater freedom.

[0078] 2. Comparative Analysis of Data Processing Method Efficiency and Closed-Loop Control Feasibility of Comparative Example 1 and Example 1 In Comparative Example 1, the traditional offline data processing method collects and integrates the mechanical and temperature information gathered during the workpiece processing, as well as the geometric morphology information obtained through 3D reconstruction after the workpiece processing is completed, to form a dataset for processing. This data processing method can only obtain one set of data per processing pass. Due to the lack of geometric morphology information on the time axis, it is impossible to analyze in real time the influence of force-thermal coupling factors on the geometric morphology of each point on the steel plate 3 during the processing. By processing and analyzing the dataset obtained by the traditional offline data processing method, open-loop control of the steel plate forming process can be achieved, that is, by optimizing process parameters before processing to improve processing quality. However, closed-loop control is not feasible for errors generated during processing.

[0079] In Example 1, the online data processing method selects common frames sampled by the first industrial camera 41, the second industrial camera 42, the thermal imager 52, and the multi-axis mechanical sensor as valid data frames. Multimodal information is extracted from these common frames from different information acquisition devices to form a dataset. This dataset contains the geometry of the steel sheet 3, its temperature, and the forming force acting on it, reflecting real-time information about the steel sheet 3 during processing. Optionally, this information acquisition system also has the capability to perform machine learning based on the acquired dataset. With a certain degree of accuracy, it can predict the processing results of the sheet 3 and the influence trends of various factors on the processing results based on the stress and heat conditions of the sheet 3. This result can be used to achieve closed-loop control of the processing process.

[0080] 3. Analysis of the processing accuracy of sheet metal 3 in Comparative Example 1 and Example 1 In Comparative Example 1, traditional multimodal information acquisition and offline data processing methods require multiple disassembly and reassembly of sheet metal 3 when acquiring and processing multimodal information. This can lead to deviations in the bending path of sheet metal 3 during these multiple disassembly and reassembly processes, such as... Figure 9 As shown, the initial bending line 91 does not coincide with the reinstalled bending line 92. This deviation in the bending path leads to a decrease in bending accuracy and, to some extent, affects the accuracy of the obtained data.

[0081] Compared with traditional multimodal information acquisition and offline data processing methods, the information acquisition system and method in Example 1 do not require multiple disassembly and assembly of sheet 3, ensuring that sheet 3 processing and information measurement are completed in one clamping, avoiding forming accuracy errors caused by repeated clamping, and reducing random errors in measurement results caused by multiple clamping. It is particularly suitable for industrial processing scenarios with high processing accuracy requirements and real-time monitoring and control.

[0082] Table 7 below shows a comparison of the measurement accuracy and efficiency between Example 1 and Comparative Example 1. It can be seen that the measurement accuracy and efficiency of Example 1 are both superior to those of Comparative Example 1.

[0083] Table 7 Comparison of measurement accuracy and efficiency between Example 1 and Comparative Example 1 project Example 1 Comparative Example 1 Time taken per measurement <1 second (online synchronous data collection) Approximately 5 minutes (including disassembly, scanning, and reconstruction) Bending angle measurement error ±0.1° ±0.3° Does the system support online feedback? yes no It should be noted that in the description of this specification, the terms "first," "second," etc., are used only for descriptive purposes and to distinguish similar objects; there is no order between them, nor should they be construed as indicating or implying relative importance. Furthermore, in the description of this specification, unless otherwise stated, "a plurality of" means two or more.

[0084] Any numerical values ​​cited herein include all values ​​ranging from a lower limit to an upper limit, increasing by one unit, with at least two units between any lower and any higher value. For example, if the quantity of a component or the value of a process variable (e.g., temperature, pressure, time, etc.) is described as being from 1 to 90, preferably from 20 to 80, more preferably from 30 to 70, the purpose is to illustrate that values ​​such as 15 to 85, 22 to 68, 43 to 51, 30 to 32 are also explicitly listed in this specification. For values ​​less than 1, a unit is appropriately considered to be 0.0001, 0.001, 0.01, 0.1, etc. These are merely examples intended for explicit expression, and it can be assumed that all possible combinations of values ​​listed between the minimum and maximum values ​​are explicitly described in this specification in a similar manner.

[0085] Unless otherwise stated, all ranges include the endpoints and all numbers between them. The terms "approximately" or "about" used with ranges apply to both endpoints of the range. Thus, "approximately 20 to 30" is intended to cover "approximately 20 to approximately 30," including at least the specified endpoints.

[0086] All articles and references disclosed herein, including patent applications and publications, are incorporated herein by reference for various purposes. The term “substantially constitutes…” used to describe a combination should include the identified elements, components, parts, or steps, as well as other elements, components, parts, or steps that do not substantially affect the essential novelty of the combination. The use of the terms “comprising” or “including” to describe combinations of elements, components, parts, or steps herein also contemplates embodiments substantially constituted by such elements, components, parts, or steps. The use of the term “may” herein is intended to indicate that any described attribute included by “may” is optional.

[0087] Multiple elements, components, parts, or steps can be provided by a single integrated element, component, part, or step. Alternatively, a single integrated element, component, part, or step can be divided into multiple separate elements, components, parts, or steps. The use of "a" or "an" to describe an element, component, part, or step does not imply the exclusion of other elements, components, parts, or steps.

[0088] It should be understood that the above description is for illustrative purposes and not for limitation. Many embodiments and applications beyond the provided examples will be apparent to those skilled in the art upon reading the above description. Therefore, the scope of this teaching should not be determined by reference to the above description, but rather by reference to the appended claims and the full scope of their equivalents. For purposes of completeness, all articles and references, including patent applications and publications, are incorporated herein by reference. The omission of any aspect of the subject matter disclosed herein in the preceding claims is not intended as a waiver of that subject matter, nor should it be construed as an indication that the inventors have not considered that subject matter as part of the disclosed inventive subject matter.

Claims

1. An information acquisition system for robot laser-assisted roll forming, comprising using a robot to perform laser-assisted roll forming on sheet metal fixed on a tooling based on predetermined processing parameters; characterized in that, The information collection system includes: An image acquisition mechanism is fixedly installed on one side of the sheet metal for acquiring geometric shape information of the sheet metal; the effective range of the image acquisition mechanism covers the entire sheet metal. A temperature acquisition mechanism is fixedly installed on the other side of the sheet metal to acquire the temperature information of the sheet metal; the effective range of the temperature acquisition mechanism covers the entire sheet metal and is at least twice the effective range of the image acquisition mechanism. A multi-axis force sensor is fixedly mounted on the robot to collect the force information of the sheet metal; A control mechanism electrically connected to the image acquisition mechanism, temperature acquisition mechanism, and multi-axis mechanical sensor is used to acquire the geometric shape information, temperature information, and force information. The control mechanism includes an online data processing unit, which is used to integrate and process the geometric shape information based on a semi-global matching algorithm, and to predict the roll forming result based on the temperature information, force information, and processing parameters using a support vector regression algorithm.

2. The information acquisition system for robot laser-assisted roll forming according to claim 1, characterized in that, The image acquisition mechanism includes a first industrial camera and a second industrial camera fixed at a certain angle and kept relatively still; the first industrial camera and the second industrial camera are set at a downward angle of 0~15° relative to the horizontal plane.

3. The information acquisition system for robot laser-assisted roll forming according to claim 2, characterized in that, The shooting range of the first industrial camera is defined as the first shooting range, and the shooting range of the second industrial camera is defined as the second shooting range; the first shooting range and the second shooting range have an overlapping portion, and the overlapping portion covers a predetermined measurement interval.

4. The information acquisition system for robot laser-assisted roll forming according to claim 3, characterized in that, The first and second shooting ranges are shaped like a first rectangle on the sheet metal. The length of the first rectangle is greater than the length of the overlapping portion, and the width of the first rectangle is equal to the width of the overlapping portion. The length of the overlapping portion is 20mm to 40mm.

5. The information acquisition system for robot laser-assisted roll forming according to claim 3, characterized in that, The sheet metal has a bending line; the lens center of the first industrial camera is defined as the first center, and the lens center of the second industrial camera is defined as the second center; the midpoint of the line connecting the first center and the second center is defined as the first midpoint, and the midpoint of the bending line located within the predetermined measurement interval is defined as the second midpoint; the line connecting the first midpoint and the second midpoint is perpendicular to the bending line.

6. The information acquisition system for robot laser-assisted roll forming according to claim 5, characterized in that, The distance between the first center and the second center is 60mm~80mm, and the vertical distance from the first center or the second center to the sheet material is 380mm~500mm.

7. The information acquisition system for robot laser-assisted roll forming according to claim 1, characterized in that, The temperature acquisition mechanism includes a first support fixedly connected to and relatively stationary, and a thermal imager. The first support is fixedly connected to the tooling.

8. The information acquisition system for robot laser-assisted roll forming according to claim 7, characterized in that, The vertical distance between the thermal imager and the sheet metal is 300mm to 700mm; the measurement temperature range of the thermal imager is 625℃ to 1900℃.

9. The information acquisition system for robot laser-assisted roll forming according to claim 1, characterized in that, The image acquisition mechanism further includes a second bracket for fixing the image acquisition mechanism and a checkerboard calibration plate for calibrating the image acquisition mechanism; the control mechanism includes a computer and a data cable for connecting the image acquisition mechanism, the temperature acquisition mechanism and the multi-axis mechanical sensor to the computer; the computer is equipped with the online data processing unit.

10. An information acquisition method for robot laser-assisted roll forming, characterized in that, The information collection method is performed using the information collection system described in any one of claims 1 to 9; the information collection method includes the following steps: Step S10: Determine and mark a predetermined measurement range on the sheet metal; the predetermined measurement range is located within the effective range of the image acquisition mechanism and the effective range of the temperature acquisition mechanism; Step S20: Open the image acquisition mechanism, temperature acquisition mechanism and multi-axis force sensor, and make the image acquisition mechanism, temperature acquisition mechanism and multi-axis force sensor electrically connected to the control mechanism respectively; adjust the position of the image acquisition mechanism and temperature acquisition mechanism so that the predetermined measurement interval is within the effective range of the image acquisition mechanism and the effective range of the temperature acquisition mechanism. Step S30: Activate the online data processing unit to collect and process the multimodal information of the sheet metal during the robot laser-assisted roll forming process; the multimodal information includes the geometric shape information, temperature information and stress information; the online processing includes integrating and processing the geometric shape information based on a semi-global matching algorithm, and predicting the roll forming result based on a support vector regression algorithm using the temperature information, stress information and processing parameters.

Images