Intelligent steel billet taking method based on multi-mode signals

Abstract

The invention discloses an intelligent steel billet taking method based on a multi-modal signal, and relates to the technical field of metallurgical industry automation, and the method comprises the following steps: extracting production takt data, and calculating and optimizing an execution time sequence of steel taking operation by analyzing the production takt data; a cooperative control model of the unmanned vehicle and the steel taking equipment is established, motion parameters of the steel taking equipment are adjusted through a cooperative control algorithm, and cooperative control parameters are generated; a fault-tolerant compensation system is switched to a laser point cloud matching mode in combination with the cooperative control parameters and the real-time working condition information, and multi-modal data are fused by using a federated Kalman filtering algorithm to generate a comprehensive signal; judging whether the steel taking task is completed or not; and a target position and a target posture are determined according to the comprehensive signal, and an actuator is driven by a control unit to execute steel taking operation. According to the invention, the comprehensiveness and accuracy of data are enhanced, and the acquisition parameters can be adaptively adjusted in a complex production environment, so that stable operation under different illumination and dust conditions is ensured.

Description

Technical Field

[0001] This invention relates to the field of automation technology in the metallurgical industry, and more specifically, to a smart billet extraction method based on multimodal signals. Background Technology

[0002] Currently, the application of technology in the steel billet tapping operation of billet heating furnaces still has many shortcomings, making it difficult to meet the production demands for high precision, high efficiency, and flexibility. Traditional steel tapping operations mainly rely on the experience and judgment of operators, manually determining the billet position and matching it with the production rhythm. This method is limited by the strong subjectivity of human factors and poor repeatability, easily leading to problems such as incorrect tapping sequence and furnace entry positioning deviations. This not only results in low tapping accuracy but also frequently causes production interruptions or delays. In high-intensity continuous production scenarios, it is difficult to maintain stable production quality and efficiency, severely restricting the continuity and reliability of steel production.

[0003] To overcome the limitations of manual operation, existing technologies have developed steel-retrieving control schemes based on automated systems. However, these systems generally employ fixed-program control modes. In actual production, billet heating furnaces often handle mixed production scenarios involving multiple billet types, such as square billets, slabs, and irregularly shaped billets, with billet specifications ranging from 3m to 12m in length. Different billet types and specifications exhibit significant differences in heating requirements and steel-retrieving cycle times. Fixed programs cannot dynamically adjust control logic based on these differences, resulting in automated systems being unable to adapt to the varying production cycle times of multiple billet specifications, hindering efficiency improvements and limiting flexible production capabilities. In the billet information detection stage, existing systems primarily rely on infrared or laser sensors to acquire information such as billet surface temperature and three-dimensional contours. However, these sensor technologies inherently suffer from a single signal, making them susceptible to interference from factors such as oxide scale on the billet surface and fluctuations in furnace temperature, leading to a billet positioning failure rate as high as 15%. This problem prevents existing systems from accurately capturing the complex shape characteristics and true temperature distribution of the billet, further exacerbating steel-retrieving positioning errors and affecting the accuracy and reliability of the steel-retrieving operation.

[0004] Furthermore, existing billet taking systems generally lack multi-modal signal fusion mechanisms, making it impossible to integrate multi-dimensional data such as billet 3D contours, surface temperature field distribution, and material coding for comprehensive analysis. The limitations of information from a single signal source make it difficult for the system to fully grasp the actual state of the billet, failing to provide accurate decision-making basis for taking operations and hindering the improvement of taking accuracy. Simultaneously, existing systems lack real-time dynamic adjustment mechanisms, failing to adapt to different billet heating states, heat conduction characteristics, and other mechanistic knowledge, and are unable to cope with complex situations such as billet state changes and process parameter fluctuations in the production environment, further reducing the stability and adaptability of taking operations.

[0005] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0006] To address the problems in related technologies, this invention proposes a smart billet extraction method based on multimodal signals to overcome the aforementioned technical issues in existing related technologies.

[0007] Therefore, the specific technical solution adopted by the present invention is as follows:

[0008] A smart billet steel extraction method based on multimodal signals, the method comprising:

[0009] S1. Based on the collected multimodal data of steel billets, extract the production cycle data, and calculate and optimize the execution sequence of steel taking operations by analyzing the production cycle data;

[0010] S2. Based on the optimized execution sequence, establish a collaborative control model for the unmanned vehicle and the steel-retrieving equipment, and adjust the motion parameters of the steel-retrieving equipment through a collaborative control algorithm to generate collaborative control parameters;

[0011] S3. Combining collaborative control parameters and real-time operating condition information, the fault-tolerant compensation system is switched to laser point cloud matching mode, and the federated Kalman filter algorithm is used to fuse multi-modal data to generate a comprehensive signal.

[0012] S4. Based on the generated integrated signal, determine whether the steel picking task is completed. If the steel picking task is not completed, return to S1. If the steel picking task is completed, execute S5.

[0013] S5. Determine the target position and target attitude based on the integrated signal, and drive the actuator to perform the steel picking operation through the control unit.

[0014] Furthermore, based on the collected multimodal data of steel billets, production cycle time data is extracted. The execution timing of steel taking operations is calculated and optimized by analyzing the production cycle time data, including:

[0015] S11. Collect multimodal data of steel billet, preprocess the collected multimodal data, and construct a three-dimensional point cloud model of steel billet based on the preprocessing results;

[0016] S12. Combining the three-dimensional point cloud model with multimodal data, a mapping database is constructed, and various billet shapes, materials and process parameters in the mapping database are extracted as steel performance indicators.

[0017] S13. Based on the extracted steel extraction performance indicators, the execution sequence of the steel extraction operation is calculated using the fuzzy PID algorithm, and the execution sequence is optimized through a preset long short-term memory network.

[0018] Furthermore, multimodal data of the steel billet is collected, and the collected multimodal data is preprocessed. Based on the preprocessing results, a three-dimensional point cloud model of the steel billet is constructed, including:

[0019] S111. Through the coordinated operation of a vision sensor, a laser rangefinder, and a coding recognition system, multimodal data of the steel billet is collected synchronously. The multimodal data includes three-dimensional contour data, surface temperature data, and material coding information.

[0020] S112. Use statistical filtering algorithms to denoise the collected multimodal data and perform multi-source data registration on the denoising results.

[0021] S113. The registration results of multi-source data are fused with geometric structure and surface texture, and combined with material coding information annotation, to construct a three-dimensional point cloud model containing geometric information, attribute information and accuracy information.

[0022] Furthermore, based on the extracted steel extraction performance indicators, the execution sequence of the steel extraction operation is calculated using a fuzzy PID algorithm, and the execution sequence is optimized through a pre-defined long short-term memory network, including:

[0023] S131. Based on the extracted steel extraction performance indicators, obtain production cycle data, analyze the production cycle data using a fuzzy PID algorithm, and calculate the execution sequence of the steel extraction operation based on the analysis results.

[0024] S132. Input the acquired production cycle data into a preset long short-term memory network, and output the cycle prediction results for future periods through temporal feature learning.

[0025] S133. Based on the beat prediction results, the execution timing is optimized using a distributed model predictive control algorithm to obtain the optimized execution timing.

[0026] Furthermore, based on the optimized execution sequence, a collaborative control model for the unmanned vehicle and the steel-retrieving equipment is established. The motion parameters of the steel-retrieving equipment are adjusted through a collaborative control algorithm, generating collaborative control parameters including:

[0027] S21. Analyze the optimized steel-retrieving operation execution sequence, identify the functional modules and corresponding interaction nodes of the unmanned vehicle and steel-retrieving equipment, generate a scene matrix, and define collaborative constraints.

[0028] S22. Based on the kinematic model of the unmanned vehicle and the steel-retrieving equipment, and combined with the cooperative constraints, construct a cooperative control model based on a distributed architecture.

[0029] S23. Based on the constructed collaborative control model, set multi-dimensional control objectives, and adjust the motion parameters of the steel taking equipment through an adaptive sliding mode control algorithm to generate collaborative control parameters.

[0030] Furthermore, based on the constructed collaborative control model, multi-dimensional control objectives are set, and the motion parameters of the steel-reclaiming equipment are adjusted through an adaptive sliding mode control algorithm to generate collaborative control parameters, including:

[0031] S231. The motion parameters, including the clamping force of the steel taking equipment, the movement speed of the telescopic arm, and the rotation attitude angle, are taken as multi-dimensional control targets. The deviation between the actual working condition information and the target state is taken as the input variable, and the parameter adjustment amount is taken as the output variable. A membership function based on fuzzy inference algorithm is constructed.

[0032] S232. Collect on-site operation experience and simulation data, combine them with the constructed membership function, establish a fuzzy control rule base that matches the multidimensional control objective, and use the max-min inference algorithm to perform inference operations on the input variables to generate the corresponding fuzzy output set.

[0033] S233. Use the centroid algorithm to defuzzify the fuzzy output set, generate control variables, and combine the adaptive adjustment mechanism to dynamically adjust the output variables of the fuzzy control rule base.

[0034] S234. Based on the dynamically adjusted output variables, adjust the motion parameters of the steel taking equipment to generate coordinated control parameters.

[0035] Furthermore, by combining collaborative control parameters with real-time operating condition information, the fault-tolerant compensation system is switched to laser point cloud matching mode, and the federated Kalman filter algorithm is used to fuse multimodal data to generate a comprehensive signal including:

[0036] S31. Analyze the generated collaborative control parameters and the collected real-time operating information, define the quantitative judgment criteria for main mode failure from multiple dimensions, and determine the trigger thresholds for each dimension in combination with the fault detection trigger conditions.

[0037] S32. Based on the determined trigger threshold, a finite state machine is used for discrimination, and the mode switching mechanism is driven based on the discrimination result to switch the fault-tolerant compensation system from the main mode to the laser point cloud matching mode.

[0038] S33. In the laser point cloud matching mode, laser point cloud data is acquired, and the laser point cloud data and multimodal data are fused through a preset federated architecture using the Kalman filter algorithm and DS evidence theory to generate a comprehensive signal.

[0039] Furthermore, fault detection triggering conditions include sensor status, positioning accuracy, environmental interference, and manual triggering.

[0040] Furthermore, based on a determined trigger threshold, a finite state machine is used for discrimination, and the discrimination result drives a mode switching mechanism to switch the fault-tolerant compensation system from the main mode to the laser point cloud matching mode, including:

[0041] S321. Perform redundancy verification on the determined trigger threshold and sort the redundancy verification results according to the preset priority rules.

[0042] S322. Based on the priority sorting result, trigger the corresponding level mode switching command, start the lidar preheating program, and simultaneously send a stop operation command to the main mode to release the corresponding resources.

[0043] S323. Based on the resource release results of the main mode, activate the lidar and start the initialization process, and switch the fault tolerance compensation system to the lidar point cloud matching mode.

[0044] Furthermore, the target position and attitude are determined based on the integrated signals, and the actuator is driven by the control unit to perform the steel-retrieving operation, including:

[0045] S51. Based on the generated integrated signal, calculate the target position and target posture of the steel taking operation. Based on the calculation results, divide the steel taking operation into different motion stages and set corresponding motion parameters for each stage. The motion stages include the standby to pre-grab stage, the pre-grab to grab stage, and the grab to lift stage.

[0046] S52. Based on the motion parameters set at each stage, the actuator is driven by the control unit to perform motion, and the motion process is dynamically corrected by combining real-time acquired multimodal data to ensure motion accuracy.

[0047] The beneficial effects of this invention are as follows:

[0048] 1. This invention constructs a multimodal perception-decision-execution closed-loop system, combining multiple sensors such as visual sensors, laser ranging arrays, and inkjet coding recognition systems, to achieve multi-dimensional data acquisition of steel billets. This overcomes the limitations of traditional single signals, improves the positioning accuracy of steel picking, and thus solves problems such as incorrect steel picking sequence and excessive furnace entry positioning deviation. The collaborative work of these multimodal sensors not only enhances the comprehensiveness and accuracy of the data, but also enables adaptive adjustment of acquisition parameters in complex production environments, thereby ensuring stable operation under different lighting and dust conditions.

[0049] 2. This invention employs dynamic production cycle technology, based on fuzzy PID algorithm and LSTM neural network, to achieve automatic adaptive production of steel billets of different shapes and specifications. This solves the problem that existing fixed program control cannot cope with the differences in cycle times for mixed production of multiple billet shapes and specifications, thus improving production efficiency. By monitoring key indicators in the production process in real time, such as production speed and defect rate, the production cycle can be dynamically adjusted to ensure the continuity and stability of production, while reducing downtime caused by changes in specifications.

[0050] 3. This invention overcomes the influence of interference factors such as oxide scale on the surface of steel billets and ambient temperature fluctuations on positioning accuracy through the fusion and collaborative control mechanism of multi-modal signals, reduces the positioning failure rate, and thus improves the reliability and stability of steel picking. This collaborative mechanism ensures the accuracy and consistency of positioning results by identifying and eliminating abnormal data through intelligent algorithms, and further enhances the anti-interference ability and stability of the system.

[0051] 4. Due to the improved steel picking accuracy and optimized production cycle, this invention reduces equipment idle time and energy consumption, achieving energy-saving and environmentally friendly effects. Through refined monitoring and optimization of energy consumption, the system can minimize energy consumption while ensuring production quality, meeting the requirements of modern industry for green production.

[0052] 5. This invention reduces reliance on human experience and judgment through intelligent multimodal signal acquisition and analysis, thereby reducing errors in manual operation and improving the level of intelligence in the production process. This intelligent analysis not only supports the application of complex algorithms but also provides operators with intuitive production status feedback through a visual interface, enabling operators to make better decisions and adjustments, and further improving the overall efficiency and quality of production. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart of a smart billet steel extraction method based on multimodal signals according to an embodiment of the present invention;

[0055] Figure 2 This is a flowchart of the execution timing optimization of a smart billet steel extraction method based on multimodal signals according to an embodiment of the present invention;

[0056] Figure 3 This is a flowchart of the collaborative control parameter generation process for a smart billet steel extraction method based on multimodal signals according to an embodiment of the present invention.

[0057] Figure 4 This is a flowchart of the integrated signal generation process for a smart billet steel extraction method based on multimodal signals according to an embodiment of the present invention.

[0058] Figure 5 This is a flowchart illustrating the execution of a smart billet steel removal method based on multimodal signals according to an embodiment of the present invention. Detailed Implementation

[0059] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention.

[0060] According to an embodiment of the present invention, a smart billet steel extraction method based on multimodal signals is provided.

[0061] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1 As shown, according to an embodiment of the present invention, a smart billet steel extraction method based on multimodal signals includes:

[0062] S1. Based on the collected multimodal data of steel billets, extract the production cycle data, and calculate and optimize the execution sequence of steel taking operations by analyzing the production cycle data.

[0063] In this optional embodiment, based on the collected multimodal data of steel billets, production cycle time data is extracted, and the execution timing of steel taking operations is calculated and optimized by analyzing the production cycle time data, including:

[0064] S11. Collect multimodal data of steel billet, preprocess the collected multimodal data, and construct a three-dimensional point cloud model of steel billet based on the preprocessing results.

[0065] In this optional embodiment, multimodal data of the steel billet is collected, and the collected multimodal data is preprocessed. Based on the preprocessing results, a three-dimensional point cloud model of the steel billet is constructed, including:

[0066] S111. Through the coordinated operation of a vision sensor, a laser rangefinder, and a coding recognition system, multimodal data of the steel billet is collected synchronously. The multimodal data includes three-dimensional contour data, surface temperature data, and material coding information.

[0067] It should be noted that the components work together as shown in Table 1.

[0068] Table 1: Equipment Selection Table

[0069] S112. Use statistical filtering algorithms to denoise the collected multimodal data and perform multi-source data registration on the denoising results.

[0070] S113. The registration results of multi-source data are fused with geometric structure and surface texture, and combined with material coding information annotation, to construct a three-dimensional point cloud model containing geometric information, attribute information and accuracy information.

[0071] It should be noted that the sensor equipment selection is shown in Table 2. Through the collaborative work of these multiple sensors and systems, the geometric features and physical properties of the steel billet can be fully captured, thereby providing basic data support for subsequent intelligent processing. In addition, it also involves the preprocessing and feature extraction of point cloud data to ensure the validity and consistency of the data, and to provide stable and reliable input for subsequent modeling and control.

[0072] Table 2: Equipment Selection Table

[0073] S12. Combining the three-dimensional point cloud model with multimodal data, a mapping database is constructed, and various billet shapes, materials and process parameters in the mapping database are extracted as steel performance indicators.

[0074] S13. Based on the extracted steel extraction performance indicators, the execution sequence of the steel extraction operation is calculated using the fuzzy PID algorithm, and the execution sequence is optimized through a preset long short-term memory network.

[0075] In this optional embodiment, based on the extracted steel extraction performance indicators, the execution sequence of the steel extraction operation is calculated using a fuzzy PID algorithm, and the execution sequence is optimized using a preset long short-term memory network, including:

[0076] S131. Based on the extracted steel extraction performance indicators, obtain production cycle data, analyze the production cycle data using a fuzzy PID algorithm, and calculate the execution sequence of the steel extraction operation based on the analysis results.

[0077] S132. Input the acquired production cycle data into a preset long short-term memory network, and output the cycle prediction results for future periods through temporal feature learning.

[0078] S133. Based on the beat prediction results, the execution timing is optimized using a distributed model predictive control algorithm to obtain the optimized execution timing.

[0079] It should be added that, such as Figure 2 As shown, a billet shape-material-process parameter mapping database is established, the steel picking interval time is calculated by analyzing the production cycle data, and the steel picking sequence is optimized by using a Long Short-Term Memory (LSTM) network. By conducting in-depth analysis of time and resource utilization in the production process, the steel picking sequence and interval are optimized to improve overall production efficiency and flexibility. The application of the LSTM network enables this step to adapt to changes under different process conditions, further enhancing the system's adaptability and predictive ability.

[0080] S2. Based on the optimized execution sequence, establish a collaborative control model for the unmanned vehicle and the steel-retrieving equipment, and adjust the motion parameters of the steel-retrieving equipment through a collaborative control algorithm to generate collaborative control parameters.

[0081] It should be added that, such as Figure 3 As shown, a collaborative control model for the unmanned vehicle and the steel-retrieving equipment is established, and the parameters of the steel-retrieving machine are adjusted through a collaborative control algorithm. Through the fuzzy control technology of the collaborative control algorithm, the position and attitude of the steel-retrieving machine can be dynamically adjusted according to the real-time changing working conditions, thereby ensuring the accuracy and reliability of the steel-retrieving process. In addition, by fusing and processing data from multiple sensors, the accuracy and timeliness of collaborative control can be ensured.

[0082] In this optional embodiment, a collaborative control model for the unmanned vehicle and the steel-retrieving equipment is established based on the optimized execution timing. The motion parameters of the steel-retrieving equipment are adjusted through a collaborative control algorithm, and the generated collaborative control parameters include:

[0083] S21. Analyze the optimized steel-retrieving operation execution sequence, identify the functional modules and corresponding interaction nodes of the unmanned vehicle and steel-retrieving equipment, generate a scene matrix, and define collaborative constraints.

[0084] It should be noted that the scenario matrix is ​​shown in Table 3, where the collaborative constraints include spatial constraints, temporal constraints, and operating condition constraints.

[0085] Table 3: Scene Matrix Table

[0086] S22. Based on the kinematic models of the unmanned vehicle and the steel-retrieving equipment, and combined with the cooperative constraints, construct a cooperative control model based on a distributed architecture.

[0087] It should be noted that in the autonomous vehicle motion model (independent control of X / Y / Z axes), the vehicle's position on the X-axis is defined as x. c (t), velocity v c (t), acceleration a c (t), then:

[0088] ;

[0089] Constraint: a c (t)≤0.5m / s² (avoid impact), v c (t)≤v cmax (v) cmax The value is determined by the weight of the steel billet, such as when it is 20t. cmax =1.5m / s).

[0090] Take the motion model of the steelmaking machine (telescopic boom X-axis + rotation angle θ), and let the position of the telescopic boom be x.e (t), with a rotation angle of θ e (t), then:

[0091] ; (k e =0.1m / V is the gain, u e (t) represents the control voltage.

[0092] ; (k θ =1° / V is the gain, u θ (t) represents the control voltage.

[0093] Constraint: |θ e (t)|≤15° (to prevent billet from colliding with the furnace body),|x e (t)|≤2m (upper limit of telescopic stroke).

[0094] The cooperative constraint equations include position synchronization constraints and force balance constraints:

[0095] Position synchronization constraint: Steel picker clamping center (x e ,y e ) and the center of the billet (x b ,y b The deviation must meet the following requirements:

[0096] ;

[0097] Force balance constraint: Clamping force F of the steel picker c The billet weight balance must be met:

[0098] ;

[0099] Where, m b Let g be the mass of the steel billet, g = 9.8 m / s², and μ = 0.3 be the coefficient of friction between the clamping jaws and the steel billet; for example: m b When =20t, Set the value to 350kN (leaving a 10% margin).

[0100] The collaborative control model is a three-layer distributed architecture, including: a perception layer, which integrates multi-sensor data (basic input), as shown in Table 4; a decision layer, which integrates collaborative logic and task planning (the core brain); and an execution layer, which standardizes equipment control interfaces (implementation). Each layer interacts through standardized interfaces to ensure flexibility and scalability, as shown in Table 5.

[0101] Table 4: Sensor Configuration Table

[0102] Table 5: Control Interface Design Table

[0103] S23. Based on the constructed collaborative control model, set multi-dimensional control objectives, and adjust the motion parameters of the steel taking equipment through an adaptive sliding mode control algorithm to generate collaborative control parameters.

[0104] In this optional embodiment, based on the constructed cooperative control model, multi-dimensional control objectives are set, and the motion parameters of the steel-retrieving equipment are adjusted through an adaptive sliding mode control algorithm to generate cooperative control parameters, including:

[0105] S231. Using motion parameters including the clamping force of the steel-retrieving equipment, the movement speed of the telescopic boom, and the rotational attitude angle as multi-dimensional control targets, and taking the deviation between the actual working conditions and the target state as input variables, and the parameter adjustment amount as output variables, a membership function based on a fuzzy inference algorithm is constructed.

[0106] It should be noted that the multidimensional control objectives are shown in Table 6; the input and output variables are shown in Table 7.

[0107] Table 6: Multidimensional Control Objectives Table

[0108] Table 7: Input / Output Variables Table

[0109] S232. Collect on-site operation experience and simulation data, combine them with the constructed membership function, establish a fuzzy control rule base that matches the multidimensional control objective, and use the max-min inference algorithm to perform inference operations on the input variables to generate the corresponding fuzzy output set.

[0110] It should be noted that, based on field operation experience and simulation data, multiple input multiple output (MIMO) fuzzy rules were established, with a total of 3×5×5=75 core rules, as shown in Table 8.

[0111] Table 8: Fuzzy Control Rule Table

[0112] S233. Use the centroid algorithm to defuzzify the fuzzy output set, generate control variables, and combine with an adaptive adjustment mechanism to dynamically adjust the output variables of the fuzzy control rule base.

[0113] It should be noted that the adaptive adjustment mechanism includes:

[0114] 1. Rule adjustment trigger conditions; rule adaptive optimization will be initiated when any of the following conditions are met:

[0115] (1) Control deviation exceeds threshold: such as clamping force deviation |eF |>10kN (lasting 50ms), or positional deviation|e x |>3mm (lasts 30ms).

[0116] (2) Sudden change in working conditions: such as billet temperature change rate > 5℃ / s (thermal shock), or billet weight change > 5t (load change).

[0117] (3) Changes in equipment status: such as the change rate of current of steel taking machine > 10% / s (increased friction).

[0118] 2. Adaptive adjustment algorithm (based on error feedback):

[0119] The output weights of fuzzy rules are dynamically adjusted using the "error correction factor method," and the steps are as follows:

[0120] (1) Define the error correction factor α (α∈[0.5,1.5]), with an initial value of α=1.0.

[0121] (2) Calculate the "accumulated error value". (e represents control deviation, such as e) F ).

[0122] (3) Adjust the correction factor:

[0123] If E > E max (e.g. E) max =50kN·s, the deviation is consistently large), then α=α+0.1 (increase the output adjustment amount to speed up convergence);

[0124] If E < E min (e.g. E) min =10kN·s, the deviation is too small), then α=α-0.1 (reduce the output adjustment amount to avoid overshoot).

[0125] (4) Update the fuzzy rule output: new output adjustment amount (For example, if the original ΔF = 7kN and α = 1.2, then the new ΔF = 8.4kN).

[0126] For example: when the billet temperature suddenly rises from 500℃ to 700℃ (high temperature H), the clamping force deviation e F =-15kN (insufficient), then: the original rule output ΔF=8kN (α=1.0); the cumulative error value E=15×0.1=1.5kN・s<E_min=10→α is adjusted to 0.9. In reality, due to the increase in temperature, the force needs to be reduced, and the correction factor α should be adjusted according to the temperature: at high temperature, α=0.9 (original ΔF=8kN→new ΔF=7.2kN, to avoid clamping too tightly).

[0127] S234. Based on the dynamically adjusted output variables, adjust the motion parameters of the steel taking equipment to generate coordinated control parameters.

[0128] S3. Combining collaborative control parameters and real-time operating condition information, the fault-tolerant compensation system is switched to laser point cloud matching mode, and the federated Kalman filter algorithm is used to fuse multi-modal data to generate a comprehensive signal.

[0129] It should be added that, such as Figure 4 As shown, the laser point cloud matching mode is a positioning mode based on 3D point cloud data collected by LiDAR. It uses algorithms to align real-time billet point clouds with pre-stored standard model point clouds (or historical valid frame point clouds) through feature alignment and spatial correlation. Its core objective is to accurately identify the 3D coordinates, attitude angles (tilt, rotation), and dimensions of billets in complex industrial environments such as high temperature, dust, and obstruction, providing millimeter-level precision positional references for processes such as gripping, transporting, and processing. This mode is essentially a complete technical process of "data preprocessing - feature extraction - matching calculation - result verification," with each step implemented as follows:

[0130] 1. Data preprocessing: Reduce the amount of data by voxel grid downsampling (e.g., 5mm voxels) (reduce the computational load by 50%-70%), and combine statistical filtering (e.g., setting a threshold of 3 times the standard deviation) to remove noise such as dust reflection points and equipment reflection points, while retaining the effective point cloud on the billet surface;

[0131] 2. Feature extraction:

[0132] Local features: Calculate the point cloud normal vector (with 10-20 neighboring points) and curvature (to distinguish between the steel billet edges and planar areas), and generate a fast point feature histogram (FPFH) for fine feature matching;

[0133] Global features: Extract the billet bounding box (minimum circumscribed cuboid) and centroid coordinates to quickly narrow down the matching search range (e.g., search within a 50mm×50mm×50mm area around the centroid).

[0134] 3. Matching Calculation: A two-step method of "coarse matching + fine matching" is adopted:

[0135] Coarse matching: Based on the initial registration of sample consistency, 300-500 sets of feature point pairs are randomly sampled, and outlier point pairs are removed by RANSAC algorithm to obtain the initial transformation matrix (translation + rotation). The matching error is controlled within 5-10mm.

[0136] Fine matching: The Iterative Closest Point (ICP) algorithm is used, with the distance from the point to the surface as the error function (to improve the matching accuracy of the planar area). The transformation matrix is ​​optimized 50-100 times, and the root mean square error (RMSE) of the final matching is ≤2mm (to meet the requirements of industrial positioning).

[0137] 4. Result verification: If RMSE < 2mm and the matching is stable for 3 consecutive frames (error fluctuation < 0.5mm), output the billet position and shape information; if the error exceeds the standard, trigger rematch (expand the search range or increase the sampling points), and if the retry fails 3 times, start the alarm.

[0138] In this optional embodiment, by combining cooperative control parameters and real-time operating condition information, the fault-tolerant compensation system is switched to laser point cloud matching mode, and the federated Kalman filter algorithm is used to fuse multimodal data to generate a comprehensive signal including:

[0139] S31. Analyze the generated collaborative control parameters and the collected real-time operating information, define the quantitative judgment criteria for main mode failure from multiple dimensions, and determine the trigger thresholds for each dimension in combination with the fault detection trigger conditions.

[0140] In this optional embodiment, the fault detection triggering conditions include sensor status, positioning accuracy, environmental interference, and manual triggering.

[0141] It should be noted that the quantitative criteria for determining the failure of the main mode include:

[0142] Sensor status anomalies: Communication interruption of the main sensor (such as a camera) (TCP connection disconnected), data packet loss rate >30% (more than 3 frames lost in 10 consecutive frames), hardware failure (lens obstruction, abnormal light source voltage); Positioning accuracy is substandard: The positioning error of the main mode output is >3mm for 5 consecutive frames, or the error variance is >1.5mm² (excessive fluctuation, unable to achieve stable positioning); Environmental interference exceeds the standard: Environmental parameters exceed the adaptability range of the main mode, such as strong light (camera exposure value >2000), dust concentration >50mg / m³ (infrared signal attenuation >40%), billet surface temperature >800℃ (exceeding the camera's temperature resistance limit); Manual triggering: The operator issues a switching command through the industrial PC (such as main mode maintenance and debugging scenarios).

[0143] S32. Based on the determined trigger threshold, a finite state machine is used for discrimination, and the mode switching mechanism is driven by the discrimination result to switch the fault-tolerant compensation system from the main mode to the laser point cloud matching mode.

[0144] In this optional embodiment, based on a determined trigger threshold, a finite state machine is used for discrimination, and based on the discrimination result, a mode switching mechanism is driven to switch the fault-tolerant compensation system from the main mode to the laser point cloud matching mode, including:

[0145] S321. Perform redundancy verification on the determined trigger threshold and sort the redundancy verification results according to the preset priority rules.

[0146] S322. Based on the priority sorting result, trigger the corresponding level mode switching command, start the lidar preheating program, and simultaneously send a stop operation command to the main mode to release the corresponding resources.

[0147] S323. Based on the resource release results of the main mode, activate the lidar and start the initialization process, and switch the fault tolerance compensation system to the lidar point cloud matching mode.

[0148] It should be further explained that the logical judgments are implemented by the finite state machine of the fault-tolerant compensation system layer to avoid erroneous switching, including:

[0149] Redundancy verification: The main mode is deemed to have failed only if both "sensor malfunction + inaccuracy" are met simultaneously, or if a single condition persists for more than 200ms. Priority sorting: Hardware failure (such as camera damage) has higher priority than environmental interference and directly triggers the switchover. Preheating mechanism: Environmental parameters (such as dust concentration) are monitored in real time. When the concentration approaches a critical value (such as 45mg / m³), the lidar is activated in advance (preheating for 50-100ms) and the template point cloud is loaded, shortening the switchover delay to <300ms.

[0150] Main mode exit: Send a stop command to the main mode module, clear cached data (such as visual frame data), and release computing threads (such as GPU resources); Laser module activation: Set LiDAR parameters (scanning frequency 30Hz, point cloud density 800 points / m²). 2 1. Scanning angle covers the billet area), initiate laser emission and data acquisition; Template loading and initial positioning: Load the standard point cloud (format PCD) of the current billet specification from an industrial database (such as MySQL), and use the valid coordinates of the last frame of the main mode (such as the billet's center of gravity X=1000mm, Y=500mm, Z=800mm) to set the initial search range (±50mm); Parameter initialization: Configure the point cloud matching parameters (SAC-IA sampling points 500, ICP iterations 80, distance threshold 2mm), and call the PCL point cloud library algorithm interface; Switching verification: Acquire the first frame of laser point cloud to complete the matching. If RMSE < 1.5mm, send a "switching complete" signal, and the laser mode takes over the positioning; If it fails, adjust the parameters (expand the search range to ±80mm) and retry, up to 3 times. If it still fails, trigger an audible and visual alarm.

[0151] S33. In the laser point cloud matching mode, laser point cloud data is acquired, and the laser point cloud data and multimodal data are fused through a preset federated architecture using the Kalman filter algorithm and DS evidence theory to generate a comprehensive signal.

[0152] It should be further explained that, through laser point cloud matching technology, the position and shape of the steel billet can be accurately located in complex environments, and Kalman filtering technology can be used to reduce noise and errors in the point cloud data. The application of DS evidence theory enables this step to comprehensively consider data from multiple sensors, improves the overall decision-making ability of the system, enables optimal decisions to be made in uncertain environments, and improves the overall performance and reliability of the system.

[0153] The federated architecture design includes: a sub-filter layer, with three parallel sub-filters processing laser point clouds (main sensor, weight 0.6), millimeter-wave radar (dust resistant, weight 0.3), and infrared sensor (high temperature resistant, weight 0.1) respectively. Each sub-filter independently executes KF and outputs a locally optimal estimate X. i (t) and error covariance P i (t);

[0154] Main filter layer: Calculates the confidence level of each sensor based on DS evidence theory (higher confidence level indicates smaller laser matching error), and obtains the global optimal estimate through weighted fusion.

[0155] ;

[0156] Information feedback: The main filter feeds back the global estimate to the sub-filters, resetting P. i (t)=P global (t), to avoid the accumulation of sub-filter errors;

[0157] Technical benefits: When a single sensor fails (e.g., the laser is blocked by dust), the system uses radar + infrared fusion positioning, increasing the error from 2mm in single laser mode to 3.5mm (still meeting industrial requirements), improving reliability by more than 60%; at the same time, infrared data is used to correct for steel billet thermal deformation (e.g., high-temperature expansion of 2mm), further improving positioning accuracy by 15%.

[0158] S4. Based on the generated integrated signal, determine whether the steel picking task is completed. If the steel picking task is not completed, return to S1. If the steel picking task is completed, execute S5.

[0159] It should be noted that S4 serves as the loop point for the entire process, ensuring the continuous operation and stability of the system under different conditions. If the steel taking task has not yet been completed, the system will return to S1 to continue waiting for new tasks. If the task has been completed, the system will proceed to the next step for summary and confirmation.

[0160] S5. Determine the target position and target attitude based on the integrated signal, and drive the actuator to perform the steel picking operation through the control unit.

[0161] In this optional embodiment, determining the target position and target attitude based on the integrated signals, and driving the actuator to perform the steel-retrieving operation through the control unit includes:

[0162] S51. Based on the generated integrated signal, calculate the target position and target posture of the steel taking operation. Based on the calculation results, divide the steel taking operation into different motion stages and set corresponding motion parameters for each stage. The motion stages include the standby to pre-grab stage, the pre-grab to grab stage, and the grab to lift stage.

[0163] It should be noted that the steel-picking action is divided into three motion stages, with different motion parameters set for each stage (based on the Jerk-limited trajectory planning algorithm to avoid impact), as shown in Table 9.

[0164] Table 9: Movement Stages Table

[0165] S52. Based on the motion parameters set at each stage, the actuator is driven by the control unit to perform motion, and the motion process is dynamically corrected by combining real-time acquired multimodal data to ensure motion accuracy.

[0166] It should be added that, such as Figure 5 As shown, the steel-retrieving position and attitude are determined based on the comprehensive signal, driving the steel-retrieving mechanism to move, and key parameters are monitored in real time to ensure quality. This step is the execution stage of the entire process, in which the processing and analysis of the comprehensive signal is crucial. Through real-time processing and analysis of multi-modal signals, the system can accurately control the movement trajectory and attitude of the steel-retrieving mechanism, thereby achieving high-precision and high-quality steel-retrieving operations. The real-time monitoring mechanism ensures the safety and controllability of the entire process, preventing potential risks and failures.

[0167] Furthermore, a smart billet extraction method based on multimodal signals constructs a multimodal perception-decision-execution closed-loop system, which includes the following key modules:

[0168] (1) Multimodal signal acquisition layer: Equipped with a high-definition camera (CCD), infrared thermal imager, laser rangefinder (accuracy ±1mm) and inkjet printing recognition system, it captures the three-dimensional shape, surface temperature distribution and material identification information of the billet in real time; through the collaborative work of these multiple sensors, it can collect data comprehensively in complex environments, ensuring the comprehensiveness and accuracy of information, and providing a solid foundation for subsequent decision-making and control.

[0169] (2) Dynamic production cycle optimization module: Establish a database of correspondence between billet shape, material and process parameters, and use fuzzy PID algorithm to calculate the optimal steel taking interval time T=ƒ(bill weight, target temperature, rolling speed) in real time; This module monitors and adjusts various parameters in the production process in real time to ensure the efficiency and stability of the production process and reduce downtime and failures caused by improper parameters.

[0170] (3) Collaborative control system: realizes the synchronous operation of unmanned cranes and steel picking equipment; through control algorithms and communication technology, ensures that each piece of equipment can work collaboratively in the production process, improves production efficiency and product quality, and reduces production bottlenecks caused by poor collaboration.

[0171] (4) Fault-tolerant compensation device: When the visual signal is interrupted, it switches to the laser point cloud matching mode and uses Kalman filtering technology to eliminate vibration interference; it can automatically switch modes under various emergencies to ensure the continuity and stability of production and reduce downtime and material waste caused by signal interruption.

[0172] In a specific embodiment, a smart billet steel removal method based on multimodal signals includes:

[0173] Example 1:

[0174] 1. Multimodal signal acquisition layer acquires billet information:

[0175] (1) Configure a vision sensor (industrial high-definition lens), a laser rangefinder (VelodyneLiDAR), and a coding recognition system to capture the three-dimensional outer contour, surface temperature distribution, and material identification information of the billet in real time;

[0176] (2) The distance to the surface of the billet is measured in real time using a laser rangefinder, and a three-dimensional point cloud model of the billet is constructed by combining the image data of the vision sensor.

[0177] (3) Use an infrared thermal imaging sensor (model FLIRE75) to monitor the surface temperature change of the billet in real time, and read the material mark of the billet through the inkjet printing system.

[0178] 2. The dynamic production rhythm modeling layer models the production rhythm:

[0179] (1) Construct a database of correspondence between billet type, material and process parameters, and record the steel performance indicators of various billet types (square billet, slab billet, special-shaped billet), materials (Q345B, Q420C, SM490YA) and process parameters (heating temperature range of 1150-1250℃);

[0180] (2) Based on the fuzzy PID algorithm, analyze the current production rhythm data and calculate the target steel taking interval time T, where T ranges from 30 to 60 seconds;

[0181] (3) Use an LSTM neural network (with 128 hidden nodes and a time step of 5 seconds) to perform deep learning training on the production rhythm data to optimize the steel taking time.

[0182] 3. Collaborative control mechanism layer performs collaborative control:

[0183] (1) Construct a collaborative control model for the unmanned vehicle (model MobileRobotsPB-1500) and the steel taking equipment (model KR16) to realize the collaborative operation of the two machines;

[0184] (2) The motion parameters of the steel taking equipment are adjusted by an adaptive fuzzy control algorithm to ensure precise coordination with the unmanned vehicle. The maximum moving speed of the steel taking equipment is 0.5 m / s and the maximum angular velocity is 30° / s.

[0185] 4. Fault tolerance compensation is performed at the fault tolerance compensation system layer:

[0186] (1) When the visual signal is interrupted, switch to laser point cloud matching mode;

[0187] (2) The Kalman filter algorithm is used to process the laser point cloud data to eliminate vibration interference, where the filter gain is 0.1;

[0188] (3) Decision-level fusion of multi-sensor data is performed through DS evidence theory to generate a comprehensive signal.

[0189] 5. Perform the steel removal operation:

[0190] (1) Determine the steel extraction position and attitude based on the integrated signal;

[0191] (2) The steel-grabbing mechanism is driven to move by the actuator control system, wherein the maximum grasping force of the steel-grabbing mechanism is 2000 N;

[0192] (3) Monitor key parameters in the steel taking process in real time to ensure the quality of steel taking, including grabbing time, displacement distance and load weight.

[0193] Example 2:

[0194] 1. Multimodal signal acquisition layer acquires billet information:

[0195] (1) Configure a vision sensor (model BasleracA2040-120um), a laser rangefinder (model RieglVQ-880), and a coding recognition system to capture the three-dimensional outer contour, surface temperature distribution, and material identification information of the billet in real time;

[0196] (2) The distance to the surface of the billet is measured in real time using a laser rangefinder, and a three-dimensional point cloud model of the billet is constructed by combining the image data of the vision sensor.

[0197] (3) Use an infrared thermal imaging sensor (model FLIRTAU640) to monitor the surface temperature change of the billet in real time, and read the material identification of the billet through a coding recognition system.

[0198] 2. The dynamic production rhythm modeling layer models the production rhythm:

[0199] (1) Construct a database of correspondence between billet type, material and process parameters, and record the steelmaking performance indicators of various billet types (tube billet, bar), materials (20MnSi, 40Mn2), and process parameters (heating temperature range of 1200-1300℃);

[0200] (2) Based on the fuzzy PID algorithm, analyze the current production rhythm data and calculate the target steel taking interval time T, where T ranges from 40 to 70 seconds;

[0201] (3) Use an LSTM neural network (with 256 hidden layer nodes and a time step of 10 seconds) to perform deep learning training on the production rhythm data to optimize the steel taking sequence.

[0202] 3. Collaborative control mechanism layer performs collaborative control:

[0203] (1) Construct a collaborative control model for the unmanned crane (model KUKAOmniMate) and the steel taking equipment (model FANUCM-710iC) to realize the collaborative operation of the two machines;

[0204] (2) The motion parameters of the steel taking equipment are adjusted by an adaptive fuzzy control algorithm to ensure precise coordination with the unmanned vehicle. The maximum moving speed of the steel taking equipment is 0.8 m / s and the maximum angular velocity is 45° / s.

[0205] 4. Fault tolerance compensation is performed at the fault tolerance compensation system layer:

[0206] (1) When the visual signal is interrupted, switch to laser point cloud matching mode;

[0207] (2) The Kalman filter algorithm is used to process the laser point cloud data to eliminate vibration interference, where the filter gain is 0.05;

[0208] (3) Decision-level fusion of multi-sensor data is performed through DS evidence theory to generate a comprehensive signal.

[0209] 5. Perform the steel removal operation:

[0210] (1) Determine the steel extraction position and attitude based on the integrated signal;

[0211] (2) The steel-retrieving mechanism is driven to move by the actuator control system, wherein the maximum grasping force of the steel-retrieving mechanism is 3000 N;

[0212] (3) Monitor key parameters in the steel taking process in real time to ensure the quality of steel taking, including grabbing time, displacement distance and load weight.

[0213] Through the above embodiments, the steel picking positioning accuracy of the present invention reaches ±2mm (40 times higher than the traditional method), which significantly improves the accuracy and reliability of the production process and reduces failures and losses caused by inaccurate positioning; the material switching and feeding time of the production line is reduced to 2.8 minutes (the industry average is 6 minutes), which greatly shortens the production cycle, improves production efficiency, and reduces production costs; the equipment idle energy consumption is reduced by 18.7%. By optimizing the production cycle and equipment operating status, energy consumption is significantly reduced, which meets the requirements of modern industry for green production and reduces the impact on the environment.

[0214] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A smart billet extraction method based on multimodal signals, characterized in that, The method includes: S1. Based on the collected multimodal data of steel billets, extract the production cycle data, and calculate and optimize the execution sequence of steel taking operations by analyzing the production cycle data; S2. Based on the optimized execution sequence, establish a collaborative control model for the unmanned vehicle and the steel-retrieving equipment, and adjust the motion parameters of the steel-retrieving equipment through a collaborative control algorithm to generate collaborative control parameters; S3. Combining collaborative control parameters and real-time operating condition information, the fault-tolerant compensation system is switched to laser point cloud matching mode, and the federated Kalman filter algorithm is used to fuse multi-modal data to generate a comprehensive signal. S4. Based on the generated integrated signal, determine whether the steel picking task is completed. If the steel picking task is not completed, return to S1. If the steel picking task is completed, execute S5. S5. Determine the target position and target attitude based on the integrated signal, and drive the actuator to perform the steel picking operation through the control unit.

2. The intelligent billet extraction method based on multimodal signals according to claim 1, characterized in that, The process of extracting production cycle time data based on the collected multimodal data of steel billets, and calculating and optimizing the execution timing of steel taking operations by analyzing the production cycle time data includes: S11. Collect multimodal data of steel billet, preprocess the collected multimodal data, and construct a three-dimensional point cloud model of steel billet based on the preprocessing results; S12. Combining the three-dimensional point cloud model with multimodal data, a mapping database is constructed, and various billet shapes, materials and process parameters in the mapping database are extracted as steel performance indicators. S13. Based on the extracted steel extraction performance indicators, the execution sequence of the steel extraction operation is calculated using the fuzzy PID algorithm, and the execution sequence is optimized through a preset long short-term memory network.

3. The intelligent billet extraction method based on multimodal signals according to claim 2, characterized in that, The process of acquiring multimodal data of steel billets, preprocessing the acquired multimodal data, and constructing a three-dimensional point cloud model of the steel billets based on the preprocessing results includes: S111. Through the coordinated operation of a vision sensor, a laser rangefinder, and a coding recognition system, multimodal data of the steel billet is collected synchronously. The multimodal data includes three-dimensional contour data, surface temperature data, and material coding information. S112. Use statistical filtering algorithms to denoise the collected multimodal data and perform multi-source data registration on the denoising results. S113. The registration results of multi-source data are fused with geometric structure and surface texture, and combined with material coding information annotation, to construct a three-dimensional point cloud model containing geometric information, attribute information and accuracy information.

4. The intelligent billet extraction method based on multimodal signals according to claim 3, characterized in that, The process of calculating the execution sequence of steel extraction operations based on extracted steel extraction performance indicators using a fuzzy PID algorithm, and optimizing the execution sequence through a pre-set long short-term memory network, includes: S131. Based on the extracted steel extraction performance indicators, obtain production cycle data, analyze the production cycle data using a fuzzy PID algorithm, and calculate the execution sequence of the steel extraction operation based on the analysis results. S132. Input the acquired production cycle data into a preset long short-term memory network, and output the cycle prediction results for future periods through temporal feature learning. S133. Based on the beat prediction results, the execution timing is optimized using a distributed model predictive control algorithm to obtain the optimized execution timing.

5. The intelligent billet extraction method based on multimodal signals according to claim 1, characterized in that, Based on the optimized execution sequence, a collaborative control model for the unmanned vehicle and the steel-retrieving equipment is established. The motion parameters of the steel-retrieving equipment are adjusted using a collaborative control algorithm, generating collaborative control parameters including: S21. Analyze the optimized steel-retrieving operation execution sequence, identify the functional modules and corresponding interaction nodes of the unmanned vehicle and steel-retrieving equipment, generate a scene matrix, and define collaborative constraints. S22. Based on the kinematic model of the unmanned vehicle and the steel-retrieving equipment, and combined with the cooperative constraints, construct a cooperative control model based on a distributed architecture. S23. Based on the constructed collaborative control model, set multi-dimensional control objectives, and adjust the motion parameters of the steel taking equipment through an adaptive sliding mode control algorithm to generate collaborative control parameters.

6. The intelligent billet extraction method based on multimodal signals according to claim 5, characterized in that, The constructed collaborative control model sets multi-dimensional control objectives and adjusts the motion parameters of the steel-retrieving equipment through an adaptive sliding mode control algorithm to generate collaborative control parameters, including: S231. The motion parameters, including the clamping force of the steel taking equipment, the movement speed of the telescopic arm, and the rotation attitude angle, are taken as multi-dimensional control targets. The deviation between the actual working condition information and the target state is taken as the input variable, and the parameter adjustment amount is taken as the output variable. A membership function based on fuzzy inference algorithm is constructed. S232. Collect on-site operation experience and simulation data, combine them with the constructed membership function, establish a fuzzy control rule base that matches the multidimensional control objective, and use the max-min inference algorithm to perform inference operations on the input variables to generate the corresponding fuzzy output set. S233. Use the centroid algorithm to defuzzify the fuzzy output set, generate control variables, and combine the adaptive adjustment mechanism to dynamically adjust the output variables of the fuzzy control rule base. S234. Based on the dynamically adjusted output variables, adjust the motion parameters of the steel taking equipment to generate coordinated control parameters.

7. The intelligent billet extraction method based on multimodal signals according to claim 1, characterized in that, The process involves combining collaborative control parameters with real-time operating information, switching the fault-tolerant compensation system to laser point cloud matching mode, and using a federated Kalman filter algorithm to fuse multimodal data to generate a comprehensive signal, including: S31. Analyze the generated collaborative control parameters and the collected real-time operating information, define the quantitative judgment criteria for main mode failure from multiple dimensions, and determine the trigger thresholds for each dimension in combination with the fault detection trigger conditions. S32. Based on the determined trigger threshold, a finite state machine is used for discrimination, and the mode switching mechanism is driven based on the discrimination result to switch the fault-tolerant compensation system from the main mode to the laser point cloud matching mode. S33. In the laser point cloud matching mode, laser point cloud data is acquired, and the laser point cloud data and multimodal data are fused through a preset federated architecture using the Kalman filter algorithm and DS evidence theory to generate a comprehensive signal.

8. The intelligent billet extraction method based on multimodal signals according to claim 7, characterized in that, The fault detection triggering conditions include sensor status, positioning accuracy, environmental interference, and manual triggering.

9. The intelligent billet extraction method based on multimodal signals according to claim 8, characterized in that, The process of using a finite state machine to make a judgment based on a determined trigger threshold, and then using the judgment result to drive a mode switching mechanism to switch the fault-tolerant compensation system from the main mode to the laser point cloud matching mode includes: S321. Perform redundancy verification on the determined trigger threshold and sort the redundancy verification results according to the preset priority rules. S322. Based on the priority sorting result, trigger the corresponding level mode switching command, start the lidar preheating program, and simultaneously send a stop operation command to the main mode to release the corresponding resources. S323. Based on the resource release results of the main mode, activate the lidar and start the initialization process, and switch the fault tolerance compensation system to the lidar point cloud matching mode.

10. The intelligent billet extraction method based on multimodal signals according to claim 1, characterized in that, The step of determining the target position and target attitude based on the integrated signal, and driving the actuator to perform the steel-picking operation through the control unit includes: S51. Based on the generated integrated signal, calculate the target position and target posture of the steel taking operation. Based on the calculation results, divide the steel taking operation into different motion stages and set corresponding motion parameters for each stage. The motion stages include the standby to pre-grab stage, the pre-grab to grab stage, and the grab to lift stage. S52. Based on the motion parameters set at each stage, the actuator is driven by the control unit to perform motion, and the motion process is dynamically corrected by combining real-time acquired multimodal data to ensure motion accuracy.

Images