Accurate stacking control system based on artificial intelligence technology

Abstract

The invention discloses a precise stacking control system based on an artificial intelligence technology, and relates to the technical field of robot automation control. Comprising a data fusion module used for preprocessing sensor data and fusing the preprocessed sensor data to obtain a fused intention label; the trajectory optimization module is used for performing parameter adjustment on cost function parameters according to the intention label, constructing a space-time constraint model, and acquiring a feasible trajectory and a corresponding risk level according to the space-time constraint model; the control response module is used for adjusting the feasible track according to the risk level to obtain the adjusted feasible track; and the verification and calibration module is used for acquiring the predictive control deviation of each adjusted feasible track through real-time simulation, and adjusting the constructed sensor network according to the predictive control deviation and the real-time braking error. The high-precision and self-adaptive man-machine collaborative stacking operation can be realized.

Description

Technical Field

[0001] The present invention relates to the field of robot automation control technology, and in particular to a precise palletizing control system based on artificial intelligence technology. Background Art

[0002] Palletizing, a critical link in logistics, warehousing, and manufacturing processes, is widely used in industries such as food, beverages, chemicals, and building materials. Traditional palletizing methods rely primarily on manual operation or automated equipment controlled by fixed programs. With the development of industry and intelligent manufacturing, automated palletizing systems are becoming mainstream. However, they still face numerous challenges in dealing with complex and changing working conditions, diverse product types, and optimizing stacking paths.

[0003] Early automated palletizing control systems often combined PLCs (Programmable Logic Controllers) with robotic motion control, implementing basic palletizing functions through pre-set paths and rules. However, these systems were limited in their ability to handle unstructured environments, identify dynamic objects, and analyze stacking stability, making them difficult to adapt to the demands of high-speed, high-flexibility, and high-precision production.

[0004] A Chinese invention patent, publication number CN119704206A, discloses a robot stability and balance control method based on an embedded deep fusion algorithm. The method includes the following steps: S1, sensor data acquisition: configuring multiple sensors to collect the robot's motion state and environmental information in real time; S2, dynamic scene understanding: real-time analysis of the collected sensor data to identify potential obstacles and changes in the environment; S3, collaborative control based on behavior prediction: predicting the future behavior of multiple robots and dynamically adjusting the collaborative strategy; S4, real-time feedback adjustment: using a multi-level feedback mechanism to compare the robot's current state with a preset stable state, and intelligently adjusting the control strategy based on real-time environmental changes and task requirements. This invention improves the system's adaptability and task execution efficiency, ensuring more intelligent and flexible collaborative work in complex scenarios.

[0005] However, during human-machine collaborative operations, when humans urgently intervene, there will be a real-time conflict between the dynamic path replanning results autonomously generated by the intelligent palletizing system and the manually input operation instructions, resulting in delayed equipment response and even malfunctions, which in turn seriously disrupt the stability of the production line rhythm. Furthermore, in close-range human-machine co-domain operations, due to the asynchronous fusion and insufficient feature coupling of multimodal perception data, the position prediction error of existing human motion prediction algorithms will increase in scenarios such as dynamic occlusion and rapid body movement, thereby increasing safety protection response delays and even threatening the safety of human-machine collaboration and the continuity of operations. Summary of the Invention

[0006] The purpose of the present invention is to provide a precise palletizing control system based on artificial intelligence technology to solve the problems raised in the above background technology.

[0007] To achieve the above objectives, the present invention provides the following technical solutions: a precise palletizing control system based on artificial intelligence technology, comprising:

[0008] A data fusion module acquires sensor data through the constructed sensor network, preprocesses the sensor data, and fuses the preprocessed sensor data to obtain a fused intent label;

[0009] The trajectory optimization module adjusts the cost function parameters according to the intention label, and constructs a spatiotemporal constraint model based on the adjusted cost function parameters. At the same time, based on the spatiotemporal constraint model, it obtains feasible trajectories and corresponding risk levels, including:

[0010] SB1: Determine the cost function: Based on the intent label and the constructed weight rule table, determine the cost function, specifically:

[0011]

[0012] in: is the comprehensive evaluation index of the trajectory, is the path length weight, is the total length of the trajectory, is the energy consumption weight, is the cumulative consumption of joint torque, is the smoothness weight, is the cumulative change of acceleration;

[0013] SB2: Obtaining a feasible trajectory: Based on the cost function, the working environment, and the state of the palletizing robot's manipulator, a corresponding optimization problem is constructed, and based on the optimization problem, an optimized trajectory is obtained, specifically:

[0014]

[0015] in: is the angle function of the j-th joint, is the coefficient of the nth term order corresponding to the jth joint, is the time variable of the nth order, is the index of the joint, is the polynomial order, is the total order of the polynomial;

[0016] SB3: Determine risk: Obtain a risk assessment coefficient of the optimized trajectory according to the optimized trajectory, and determine a risk level of the optimized trajectory according to the risk assessment coefficient;

[0017] A control response module adjusts the feasible trajectory according to the risk level to obtain an adjusted feasible trajectory;

[0018] The verification and calibration module obtains the predicted control deviation of each adjusted feasible trajectory through real-time simulation, and adjusts the constructed sensor network according to the predicted control deviation and the real-time braking error.

[0019] Furthermore, the fused intent labels are obtained, including:

[0020] SA1: Build a sensor network: Acquire asynchronous event stream data through event-driven cameras, obtain low-frequency coordinate data through UWB mapping base stations and UWB tags, and obtain six-dimensional force / torque and pressure thermal maps through distributed fiber optic force sensor arrays;

[0021] SA2: Data preprocessing: Using the constructed adaptive Kalman filter model, the asynchronous event stream data and the low-frequency coordinate data are subjected to spatiotemporal alignment processing to obtain preprocessed sensor data;

[0022] SA3: Obtain fusion intent label: Determine the fusion intent label through the constructed knowledge graph and preprocessed sensor data.

[0023] Furthermore, the pre-processed sensor data is obtained, including:

[0024] SA2.1: Noise Modeling: Initialize the process noise covariance matrix through static calibration experiments, UWB mapping base stations and UWB tags. Specifically:

[0025]

[0026] in: is the process noise covariance matrix, is the noise level in the X direction during UWB positioning. is the noise level in the Y direction during UWB positioning. The noise level in the Z direction during UWB positioning.

[0027] The illumination of the event-driven camera is adjusted and tested to obtain the time trigger rate, and the observation noise covariance matrix is ​​initialized according to the time trigger rate, specifically:

[0028]

[0029] in: is the observation noise covariance matrix in a dark light environment, is the observation noise covariance matrix under normal lighting conditions, is the noise amplification factor, is the standard deviation of the observation noise of a single pixel;

[0030] SA2.2: Matrix Adjustment: Adjust the process noise covariance matrix and the observation noise covariance matrix based on the received signal strength corresponding to the low-frequency coordinate data and the real-time acceleration corresponding to the asynchronous event stream data, as follows:

[0031] The received signal strength within a preset continuous time is compared with a preset signal strength threshold, and according to the comparison result, the process noise covariance matrix and the observation noise covariance matrix are adjusted, specifically:

[0032] When the received signal strength within a preset continuous time is less than a preset signal strength threshold, the process noise covariance matrix is ​​increased and the observation noise covariance matrix is ​​decreased; otherwise, the process noise covariance matrix and the observation noise covariance matrix remain unchanged;

[0033] The real-time acceleration is compared with a preset acceleration threshold, and the process noise covariance matrix and the observation noise covariance matrix are adjusted according to the comparison result, specifically:

[0034] When the real-time acceleration is greater than a preset acceleration threshold, the process noise covariance matrix is ​​increased. At the same time, when the environment of the event-driven camera is a dark environment, the observation noise covariance matrix is ​​increased. When the environment of the event-driven camera is a normal lighting environment, the observation noise covariance matrix remains unchanged; otherwise, the process noise covariance matrix and the observation noise covariance matrix remain unchanged.

[0035] SA2.3: Spatiotemporal alignment: Construct an adaptive Kalman filter model based on the adjusted process noise covariance matrix and the observation noise covariance matrix, and perform spatiotemporal processing on the asynchronous event stream data and low-frequency coordinate data based on the adaptive Kalman filter model.

[0036] Furthermore, the asynchronous event stream data and the low-frequency coordinate data are subjected to spatiotemporal processing, including:

[0037] SA2.3.1: Time alignment: Based on the PTP timestamp in the low-frequency coordinate data and the timestamp in the asynchronous event stream data, obtain the time alignment error between the low-frequency coordinate data and the asynchronous event stream data, and determine the object position offset based on the time alignment error. At the same time, based on the coordinates in the asynchronous event stream data with the minimum time alignment error and the object position offset, correct the coordinates in the asynchronous event stream data, specifically:

[0038]

[0039] in: is the corrected X-axis coordinate in the asynchronous event stream data. is the corrected Y-axis coordinate in the asynchronous event stream data. is the original X-axis coordinate in the asynchronous event stream data, is the original Y-axis coordinate in the asynchronous event stream data, is the motion direction angle, is the object position offset caused by time alignment error;

[0040] SA2.3.2: Spatial Alignment: Use the constructed transformation matrix to transform the coordinates in each sensor coordinate system into the coordinates in the pallet center coordinate system. Specifically:

[0041]

[0042] in: is the coordinate of the object in the coordinate system of the center of the pallet, is the transformation matrix from the sensor to the pallet center coordinate system, The original measured coordinates of the object in the sensor.

[0043] Furthermore, the transformation matrix is ​​determined by the least squares method, including:

[0044] SA2.3.2.1: Through calibration experiments, obtain the average coordinates of the fixed-point object in the pallet center coordinate system and the sensor. Specifically:

[0045]

[0046] in: is the average coordinate of the fixed-point object in the coordinate system of the center of the pallet, is the average coordinate of the original measurement of the fixed-point object in the sensor, is the number of calibration poses, is the real coordinate of the calibration plate in the pallet center coordinate system at the i-th pose, is the coordinate of the calibration plate measured in the sensor at the i-th pose, is the pose index;

[0047] SA2.3.2.2: Determine the correlation between the pallet center coordinate system and the sensor coordinates based on the average coordinates of the fixed-point object in the pallet center coordinate system and the sensor coordinates, specifically:

[0048]

[0049] in: is the covariance matrix, is the average coordinate of the fixed-point object in the coordinate system of the center of the pallet, is the average coordinate of the original measurement of the fixed-point object in the sensor, is the number of calibration poses, is the real coordinate of the calibration plate in the pallet center coordinate system at the i-th pose, is the coordinate of the calibration plate measured in the sensor at the i-th pose, is the pose index, is an inverted symbol;

[0050] SA2.3.2.3: Construct a rotation matrix based on the covariance matrix and the result matrix of the SVD decomposition, specifically:

[0051]

[0052] in: is the covariance matrix, is the inverted symbol, is the rotation matrix, 、 is the orthogonal matrix of the SVD decomposition result, is the singular value diagonal matrix of the SVD decomposition result;

[0053] SA2.3.2.4: Determine the translation vector based on the rotation matrix and the average coordinates of the fixed-point object in the pallet center coordinate system and the sensor. Specifically,

[0054]

[0055] in: is the translation vector, is the average coordinate of the fixed-point object in the coordinate system of the center of the pallet, is the average coordinate of the original measurement of the fixed-point object in the sensor, is the rotation matrix;

[0056] SA2.3.2.5: Based on the rotation matrix and translation vector, obtain the transformation matrix from the sensor to the pallet center coordinate system, specifically:

[0057]

[0058] in: is the transformation matrix from the sensor to the pallet center coordinate system, is the translation vector, is the rotation matrix.

[0059] Furthermore, the fusion intent labels are determined, including:

[0060] SA3.1: Data Encoding Processing: Construct a spatial position encoding vector based on the asynchronous event stream data and the set voxel grid. Construct a motion feature vector based on the continuous frame coordinates, instantaneous velocity, and instantaneous acceleration in the low-frequency coordinate data. Obtain the steady-state component and fluctuation intensity from the force data window in the six-dimensional force / torque and pressure thermal map to construct a force feature vector.

[0061] SA3.2: Construct joint features: Based on the spatial position encoding vector, motion feature vector, and force feature vector, and using the embedding vector in the knowledge graph, a joint feature vector is constructed. Specifically,

[0062]

[0063] in: is the joint eigenvector, is the spatial position encoding vector, is the motion feature vector, is the force perception eigenvector, Embed vectors for knowledge graphs;

[0064] SA3.3: Determine intent label: Obtain intent label through Transformer encoder and the joint feature vector, specifically:

[0065]

[0066] in: is the intent label, is the normalized exponential function, is the output weight matrix, is the Transformer encoder function, is the joint feature direction.

[0067] Furthermore, the risk level of the optimized trajectory is determined, including:

[0068] SB3.1: Obtaining a risk assessment coefficient: Obtain a risk assessment coefficient for each optimized trajectory based on the closest distance between the optimized trajectory and the nearest obstacle, the actual energy consumption during the optimized trajectory operation, and the reference energy consumption, specifically:

[0069]

[0070] in: To optimize the collision probability between the trajectory and the nearest obstacle, is the sensitivity coefficient, To optimize the minimum safe distance between the trajectory and the nearest obstacle, is the comprehensive risk value, is the collision risk weight, is the energy consumption risk weight, In order to optimize the actual energy consumption during trajectory execution, is the benchmark energy consumption value, is the deviation risk weight, is the difference between the actual trajectory and the reference trajectory;

[0071] SB3.2: Risk level classification: Compare the risk assessment coefficient with the preset assessment threshold range and determine the risk level based on the comparison results, specifically:

[0072] When the risk assessment coefficient is less than the lower limit threshold of the preset assessment threshold, the risk level is low risk; when the risk assessment coefficient is within the preset assessment threshold range, the risk level is medium risk; otherwise, the risk level is high risk.

[0073] Furthermore, the adjusted feasible trajectory is obtained, including:

[0074] SC1: Determine the response strategy: Based on the risk level, determine the hierarchical processing operations, specifically:

[0075] When the risk level is low, the PID controller is used to adjust the joint speed of the palletizing robot; when the risk level is medium, the current operating state is maintained; when the risk level is high, the MPC controller is used to obtain the predicted collision time and adjust the joint acceleration curve of the palletizing robot;

[0076] SC2: Determine the adjustment torque: The joint speed of the palletizing robot is adjusted by the PID controller to obtain the adjustment torque, specifically:

[0077]

[0078] in: is the output torque of the j-th joint, is the proportional gain coefficient, is the differential gain coefficient, is the integral gain coefficient, is the target angle of the j-th joint, is the actual angle of the j-th joint, is the target angular velocity of the j-th joint, is the actual angular velocity of the jth joint, is the index of the joint;

[0079] SC3: Braking acceleration curve adjustment: The response speed of the braking acceleration curve is obtained through the predicted collision time of the MPC controller, and the braking acceleration curve is adjusted according to the response speed, specifically:

[0080]

[0081] in: is the braking acceleration that varies with time, is the maximum permissible braking acceleration, is the base of natural logarithms, is the time after braking starts, is the time constant, To predict the collision time.

[0082] Furthermore, adjustments are made to the constructed sensor network, including:

[0083] SD1: Determine the prediction deviation: Based on the real-time position of the adjusted feasible trajectory and the predicted position in the real-time simulation, obtain the prediction deviation, specifically:

[0084]

[0085] in: is the forecast deviation at time t, is the smoothing factor, is the actual measurement deviation at time t, is the prediction deviation at time t-1;

[0086] SD2: Determine a response signal: Based on the prediction deviation and the preset time length, compare the prediction deviation within the preset time with the preset deviation threshold, and determine a response signal based on the comparison result, specifically:

[0087] When the prediction deviations within the preset time are all greater than the preset deviation threshold, an adjustment signal is triggered and the next step SD3 is executed; otherwise, no adjustment signal is triggered;

[0088] SD3: Base station adjustment: The original position of the UWB base station is determined by the Transformer encoder to determine the adjustment displacement of the UWB base station, specifically:

[0089]

[0090] in: Function to calculate tag coordinates for UWB base station location, is the total number of calibration measurements for the encoder, is the index of the encoder’s calibration measurement times, is the initial position of the UWB base station measured for the mth time, The displacement that needs to be adjusted for the UWB base station, is the inverse position of the mth encoder.

[0091] Compared with the prior art, the present invention has the following beneficial effects:

[0092] First, the present invention dynamically adjusts the noise covariance matrix through adaptive Kalman filtering and combines it with a spatiotemporal alignment algorithm to effectively solve the positioning deviation problem caused by sensor data asynchrony in traditional systems under dynamic occlusion and rapid motion scenarios, thereby reducing prediction errors in human-machine collaboration.

[0093] Second, the present invention uses collision probability and energy consumption deviation to obtain the corresponding risk level in real time and adopts a corresponding hierarchical control strategy. That is, when the risk is low, the PID controller fine-tunes the joint speed, and when the risk is high, the MPC controller generates a braking curve, thereby shortening the emergency braking response time and avoiding interruptions to the production line cycle.

[0094] Third, the present invention generates intent labels through a Transformer encoder embedded in a knowledge graph and dynamically adjusts the cost function weights, so that the generated polynomial trajectory can not only meet efficiency requirements but also adapt to sudden changes in working conditions, thereby improving the overall stacking stability of the system.

[0095] Fourthly, the present invention uses real-time simulation to obtain the predicted deviation between the real-time position and the predicted position, and reversely optimizes the layout of the UWB base station, thereby solving the cumulative error problem caused by sensor drift in traditional systems and improving the positioning accuracy of long-term operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 This is a schematic diagram of the system flow of the precise palletizing control system of the present invention;

[0097] Figure 2 This is the effect diagram after data alignment optimization in the present invention;

[0098] Figure 3 This is a diagram showing the optimization effect of coordinate conversion error under the pallet coordinates of the present invention;

[0099] Figure 4 is a distribution diagram of risk levels in the present invention;

[0100] Figure 5 This is a comparison chart of the emergency braking performance in the present invention;

[0101] Figure 6 This is a comparison diagram of the position adjustment of the UWB base station in the present invention. DETAILED DESCRIPTION

[0102] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0103] During human-machine collaborative operations, if human intervention is urgent, there will be a real-time conflict between the dynamic path replanning results autonomously generated by the intelligent palletizing system and the manually input operation instructions. This will cause equipment response delays and even malfunctions, which will seriously disrupt the stability of the production line cycle. Furthermore, in close-range human-machine co-domain operations, due to the asynchronous fusion and insufficient feature coupling of multimodal sensor data, the position prediction error of existing human motion prediction algorithms will increase in scenarios such as dynamic occlusion and rapid limb movement. This will increase safety protection response delays and even threaten the safety of human-machine collaboration and the continuity of operations. This technical solution, however, uses a multimodal sensor network to collect environmental data in real time. After performing spatiotemporal alignment and noise suppression using an adaptive Kalman filter, the data is fused to generate corresponding intent labels. This intent label is then used to dynamically adjust the cost function for trajectory optimization, and the corresponding feasible trajectory is fitted to assess the risk level. The multimodal sensor network is calibrated based on the risk level response and real-time simulation feedback, enabling high-precision, adaptive human-machine collaborative palletizing operations.

[0104] Example 1

[0105] refer to Figure 1This embodiment provides a precise palletizing control system based on artificial intelligence technology, which includes a data fusion module, a trajectory optimization module, a control response module, and a verification and calibration module. Specifically, the data fusion module is used to preprocess the sensor data obtained by the sensor network, and fuse the preprocessed sensor data to obtain the fused intention label. The trajectory optimization module is used to adjust the cost function parameters in the spatiotemporal constraint model according to the fused intention label, and obtain multiple feasible trajectories and the risk level corresponding to each feasible trajectory according to the spatiotemporal constraint model after parameter adjustment. The control response module is used to grade the corresponding feasible trajectory according to the risk level corresponding to each feasible trajectory and execute the corresponding trajectory action. The verification and calibration module is used to obtain the predicted control deviation corresponding to each feasible trajectory through real-time simulation and multiple feasible trajectories, and adjust the sensor network according to the predicted control deviation and the real-time braking error.

[0106] In this embodiment, the data fusion module performs corresponding data preprocessing based on the sensor data obtained from the constructed sensor network, and fuses the preprocessed sensor data to obtain a fused intent label. The details are as follows:

[0107] Step SA1: Build a sensor network. This involves installing event-driven cameras on the top and sides of the palletizing robot's arm workspace to capture asynchronous pixel brightness change data. This data is then used to construct an asynchronous event stream using the corresponding timestamps, pixel coordinates, and polarity.

[0108] Furthermore, multiple UWB mapping base stations are evenly distributed throughout the palletizing workspace to cover the entire area, forming a multilateral positioning network. UWB tags are also placed on the palletizing robot's end effector (e.g., gripper or suction cup) and the objects to be palletized. Specifically, through signal transmission between the UWB tag and the UWB mapping base station, the corresponding signal transmission time series is acquired. This is achieved by constructing low-frequency coordinate data with a PTP timestamp using the local timestamp of the corresponding UWB tag and the time of arrival at the corresponding mapping point.

[0109] Furthermore, a distributed fiber optic force sensor array is set inside the end effector of the palletizing robot's robotic arm. Through the distributed fiber optic force sensor array, the pressure distribution of the palletizing robot during grasping is monitored, and object sliding (such as sudden change in pressure gradient) or deformation (such as pressure asymmetry) is identified, and the corresponding six-dimensional force / torque and pressure thermal map is obtained.

[0110] Step SA2: Data preprocessing. This involves using the constructed adaptive Kalman filter model to perform spatiotemporal alignment on the asynchronous event stream data and low-frequency coordinate data obtained in step SA1 to eliminate noise between sensors such as event-driven cameras, UWB mapping base stations, and UWB tags, and obtain preprocessed sensor data. The details are as follows:

[0111] Step SA2.1: Noise modeling. This involves determining the noise level during the UWB positioning process through static calibration experiments and the UWB tags and UWB mapping base stations set up in step SA1. Based on the noise level during the UWB positioning process, the process noise covariance matrix is ​​initialized. Specifically,

[0112]

[0113] in: is the process noise covariance matrix, is the noise level in the X direction during UWB positioning. is the noise level in the Y direction during UWB positioning. It is the process noise magnitude in the Z direction during the UWB positioning process.

[0114] Furthermore, the event-driven camera set in step SA1 is tested under normal lighting (500 lux) and dark lighting (50 lux) to obtain the corresponding time trigger rate. The observation noise covariance matrix is ​​initialized based on the time trigger rate of the event-driven camera in the dark environment. Specifically, it is:

[0115]

[0116] in: is the observation noise covariance matrix in a dark light environment, is the observation noise covariance matrix under normal lighting conditions, is the noise amplification factor, is the standard deviation of the observation noise of a single pixel.

[0117] Step SA2.2: Matrix Adjustment. That is, based on the received signal strength obtained in real time in the UWB signal system, determine the received signal strength obtained in the UWB signal system within a preset continuous time, compare the received signal strength within the preset continuous time with a preset signal strength threshold, and adjust the observation noise covariance matrix and the observation noise covariance matrix determined in step SA2.1 based on the comparison result. Based on the comparison result, determine the final observation noise covariance matrix and the observation noise covariance matrix, specifically:

[0118] If the received signal strength within a preset continuous time is less than a preset signal strength threshold, then the UWB signal system is blocked by an object. In this case, the process noise covariance matrix determined in step SA2.1 is increased, and the observation noise covariance matrix determined in step SA2.1 is decreased. Otherwise, the process noise covariance matrix and the observation noise covariance matrix determined in step SA2.1 remain unchanged.

[0119] Furthermore, the real-time acceleration corresponding to the end effector of the palletizing robot's manipulator arm is compared with the preset acceleration threshold, and based on the comparison result, the process noise covariance matrix and the observation noise covariance matrix determined in step SA2.1 are adjusted, specifically:

[0120] When the real-time acceleration is greater than the preset acceleration threshold, the process noise covariance matrix determined in step SA2.1 is increased, and the environment corresponding to the event-driven camera is determined. Specifically, when the environment is a low-light environment, the observation noise covariance matrix determined in step SA2.1 is increased. When the environment is a normal-light environment, the observation noise covariance matrix determined in step SA2.1 remains unchanged. Furthermore, when the real-time acceleration is not greater than the preset acceleration threshold, the process noise covariance matrix and the observation noise covariance matrix determined in step SA2.1 remain unchanged.

[0121] Step SA2.3: Spatiotemporal Alignment. This involves constructing an adaptive Kalman filter model based on the final process noise covariance matrix and observation noise covariance matrix determined in step SA2.2. This adaptive Kalman filter model is then used to perform spatiotemporal processing on the low-frequency coordinate data and asynchronous event stream data. The details are as follows:

[0122] Step SA2.3.1: Time alignment. This involves time-aligning the delayed data in the UWB signal system and the event stream in the event-driven camera based on the PTP timestamps in the low-frequency coordinate data. These data are then aligned to a unified timeline using nearest neighbor interpolation.

[0123] Specifically, according to the PTP timestamp in the low-frequency coordinate data and the timestamp in the asynchronous event stream data, the time alignment error between the low-frequency coordinate data and the asynchronous event stream data is obtained, specifically:

[0124]

[0125] in: is the time alignment error between the low-frequency coordinate data and the asynchronous event stream data, is the timestamp in the asynchronous event stream data, It is the PTP timestamp in the low-frequency coordinate data.

[0126] During the specific implementation process, the PTP timestamp in the low-frequency coordinate data is 100.00035s, and the timestamp in the asynchronous event stream data is 99.9991s, so the corresponding time alignment error is 0.00125s.

[0127] Furthermore, the corresponding object position offset is determined based on the obtained time alignment error, specifically:

[0128]

[0129] in: is the object position offset caused by time alignment error, is the instantaneous velocity of the object, is the time alignment error between the low-frequency coordinate data and the asynchronous event stream data, is the acceleration of the robot arm during motion.

[0130] During the specific implementation, the instantaneous speed of the robot arm is 0.4m / s, and the acceleration of the robot arm during movement is 0.1m / s. 2 , the corresponding object position offset is 0.5mm.

[0131] From all the acquired time alignment errors, the minimum time alignment error is determined, and the timestamp corresponding to the minimum time alignment error is determined. At the same time, based on the asynchronous event stream data corresponding to the timestamp and the determined object position offset, the coordinates of the asynchronous event stream data are corrected to obtain the corrected coordinates of the asynchronous event stream data, specifically:

[0132]

[0133] in: is the corrected X-axis coordinate in the asynchronous event stream data. is the corrected Y-axis coordinate in the asynchronous event stream data. is the original X-axis coordinate in the asynchronous event stream data, is the original Y-axis coordinate in the asynchronous event stream data, is the motion direction angle, is the object position offset caused by time alignment error.

[0134] During the specific implementation process, the original event coordinates in the asynchronous event stream data are (100, 200), and the motion direction angle is 45°. The corresponding coordinates of the corrected asynchronous event stream data are (102.5, 202.5).

[0135] Step SA2.3.2: Spatial alignment. This involves converting the global coordinate system in the UWB signal system and the local coordinate system in the event-driven camera to the pallet center coordinate system using a calibration matrix.

[0136] Specifically, according to the constructed transformation matrix, the coordinates in each sensor coordinate system are converted to the corresponding coordinates in the pallet center coordinate system, specifically:

[0137]

[0138] in: is the coordinate of the object in the coordinate system of the center of the pallet, is the transformation matrix from the sensor to the pallet center coordinate system, The original measured coordinates of the object in the sensor.

[0139] In the specific implementation process, the transformation matrix from the sensor to the pallet center coordinate system in this embodiment is: , where the original measured coordinates of the object in the sensor are , then the coordinates of the corresponding object in the coordinate system of the center of the pallet are .

[0140] refer to Figure 2 , Figure 2 This is the effect diagram after data alignment optimization in this embodiment, Figure 2 The average time alignment error for the original data was 15.2ms, but after optimization, it was reduced to 1.8ms, resulting in an 88.2% reduction. Furthermore, the standard deviation decreased from 3.5ms to 0.6ms, and the maximum error decreased from 18.7ms to 2.4ms. This spatiotemporal alignment optimization method effectively resolved the time synchronization issue between the UWB base station and the event-driven camera.

[0141] refer to Figure 3 , Figure 3 This is the coordinate conversion error optimization effect diagram under the pallet coordinate in this embodiment, Figure 3 It can be seen that the peak value of the positioning deviation of the original data is 10.0mm. After SVD optimization, the peak value of the positioning deviation is reduced to 1.5m, that is, the peak error is reduced by 85%. The error curve after optimization is also reduced as a whole, and the fluctuation range is greatly converged. Figure 3 As can be seen from the line graph at the bottom, the error ratio before and after optimization is less than 20% in more than 80% of the time periods. This means that SVD optimization significantly improves the stability in the time dimension.

[0142] Step SA3: Obtain fusion intent labels. That is, based on the constructed knowledge graph and the time-aligned sensor data obtained in step SA2.3, determine the corresponding fusion intent labels. The details are as follows:

[0143] Step SA3.1: Data encoding processing: The asynchronous event stream data, low-frequency coordinate data, six-dimensional force / torque and pressure thermal map obtained in step SA1 are encoded accordingly to obtain corresponding encoded data.

[0144] In this embodiment, the corresponding voxel matrix is ​​obtained based on the asynchronous event stream data and the generated voxel grid, and the spatial position encoding vector corresponding to the asynchronous event stream data is determined through the constructed 3D convolution model and the obtained voxel matrix.

[0145] Furthermore, the corresponding instantaneous velocity and instantaneous acceleration are determined based on the continuous frame coordinates in the low-frequency coordinate data, specifically:

[0146]

[0147] in: For the The instantaneous speed corresponding to the frame, For the The UWB coordinates corresponding to the frame, For the The UWB coordinates corresponding to the frame, is the time frame index in the low-frequency coordinate data, For the The instantaneous acceleration corresponding to the frame, For the The instantaneous speed corresponding to the frame, For the The instantaneous speed corresponding to the frame.

[0148] Specifically, a corresponding motion feature vector is constructed based on the frame coordinates in the low-frequency coordinate data and the instantaneous velocity and instantaneous acceleration corresponding to the frame.

[0149] Furthermore, based on the force data window in the six-dimensional force / torque and pressure thermodynamic map, the steady-state component and fluctuation intensity of the force data are obtained, specifically:

[0150]

[0151] in: is the steady-state component of the grip force, is the total number of sampling points, For the The force measurement value of the sampling points, is the index of the sampling point, is the fluctuation intensity of the grip force.

[0152] Specifically, the obtained steady-state component and fluctuation intensity are decomposed to obtain their decomposition vectors in different directions, namely the steady-state component in the X direction, the steady-state component in the Y direction, the fluctuation intensity in the X direction, and the fluctuation intensity in the Y direction. In other words, the corresponding force perception feature vector is constructed based on the steady-state component in the X direction, the steady-state component in the Y direction, the fluctuation intensity in the X direction, and the fluctuation intensity in the Y direction.

[0153] Step SA3.2: Construct joint features. This involves fusing the spatial position encoding vector, motion feature vector, force feature vector, and embedded vector in the constructed knowledge graph obtained in step SA3.1 to construct a joint feature vector. Specifically,

[0154]

[0155] in: is the joint eigenvector, is the spatial position encoding vector, is the motion feature vector, is the force perception eigenvector, Embedding vector for knowledge graph.

[0156] Step SA3.3: Determine the intent label. That is, based on the joint feature vector obtained in step SA3.2, the corresponding intent label is obtained through the Transformer encoder. Specifically,

[0157]

[0158] in: is the intent label, is the normalized exponential function, is the output weight matrix, is the Transformer encoder function, is the joint eigenvector.

[0159] Specifically, through the Transformer encoder and the joint feature vector, the intention label obtained includes normalized indicators corresponding to emergency braking, obstacle avoidance and continued operation.

[0160] In this embodiment, the trajectory optimization module adjusts the cost function parameters in the spatiotemporal constraint model based on the intent label obtained in step SA3.3, and obtains multiple feasible trajectories and the risk level corresponding to each feasible trajectory based on the adjusted spatiotemporal constraint model. The details are as follows:

[0161] Step SB1: Determine the cost function. This involves combining the normalized metrics corresponding to emergency braking, obstacle avoidance, and continued maneuvers obtained in step SA3.3 with the constructed weight rule table (specific settings are determined based on actual needs and are not detailed in this embodiment) to obtain the corresponding path length weight, energy consumption weight, and smoothness weight.

[0162] Specifically, based on the determined path length weight, energy consumption weight, and smoothness weight, the path length cost, energy consumption cost, and smoothness cost are comprehensively processed to determine the corresponding cost function, which is:

[0163]

[0164] in: is the comprehensive evaluation index of the trajectory, is the path length weight, is the total length of the trajectory, is the energy consumption weight, is the cumulative consumption of joint torque, is the smoothness weight, is the cumulative change in acceleration.

[0165] In the specific implementation process, the pre-built weight rule table is shown in Table 1 below, specifically:

[0166] Table 1: Weight rules table

[0167] Intent Type Path length weight Energy consumption weight Smoothness weight emergency braking 0.3 0.2 0.5 Continue operation 0.7 0.1 0.2 Obstacle avoidance and detour 0.9 0 0.1

[0168] Step SB2: Obtain a feasible trajectory. This involves constructing an optimization problem based on the cost function determined in step SB1, the current working environment (e.g., the pallet's geometry and obstacle locations), and the palletizing robot's arm state (e.g., the current joint angles and end-effector speed). Specifically, the problem is:

[0169]

[0170] in: is the comprehensive evaluation index of the trajectory, is the trajectory parameter vector, is the joint angular acceleration, is the terminal linear velocity, is the end effector position, is the obstacle position, is the Euclidean distance.

[0171] Furthermore, according to the constructed optimization problem, the corresponding optimization trajectory is obtained, specifically:

[0172]

[0173] in: is the angle function of the j-th joint, is the coefficient of the nth term order corresponding to the jth joint, is the time variable of the nth order, is the index of the joint, is the polynomial order, is the total order of the polynomial.

[0174] Step SB3: Determine the risk. That is, based on the optimized trajectory obtained in step SB2, obtain the risk assessment coefficient corresponding to each optimized trajectory, and determine the corresponding risk level based on the risk assessment coefficient. The details are as follows:

[0175] Step SB3.1: Obtain the risk assessment coefficient. This means determining the minimum distance between the optimized trajectory and the nearest obstacle based on the optimized trajectory obtained in step SB2. Furthermore, based on the minimum distance between the optimized trajectory and the nearest obstacle, the actual energy consumption during the optimized trajectory execution, and the reference energy consumption, the corresponding risk assessment coefficient for each optimized trajectory is obtained. Specifically, it is:

[0176]

[0177] in: To optimize the collision probability between the trajectory and the nearest obstacle, is the sensitivity coefficient, To optimize the minimum safe distance between the trajectory and the nearest obstacle, is the comprehensive risk value, is the collision risk weight, is the energy consumption risk weight, In order to optimize the actual energy consumption during trajectory execution, is the benchmark energy consumption value, is the deviation risk weight, is the difference between the actual trajectory and the reference trajectory.

[0178] During the specific implementation process, the sensitivity coefficient in this embodiment is set to 10, where the minimum safe distance between optimized trajectory A and the nearest obstacle is 0.1m, and the minimum safe distance between optimized trajectory B and the nearest obstacle is 0.07m. Then, the collision probability between optimized trajectory A and the nearest obstacle is 0.63, and the collision probability between optimized trajectory B and the nearest obstacle is 0.05.

[0179] Step SB3.2: Classify the risk level. This involves comparing the risk assessment coefficient obtained in step SB3.1 with the preset assessment threshold range, and determining the corresponding risk level based on the comparison result. Specifically:

[0180] When the obtained risk assessment coefficient is less than the lower limit of the preset assessment threshold, the risk level corresponding to the obtained risk assessment coefficient is low risk. When the obtained risk assessment coefficient is within the preset assessment threshold range, the risk level corresponding to the obtained risk assessment coefficient is medium risk. Conversely, when the obtained risk assessment coefficient is greater than the upper limit of the preset assessment threshold, the risk level corresponding to the obtained risk assessment coefficient is high risk.

[0181] In this implementation, the preset assessment threshold range is set to [0.3, 0.6]. Specifically, the collision probability between optimized trajectory A and the nearest obstacle is 0.63, so the risk level corresponding to optimized trajectory A is high. Furthermore, the collision probability between optimized trajectory B and the nearest obstacle is 0.05, so the risk level corresponding to optimized trajectory B is low.

[0182] refer to Figure 4 , Figure 4 is the distribution diagram of risk levels in this embodiment, Figure 4 It can be seen that 65% of the scenarios are in a low-risk state, 10% of the scenarios are in a high-risk state, and 25% of the scenarios are in a medium-risk state.

[0183] In this embodiment, the control response module makes corresponding adjustments based on the risk level determined in step SB3.2, and executes the adjusted trajectory instructions through a PID controller or an MPC controller.

[0184] Step SC1: Determine the response strategy. That is, perform corresponding grading based on the risk level determined in step SB3.2, specifically:

[0185] When the risk level is low, the PID controller adjusts the palletizing robot's joint speed. When the risk level is medium, no action is taken and the current operating state is maintained. When the risk level is high, the MPC controller obtains the predicted collision time and adjusts the palletizing robot's joint acceleration curve.

[0186] Step SC2: Determine the adjustment torque. That is, adjust the joint speed of the palletizing robot through the PID controller to obtain the corresponding adjustment torque, specifically:

[0187]

[0188] in: is the output torque of the j-th joint, is the proportional gain coefficient, is the differential gain coefficient, is the integral gain coefficient, is the target angle of the j-th joint, is the actual angle of the j-th joint, is the target angular velocity of the j-th joint, is the actual angular velocity of the jth joint, is the index of the joint.

[0189] In the specific implementation process, the target angle of the second joint is 0.5 rad, the actual angle is 0.48 rad, and the actual speed of the second joint is 0.01 rad / s, while the target speed is 0 rad / s. The corresponding adjustment torque is 0.952 N / cm2.

[0190] Step SC3: Adjust the braking acceleration curve. This involves obtaining the response speed of the corresponding braking acceleration curve based on the predicted collision time predicted by the MPC controller, and adjusting the braking acceleration curve based on the response speed. Specifically,

[0191]

[0192] in: is the braking acceleration that varies with time, is the maximum permissible braking acceleration, is the base of natural logarithms, is the time after braking starts, is the time constant, To predict the collision time.

[0193] During the specific implementation process, the MPC controller determined that the predicted collision time was 0.6s, the corresponding time constant was 0.2s, and the maximum allowable braking acceleration was 5rad / s 2 , then the corresponding braking acceleration at 0.1s is 1.97rad / s 2 .

[0194] refer to Figure 5 , Figure 5 is a comparison chart of emergency braking performance in this embodiment, Figure 5 The MPC controller achieves 95% of the maximum acceleration (10 m / s²) within 100 ms, 40% faster than the traditional PID controller (167 ms). Furthermore, within the 300 ms predicted collision time, the MPC controller completes 72% of the braking distance, while the traditional PID controller only completes 58%.

[0195] In this embodiment, the verification and calibration module compares the actual position corresponding to the trajectory instruction adjusted in the control response module with the predicted position in the real-time simulation, and adjusts the position of the UWB mapping base station set in the data fusion module based on the comparison result. The details are as follows:

[0196] Step SD1: Determine the prediction deviation. That is, based on the adjusted trajectory command in the control response module and the position difference between the actual position and the predicted position corresponding to the real-time simulation at each time point, obtain the corresponding prediction deviation, specifically:

[0197]

[0198] in: is the forecast deviation at time t, is the smoothing factor, is the actual measurement deviation at time t, is the prediction deviation at time t-1.

[0199] During the specific implementation process, the actual deviation corresponding to 100ms is 3.7mm, and the predicted deviation is 3mm. The actual deviation corresponding to 101ms is 3.5mm, and the predicted deviation is 3.14mm. The predicted deviation corresponding to 100ms is 3.14mm, and the predicted deviation corresponding to 101ms is 3.21mm.

[0200] Step SD2: Determine the response signal. That is, based on the predicted deviation and the preset time obtained in step SD1, compare the predicted deviation within the preset time with the preset deviation threshold, and determine the corresponding response signal based on the comparison result. Specifically:

[0201] When the prediction deviations within the preset time are all greater than the preset deviation threshold, an adjustment signal is triggered, and the next step SD3 is executed to adjust the position of the UWB mapping base station set in the data fusion module. Otherwise, no adjustment signal is triggered.

[0202] Step SD3: Base station adjustment. This involves using the Transformer encoder to inversely decode the UWB base station position and the initial position of the UWB base station before adjustment to determine the displacement that needs to be adjusted for the UWB base station. Specifically,

[0203]

[0204] in: Function to calculate tag coordinates for UWB base station location, is the total number of calibration measurements for the encoder, is the index of the encoder’s calibration measurement times, is the initial position of the UWB base station measured for the mth time, The displacement that needs to be adjusted for the UWB base station, is the inverse position of the mth encoder.

[0205] Furthermore, according to the determined displacement of the UWB base station that needs to be adjusted, the position of the UWB mapping base station set in the data fusion module is adjusted, and steps SA1 to SD3 are repeated until the position of the UWB base station does not need to be adjusted.

[0206] refer to Figure 6 , Figure 6 is a comparison diagram of the position adjustment of the UWB base station in this embodiment, Figure 6 It can be seen that: about 25% of the UWB base stations have a displacement greater than 10cm, and the maximum displacement is 15cm. At the same time, the standard deviation of the triangle edges formed by the optimized UWB base stations is reduced by 37%. Therefore, the Transformer encoder effectively improves the geometric relationship of the UWB base stations.

[0207] Example 2

[0208] This embodiment provides a precise palletizing control system based on artificial intelligence technology. Its specific implementation method is the same as that of Example 1, except that, in step SA2.3.2, the corresponding transformation matrix is ​​obtained by the least squares method. The following examples illustrate the present invention with reference to the specific implementation methods of this embodiment.

[0209] In this embodiment, the corresponding transformation matrix is ​​obtained by using the least squares method and the coordinates in the global coordinate system of the UWB signal system, the coordinates in the local coordinate system of the event-driven camera, and the coordinates in the coordinate system of the end-of-arm of the distributed fiber optic force sensor array. Specifically, it is as follows:

[0210] Step SA2.3.2.1: Through calibration experiments, install a composite calibration plate at the end of the palletizing robot's robotic arm. Using the coordinates of the calibration plate in the pallet center coordinate system and the coordinates measured by each sensor, obtain the average coordinates of the fixed-point object in the pallet center coordinate system and the sensor coordinate system. Specifically:

[0211]

[0212] in: is the average coordinate of the fixed-point object in the coordinate system of the center of the pallet, is the average coordinate of the original measurement of the fixed-point object in the sensor, is the number of calibration poses, is the real coordinate of the calibration plate in the pallet center coordinate system at the i-th pose, is the coordinate of the calibration plate measured in the sensor at the i-th pose, is the pose index.

[0213] Step SA2.3.2.2: Based on the average coordinates of the fixed-point object in the pallet center coordinate system and the sensor obtained in step SA2.3.2.1, determine the correlation between the pallet center coordinate system and the sensor coordinates, specifically:

[0214]

[0215] in: is the covariance matrix, is the average coordinate of the fixed-point object in the coordinate system of the center of the pallet, is the average coordinate of the original measurement of the fixed-point object in the sensor, is the number of calibration poses, is the real coordinate of the calibration plate in the pallet center coordinate system at the i-th pose, is the coordinate of the calibration plate measured in the sensor at the i-th pose, is the pose index, Inverted symbol.

[0216] Step SA2.3.2.3: Perform SVD decomposition based on the covariance matrix determined in step SA2.3.2.2, and construct a rotation matrix based on the result matrix of the SVD decomposition, specifically:

[0217]

[0218] in: is the covariance matrix, is the inverted symbol, is the rotation matrix, 、 is the orthogonal matrix of the SVD decomposition result, is the diagonal matrix of singular values ​​of the SVD decomposition result.

[0219] Step SA2.3.2.4: Based on the rotation matrix constructed in step SA2.3.2.3 and the average coordinates of the fixed-point object in the pallet center coordinate system and the sensor obtained in step SA2.3.2.1, determine the translation vector, specifically:

[0220]

[0221] in: is the translation vector, is the average coordinate of the fixed-point object in the coordinate system of the center of the pallet, is the average coordinate of the original measurement of the fixed-point object in the sensor, is the rotation matrix.

[0222] Step SA2.3.2.5: Based on the rotation matrix constructed in step SA2.3.2.3 and the translation vector determined in step SA2.3.2.4, obtain the transformation matrix from the sensor to the pallet center coordinate system, specifically:

[0223]

[0224] in: is the transformation matrix from the sensor to the pallet center coordinate system, is the translation vector, is the rotation matrix.

[0225] In the process of specific implementation, the covariance matrix obtained is , then the corresponding orthogonal matrices of the SVD decomposition results are: and , so the corresponding rotation matrix is Furthermore, in this embodiment, the average coordinates of the fixed-point object in the pallet center coordinate system are , the average coordinates of the fixed-point object originally measured in the sensor are , then the corresponding translation vector is , so the corresponding transformation matrix from the sensor to the pallet center coordinate system is .

[0226] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is limited by the accompanying embodiments and their equivalents.

Claims

1. A precise palletizing control system based on artificial intelligence technology, characterized in that: Includes: A data fusion module acquires sensor data through the constructed sensor network, preprocesses the sensor data, and fuses the preprocessed sensor data to obtain a fused intent label; The trajectory optimization module adjusts the cost function parameters according to the intention label, and constructs a spatiotemporal constraint model based on the adjusted cost function parameters. At the same time, based on the spatiotemporal constraint model, it obtains feasible trajectories and corresponding risk levels, including: SB1: Determine the cost function: Based on the intent label and the constructed weight rule table, determine the cost function, specifically: ,in: is the comprehensive evaluation index of the trajectory, is the path length weight, is the total length of the trajectory, is the energy consumption weight, is the cumulative consumption of joint torque, is the smoothness weight, is the cumulative change of acceleration; SB2: Obtaining a feasible trajectory: Based on the cost function, the working environment, and the state of the palletizing robot's manipulator, a corresponding optimization problem is constructed, and based on the optimization problem, an optimized trajectory is obtained, specifically: ,in: is the angle function of the j-th joint, is the coefficient of the nth term order corresponding to the jth joint, is the time variable of the nth order, is the index of the joint, is the polynomial order, is the total order of the polynomial; SB3: Determine risk: Obtain a risk assessment coefficient of the optimized trajectory according to the optimized trajectory, and determine a risk level of the optimized trajectory according to the risk assessment coefficient; A control response module adjusts the feasible trajectory according to the risk level to obtain an adjusted feasible trajectory; The verification and calibration module obtains the predicted control deviation of each adjusted feasible trajectory through real-time simulation, and adjusts the constructed sensor network according to the predicted control deviation and the real-time braking error.

2. The precise palletizing control system based on artificial intelligence technology according to claim 1 is characterized in that: Get the fused intent label, including: SA1: Build a sensor network: Acquire asynchronous event stream data through event-driven cameras, obtain low-frequency coordinate data through UWB mapping base stations and UWB tags, and obtain six-dimensional force / torque and pressure thermal maps through distributed fiber optic force sensor arrays; SA2: Data preprocessing: Using the constructed adaptive Kalman filter model, the asynchronous event stream data and the low-frequency coordinate data are subjected to spatiotemporal alignment processing to obtain preprocessed sensor data; SA3: Obtain fusion intent label: Determine the fusion intent label through the constructed knowledge graph and preprocessed sensor data.

3. The precise palletizing control system based on artificial intelligence technology according to claim 2 is characterized in that: Get pre-processed sensor data, including: SA2.1: Noise Modeling: Initialize the process noise covariance matrix through static calibration experiments, UWB mapping base stations and UWB tags. Specifically: ,in: is the process noise covariance matrix, is the noise level in the X direction during UWB positioning. is the noise level in the Y direction during UWB positioning. The noise level in the Z direction during UWB positioning. The illumination of the event-driven camera is adjusted and tested to obtain the time trigger rate, and the observation noise covariance matrix is ​​initialized according to the time trigger rate, specifically: ,in: is the observation noise covariance matrix in a dark light environment, is the observation noise covariance matrix under normal lighting conditions, is the noise amplification factor, is the standard deviation of the observation noise of a single pixel; SA2.2: Matrix Adjustment: Adjust the process noise covariance matrix and the observation noise covariance matrix based on the received signal strength corresponding to the low-frequency coordinate data and the real-time acceleration corresponding to the asynchronous event stream data, as follows: The received signal strength within a preset continuous time is compared with a preset signal strength threshold, and according to the comparison result, the process noise covariance matrix and the observation noise covariance matrix are adjusted, specifically: When the received signal strength within a preset continuous time is less than a preset signal strength threshold, the process noise covariance matrix is ​​increased and the observation noise covariance matrix is ​​decreased; otherwise, the process noise covariance matrix and the observation noise covariance matrix remain unchanged; The real-time acceleration is compared with a preset acceleration threshold, and the process noise covariance matrix and the observation noise covariance matrix are adjusted according to the comparison result, specifically: When the real-time acceleration is greater than a preset acceleration threshold, the process noise covariance matrix is ​​increased. At the same time, when the environment of the event-driven camera is a dark environment, the observation noise covariance matrix is ​​increased. When the environment of the event-driven camera is a normal lighting environment, the observation noise covariance matrix remains unchanged; otherwise, the process noise covariance matrix and the observation noise covariance matrix remain unchanged. SA2.3: Spatiotemporal alignment: Construct an adaptive Kalman filter model based on the adjusted process noise covariance matrix and the observation noise covariance matrix, and perform spatiotemporal processing on the asynchronous event stream data and low-frequency coordinate data based on the adaptive Kalman filter model.

4. The precise palletizing control system based on artificial intelligence technology according to claim 3 is characterized in that: Performing spatiotemporal processing on the asynchronous event stream data and the low-frequency coordinate data includes: SA2.3.1: Time alignment: Based on the PTP timestamp in the low-frequency coordinate data and the timestamp in the asynchronous event stream data, obtain the time alignment error between the low-frequency coordinate data and the asynchronous event stream data, and determine the object position offset based on the time alignment error. At the same time, based on the coordinates in the asynchronous event stream data with the minimum time alignment error and the object position offset, correct the coordinates in the asynchronous event stream data, specifically: ,in: is the corrected X-axis coordinate in the asynchronous event stream data. is the corrected Y-axis coordinate in the asynchronous event stream data. is the original X-axis coordinate in the asynchronous event stream data, is the original Y-axis coordinate in the asynchronous event stream data, is the motion direction angle, is the object position offset caused by time alignment error; SA2.3.2: Spatial Alignment: Use the constructed transformation matrix to transform the coordinates in each sensor coordinate system into the coordinates in the pallet center coordinate system. Specifically: ,in: is the coordinate of the object in the coordinate system of the center of the pallet, is the transformation matrix from the sensor to the pallet center coordinate system, The original measured coordinates of the object in the sensor.

5. The precise palletizing control system based on artificial intelligence technology according to claim 4 is characterized in that: By using the least squares method, the transformation matrix is ​​determined, including: SA2.3.2.1: Through calibration experiments, obtain the average coordinates of the fixed-point object in the pallet center coordinate system and the sensor. Specifically: ,in: is the average coordinate of the fixed-point object in the coordinate system of the center of the pallet, is the average coordinate of the original measurement of the fixed-point object in the sensor, is the number of calibration poses, is the real coordinate of the calibration plate in the pallet center coordinate system at the i-th pose, is the coordinate of the calibration plate measured in the sensor at the i-th pose, is the pose index; SA2.3.2.2: Determine the correlation between the pallet center coordinate system and the sensor coordinates based on the average coordinates of the fixed-point object in the pallet center coordinate system and the sensor coordinates, specifically: ,in: is the covariance matrix, is the average coordinate of the fixed-point object in the coordinate system of the center of the pallet, is the average coordinate of the original measurement of the fixed-point object in the sensor, is the number of calibration poses, is the real coordinate of the calibration plate in the pallet center coordinate system at the i-th pose, is the coordinate of the calibration plate measured in the sensor at the i-th pose, is the pose index, is an inverted symbol; SA2.3.2.3: Construct a rotation matrix based on the covariance matrix and the result matrix of the SVD decomposition, specifically: ,in: is the covariance matrix, is the inverted symbol, is the rotation matrix, 、 is the orthogonal matrix of the SVD decomposition result, is the singular value diagonal matrix of the SVD decomposition result; SA2.3.2.4: Determine the translation vector based on the rotation matrix and the average coordinates of the fixed-point object in the pallet center coordinate system and the sensor. Specifically, ,in: is the translation vector, is the average coordinate of the fixed-point object in the coordinate system of the center of the pallet, is the average coordinate of the original measurement of the fixed-point object in the sensor, is the rotation matrix; SA2.3.2.5: Based on the rotation matrix and translation vector, obtain the transformation matrix from the sensor to the pallet center coordinate system, specifically: ,in: is the transformation matrix from the sensor to the pallet center coordinate system, is the translation vector, is the rotation matrix.

6. The precise palletizing control system based on artificial intelligence technology according to claim 2 is characterized in that: Determine the fusion intent label, including: SA3.1: Data Encoding Processing: Construct a spatial position encoding vector based on the asynchronous event stream data and the set voxel grid. Construct a motion feature vector based on the continuous frame coordinates, instantaneous velocity, and instantaneous acceleration in the low-frequency coordinate data. Obtain the steady-state component and fluctuation intensity from the force data window in the six-dimensional force / torque and pressure thermal map to construct a force feature vector. SA3.2: Construct joint features: Based on the spatial position encoding vector, motion feature vector, and force feature vector, and using the embedding vector in the knowledge graph, a joint feature vector is constructed. Specifically, ,in: is the joint eigenvector, is the spatial position encoding vector, is the motion feature vector, is the force perception eigenvector, Embed vectors for knowledge graphs; SA3.3: Determine intent label: Obtain intent label through Transformer encoder and the joint feature vector, specifically: ,in: is the intent label, is the normalized exponential function, is the output weight matrix, is the Transformer encoder function, is the joint feature direction.

7. The precise palletizing control system based on artificial intelligence technology according to claim 1 is characterized in that: Determining the risk level of the optimized trajectory includes: SB3.1: Obtaining a risk assessment coefficient: Obtain a risk assessment coefficient for each optimized trajectory based on the closest distance between the optimized trajectory and the nearest obstacle, the actual energy consumption during the optimized trajectory operation, and the reference energy consumption, specifically: ,in: To optimize the collision probability between the trajectory and the nearest obstacle, is the sensitivity coefficient, To optimize the minimum safe distance between the trajectory and the nearest obstacle, is the comprehensive risk value, is the collision risk weight, is the energy consumption risk weight, In order to optimize the actual energy consumption during trajectory execution, is the benchmark energy consumption value, is the deviation risk weight, is the difference between the actual trajectory and the reference trajectory; SB3.2: Risk level classification: Compare the risk assessment coefficient with the preset assessment threshold range and determine the risk level based on the comparison results, specifically: When the risk assessment coefficient is less than the lower limit threshold of the preset assessment threshold, the risk level is low risk; when the risk assessment coefficient is within the preset assessment threshold range, the risk level is medium risk; otherwise, the risk level is high risk.

8. The precise palletizing control system based on artificial intelligence technology according to claim 1 is characterized in that: Get the adjusted feasible trajectory, including: SC1: Determine the response strategy: Based on the risk level, determine the hierarchical processing operations, specifically: When the risk level is low, the PID controller is used to adjust the joint speed of the palletizing robot; when the risk level is medium, the current operating state is maintained; when the risk level is high, the MPC controller is used to obtain the predicted collision time and adjust the joint acceleration curve of the palletizing robot; SC2: Determine the adjustment torque: The joint speed of the palletizing robot is adjusted by the PID controller to obtain the adjustment torque, specifically: ,in: is the output torque of the j-th joint, is the proportional gain coefficient, is the differential gain coefficient, is the integral gain coefficient, is the target angle of the j-th joint, is the actual angle of the j-th joint, is the target angular velocity of the j-th joint, is the actual angular velocity of the jth joint, is the index of the joint; SC3: Braking acceleration curve adjustment: The response speed of the braking acceleration curve is obtained through the predicted collision time of the MPC controller, and the braking acceleration curve is adjusted according to the response speed, specifically: ,in: is the braking acceleration that varies with time, is the maximum permissible braking acceleration, is the base of natural logarithms, is the time after braking starts, is the time constant, To predict the collision time.

9. The precise palletizing control system based on artificial intelligence technology according to claim 1 is characterized in that: Adjustments to the constructed sensor network include: SD1: Determine the prediction deviation: Based on the real-time position of the adjusted feasible trajectory and the predicted position in the real-time simulation, obtain the prediction deviation, specifically: ,in: is the forecast deviation at time t, is the smoothing factor, is the actual measurement deviation at time t, is the prediction deviation at time t-1; SD2: Determine a response signal: Based on the prediction deviation and the preset time length, compare the prediction deviation within the preset time with the preset deviation threshold, and determine a response signal based on the comparison result, specifically: When the prediction deviations within the preset time are all greater than the preset deviation threshold, an adjustment signal is triggered and the next step SD3 is executed; otherwise, no adjustment signal is triggered; SD3: Base station adjustment: The original position of the UWB base station is determined by the Transformer encoder to determine the adjustment displacement of the UWB base station, specifically: ,in: Function to calculate tag coordinates for UWB base station location, is the total number of calibration measurements for the encoder, is the index of the encoder’s calibration measurement times, is the initial position of the UWB base station measured for the mth time, The displacement that needs to be adjusted for the UWB base station, is the inverse position of the mth encoder.

Images