Robot multimodal perception and motion coordinated control method and device

Abstract

The present application discloses a robot multimodal perception and motion coordination control method and device. The robot method includes: obtaining multiple types of sensor information as multimodal perception data according to a preset cycle through multiple types of sensors arranged in the robot operation area, and the multimodal perception data includes laser radar data, infrared sensor data and visual sensor data. A conveying device is provided in the robot operation area; based on the multimodal perception data, analyzing the transmission parameters of the conveyor belt of the conveying device and the cargo parameters of the goods on the conveyor belt; dynamically adjusting the historical motion parameters of the robot's mechanical arm according to the transmission parameters and the cargo parameters to obtain the optimal motion coordination parameters suitable for the conveying device; constructing and executing the motion control instructions corresponding to the optimal motion coordination parameters to operate the goods on the conveyor belt. Therefore, the use of the embodiments of the present application can improve sorting efficiency, reduce maintenance costs, and ensure the stability of the entire production process.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a method and device for multimodal perception and motion coordinated control of a robot. Background Art

[0002] Automated assembly lines are widely used in modern industrial production. For example, in logistics sorting centers, sorting robots are widely used to quickly sort goods on conveyor belts by category. These robots use their precise mechanical arms to grab goods from conveyor belts and place them in designated locations, achieving efficient and accurate logistics sorting.

[0003] Currently, traditional sorting robots typically use fixed control parameters to perform their tasks. These parameters include the robot's motion trajectory, the timing of the grasping action, and synchronization with the conveyor belt speed. In practice, the robot performs sorting actions based on a pre-set program, according to the fixed position and speed of the goods on the conveyor belt. This control method works effectively when the conveyor belt speed and the position of the goods are relatively stable.

[0004] However, over time, conveyor equipment may experience operational changes due to aging, wear, or other factors, such as fluctuations in conveyor belt speed or shifting cargo positions on the conveyor. In these cases, sorting robots, relying on fixed parameters, are unable to perceive and adapt to these dynamic changes in real time. This can lead to inaccurate grasping of cargo and even damage to the cargo or the equipment itself, reducing sorting efficiency, increasing maintenance costs, and impacting the stability of the entire production process. Summary of the Invention

[0005] The embodiments of this application provide a method and apparatus for multimodal perception and coordinated motion control of a robot. To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is provided below. This summary is not intended to be a comprehensive review, identify key or important elements, or delineate the scope of protection for these embodiments. Its sole purpose is to present some concepts in a simplified form, serving as a prelude to the detailed description that follows.

[0006] In a first aspect, an embodiment of the present application provides a robot multimodal perception and motion coordinated control method, which is applied to a robot and includes:

[0007] Multiple types of sensor information are acquired as multimodal perception data according to preset cycles by multiple types of sensors deployed in the robot's operating area. The multimodal perception data includes lidar data, infrared sensor data, and visual sensor data. A transmission device is provided in the robot's operating area.

[0008] Analyze the conveyor belt transmission parameters of the conveyor equipment and the cargo parameters of the cargo on the conveyor belt based on multimodal sensing data;

[0009] Dynamically adjust the robot's arm's historical motion parameters based on the conveying parameters and cargo parameters to obtain the optimal motion coordination parameters suitable for the conveying equipment;

[0010] Construct and execute motion control instructions corresponding to the optimal motion coordination parameters to operate the goods on the conveyor belt.

[0011] In a second aspect, an embodiment of the present application provides a robot multimodal perception and motion coordinated control device, the device comprising:

[0012] A sensor information acquisition module is used to acquire multiple types of sensor information as multimodal perception data according to a preset period through multiple types of sensors deployed in the robot's operating area. The multimodal perception data includes lidar data, infrared sensor data, and visual sensor data. A transmission device is provided in the robot's operating area.

[0013] A parameter analysis module, configured to analyze transmission parameters of a conveyor belt of a conveying device and parameters of goods on the conveyor belt based on multimodal sensing data;

[0014] The motion parameter adjustment module is used to dynamically adjust the historical motion parameters of the robot's manipulator arm according to the transmission parameters and cargo parameters to obtain the optimal motion coordination parameters suitable for the transmission equipment;

[0015] The instruction execution module is used to construct and execute motion control instructions corresponding to the optimal motion coordination parameters to operate the goods on the conveyor belt.

[0016] The technical solutions provided by the embodiments of the present application may have the following beneficial effects:

[0017] In the embodiments of the present application, on the one hand, through multimodal perception data, the robot can obtain the operating status of the conveyor belt and detailed information about the goods in real time, thereby dynamically adjusting the historical motion parameters of the robotic arm to achieve accurate and efficient sorting operations. Accurate sorting operations can significantly improve sorting efficiency, reduce damage to goods or sorting errors caused by misoperation, and thus reduce the costs incurred by frequent maintenance and repair of equipment. On the other hand, by dynamically adjusting the historical motion parameters of the robotic arm to adapt to the real-time operating status of the conveyor belt and the characteristics of the goods, the robot can maintain stable operating performance under different working conditions. This adaptive capability can effectively avoid operation interruptions or erroneous operations caused by problems such as changes in conveyor belt speed, cargo position offset, or conveyor belt wear, thereby ensuring the stability of the entire production process.

[0018] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0020] Figure 1 This is a schematic diagram of a method flow of a robot multimodal perception and motion coordinated control method provided by an embodiment of the present application;

[0021] Figure 2 This is a schematic diagram of the system structure of a robot multimodal perception and motion coordinated control system provided by an embodiment of the present application;

[0022] Figure 3 It is a signal diagram between a time domain signal and a frequency domain signal provided in an embodiment of the present application;

[0023] Figure 4 This is a model architecture diagram of a pre-trained motion parameter analysis model provided in an embodiment of the present application;

[0024] Figure 5 1 is a flow chart of a model training method for an action parameter analysis model provided in an embodiment of the present application;

[0025] Figure 6 This is a schematic structural diagram of a robot multimodal perception and motion coordinated control device provided in an embodiment of the present application;

[0026] Figure 7 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following description and the drawings sufficiently illustrate specific embodiments of the application to enable those skilled in the art to practice them.

[0028] It should be clear that the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0029] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0030] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances. In addition, in the description of this application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0031] Currently, traditional sorting robots typically use fixed control parameters to perform their tasks. These parameters include the robot's motion trajectory, the timing of the grasping action, and synchronization with the conveyor belt speed. In practice, the robot performs sorting actions based on a pre-set program, according to the fixed position and speed of the goods on the conveyor belt. This control method works effectively when the conveyor belt speed and the position of the goods are relatively stable.

[0032] The inventors recognized that over time, conveyor equipment may experience changes in operating parameters due to aging, wear, or other factors. This can include fluctuations in conveyor belt speed or shifts in the position of goods on the conveyor belt. In such cases, sorting robots, relying on fixed parameters, are unable to perceive and adapt to these dynamic changes in real time. This can lead to inaccurate grasping of goods and may even damage the goods or the equipment itself, reducing sorting efficiency, increasing maintenance costs, and impacting the stability of the entire production process.

[0033] In order to solve the above problems, the present application provides a robot multimodal perception and motion coordinated control method and device to solve the problems existing in the above-mentioned related technical problems. In the embodiment of the present application, on the one hand, through multimodal perception data, the robot can obtain the operating status of the conveyor belt and detailed information of the goods in real time, so as to dynamically adjust the historical motion parameters of the robot arm to achieve accurate and efficient sorting operations. Accurate sorting operations can significantly improve sorting efficiency, reduce damage to goods or sorting errors caused by misoperation, and thus reduce the cost of frequent maintenance and repair of equipment. On the other hand, by dynamically adjusting the historical motion parameters of the robot arm to adapt to the real-time operating status of the conveyor belt and the characteristics of the goods, the robot can maintain stable operating performance under different working conditions. This adaptive ability can effectively avoid operation interruptions or erroneous operations caused by problems such as changes in conveyor belt speed, cargo position offset or conveyor belt wear, thereby ensuring the stability of the entire production process. The following is a detailed description using an exemplary embodiment.

[0034] The following will be combined with the Figure 1-Attached Figure 5 This paper introduces in detail the robot multimodal perception and motion coordinated control method provided by the embodiments of this application. This method can be implemented by a computer program and can be run on a robot multimodal perception and motion coordinated control device based on the von Neumann architecture. This computer program can be integrated into an application or run as a standalone tool application.

[0035] See Figure 1 , provides a flow chart of a robot multimodal perception and motion coordinated control method for an embodiment of the present application, which is applied to a robot. Figure 1 As shown, the method of the embodiment of the present application includes the following steps:

[0036] S101, acquiring multiple types of sensor information as multimodal perception data according to a preset period using multiple types of sensors deployed in the robot operation area, where a transmission device is provided in the robot operation area. The multimodal perception data includes lidar data, infrared sensor data, and visual sensor data.

[0037] The robot's operating area is the spatial range within which the robot performs its tasks, such as an automated production line. Multi-type sensors are various types of data acquisition devices installed within the robot's operating area to acquire environmental information. A preset period refers to the time interval between sensor data collection, a fixed, pre-set duration. Multimodal perception data refers to data acquired by multiple different types of sensors, reflecting the state of the environment from multiple perspectives. LiDAR (Light Detection and Ranging) is a sensor that uses lasers for distance measurement and environmental modeling. LiDAR data refers to the distance information and three-dimensional point cloud data collected by the LiDAR. Infrared sensors acquire information by detecting infrared radiation emitted by objects. Infrared sensor data refers to information such as temperature and reflectivity collected by the infrared sensor. Visual sensors, typically cameras, are used to capture images or video data. Visual sensor data refers to the two-dimensional image or video information captured by the camera. Conveyor equipment refers to mechanical devices used to transport goods, typically including conveyor belts and conveyors.

[0038] In some embodiments of the present application, a variety of sensors are installed in the robot's operating area, including laser radar, infrared sensors, and visual sensors. These sensors are distributed around the conveying equipment and can cover the conveyor belt and the goods on it. Laser radar is used to sense the spatial position of the conveyor belt and the goods. Infrared sensors are used to determine the wear of the conveyor belt and the thermal characteristics of the goods. Visual sensors are used to capture image information of the conveyor belt and the goods to identify the appearance characteristics of the goods, such as color, shape, and size. All sensors synchronously collect data according to a preset time period to obtain multimodal perception data. The preset time period can be 3 seconds.

[0039] For example Figure 2 As shown, Figure 2 This application provides a schematic diagram of the system architecture of a multimodal perception and motion coordination control system for robots. The system includes a robot, a conveyor, a robotic arm, and multiple sensors, including lidar, visual sensors, and infrared sensors. After the system is started, sensors deployed throughout the robot's operating area acquire information at preset intervals as multimodal perception data.

[0040] S102, analyzing the conveying parameters of the conveyor belt of the conveying equipment and the cargo parameters of the cargo on the conveyor belt based on the multimodal sensing data;

[0041] Among them, the transmission parameters are parameters that describe the operating status of the transmission equipment, including the running speed, inclination angle and wear data of the conveyor belt; the cargo parameters refer to the characteristic parameters that describe the cargo on the conveyor belt, including the center coordinates, size and relative position of the cargo on the conveyor belt.

[0042] In some embodiments of the present application, a high-definition camera installed in front of the conveyor belt captures images of the goods on the conveyor belt at a frequency of 30 frames per second and records the appearance characteristics of the goods. A laser radar installed on the side of the conveyor belt scans the conveyor belt and the goods in real time to generate high-precision three-dimensional point cloud data for measuring the center coordinates, size, and relative position of the goods on the conveyor belt to the edge of the conveyor belt. The infrared sensor arranged on the side of the conveyor belt detects the temperature change and wear of the conveyor belt and monitors the thermal characteristics of the goods. The marking points on the conveyor belt are captured by a visual sensor, and the real-time running speed of the conveyor belt is calculated in combination with the scanning data of the laser radar. For example, the displacement of the marking point in two consecutive frames of images is divided by the time interval to obtain the conveyor belt speed. The height difference on both sides of the conveyor belt is measured by the laser radar, and the inclination angle of the conveyor belt is calculated in combination with the length of the conveyor belt.

[0043] S103, dynamically adjusting the historical motion parameters of the robot's manipulator arm based on the conveying parameters and the cargo parameters to obtain optimal motion coordination parameters suitable for the conveying equipment;

[0044] The historical motion parameters of the robot arm refer to the motion parameters used by the robot arm in previous operations.

[0045] In some embodiments of the present application, the historical motion parameters of the robot's robotic arm are dynamically adjusted according to the transmission parameters and cargo parameters to obtain the optimal motion coordination parameters suitable for the transmission equipment. The specific process includes: converting the running speed and tilt angle into frequency domain features through dynamic Fourier transform; constructing a wear heat map of the conveyor belt based on wear data; fusing the frequency domain features with the wear heat map to obtain an environment coding vector; using the attention mechanism to encode the center coordinates, size and relative position to generate a cargo coding vector for each cargo; inputting the environment coding vector and the cargo coding vector into a pre-trained motion parameter analysis model to output the current motion parameters of the robot's robotic arm; when the current motion parameters are inconsistent with the historical motion parameters of the robot's robotic arm, replacing the historical motion parameters with the current motion parameters to obtain the optimal motion coordination parameters suitable for the transmission equipment.

[0046] The dynamic Fourier transform is a mathematical tool that converts time-domain signals into frequency-domain signals and is used to analyze the signal's frequency components. Frequency-domain features refer to the representation of a signal in the frequency domain, including information such as frequency components, amplitude, and phase. Wear data describes the extent of conveyor belt wear, including wear depth, temperature changes, and reflectivity changes. A wear heatmap is a visualization tool that uses color to represent the degree of wear in different areas of the conveyor belt. The environment encoding vector is a vector that comprehensively represents the operating status and wear of the conveyor belt. It is generated by fusing frequency-domain features with the spatial information of the wear heatmap. The attention mechanism is a neural network technology that mimics human attention. It can be used to identify key features of goods (such as center coordinates, size, and relative position) and generate a goods encoding vector. This attention mechanism enables robots to more accurately identify and manipulate goods. The goods encoding vector integrates key features of goods, providing relevant information for adjusting the robot's motion parameters. The motion parameter analysis model is a pre-trained neural network model that learns the relationship between conveyor parameters, cargo parameters, and robot arm motion parameters, dynamically adjusting the robot arm's motion parameters to adapt to different operating environments.

[0047] In the embodiments of this application, by integrating frequency domain features and wear heat maps, the robot can more comprehensively understand the operating status and wear of the conveyor belt. This multi-dimensional perception enables the robot to accurately identify the location and status of the goods, reducing sorting errors caused by misjudgment. At the same time, the motion parameter analysis model can quickly output the optimal motion parameters, allowing the robot to adjust its movements in real time to adapt to changes in conveyor belt speed and the dynamic position of the goods, thereby improving sorting efficiency.

[0048] In some embodiments of the present application, the specific process of converting the running speed and tilt angle into frequency domain features through dynamic Fourier transform is: filtering, denoising and normalizing the running speed and tilt angle that change with time to obtain the preprocessed time domain signal of the transmission equipment; using the dynamic Fourier transform expression to convert the time domain signal of the transmission equipment into a frequency domain signal to obtain the spectrum data of the transmission equipment; identifying the main frequency components, the amplitude of each frequency component and the phase of each frequency component in the spectrum data of the transmission equipment as the frequency domain information of the running speed and tilt angle; fusing the frequency domain information of the running speed and tilt angle to obtain frequency domain features.

[0049] Among them, the dynamic Fourier transform expression is:

[0050]

[0051] in, is the frequency domain signal The complex value of the frequency components, It is the first time domain signal of the transmitting device. The value of the sampling point, is the total number of sampling points, The sampling index of the time domain signal, is the frequency index of the frequency domain signal, Is a complex exponential function used to convert time domain signals to frequency domain.

[0052] For example, a low-pass filter is used to remove high-frequency noise and normalize the signal to the range of [0,1]. Fast Fourier transform (FFT) is used to convert the pre-processed time domain signal into a frequency domain signal. The main frequency components, amplitude and phase of each frequency component are extracted from the frequency domain signal. For example, the signal diagram between the time domain signal and the frequency domain signal is shown in Figure 1. Figure 3 shown.

[0053] In some embodiments of the present application, the specific process of constructing a wear heat map of a conveyor belt based on wear data is as follows: extracting the wear depth of the conveyor belt, the temperature change of the wear area, and the reflectivity change of the wear area from the wear data; dividing the surface of the conveyor belt into multiple small grids based on preset grid division parameters, and each small grid corresponds to a wear data point; normalizing the wear depth of the conveyor belt, the temperature change of the wear area, and the reflectivity change of the wear area, and mapping them to each small grid to form a wear data matrix; generating a grid layer representing the degree of wear by spatially smoothing the wear data matrix; assigning different color shades to the grid layer according to the size of the wear data in the wear data matrix to obtain a wear heat map of the conveyor belt.

[0054] For example, the generated wear heat map can visually display the wear of the conveyor belt, helping maintenance personnel to identify potential problems in advance and reduce the frequency of sudden failures. Furthermore, combined with the robot control system, motion parameters can be dynamically adjusted to optimize the production process and improve production efficiency.

[0055] In some embodiments of the present application, the specific process of fusing frequency domain features with the wear heat map to obtain an environmental coding vector includes: extracting the spatial information of the wear heat map; mapping the spatial information of the wear heat map to the frequency domain features to splice the frequency domain features with the spatial information of the wear heat map to obtain a spliced ​​feature; encoding the spliced ​​feature to obtain an environmental coding vector.

[0056] Among them, the pre-trained action parameter analysis model includes a weight matrix loading layer, a quantization layer, an attention layer, a dot product operation layer, a feature mapping layer, and a cropping layer, for example Figure 4 shown.

[0057] In some embodiments of the present application, the environment coding vector and the cargo coding vector are input into a pre-trained motion parameter analysis model, and the specific process of outputting the current motion parameters of the robot's robotic arm includes: the weight matrix loading layer loads the pre-trained weight matrix; the quantization layer uses the pre-trained weight matrix to quantize the environment coding vector and the cargo coding vector respectively to obtain a query vector, a key vector, and a value vector; the attention layer calculates the attention score based on the query vector, the key vector, and the value vector; the attention score is used to measure the importance of the environment coding vector to the cargo coding vector; the dot product operation layer performs dot product operations on the environment coding vector and the cargo coding vector with the attention score respectively and sums them to obtain a joint feature containing the interaction information between the environment and the cargo; the feature mapping layer maps the joint feature to a preset robotic arm motion space to obtain the dimension and the robotic arm degree of freedom matching amount; the cropping layer uses the pre-stored robotic arm physical constraints to crop the dimension and the robotic arm degree of freedom matching amount to obtain the current motion parameters of the robot's robotic arm.

[0058] The quantification formula is:

[0059] ;

[0060] in, is the query vector, is the key vector, is a value vector, ( ) is the pre-trained weight matrix for the model’s query vector, key vector, and value vector, is the environment encoding vector, is the cargo encoding vector;

[0061] The attention score calculation formula is:

[0062] in, is the attention score, is the dimension of the key vector, which is used to scale the dot product result; the mapping formula is: ;

[0063] in, is the matching quantity between the dimension and the degree of freedom of the robot arm, is the output layer weight matrix, is the output layer bias term, is a joint feature; the expression for clipping is:

[0064] ;

[0065] in, is the maximum angular velocity allowed for each joint of the robotic arm, is the clipping function of the model.

[0066] S104: construct and execute motion control instructions corresponding to the optimal motion coordination parameters to operate the goods on the conveyor belt.

[0067] Among them, the motion control instructions are specific control instructions generated based on the optimal motion coordination parameters. The motion control instructions convert the optimal motion coordination parameters into specific commands that the robot can execute, ensuring that the robotic arm completes the task according to the preset motion trajectory and parameters.

[0068] In some embodiments of the present application, specific motion control instructions are generated based on optimal motion coordination parameters. These instructions require converting abstract parameters into specific commands that the robot can execute. For example, speed parameters are converted into motor speed instructions, and joint angles are converted into target position instructions for joint motors. Based on information such as the position and size of the goods on the conveyor belt and combined with the optimal motion coordination parameters, the specific motion sequence for the robot arm to complete the task is determined. For example, the motion sequence may include moving over the goods, descending to the grasping height, grasping the goods, ascending, moving to the target position, and placing the goods. The constructed motion control instructions are sent to the robot control system, typically via a network interface or a directly connected controller. After receiving the motion control instructions, the robot control system parses the instructions and extracts the specific motion parameters. For example, the control system parses the target position coordinates, speed, acceleration, and other information that the robot arm needs to move to. Based on the parsed instructions, the robot control system controls the motors and actuators of each joint of the robot arm to execute the action according to the preset motion parameters. For example, according to the instructions, the robot arm moves over the goods on the conveyor belt, adjusts the posture of the gripper, and then grasps the goods with appropriate force.

[0069] In the embodiments of the present application, on the one hand, through multimodal perception data, the robot can obtain the operating status of the conveyor belt and detailed information about the goods in real time, thereby dynamically adjusting the historical motion parameters of the robotic arm to achieve accurate and efficient sorting operations. Accurate sorting operations can significantly improve sorting efficiency, reduce damage to goods or sorting errors caused by misoperation, and thus reduce the costs incurred by frequent maintenance and repair of equipment. On the other hand, by dynamically adjusting the historical motion parameters of the robotic arm to adapt to the real-time operating status of the conveyor belt and the characteristics of the goods, the robot can maintain stable operating performance under different working conditions. This adaptive capability can effectively avoid operation interruptions or erroneous operations caused by problems such as changes in conveyor belt speed, cargo position offset, or conveyor belt wear, thereby ensuring the stability of the entire production process.

[0070] See Figure 5 , provides a flow chart of a model training method for a pre-trained reinforcement learning model in an embodiment of the present application. Figure 5 As shown, the method of the embodiment of the present application may include the following steps:

[0071] S201, simulating different conveyor belt working conditions in the robot operation area in a simulation environment to obtain sample conveying parameters;

[0072] The simulation environment refers to one or more computer system models created using software tools. Within the simulation environment of the robot's operating area, different conveyor belt operating conditions can be simulated, including conveyor belt speed, inclination angle, and load conditions, providing a safe and controllable platform for the development and testing of robot control strategies. Sample conveyor parameters refer to the specific numerical values ​​of the conveyor belt's operating conditions simulated in the simulation environment, such as conveyor belt speed, inclination angle, and degree of wear. These parameters are used to train and validate the robot's motion parameter analysis model, helping the robot learn how to adjust motion parameters under different conveyor belt conditions.

[0073] S202, during the simulation process, randomly generate sample cargo parameters;

[0074] S203, recording the robot arm joint angular velocity sequence input by the user during the simulation process as a supervision signal;

[0075] S204, constructing a model training sample based on the supervision signal, sample transmission parameters, and sample cargo parameters;

[0076] In some embodiments of the present application, the specific process of constructing a model training sample based on a supervisory signal, sample transmission parameters, and sample cargo parameters includes: constructing a sample environment coding vector based on the sample transmission parameters; constructing a sample cargo coding vector based on the sample cargo parameters; quantizing the sample environment coding vector and the sample cargo coding vector into a sample query vector, a sample key vector, and a sample value vector, respectively; and using a supervisory signal to label the sample query vector, the sample key vector, and the sample value vector to obtain a model training sample.

[0077] For example, a robot model is imported into simulation software (such as Delmia or MATLAB), and the joint types and range of motion are configured to ensure the model matches the actual physical robot. Elements such as conveyor belts and cargo are added to the simulation environment, and initial conveyor belt parameters (such as speed and tilt angle) are set to construct a complete robot operating area. Within the simulation environment, different operating conditions are simulated by modifying conveyor belt speed, tilt angle, load, and other parameters. For example, the conveyor belt speed can be set to 0.5 m / s, 1 m / s, and 1.5 m / s, and the tilt angles can be set to 0 degrees, 5 degrees, and 10 degrees. Under each operating condition, the conveyor belt operating parameters, including speed, tilt angle, and wear, are recorded to form sample conveyor parameters. During the simulation, parameters such as cargo position, size, and weight are randomly generated to simulate the variety of cargo encountered in actual production. Within the simulation environment, user-entered robot arm joint angular velocity sequences based on the current conveyor belt conditions and cargo parameters are recorded as supervisory signals. The sample conveyor parameters, sample cargo parameters, and supervisory signals are integrated to form sample data for training the robot motion parameter analysis model.

[0078] S205, using a neural network to create and initialize an action parameter analysis model, the model including a weight matrix loading layer, a quantization layer, an attention layer, a dot product operation layer, a feature mapping layer, and a cropping layer;

[0079] S206, using the model training sample to perform machine learning on the initialized motion parameter analysis model, so that the motion parameter analysis model learns the weighted relationship between the sample transmission parameters, the sample cargo parameters, and the supervisory signal, and obtains a pre-trained weight matrix;

[0080] S207, using the model loss function of the action parameter analysis model to quantify the loss value of the pre-trained weight matrix;

[0081] Among them, the loss function of the model is:

[0082]

[0083] in, is the loss value, is the total number of samples for model training, It is The predicted action parameters of samples, It is The real action parameters represented by the supervision signal of samples; among them, is the dimension of attention score, It is The predicted value of attention score, It is The true value of the attention score, is the regularization coefficient, which is used to control the strength of regularization. is the pre-trained weight matrix, where Represents an arbitrary layer in a model.

[0084] S208: When the loss value reaches the minimum, a pre-trained motion parameter analysis model is obtained.

[0085] In the embodiments of the present application, on the one hand, through multimodal perception data, the robot can obtain the operating status of the conveyor belt and detailed information about the goods in real time, thereby dynamically adjusting the historical motion parameters of the robotic arm to achieve accurate and efficient sorting operations. Accurate sorting operations can significantly improve sorting efficiency, reduce damage to goods or sorting errors caused by misoperation, and thus reduce the costs incurred by frequent maintenance and repair of equipment. On the other hand, by dynamically adjusting the historical motion parameters of the robotic arm to adapt to the real-time operating status of the conveyor belt and the characteristics of the goods, the robot can maintain stable operating performance under different working conditions. This adaptive capability can effectively avoid operation interruptions or erroneous operations caused by problems such as changes in conveyor belt speed, cargo position offset, or conveyor belt wear, thereby ensuring the stability of the entire production process.

[0086] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0087] See Figure 6 , which shows a schematic diagram of the structure of a robot multimodal perception and motion coordinated control device provided by an exemplary embodiment of the present application. The robot multimodal perception and motion coordinated control device can be implemented as all or part of an electronic device through software, hardware, or a combination of both. The device 1 includes a sensor information acquisition module 10, a parameter analysis module 20, an action parameter adjustment module 30, and an instruction execution module 40.

[0088] The sensor information acquisition module 10 is used to acquire multiple types of sensor information as multimodal perception data according to a preset period through multiple types of sensors deployed in the robot operation area. The multimodal perception data includes lidar data, infrared sensor data, and visual sensor data. A transmission device is provided in the robot operation area.

[0089] a parameter analysis module 20 for analyzing, based on the multimodal sensing data, conveying parameters of the conveyor belt of the conveying equipment and cargo parameters of the cargo on the conveyor belt;

[0090] The motion parameter adjustment module 30 is used to dynamically adjust the historical motion parameters of the robot's manipulator arm according to the conveying parameters and the cargo parameters to obtain the optimal motion coordination parameters suitable for the conveying equipment;

[0091] The instruction execution module 40 is used to construct and execute motion control instructions corresponding to the optimal motion coordination parameters to operate the goods on the conveyor belt.

[0092] It should be noted that the robot multimodal perception and motion collaborative control device provided in the above embodiment only uses the division of the above functional modules as an example when executing the robot multimodal perception and motion collaborative control method. In actual applications, the above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the robot multimodal perception and motion collaborative control device provided in the above embodiment and the robot multimodal perception and motion collaborative control method embodiment belong to the same concept. The implementation process is detailed in the method embodiment and will not be repeated here.

[0093] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0094] In the embodiments of the present application, on the one hand, through multimodal perception data, the robot can obtain the operating status of the conveyor belt and detailed information about the goods in real time, thereby dynamically adjusting the historical motion parameters of the robotic arm to achieve accurate and efficient sorting operations. Accurate sorting operations can significantly improve sorting efficiency, reduce damage to goods or sorting errors caused by misoperation, and thus reduce the costs incurred by frequent maintenance and repair of equipment. On the other hand, by dynamically adjusting the historical motion parameters of the robotic arm to adapt to the real-time operating status of the conveyor belt and the characteristics of the goods, the robot can maintain stable operating performance under different working conditions. This adaptive capability can effectively avoid operation interruptions or erroneous operations caused by problems such as changes in conveyor belt speed, cargo position offset, or conveyor belt wear, thereby ensuring the stability of the entire production process.

[0095] The present application also provides a computer-readable medium having program instructions stored thereon, which, when executed by a processor, implement the robot multimodal perception and motion coordinated control method provided by the above-mentioned various method embodiments.

[0096] The present application also provides a computer program product containing instructions, which, when run on a computer, enables the computer to execute the robot multimodal perception and motion coordinated control method of each of the above method embodiments.

[0097] See Figure 7 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 1000 may include: at least one processor 1001 , at least one network interface 1004 , a user interface 1003 , a memory 1005 , and at least one communication bus 1002 .

[0098] The communication bus 1002 is used to implement the connection and communication between these components.

[0099] The user interface 1003 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.

[0100] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0101] The processor 1001 may include one or more processing cores. The processor 1001 utilizes various interfaces and circuits to connect various components within the electronic device 1000. It executes instructions, programs, code sets, or instruction sets stored in the memory 1005, and accesses data stored in the memory 1005 to perform various functions and process data within the electronic device 1000. Optionally, the processor 1001 may be implemented in hardware using at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 1001 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display; and the modem handles wireless communications. It is understood that the modem may also be implemented independently of the processor 1001 and implemented on a separate chip.

[0102] Among them, the memory 1005 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 1005 may also be optionally at least one storage system located away from the aforementioned processor 1001. As Figure 7 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a robot multimodal perception and motion collaborative control application.

[0103] exist Figure 7In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and obtain user input data; and the processor 1001 can be used to call the robot multimodal perception and motion coordinated control application stored in the memory 1005 and specifically perform the following operations:

[0104] Multiple types of sensor information are acquired as multimodal perception data according to preset cycles by multiple types of sensors deployed in the robot's operating area. The multimodal perception data includes lidar data, infrared sensor data, and visual sensor data. A transmission device is provided in the robot's operating area.

[0105] Analyze the conveyor belt transmission parameters of the conveyor equipment and the cargo parameters of the cargo on the conveyor belt based on multimodal sensing data;

[0106] Dynamically adjust the robot's arm's historical motion parameters based on the conveying parameters and cargo parameters to obtain the optimal motion coordination parameters suitable for the conveying equipment;

[0107] Construct and execute motion control instructions corresponding to the optimal motion coordination parameters to operate the goods on the conveyor belt.

[0108] In one embodiment, when dynamically adjusting the historical motion parameters of the robot's manipulator arm based on the transport parameters and cargo parameters to obtain optimal motion coordination parameters suitable for the transport equipment, the processor 1001 specifically performs the following operations:

[0109] The running speed and tilt angle are converted into frequency domain features through dynamic Fourier transform;

[0110] Based on the wear data, a wear heat map of the conveyor belt is constructed;

[0111] The frequency domain features are fused with the wear heat map to obtain the environmental coding vector;

[0112] The attention mechanism is used to encode the center coordinates, size, and relative position to generate a cargo encoding vector for each cargo;

[0113] Input the environment encoding vector and the cargo encoding vector into the pre-trained motion parameter analysis model to output the current motion parameters of the robot's manipulator arm;

[0114] When the current motion parameters are inconsistent with the historical motion parameters of the robot's manipulator, the historical motion parameters are replaced with the current motion parameters to obtain the optimal motion coordination parameters suitable for the conveying device.

[0115] In one embodiment, when the processor 1001 converts the running speed and the tilt angle into frequency domain features through dynamic Fourier transform, it specifically performs the following operations:

[0116] Filter, denoise, and normalize the time-varying running speed and tilt angle to obtain the pre-processed time domain signal of the transmission equipment;

[0117] Use dynamic Fourier transform expression to convert the time domain signal of the transmission device into a frequency domain signal to obtain the spectrum data of the transmission device;

[0118] Identify the main frequency components, the amplitude of each frequency component, and the phase of each frequency component in the spectrum data of the transmitting device as frequency domain information of the running speed and tilt angle;

[0119] The frequency domain information of running speed and tilt angle are fused to obtain frequency domain features.

[0120] In one embodiment, when the processor 1001 constructs a wear heat map of the conveyor belt based on the wear data, the processor 1001 specifically performs the following operations:

[0121] Extract the wear depth of the conveyor belt, the temperature change of the worn area, and the reflectivity change of the worn area from the wear data;

[0122] The surface of the conveyor belt is divided into multiple small grids based on preset grid division parameters, and each small grid corresponds to a wear data point;

[0123] The wear depth of the conveyor belt, the temperature change of the wear area, and the reflectivity change of the wear area are normalized and mapped to each small grid to form a wear data matrix;

[0124] By performing spatial smoothing on the wear data matrix, a raster layer representing the degree of wear is generated;

[0125] According to the size of the wear data in the wear data matrix, different color shades are assigned to the raster layer to obtain the wear heat map of the conveyor belt.

[0126] In one embodiment, when the processor 1001 fuses the frequency domain features with the wear heat map to obtain the environment coding vector, it specifically performs the following operations:

[0127] Extract spatial information of wear heat map;

[0128] Mapping the spatial information of the wear thermogram onto the frequency domain features, so as to splice the frequency domain features with the spatial information of the wear thermogram to obtain spliced ​​features;

[0129] Encode the concatenated features to obtain the environment encoding vector.

[0130] In one embodiment, when the processor 1001 inputs the environment code vector and the cargo code vector into the pre-trained motion parameter analysis model and outputs the current motion parameters of the robot's manipulator arm, the processor 1001 specifically performs the following operations:

[0131] The weight matrix loading layer loads the pre-trained weight matrix;

[0132] The quantization layer uses the pre-trained weight matrix to quantize the environment encoding vector and the cargo encoding vector to obtain the query vector, key vector, and value vector;

[0133] The attention layer calculates an attention score based on the query vector, key vector, and value vector. The attention score is used to measure the importance of the environment encoding vector to the item encoding vector.

[0134] The dot product operation layer performs dot product operations on the environment encoding vector and the cargo encoding vector with the attention score and sums them up to obtain a joint feature containing the interaction information between the environment and the cargo;

[0135] The feature mapping layer maps the joint features to the preset robotic arm motion space to obtain the matching quantity of the dimension and the robotic arm's degrees of freedom;

[0136] The clipping layer uses the pre-existing physical constraints of the robot arm to clip the dimensions and the matching amount of the robot arm's degrees of freedom to obtain the current motion parameters of the robot's robot arm.

[0137] In one embodiment, when executing the generation of the pre-trained motion parameter analysis model, the processor 1001 specifically performs the following operations:

[0138] Simulate different conveyor belt working conditions in the robot operation area in the simulation environment to obtain sample conveying parameters;

[0139] During the simulation, sample cargo parameters are randomly generated;

[0140] Record the robot arm joint angular velocity sequence input by the user during the simulation as a supervision signal;

[0141] Construct model training samples based on supervision signals, sample transmission parameters, and sample cargo parameters;

[0142] A neural network is used to create and initialize an action parameter analysis model, which includes a weight matrix loading layer, a quantization layer, an attention layer, a dot product operation layer, a feature mapping layer, and a cropping layer.

[0143] Using model training samples to perform machine learning on the initialized motion parameter analysis model, so that the motion parameter analysis model learns the weighted relationship between the sample transmission parameters, the sample cargo parameters, and the supervision signal, and obtains a pre-trained weight matrix;

[0144] The model loss function of the action parameter analysis model is used to quantify the loss value of the pre-trained weight matrix;

[0145] When the loss value reaches the minimum, the pre-trained motion parameter analysis model is obtained.

[0146] In one embodiment, the processor 1001 performs the following operations when constructing a model training sample based on the supervisory signal, the sample transmission parameters, and the sample cargo parameters:

[0147] Constructing a sample environment encoding vector based on the sample transmission parameters;

[0148] Construct a sample cargo encoding vector based on the sample cargo parameters;

[0149] Quantize the sample environment encoding vector and the sample cargo encoding vector into a sample query vector, a sample key vector, and a sample value vector respectively;

[0150] The sample query vector, sample key vector, and sample value vector are labeled using supervisory signals to obtain model training samples. The loss function of the model is:

[0151]

[0152] in, is the loss value, is the total number of samples for model training, It is The predicted action parameters of samples, It is The real action parameters represented by the supervision signal of samples; among them, is the dimension of attention score, It is The predicted value of attention score, It is The true value of the attention score, is the regularization coefficient, which is used to control the strength of regularization. is the pre-trained weight matrix, where Represents an arbitrary layer in a model.

[0153] In the embodiments of the present application, on the one hand, through multimodal perception data, the robot can obtain the operating status of the conveyor belt and detailed information about the goods in real time, thereby dynamically adjusting the historical motion parameters of the robotic arm to achieve accurate and efficient sorting operations. Accurate sorting operations can significantly improve sorting efficiency, reduce damage to goods or sorting errors caused by misoperation, and thus reduce the costs incurred by frequent maintenance and repair of equipment. On the other hand, by dynamically adjusting the historical motion parameters of the robotic arm to adapt to the real-time operating status of the conveyor belt and the characteristics of the goods, the robot can maintain stable operating performance under different working conditions. This adaptive capability can effectively avoid operation interruptions or erroneous operations caused by problems such as changes in conveyor belt speed, cargo position offset, or conveyor belt wear, thereby ensuring the stability of the entire production process.

[0154] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program for multimodal perception and coordinated motion control of the robot can be stored in a computer-readable storage medium. When executed, the program can include the processes of the above-described method embodiments. The storage medium for the program for multimodal perception and coordinated motion control of the robot can be a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0155] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A robot multimodal perception and motion coordinated control method, characterized in that: Applied to a robot, the method comprises: Acquiring multiple types of sensor information as multimodal perception data according to a preset period through multiple types of sensors deployed in the robot operation area, wherein a transmission device is provided in the robot operation area; wherein the multimodal perception data includes lidar data, infrared sensor data, and visual sensor data; Analyzing, based on the multimodal sensing data, conveying parameters of a conveyor belt of the conveying equipment and cargo parameters of cargo on the conveyor belt; Dynamically adjust historical motion parameters of the robot's manipulator arm according to the conveying parameters and the cargo parameters to obtain optimal motion coordination parameters suitable for the conveying equipment; The conveying parameters include the running speed, inclination angle and wear data of the conveyor belt; the cargo parameters include the center coordinates, size and relative position of the cargo on the conveyor belt to the edge of the conveyor belt; The dynamically adjusting the historical motion parameters of the robot's manipulator arm according to the conveying parameters and the cargo parameters to obtain the optimal motion coordination parameters suitable for the conveying equipment includes: Converting the running speed and tilt angle into frequency domain features through dynamic Fourier transform; constructing a wear heat map of the conveyor belt based on the wear data; Fusing the frequency domain features with the wear thermodynamic map to obtain an environmental coding vector; The center coordinates, size, and relative position are encoded using an attention mechanism to generate a cargo encoding vector for each cargo; Inputting the environment encoding vector and the cargo encoding vector into a pre-trained motion parameter analysis model, and outputting the current motion parameters of the robot's manipulator arm; In the case where the current motion parameters are inconsistent with the historical motion parameters of the robot's manipulator, the historical motion parameters are replaced with the current motion parameters to obtain the optimal motion coordination parameters suitable for the conveying device; wherein, The step of fusing the frequency domain features with the wear thermogram to obtain an environment coding vector includes: Extracting spatial information of the wear thermodynamic map; Mapping the spatial information of the wear thermogram onto the frequency domain features, so as to splice the frequency domain features with the spatial information of the wear thermogram to obtain spliced ​​features; Encoding the splicing features to obtain an environment encoding vector; Construct and execute motion control instructions corresponding to the optimal motion coordination parameters to operate on the goods on the conveyor belt.

2. The method according to claim 1, characterized in that The converting the running speed and the tilt angle into frequency domain features through dynamic Fourier transform comprises: Filtering, denoising and normalizing the running speed and tilt angle that change with time to obtain a preprocessed time domain signal of the transmission equipment; Convert the transmission device time domain signal into a frequency domain signal using a dynamic Fourier transform expression to obtain transmission device spectrum data; Identifying the main frequency components, the amplitude of each frequency component, and the phase of each frequency component in the spectrum data of the transmitting device as frequency domain information of the running speed and tilt angle; The frequency domain information of the running speed and the tilt angle are fused to obtain frequency domain features.

3. The method according to claim 1, characterized in that The step of constructing a wear heat map of the conveyor belt according to the wear data includes: extracting the wear depth of the conveyor belt, the temperature change of the worn area, and the reflectivity change of the worn area from the wear data; Dividing the surface of the conveyor belt into a plurality of small grids based on preset grid division parameters, each small grid corresponding to a wear data point; Normalizing the wear depth of the conveyor belt, the temperature change of the wear area, and the reflectivity change of the wear area, and mapping them onto each small grid to form a wear data matrix; Generate a raster layer representing the degree of wear by performing spatial smoothing processing on the wear data matrix; According to the size of the wear data in the wear data matrix, different color shades are assigned to the grid layer to obtain a wear heat map of the conveyor belt.

4. The method according to claim 1, wherein The pre-trained motion parameter analysis model includes a weight matrix loading layer, a quantization layer, an attention layer, a dot product operation layer, a feature mapping layer, and a cropping layer; Inputting the environment coding vector and the cargo coding vector into a pre-trained motion parameter analysis model to output current motion parameters of the robot's manipulator arm includes: The weight matrix loading layer loads the pre-trained weight matrix; The quantization layer uses a pre-trained weight matrix to quantize the environment encoding vector and the cargo encoding vector respectively to obtain a query vector, a key vector, and a value vector; The attention layer calculates an attention score based on the query vector, the key vector, and the value vector; the attention score is used to measure the importance of the environment encoding vector to the cargo encoding vector; The dot product operation layer performs dot product operations on the environment encoding vector and the cargo encoding vector with the attention score respectively and sums them to obtain a joint feature containing interaction information between the environment and the cargo; The feature mapping layer maps the joint features to a preset robotic arm motion space to obtain a matching amount of dimensions and robotic arm degrees of freedom; The clipping layer uses pre-stored physical constraints of the robot arm to clip the dimension and the matching amount of the robot arm's degree of freedom to obtain the current motion parameters of the robot's robot arm.

5. The method according to claim 4, characterized in that The quantization formula of the quantization layer is: ； in, is the query vector, is the key vector, is a value vector, ( ) is the pre-trained weight matrix for the model’s query vector, key vector, and value vector, is the environment encoding vector, is the cargo encoding vector; The attention score calculation formula is: in, is the attention score, is the dimension of the key vector, which is used to scale the dot product result; the mapping formula is: ; in, is the matching quantity between the dimension and the degree of freedom of the robot arm, is the output layer weight matrix, is the output layer bias term, is a joint feature; the expression for clipping is: ； in, is the maximum angular velocity allowed for each joint of the robotic arm, is the clipping function of the model.

6. The method according to any one of claims 2 to 5, characterized in that: The following steps are used to generate a pre-trained motion parameter analysis model, including: Simulating different conveyor belt working conditions in the robot operation area in a simulation environment to obtain sample conveying parameters; During the simulation, sample cargo parameters are randomly generated; Record the robot arm joint angular velocity sequence input by the user during the simulation as a supervision signal; Constructing a model training sample according to the supervision signal, the sample transmission parameter, and the sample cargo parameter; A neural network is used to create and initialize an action parameter analysis model, which includes a weight matrix loading layer, a quantization layer, an attention layer, a dot product operation layer, a feature mapping layer, and a cropping layer. Using the model training samples to perform machine learning on the initialized action parameter analysis model, so that the action parameter analysis model learns the weighted association relationship between the sample transmission parameters, the sample cargo parameters, and the supervisory signal, and obtains a pre-trained weight matrix; quantifying the loss value of the pre-trained weight matrix by using a model loss function of the action parameter analysis model; When the loss value reaches the minimum, the pre-trained motion parameter analysis model is obtained.

7. The method according to claim 6, characterized in that The constructing of a model training sample according to the supervisory signal, the sample transmission parameter, and the sample cargo parameter includes: constructing a sample environment coding vector according to the sample transmission parameters; Constructing a sample cargo encoding vector according to the sample cargo parameters; quantizing the sample environment code vector and the sample cargo code vector into a sample query vector, a sample key vector, and a sample value vector, respectively; The sample query vector, sample key vector, and sample value vector are labeled using the supervisory signal to obtain a model training sample; wherein the loss function of the model is: in, is the loss value, is the total number of samples for model training, It is The predicted action parameters of samples, It is The real action parameters represented by the supervision signal of samples; among them, is the dimension of attention score, It is The predicted value of attention score, It is The true value of the attention score, is the regularization coefficient, which is used to control the strength of regularization. is the pre-trained weight matrix, where Represents an arbitrary layer in a model.

8. A robot multimodal perception and motion coordinated control device implemented using the method according to any one of claims 1 to 7, characterized in that: The device comprises: a sensor information acquisition module, configured to acquire, as multimodal perception data, a plurality of types of sensor information from a plurality of types of sensors deployed in a robot operating area according to a preset period, wherein the multimodal perception data includes lidar data, infrared sensor data, and visual sensor data; and a parameter analysis module, configured to analyze, based on the multimodal sensing data, conveying parameters of the conveyor belt of the conveying equipment and cargo parameters of the cargo on the conveyor belt; an action parameter adjustment module, configured to dynamically adjust historical action parameters of the robot's manipulator arm according to the conveying parameters and the cargo parameters, to obtain optimal motion coordination parameters suitable for the conveying equipment; The instruction execution module is used to construct and execute the motion control instructions corresponding to the optimal motion coordination parameters to operate the goods on the conveyor belt.

Images