211 results about "Robot kinematics" patented technology

The invention relates to the technical field of path planning, and discloses a robot path planning method, which comprises the following steps that a hybrid cost map is constructed, and the hybrid cost map comprises a static obstacle field, a dynamic obstacle probability prediction field and a terrain energy consumption field; constructing an anisotropic heuristic function, and searching on the mixed cost map to obtain an initial path; parameterizing the initial path into a group of piecewise polynomial curves, constructing a joint optimization target which takes the total curvature, the piecewise polynomial curves and the space-time overlapping integral of a dynamic obstacle probability prediction field, and performing trajectory optimization under the condition of meeting the kinematics constraint of the robot to obtain a smooth space-time trajectory; when the task is executed, the information divergence is continuously calculated, and when the information divergence exceeds a preset threshold value, the current state of the robot serves as a new starting point, and the complete path planning process is executed again. According to the method, the future collision risk is prospectively avoided, the energy consumption of the robot is considered, and the comprehensive quality of the path is improved from the source.

The invention discloses a covariance self-adaptive control method for a quadruped robot for spine joint adjustment. The method comprises the following steps that 1, a spine quadruped robot kinematics model with spine joints, a robot trunk dynamics model with the spine joints and a robot leg multi-rigid-body dynamics model are established; 2, based on the energy stability margin theorem and the spine joint coupling effect, a posture control method of the spine joint quadruped robot is provided by combining a covariance self-adaptive evolutionary strategy, and the angle, rigidity and damping coefficient of spine joints are estimated and adjusted according to the terrain gradient; 3, extracting the spatial relationship between the trunk mass center and the foot ends of the robot as an expected reference, updating the expected mass center position in real time, and constructing a model prediction control method to control the posture of the spinal joint quadruped robot; 4, the position of the foot falling point of the swing leg is designed in advance in combination with the additional freedom degree provided by the spine joint and MPC prediction state information. The motion stability of the robot in the complex terrain is improved.

The invention relates to the technical field of robot control, and particularly discloses a robot indoor moving path planning method and system. An environment sensing module collects environment data through a multi-mode sensor array and constructs a three-dimensional semantic map; the dynamic decision-making module generates a candidate path set based on the three-dimensional semantic map data output by the environment perception module, and transmits a decision-making result to the path optimization module through a priority arbitration mechanism; the path optimization module performs multi-objective optimization according to candidate paths of the robot kinematics constraint and dynamic decision module; the execution control module is used for converting an optimized path into a bottom layer driving instruction through kinematics calculation and feeding back an execution state to the dynamic decision-making module in real time to form closed-loop control. A dynamic environment modeling mechanism is adopted, through variable resolution perception and semantic map construction, a layered decision-making framework and multi-target collaborative optimization are adopted, and the dynamic environment modeling mechanism is optimized; and the adaptive control system is selected to significantly improve the environmental adaptability and reliability of the system.

The invention provides a robot positioning prediction method and device, a medium and equipment, and relates to the field of industrial robot motion control. According to the method, on one hand, a physical-data double-branch collaborative network is designed, meanwhile, a DH parameter method is used for predicting and obtaining a theoretical position coordinate of a target robot, an improved Transform network is used for predicting and obtaining a coordinate compensation value of the target robot, so that a theoretical value and the compensation value are decoupled, and the limitation of single model establishment is avoided; and then, the two are fused, and high-precision end point location prediction for the target robot is realized by combining priori knowledge of robot kinematics and nonlinear expression ability of deep learning. And on the other hand, a spatial physical information mixed loss function is provided, and a spatial topological structure output by the DH model is used to guide the distribution of predicted values of the whole position prediction model, so that the geometric rationality of target robot end point location prediction is improved.

The invention relates to the technical field of robot navigation, and discloses an automatic path-finding intelligent robot and a path-finding method thereof. According to the method, multi-dimensional spatial data including obstacle distribution, ground features and dynamic target motion information is acquired through an environment sensing device, spatial discretization processing is performed on the data, and a topological map which has a hierarchical structure and includes a node connection relation and a region passing weight is generated. Then, according to the node connection relation of the topological map, a path planning model based on manifold learning is established, and an optimal path candidate set is determined by calculating the manifold distance between adjacent nodes; then extracting a key node sequence in the candidate set, dynamically optimizing a path in combination with a region passing weight, and generating a preliminary navigation track; and finally, inputting the initial track into a motion control model, carrying out smooth processing according to kinematics constraint of the robot, and outputting a final execution path. According to the invention, the robot can efficiently find the way in a complex environment.

The invention discloses an assemblage multi-robot collaborative welding system in ship sections, and relates to the technical field of industrial robot collaborative control, the assemblage multi-robot collaborative welding system comprises an industrial internet platform, the industrial internet platform is in communication connection with the following modules: a visual inspection modeling module, which is used for scanning the surface of a workpiece through a 3D camera, constructing a point cloud model, and establishing a point cloud model; matching the point cloud model with a design drawing, and identifying a deviation between an actual workpiece and the drawing; and the path planning and correcting module is used for planning a collision-free welding path in combination with the visual detection data and the motion range of the robot. Through high-precision visual inspection scanning and an SLAM matching algorithm, a workpiece point cloud model is constructed in real time and is accurately aligned with a design drawing, the welding seam position and the workpiece geometric deviation can be accurately recognized, a collision-free welding path is generated in combination with robot kinematics constraint and a safety boundary, it is ensured that the pose of the tail end of a welding gun is attached to the surface of the workpiece, and the welding quality is improved. Welding deviation is reduced, and process stability is ensured.

The invention provides a configuration optimization method before capturing of a space double-arm robot, and the method comprises the steps: constructing a kinematics and dynamics model of the space double-arm robot according to a three-dimensional model of the space double-arm robot; constructing a kinetic model of the to-be-captured target by using the motion parameters of the to-be-captured target in the space; according to the dynamic model of the space double-arm robot, the relative mass of the space double-arm robot under non-centering collision is obtained, and a maximum pre-contact collision force performance evaluation index is constructed; further constructing a double-arm closed chain inertia matching index performance evaluation index; constructing a configuration optimization model before the space double-arm robot captures the target; and through a multi-target optimization algorithm, the optimal configuration of the space double-arm robot before capturing the target is solved. The pre-contact collision force peak value of the double-arm robot during target capturing can be effectively reduced, and meanwhile the operability of a closed chain system formed after capturing is guaranteed.

The invention discloses a physical man-machine interaction contact safety protection method and system based on a Maxwell model, and the method comprises the steps: building a human body physical model, and expressing the effective mass, elastic coefficient and maximum allowable impact force physical attributes of each part of a human body; kinematics, statics and dynamics models of the robot are established, and the relation between the joint space displacement, speed, torque and mass of the robot and the tail end speed, tail end rigidity and effective mass is expressed; a physical man-machine interaction model is established, and the relation between the joint displacement, speed and mass of the robot and the physical man-machine interaction impact force is expressed; physical man-machine interaction contact characteristics are calculated, and impact duration, the maximum impact force, a contact damping coefficient and a recovery coefficient are calculated; and 5, robot safety protection measures are formulated, and the safety of physical man-machine interaction is guaranteed. According to the method, the solution of a Hertz contact stiffness parameter is avoided, the safety of physical man-machine interaction can be evaluated through the impact duration and the maximum impact force, and a theoretical basis is provided for the design of a robot man-machine interaction system.

The invention discloses a trajectory reduction and map reconstruction method capable of solving a time drift error and application, and relates to the technical field of data processing. The method comprises the following steps: constructing a robot kinematics model constraint, fusing real-time synchronous data streams of multiple sensors, and combining a dynamic sensor weight distribution function and Kalman filtering to generate an initial motion track; adjusting the trajectory based on dead reckoning and a pipeline kinematics model in a non-feature region, matching pipeline feature points through an ICP algorithm to calculate a pose deviation, and correcting a trajectory error by using Kalman filtering; iMU, vision and pipeline geometric constraints are optimized and fused through a sliding window graph, and an optimal track is obtained; generating a three-dimensional digital twinborn model with defect mapping in combination with laser point cloud and Poisson reconstruction; according to the method, the problems of large sensor accumulative error, feature deficiency positioning failure and low modeling precision are solved, high-precision positioning in a complex pipeline environment is realized, the detection efficiency is remarkably improved, and the manual dependence is reduced.

The invention discloses a multi-constraint path planning method and device based on intelligent heuristic search, and is suitable for a mobile robot in a complex and narrow environment. According to the method, a multi-source sensor is used for environment perception and fusion mapping, and a grid map is generated; a neural heuristic search algorithm based on a Mama structure and a selective state space model (Selective SSM) is adopted on the cost map, and a guide graph is generated; a continuous and collision-free initial path is generated through self-adaptive step size control in combination with robot kinematics constraints and channel characteristics; further establishing a multi-target constraint model based on a robot center Euclidean symbol distance field (RC-ESDF), and executing path smoothing and safety optimization by using a priori-guided particle swarm optimization algorithm (Prior-guided PSO) to obtain an optimal path meeting dynamic consistency; the system corrects the path in real time through a closed-loop feedback mechanism and triggers dynamic re-planning, so that the path feasibility, safety and real-time adaptive capacity of the robot in a limited environment are remarkably improved.

The invention relates to the technical field of spraying control, and discloses a wind power tower wall surface spraying method and system, and the method comprises the steps: scanning a wind power tower wall surface through a sensor, obtaining the surface contour data of the wind power tower wall surface, and forming reference geometric information; curvature distribution characteristics of the reference geometric information are extracted, key path points are extracted according to the curvature distribution characteristics, robot kinematics constraints are introduced to carry out smoothing and coverage completeness integrated processing on the key path points, and therefore a spraying path is generated; establishing a mapping relation between the curvature of the spraying path and nozzle parameters, and setting initial values of atomization pressure and coating flow based on the mapping relation; in the spraying execution process, the real-time thickness monitoring data and the preset thickness are fused to generate a parameter adjusting instruction, and an actuator is driven to conduct online correction on spraying parameters; according to the method, accurate adaptation of the spraying path and the outline of the tower drum and dynamic matching of spraying parameters are achieved, it is ensured that the thickness of the coating always fits the preset standard, and the stability and controllability of the spraying process are improved.

The invention provides a ship active stabilization control method and system, and relates to the field of ships. The ship active stabilization control system comprises a stabilization execution mechanism which comprises an upper platform constructed according to a ship body of a ship and a lower platform constructed according to multiple sets of water pressing plate execution mechanisms, and the multiple sets of water pressing plate execution mechanisms are arranged at any positions below waterlines of the two sides of the ship correspondingly; each water pressing plate executing mechanism comprises a water pressing plate assembly and a driving cylinder connected with the water pressing plate assembly; the data acquisition unit is configured to acquire ship body attitude and motion state information and environment data of the ship; the stabilization controller is configured to calculate and generate correction torque according to the parallel robot kinematic mechanics mechanism model, the ship posture and motion state information and the environment data, and distribute the correction torque to an execution instruction corresponding to each driving cylinder; and the driving cylinders drive the corresponding water pressing plate assemblies to execute water pressing movement or return movement according to the execution instructions.

The invention discloses an industrial robot grinding and polishing compliance control method for reinforcement learning optimization. The method comprises the steps that S1, a five-degree-of-freedom industrial robot kinematics model is established; s2, an environment is created, an impedance control module is built, and expected polishing force and an expected position are set; s3, creating an intelligent agent, and setting a DQN hyper-parameter by adopting a DQN algorithm; s4, the action of the intelligent agent is output to a damping parameter Bd port of the impedance controller, and the damping parameter of the impedance controller is optimized; s5, training options, the maximum training turn number and the like are set, training is started, and corresponding agents are stored; and S6, the trained intelligent agent is used, and constant-force grinding of the robot is completed. According to the method, the optimal impedance parameter can be determined according to the environment, the relation between the overshoot and the adjustment time is balanced, it is guaranteed that the adjustment time is short while the overshoot is restrained, and grinding and polishing compliance control over the industrial robot is completed.

The invention discloses an industrial robot tail end vibration response calculation method and system, solves the problem of how to improve industrial robot tail end dynamic response prediction accuracy, and belongs to the field of industrial robot dynamics analysis. According to the method, elements of the industrial robot are regarded as rigid bodies or flexible bodies, all assemblies are integrated into a rigid-flexible coupling dynamic model in combination with a robot kinematic model, step-by-step parameter identification is conducted on the decoupled rigid body dynamic model and structural dynamic model, inertial parameters are identified by considering the influence of nonlinear joint friction, and the method is suitable for industrial robot control. Further identifying a rotational inertia matrix in the robot milling kinetic model; joint elastic parameters are identified by considering the influence of non-linear joint rigidity, and a damping matrix in a robot milling kinetic model is further identified. And the torque fluctuation of each joint is predicted, the vibration response of each joint is calculated in combination with a robot milling kinetic model, and finally the robot tail end vibration response is calculated in combination with robot kinematics.

The invention discloses a robot tracking control method, system and device adapted to an irregular discrete trajectory, and a storage medium, and belongs to the technical field of robots, the method comprises the following steps: dynamically selecting discrete points according to the change of a reference trajectory to quantify the curvature change of the reference trajectory; according to the curvature change of the reference trajectory, dynamically selecting an appropriate prediction step length; discretizing the robot kinematic model to obtain a robot discrete kinematic model; defining a trajectory tracking error function, and constructing a trajectory tracking target function in combination with the robot discrete kinematics model; and solving the target function, obtaining a robot trajectory tracking controller, and enabling the robot to move according to a reference trajectory, thereby completing a trajectory tracking target. The robot trajectory tracking control method with the dynamic adaptive capacity is provided for discrete reference trajectories which change irregularly, the robot trajectory tracking precision and the driving efficiency can be improved, and the robot trajectory tracking control method has important practical value.

The invention relates to an autonomous obstacle avoidance control method, device and equipment for an underground cable intelligent laying robot and a medium. The method comprises the following steps: acquiring underground environment data through a multi-source sensor array, performing multi-modal fusion on the position, shape and material information of an obstacle in the underground environment data, and generating an environment sensing model; classifying and identifying the obstacles based on the environment perception model, and outputting a classification result containing threat levels; wherein categories in classification identification comprise rigid obstacles, flexible obstacles and dynamic obstacles; according to a classification result and a robot kinematics constraint condition, generating an obstacle avoidance feasibility boundary; and generating a self-adaptive obstacle avoidance strategy based on the obstacle avoidance feasibility boundary, wherein the self-adaptive obstacle avoidance strategy is used for indicating the laying robot to output a control instruction. By adopting the method, the intelligent laying robot can make high-precision obstacle avoidance decisions on complex and diversified obstacles in an unstructured underground space.

The invention discloses a robot rigidity modeling method and system based on joint module nonlinear rigidity parameter fitting, and the method comprises the steps: modeling the torsional rigidity of a joint module, and expressing the relation between the torsional angle of the joint module and the joint torque through a cubic polynomial function; modeling a robot rigidity model, and expressing a relation between a joint space rigidity matrix and a robot tail end rigidity matrix through robot kinematics; identifying a torsion angle and a torque of a joint module, and acquiring the torque of the joint module through a torsion sensor or based on joint current identification; a static flexibility experiment is conducted, and the corresponding relation between the tail end force vector and the tail end elastic deformation vector of the robot is obtained; non-linear stiffness parameter fitting identification is carried out, and non-linear stiffness parameter identification is realized through a least square problem model; according to the method, non-linear rigidity parameter identification of the joint module is achieved, whole machine rigidity modeling is completed on the basis, and a basis is provided for rigidity compensation of a robot and improvement of positioning precision.

The invention discloses an agricultural robot trajectory planning method and system with optimal energy consumption under collision constraint. The method comprises the following steps: acquiring historical state sequence data and environmental map information of a target machine and peripheral machines in real time by a self-machine; identifying the operation intention of the target machine, and outputting a multi-modal prediction trajectory; the method comprises the following steps of: generating a global coarse reference track covering an operation area by taking completion of an own operation task as a main target; generating a space-time safety corridor by adopting a space-time inheritance strategy, and reserving a safety space for the dynamic obstacle; constructing an optimization problem cost function, and comprehensively considering reference trajectory cost, collision risk cost and energy consumption cost; and under the boundary constraint of the safety corridor and the kinematics constraint of the robot, solving the optimization problem by using a numerical optimization solver to obtain an optimized local trajectory. According to the method, safe, efficient and energy-saving collaborative operation can be realized by fusing multi-modal trajectory prediction of the dynamic obstacle, refining energy consumption modeling and risk perception planning.

The invention discloses a motion state determination method, system and device based on human upper limb inverse kinematics and a medium, and relates to the field of robot kinematics control, and the method comprises the steps: obtaining an upper limb working space of a human upper limb; according to the upper limb working space, discrete sampling is carried out based on a Monte Carlo sampling method, and an initial discrete point set is determined; performing sampling density adjustment on the sensitive area, the boundary area and the singular area in the initial discrete point set to obtain a final discrete point set; determining a joint angle set corresponding to each discrete point, and constructing a joint angle database; retrieving a joint angle database based on a kd-tree method, and determining an initial joint angle; local optimization is carried out based on a gradient descent method, and a joint angle corresponding to each tail end position is determined; determining the motion state of the upper limbs of the human body; the joint angle corresponding to each tail end pose of the human upper limb can be quickly and accurately determined, and then the current motion state of the human upper limb is determined.

The invention discloses an industrial robot motion pose error calibration method based on constraint solution. The industrial robot motion pose error calibration method comprises the steps that S1, joint angles, tail end coordinates and temperature data of multiple measurement points are collected; s2, correcting the kinematic model by using the thermal elongation and constructing a parameter identification model; s3, constructing a mixed integer nonlinear constraint pool according to mechanical limitation; s4, introducing a semi-definite programming slack variable to convert the non-convex constraint into a linear matrix inequality; s5, combining a primal-dual interior point method and an alternating direction multiplier method to iteratively solve the optimal kinematics parameter correction; s6, introducing a square root information filtering improvement mechanism to correct a state vector square root factor, and calculating an updated nominal kinematics parameter; and S7, based on the condition number of the error Jacobian matrix, judging observability and repeatedly calibrating. According to the method, the problem of precision reduction caused by thermal errors is solved, and the identification precision, the numerical stability and the self-adaptive calibration capability of the kinematics parameters of the robot are improved.

The invention relates to the technical field of agricultural automation, in particular to a robot pollination and picking method based on artificial intelligence machine position adjustment, which comprises the following steps: (1) a target detection stage; (2) a machine position adjusting stage; and (3) a mechanical arm control stage: implementing a hierarchical motion control strategy to finish accurate pollination or picking operation. A cross-modal collaborative decision-making system is adopted, visual perception information and a multi-dimensional decision-making architecture of mechanical arm motion constraints are fused, and a target detection result, robot kinematics parameters and mechanical arm working space constraints are unified in a three-dimensional task space for optimization calculation; the system establishes a mapping model from a visual feature space to a motion space to realize integrated decision of observation quality evaluation and motion planning.

The invention discloses a robot joint positioning method and device and a storage medium, and belongs to the field of robot control. The method comprises the steps of performing article recognition on environment image data to obtain article data, and inputting the article data into a robot motion learning model in combination with joint pose data to perform coordinate transformation calculation to obtain relative position data of an article and a joint; according to the relative position data and robot executable motion data obtained by a joint power parameter table of the robot, adjusting a joint power adjusting coefficient of the robot to generate a robot joint control parameter, and performing dynamic smooth calibration on the joint control parameter to obtain a variable structure control signal; acquiring an impedance control signal according to the object contact force data and the variable structure control signal; carrying out weight merging on the impedance control signal and the variable structure control signal to obtain a joint driving signal; and the joint driving signal is sent to the robot, so that the robot executes corresponding operation according to the joint driving signal.

The invention discloses an inverse kinematics method for a redundant seven-degree-of-freedom robot. The method specifically comprises the following steps: step 1, establishing forward kinematics modeling of the seven-degree-of-freedom robot; step 2, inputting a target pose, and defining a tail end pose error of the seven-degree-of-freedom robot as a Riemannian distance on a three-dimensional special Euclidean group manifold; 3, adopting a three-stage hybrid optimization strategy; 4, carrying out constraint processing between joints; and 5, verifying convergence and robustness. The adaptive differential evolution algorithm is used as a core optimizer, parameters of the adaptive differential evolution algorithm can be dynamically adjusted along with an iteration process, and global exploration and local development capabilities are balanced; and a dynamic parameter adjustment strategy of nonlinear progressive increase of a scaling factor and linear progressive decrease of a crossover probability is designed, and a DE / current-to-pbest variation strategy is adopted, so that the algorithm pays attention to global exploration in the initial stage of iteration and is biased to fine development in the later stage, and the convergence speed and the global search capability are effectively improved.

The application discloses a double-robot path planning method for rocket tank weld detection based on binocular vision, which comprises the following steps: firstly, a measuring instrument and a weighted moving average filter are used to calibrate the kinematic parameters of a robot; then, a Monte Carlo method is used to solve the double-robot collaborative space, so as to determine the range of the double-robot collaborative working interval and the rocket tank products that can be detected; the running area of each joint is determined by using the collaborative working space, the running range of each joint is reset, the double-vision technology is used to reduce the clamping error of the rocket tank to be detected and identify the model of the rocket tank to be detected; in addition, a polynomial interpolation method is used to interpolate and plan the double-robot path, so as to automatically generate the rocket tank path trajectory and the trajectory number corresponding to the model; finally, the communication between the robot control system and the detection software is established, so as to realize the automatic calling of the rocket tank detection path trajectory and the automatic process detection.

The application discloses a hybrid robot kinematics analysis method, which is applied to a hybrid robot and comprises the following steps: acquiring the position coordinates of key points in parallel structure parts in an upper motion platform and a lower motion platform based on a base coordinate system by using a geometric analysis method; acquiring the rotation angles of upper universal joints and lower universal joints; establishing DH models of series structure parts in the upper motion platform and the lower motion platform based on the position coordinates and the rotation angles; determining DH parameter tables of upper models and lower models based on the DH models; solving the pose of a terminal sleeve and a puncture needle based on the DH parameter tables, and obtaining a conversion matrix of any point on the puncture needle relative to the base coordinate system. The application can realize the technical effect that kinematics analysis of the hybrid robot is more efficient, and thus solves the problem that the kinematics analysis method in the prior art is not applicable to kinematics analysis of the hybrid robot.

This invention discloses a method, system, device, and storage medium for generating three-dimensional cutting trajectories, comprising the following processes: acquiring a CAD model of the finished product, extracting cutting trajectory lines, and generating a three-dimensional point cloud of the CAD model; scanning the workpiece to be cut to generate a three-dimensional point cloud of the workpiece; registering the three-dimensional point cloud of the CAD model and the three-dimensional point cloud of the workpiece to be cut, calculating the transformation matrix, and performing coordinate system transformation on the CAD model according to the transformation matrix; generating a set of cutting poses based on the cutting trajectory lines and the transformation matrix, performing translation and rotation adjustments, smoothing, sorting, deformation compensation, and robot kinematics checks on the set of cutting poses to generate a set of cutting trajectories containing multiple cutting trajectories; and generating a cutting program based on a recognizable syntax based on the set of cutting trajectories containing multiple cutting trajectories.

The invention relates to the technical field of robot path control, and discloses a robot path control method based on automatic operation. The method comprises the following steps: acquiring three-dimensional point cloud data of a robot working environment, and generating an environment grid map through spatial rasterization processing; a dynamic obstacle track is extracted based on the map, and a dynamic obstacle avoidance constraint condition is constructed in combination with kinematics parameters of the robot. After the path search space is initialized according to the constraint condition, a candidate path set is generated by adopting a heuristic strategy, and an optimized path meeting a preset threshold value is screened out through smoothness and energy consumption evaluation. And inputting the optimized path into a track generator to generate a continuous motion track of each joint of the robot. And pose deviation is monitored in real time during operation, and track re-planning is triggered if the deviation exceeds a fault-tolerant threshold value. According to the method, the environment sensing precision and the path generation efficiency are improved, and the stable operation of the robot in a complex dynamic environment is ensured.

This disclosure provides a method and system for safe control of two-wheeled robots against periodic Markov DoS attacks, relating to the field of robot safety control technology. The method includes: constructing a kinematic model of the two-wheeled robot; integrating and fusing the kinematic model and system signals through a communication network to construct a networked two-wheeled robot control system (NCSs); linearizing and discretizing the NCSs to obtain a discrete-time linear state-space model, and constructing the motion target of the two-wheeled robot; determining the attack type; using switched system theory to transform the attacked NCSs into a cyclic Markov system with modal dwell time; setting a controller; calculating the control gain based on the subsystem constraints of the switched system to control the two-wheeled robot; tracking the target trajectory of the two-wheeled robot; and achieving safe control of the networked two-wheeled robot system against periodic Markov DoS attacks.

The invention discloses an industrial robot and environment contact event identification method based on cross-modal learning, and the method comprises the steps: collecting a visual image and a force sense time sequence of the tail end of a robot, and extracting visual features and force sense features through convolution operation and continuous wavelet transformation; a force sense feature driving offset generation network is utilized to calculate a time offset scalar, time dimension deformable convolution is executed on visual features, and time-space accurate alignment of visual and tactile features is achieved; according to kinematics parameters of the robot, kinetic theoretical stress is calculated, a physical residual vector is obtained through difference between the kinetic theoretical stress and actually-measured force, and inertia force interference generated by motion of the robot is eliminated; mapping the physical residual error, fusing the mapped physical residual error with the aligned visual tactile features, and inputting the fused physical residual error into a neural network to identify a contact type; according to the method, a physical consistency constraint and anti-occlusion reconstruction mechanism is introduced in a model training stage, so that the problems that multi-modal data are asynchronous, and the model lacks physical interpretability and visual occlusion are effectively solved, and the accuracy and robustness of event recognition are improved.

This system increases the number of processes that can be automatically annotated, saving labor, and also detects feature areas in images that are not normally detected during processing. [Solution] A robot control device is used, comprising: a coordinate calculation unit that calculates coordinates from data of an object and a robot manipulator in a simulation environment; a first learning model generation unit that generates a first learning model including data of a first estimated position of the object and the robot manipulator in the simulation environment using first teaching data including coordinates on a simulation image; a second learning model generation unit that generates a second learning model including data of a second estimated position of the object and the robot manipulator in real space based on second teaching data generated using data of the object and the robot manipulator in real space; and a robot control unit that controls the operation of the robot manipulator based on the data of the second estimated position.