361 results about "Visual servoing" patented technology

The invention discloses a multi-modal perception humanoid robot action adaptive control method and system, and belongs to the technical field of computers, and the method comprises the steps: obtaining first multi-modal data; the first multi-modal data comprises first image data, first point cloud data and first force sense data; performing data fusion on the first image data and the first point cloud data, and constructing a three-dimensional model of the scene in real time; generating an optimal path according to the real-time environment data of the three-dimensional model, converting the optimal path into a joint instruction of the humanoid robot, and generating an action control strategy; according to the action control strategy, arm movement of the robot is adjusted through visual servo control, and first operation data are obtained; the path and action of the robot are adjusted in a closed-loop control mode, and second operation data are obtained; and reinforcement learning and regular evaluation are carried out through a DDPG algorithm, and an action control strategy is continuously optimized. According to the invention, multi-modal sensing robot motion adaptive control with strong adaptive capability, high operation precision and good robustness can be realized.

The invention relates to the technical field of marine monitoring, in particular to a marine target visual servo tracking method, which comprises the steps of multi-modal sensor fusion, a self-adaptive visual tracking algorithm, a servo control and visual collaboration mechanism and a shielding processing and target re-identification strategy. The problem that tracking is unstable under the conditions of illumination change, ship body shaking, target shielding and the like in a traditional method is solved. The IMU, the GNSS and the visual data are fused through extended Kalman filtering, ship body shaking is compensated, and the target state estimation precision is improved; the improved D-Fi ne target detection model is combined with an online feature updating mechanism to dynamically adapt to the appearance change of the target; the prediction and correction control strategy and the double-closed-loop PI D controller cooperate to adjust the camera holder, and the tracking delay is reduced; the multi-clue shielding detection and space-time joint feature matching technology ensures accurate re-identification of the target after shielding is removed. The real-time performance and robustness of the tracking system on an embedded platform are improved, and an efficient and stable target tracking solution is provided.

A human-guided vision-force fused impedance iterative learning control method for a robotic arm, comprising: analyzing a robot-environment interaction dynamics equation, solving a visual servo acceleration model, and making use of the equation to establish a human-robotic arm-environment interaction dynamics model in an image feature space; acquiring an image feature position and speed curve of a human-guided robot completing an assembly task, and using dynamic movement primitives for coding and generalization; and designing an impedance iterative learning controller which uses image feature tracking errors as control input, learning impedance characteristics when the human-guided robot performs a contact operation, identifying unknown contact dynamics under the interaction between the robot and the environment, and counteracting identified contact interference in the feature space, so as to implement a flexible assembly operation. The control method solves the problems in existing assembly operations that human-robotic arm-environment coupling nonlinear dynamics, unknown contact dynamics of intensive contact assembly tasks and poor generalization of assembly scenarios require relearning for different scenarios, etc.

The invention relates to the technical field of automatic workpiece grabbing and visual servo control, and discloses a workpiece grabbing method and system based on visual feedback, and the method comprises the steps: obtaining a real-time image through a front-view camera, a side-view camera and a top-view camera, and extracting surface features through a feature fusion network; inputting the features into a pose estimation model based on a particle filtering framework, and iteratively updating the pose in combination with a multi-dimensional observation likelihood function to generate prediction parameters; constructing a multi-target path planning model based on the parameters, and optimizing the path by adopting a dynamic planning algorithm with the shortest path and the minimum joint movement as targets; a hierarchical control model is established, a strategy layer performs global planning, an adjustment layer corrects a local path, and an execution layer realizes trajectory tracking through visual servo control and outputs a control instruction to complete grabbing. Through multi-view perception, robust pose estimation, global optimization path planning and hierarchical control, the precision, efficiency and stability of workpiece grabbing in a complex environment are improved.

The invention discloses a visual servo double-arm robot migration simulation learning method from simulation to reality, which comprises the following steps: S110, constructing a virtual simulation scene according to a real task scene, and establishing a relation between the virtual simulation scene and the real task scene; s120, the collected teaching mechanical arm and executing mechanical arm operation data are replayed and optimized in the virtual simulation scene, so that a data set used for final training is constructed; s130, a deep imitation learning network is designed and achieved, input of the deep imitation learning network comprises visual data obtained in the real task scene and state data of all joint motors of the teaching mechanical arm, and output of the deep imitation learning network is predicted states of all joint motors of the execution mechanical arm at the next moment; and S140, migrating the fully trained and converged deep imitation learning network from a virtual simulation scene to a double-arm robot in a real task scene. According to the invention, the exploration efficiency and generalization ability of the two-arm robot learning network are improved.

The invention relates to the technical field of welding automation and robot visual perception, and discloses a robot intelligent welding method and system.The method comprises the steps that a three-dimensional point cloud image is constructed through a laser triangulation method; filtering and enhancing the point cloud image, and extracting a weld point cloud set by adopting a U-Net semantic segmentation model; carrying out space path identification and attitude regression based on the set to obtain a track element of a position-attitude pair; fitting a path by adopting a cubic B-spline curve and interpolating to generate a continuous welding track; and in the welding process, visual feedback is combined, and proportional differential control logic is introduced for closed-loop track error compensation. Compared with the technical problem that in the prior art, weld joint path recognition and accurate tracking cannot be stably achieved under complex structures such as curved surfaces and variable cross sections, a perception control closed loop based on visual servo is constructed, high-precision automatic welding of the complex weld joint structures is achieved, and the track fitting precision is improved.

The invention relates to a dynamic cooperative path planning and autonomous landing control method for an unmanned aerial vehicle and an unmanned ship, and the method comprises the following steps: collecting the state data of the unmanned ship and the unmanned aerial vehicle and the high sea condition environment data, and carrying out the data fusion processing; an improved fast extended random tree algorithm is adopted to plan a safe navigation path for the unmanned ship, the generated path of the unmanned ship is optimized based on an unmanned ship motion model considering environmental interference, and a model prediction control algorithm is adopted to plan a path for the unmanned ship to fly to a predicted landing point of the unmanned ship. Dynamic cooperation between the unmanned ship and the unmanned aerial vehicle is realized through information interaction between the unmanned ship and the unmanned aerial vehicle; the unmanned aerial vehicle recognizes an unmanned ship landing identifier through a visual servo guide system, calculates the position and attitude deviation relative to a landing point, adjusts flight control parameters according to interference conditions by using an adaptive control algorithm, and performs trajectory prediction compensation in combination with an unmanned ship motion model to realize autonomous landing. Compared with the prior art, safe landing of the unmanned aerial vehicle on the unmanned ship under the high sea condition is achieved.

The invention discloses a high-precision laser positioning unpacking path planning system, and belongs to the technical field of robot automatic control and industrial automation. The system comprises a data synchronization module used for multi-sensor hardware synchronization and data alignment; the high-precision positioning module is used for outputting a precise pose based on environment skeleton characteristics and sliding window optimization; the semantic map construction module is used for fusing vision and laser data to generate a dynamic semantic grid map; the global path planning module is used for planning a smooth path in the skeleton channel by utilizing a mixed potential field improved A * algorithm; the motion control module is used for realizing closed-loop motion control and safety monitoring through model prediction and tracking; and the operation execution module is used for finishing millimeter-level precise stopping of an operation point by adopting visual servo and triggering unpacking operation. According to the method, the positioning robustness under dynamic shielding is improved through the environmental skeleton features, the safety and the high efficiency of the path are ensured by utilizing semantic understanding and intelligent planning, and the full-process automation from navigation to precise operation is realized.

The invention belongs to the technical field of position tracking control of rigid multi-joint mechanical arms, and discloses a fuzzy self-adaptive visual servo control method and system for a multi-joint mechanical arm system for placing bobbins in a textile workshop, and the method comprises the steps: constructing an image error model according to the positions of feature points on an image plane; the fuzzy logic system is used for processing uncertain items in the system, the instruction filtering and fuzzy self-adaption technology is used for achieving tracking control over the expected trajectory of the system, the problem of calculation complexity is solved, and adverse effects caused by filtering errors are eliminated. In the process of changing the bobbins, the bobbins are placed on a creel through a multi-degree-of-freedom robotic arm, a clamping jaw at the tail end of the robotic arm is required to accurately grab the bobbins for feeding and discharging, and the requirement for accurate and rapid control of the robotic arm is met, so that research on visual servo control of the robotic arm is of great significance. The method has a good control effect, the problem of calculation complexity in the design process of the controller is solved, the trajectory tracking error is smaller, and the system robustness is higher.

The invention discloses a fluid loading and unloading arm dynamic flange automatic docking method based on image visual servo, which comprises the following steps: initializing the tail end position of a loading and unloading arm, and controlling the loading and unloading arm to move until complete image information of a flange to be docked can be acquired; the method comprises the following steps: acquiring an image of a flange to be butted, and extracting initial candidate circles with circle features in the image by adopting a Hough circle detection algorithm; through an RANSAC (Random Sample Consensus) algorithm, performing refined fitting to obtain a refined candidate circle; extracting the center coordinates of the positioning holes from the refined candidate circle; according to the circle center coordinate of the positioning hole and the butt joint state coordinate position, calculating the current time deviation; taking the minimum deviation as a target, solving an angular motion angular velocity instruction of each joint of the loading and unloading arm, driving each joint of the loading and unloading arm to adjust the pose until the deviation converges, and completing the butt joint of the flange at the tail end of the loading and unloading arm and the to-be-butt-jointed flange. The method is suitable for millimeter-level accurate butt joint and control of loading and unloading arm flanges in the dynamic marine environment, and the operation efficiency and safety in the complex environment are remarkably improved.

The invention discloses a robot dynamic grabbing method based on a sliding window self-attention mechanism, and the method comprises the steps: firstly installing a camera, and calculating a transformation matrix between the camera and a mechanical arm coordinate system based on a hand-eye calibration technology; processing the static data set by adopting affine transformation to generate a grabbed dynamic data set with speed information; then, a dynamic grabbing network model based on a sliding window multi-head self-attention mechanism is constructed, and a loss function optimization model considering the object movement speed and the grabbing posture at the same time is designed; and finally, the trained model is deployed in a robot grabbing environment, a visual servo grabbing control scheme based on the position is designed, and follow-up movement and grabbing action of an end effector of a mechanical arm on a to-be-grabbed object are achieved. According to the dynamic grabbing method of the robot, the grabbing posture and the movement speed of the object can be quickly and accurately predicted based on a group of continuous single-channel depth images collected by the camera, and the grabbing problem of a universal object in a complex dynamic environment is solved.

The invention relates to a mechanical arm grabbing method and system based on dynamic visual servo, and belongs to the technical field of automatic control. The method comprises the following steps: acquiring an initial depth image of a target object acquired by a depth camera; determining the shape and position of the target object based on the initial depth image, and performing feature extraction on the initial depth image based on a set feature extraction network to obtain initial key feature points of the target object; the shape and the position of the target object and the initial key feature points of the target object are input into a set track model, and an initial grabbing path of the mechanical arm is determined; and the mechanical arm is controlled to grab the target object according to the initial grabbing path, and in the grabbing process, the initial grabbing path is dynamically adjusted for grabbing. In this way, the grabbing success rate can be increased, and the risk that the target object is damaged is reduced.

The invention discloses a visual servo control method and system for assembling multi-shaft-hole parts through multiple mechanical arms in a tight coordination mode, and belongs to the field of multi-robot cooperative control. The invention aims to solve the problems of visual feature extraction and collaborative assembly control in the task of tightly coordinating and assembling large-size multi-shaft-hole parts by multiple mechanical arms. The method is technically characterized in that a shaft hole assembly task image data set containing different hole shapes is established and used for training a positioning neural network and a relative state estimation neural network, and a state classification neural network is used for dividing the states of shafts and holes in images into three types of shielding, separation and overlapping; the positioning neural network is used for determining the positions of the shaft and the hole in the image. A multi-mechanical-arm visual feature fusion method based on the virtual acting force is provided, the virtual acting force acting on an object is calculated according to position information and relative state information of a shaft and a hole in an image, and the virtual acting force is input into a cooperative controller of the multiple mechanical arms to achieve cooperative assembly movement of the multiple mechanical arms. And natural combination of visual features and a multi-mechanical-arm cooperative controller is achieved. Finally, the method provided by the invention is verified in the task of tightly coordinating and assembling the double-shaft-hole parts by the double mechanical arms with the assembling clearance of 0.2 mm, the success rate reaches 85%, and the effectiveness of the method is verified.

The invention discloses a mechanical arm visual servo control method and equipment based on target condition reinforcement learning, and belongs to the technical field of mechanical arm control. A mechanical arm environment is built, a camera is additionally arranged at the tail end of a mechanical arm, a computer is used for designing a safety reinforcement learning model, then online learning is carried out according to a depth deterministic strategy gradient algorithm and a post-experience playback algorithm, and the mechanical arm environment is matched. The position of the feature point in the camera picture serves as input to control the mechanical arm to move and lock the two-dimensional code attached to the surface of the object, and finally the visual servo task is completed. The method is simple in step, low in environment requirement and rapid in response of the mechanical arm, and compared with a traditional control method based on pseudo-inverse, the method has remarkable effect improvement in the aspects of the number of iterations and robustness.

The invention provides an unmanned aerial vehicle visual servo intrusion target tracking and intercepting method based on time delay compensation. The method comprises the following steps: acquiring a latest image of a tracking target through a camera on an unmanned aerial vehicle; based on the latest image, determining the position of the tracking target in an image coordinate system of the latest image as a first position; determining a position error of the first position by adopting a prediction algorithm based on improved Kalman filtering based on the delay time of collecting the latest image; based on the position error, determining a control parameter used for visual servo tracking of the corresponding tracking target; and based on the first position and the control parameter, controlling the unmanned aerial vehicle to fly to track and intercept the corresponding tracking target. Therefore, the position error compensation is performed through the delay time of the latest image, and the unmanned aerial vehicle is controlled to track the corresponding target based on the position error, so that the tracking accuracy is improved, the tracking interception error is reduced, and the interception success rate is improved.

The invention relates to a mechanical arm visual servo control method and system based on deep reinforcement learning. Firstly, real-time state observation and collision detection are conducted on the mechanical arm, and an observation result is obtained; then, a deep reinforcement learning controller is built by means of a maximum entropy deep reinforcement learning SAC algorithm network architecture, and an action space, a state space and a reward function are built according to an observation result and a target position; and then, outputting a first action representation based on the construction content, optimizing a controller according to a calculation result of the first action representation and the reward function, smoothly optimizing the first action representation by using a real-time track interpolation strategy to obtain a second action representation, and controlling the mechanical arm joint according to the second action representation. And the process is continuously repeated until the mechanical arm successfully arrives at the target position. Compared with the prior art, the method has the advantages of high control real-time performance, large application range and the like.

The invention discloses a robot automatic grabbing path planning method based on visual identification, and relates to the technical field of intelligent grabbing. The method comprises the following steps: acquiring multi-view visual data of a target scene, and generating a scene three-dimensional compact reconstruction model through a multi-modal image fusion algorithm; performing target detection and feature extraction on the model, and screening an optimal capture point in combination with a visual attention mechanism; constructing a dynamic environment obstacle probability map, updating an obstacle state through time sequence visual tracking, and quantifying an interference weight; an initial grabbing path is planned based on an improved fast expansion random tree algorithm, path smoothness constraints and robot joint movement limit parameters are introduced, and path nodes are optimized through a Bezier curve; visual servo feedback and path deviation prediction are fused, path parameters are corrected in real time, and a continuous movement track is generated. The method effectively adapts to the dynamic environment, gives consideration to path safety, smoothness and mechanical arm motion characteristics, and remarkably improves the grabbing success rate and operation reliability.

The invention discloses a method and device for controlling grabbing of dynamic strips based on visual servo guidance and a test platform. According to the method for grabbing the dynamic strip-shaped object based on visual servo guidance control, the mechanical arm is guided and controlled to grab the dynamic strip-shaped object by adopting an image-based visual servo method, and real-time tracking of the dynamic pose of the strip-shaped object and immediate response of the action of the mechanical arm are realized through continuous sensing, processing and tight coupling of execution links; therefore, a more accurate, stable and efficient grabbing effect can be achieved in a highly dynamic and uncertain working environment; a test platform capable of simulating dynamic swing of a real strip-shaped object is also built, and a prediction control method and a visual servo control method are combined, so that effective verification and optimization of the high-precision and real-time grabbing process of the dynamic strip-shaped object are realized; the platform can be used for systematically researching the grabbing capability under various disturbance conditions in a laboratory environment, and a solid technical foundation is laid for automatic and intelligent operation in practical application.

The invention discloses a mobile robot accurate docking method based on edge calculation, and aims to solve the problems that the dynamic docking precision is reduced and the collision risk is increased due to micro-motion or drifting of a target station. According to the method, unified time reference alignment is carried out on data of a camera, a laser radar, an inertial measurement unit, an ultra-wideband range finder, a station encoder and a programmable logic controller at an edge node, and a three-dimensional special Euclidean group equivariant multi-source fusion network is used for outputting relative pose estimation and covariance; the estimation in the time window is further used as a condition to be input into a conditional diffusion short-time prediction model to obtain a time-varying mean value and a time-varying covariance, an anisotropic probability tube is constructed, and prediction-measurement joint correction is carried out based on a score function; scenarized opportunity constraints are constructed under correction probability tube constraints, a tubular nonlinear model is adopted to predict, control and solve a reference trajectory, an actuator command is generated in combination with depth visual servo and compliance control, and the technical effects of high precision, robustness and safe docking under the station dynamic disturbance condition are achieved.

The invention discloses a mechanical arm anti-interference control method and system based on visual servo, and relates to the technical field of mechanical arm anti-interference control, and the method comprises the steps: collecting an initial image through an industrial camera during the operation of a mechanical arm, and carrying out the preprocessing; selecting a pixel point from the central area of the initial image as a starting point by using a random walk algorithm, generating a step length in a random direction, calculating the next coordinate of the starting point, cutting, obtaining a walk path set, carrying out pixel value disturbance, generating a pseudo-abnormal image, and carrying out two-dimensional position coding based on the pseudo-abnormal image to obtain an embedded sequence vector; based on the embedded sequence vector, using a repair model to obtain a repair image, and generating a reconstructed image in combination with a deconvolution operation; by introducing the random walk algorithm, the multi-head self-attention mechanism, the Gaussian pyramid and the sparse dictionary, the perceptual robustness, the task execution stability and the generalization ability of the mechanical arm in the disturbed environment are remarkably improved.

The invention provides a lift wing unmanned aerial vehicle high maneuvering target interception method based on visual servo, which is used for using a lift wing unmanned aerial vehicle to cope with an invasion target moving at a high speed and using a lift wing platform. Lift wing modeling based on a rotation matrix and high-precision and high-dynamic autonomous interception control based on visual servo are introduced to serve as key technologies for supporting interception of the lift wing unmanned aerial vehicle. The effectiveness of the algorithm under airborne perception and control is verified, and the feasibility of the algorithm in practical application is consolidated. The contributions jointly promote the development of an unmanned aerial vehicle autonomous interception technology, and an important tool and method are provided for effectively coping with long-distance high-speed target intrusion threats. In addition, an IMU (Inertial Measurement Unit) measurement model and an image delay measurement model continue to be used, and a delay state estimation problem is researched.

The invention belongs to the technical field related to robot visual control, and discloses a method and system for cooperatively controlling a mechanical arm to move based on visual servo and force sense admittance. The method comprises the steps that a mechanical arm moves to an assembly target position, and an image collection device collects an image of a target point at the assembly target position as an expected image; the tail end of the mechanical arm clamps the to-be-assembled workpiece to move to an assembly target position, an image collection device collects an image of a target point in real time in the moving process of the tail end of the mechanical arm, and the image collected in real time is compared with an expected image so as to calculate the movement speed vc of the tail end of the mechanical arm; when the to-be-assembled workpiece is in contact with the pin at the assembly target position, the force sensor measures the force and the torque during contact, and the movement speed vf of the tail end of the mechanical arm is calculated through the force and the torque; and calculating the movement speed of the tail end of the mechanical arm at the next moment by utilizing the movement speeds vc and vf of the mechanical arm. According to the invention, the problem that visual control and force control cannot be cooperatively controlled in the prior art is solved.

The invention relates to the technical field of intelligent control, and discloses an intelligent control method and system for an electromechanical module intelligent installation robot. The method comprises the following steps: fusing multi-sensor data through a Kalman filtering algorithm to generate a standardized space state vector, carrying out three-dimensional path planning on a BIM model based on the vector to obtain a path instruction set, and carrying out precise positioning correction on the path instruction set by adopting a visual servo algorithm to obtain a corrected motion instruction. A synchronous control instruction is generated through synchronous coordination processing of multiple actuators, and finally safe obstacle avoidance evaluation processing is performed in combination with laser radar scanning data to obtain a safe execution instruction, so that intelligent precision installation of the electromechanical module is realized. The technical problems of inaccurate multi-sensor data fusion, lack of three-dimensional intelligence in path planning, insufficient visual servo precision, difficulty in synchronous coordination of multiple actuators and lack of predictability in safe obstacle avoidance in intelligent installation of the electromechanical module are solved.

The invention belongs to the technical field of intelligent and robot operation, and discloses a collaborative robot closed-loop operation method based on scene perception and thinking chain reasoning, which comprises the following steps of: firstly, generating structured semantic representation containing confidence by utilizing a visual language model of LoRA efficient fine tuning and fusing multi-view data, and providing semantic understanding for a robot; secondly, utilizing a three-dimensional diffusion model to generate explicit geometric priori under the condition of no real CAD, and correspondingly completing PnP initial pose estimation in combination with 2D-3D; then, carrying out pose optimization by adopting a tracking-refining strategy, and triggering reinitialization based on geometric prior when a residual error exceeds a threshold value; and finally, decomposing a global task by utilizing a thinking chain reasoning module, and realizing'perception-reasoning-execution 'closed-loop control based on visual servo in cooperation with an execution module. The method realizes effective combination of large model reasoning and physical execution, has strong robustness under dynamic disturbance, and is suitable for intelligent manufacturing and man-machine cooperation scenes.

The invention belongs to the technical field of robot application, and particularly relates to an acupoint recognition and mechanical arm control method for assisting expectoration, which uses a lightweight Inverted Residual convolution module containing depth separable convolution and residual connection to carry out high-precision recognition on 13 acupoints on the back, and on the basis of RGB three-channel image input, the acupoint recognition and mechanical arm control method for assisting expectoration is realized. A spine central axis channel is added, and a user-defined loss function including MSE loss, spine symmetry constraint and acupoint position constraint is designed to optimize the deep learning model, so that the accuracy and symmetry of acupoint recognition are further improved; after identification is completed, the system combines an improved near-end strategy optimization reinforcement learning algorithm and a visual servo technology to design a self-defined reward function which comprehensively considers distance reward, angle reward, cross punishment and target completion reward. And a voice coil motor of an end effector of the mechanical arm is guided to approach the target acupuncture point and align with the normal of the back of the human body, so that accurate control and dynamic motion planning of the voice coil motor are realized, and auxiliary sputum excretion actions are effectively executed.

A robot visual servo control method based on an exponential finite time neural network solves the visual servo control problem of a robot and comprises the steps that firstly, a visual servo kinematical equation is written based on the geometric principle of visual servo, then the target point position or the target track of the visual servo is set, and the joint expectation angle of the robot is given; secondly, describing a robot visual servo control problem as a quadratic programming problem, and constructing a novel exponentiation finite time neural network solver for solving the quadratic programming visual servo control problem based on a KKT condition met by a quadratic programming optimal solution; the quadratic programming type visual servo control problem is solved under the conditions of no noise interference and no noise interference in sequence, and it is verified that the method has the advantages of being higher in convergence speed and better in visual servo control robustness.

The invention discloses a 6D pose detection and visual servo method of a bolt based on 3D template matching. The method comprises the following steps: S1, establishing a CAD model of the bolt; s2, acquiring a bolt image, and performing image preprocessing; s3, performing bolt feature information detection to obtain feature points, feature lines and feature surfaces; s4, bolt initial point matching is carried out, and matched feature information is stored; s5, performing tracking and 6D pose estimation on the bolt, and performing subtraction on the estimated 6D pose and a set expected pose as input of a visual servo algorithm; s6, acquiring the movement speed of the depth camera; s7, the movement speed of the mechanical arm is obtained and sent to the mechanical arm, and the mechanical arm is made to move; s8, the steps S5-S8 are repeated until the pose error in the step S5 is smaller than the pose error threshold value; and S9, a move L instruction is used for enabling an end effector installed at the tail end of the mechanical arm to conduct bolt alignment. According to the invention, the bolt can be stably tracked and the 6D pose of the bolt can be detected in real time.

The invention relates to the field of industrial robot visual servo control, in particular to a visual servo grabbing dynamic compensation system. The system comprises an image acquisition module, a motion analysis module, a local modeling module, a prediction module and a compensation execution module. A target image sequence is continuously captured, a motion track is extracted, a non-parametric local motion model is established based on recent data, the position of a target at the future grabbing moment is predicted, and a compensation signal is generated according to a prediction result to adjust the grabbing track of a robot. According to the method, the motion model is constructed by adopting a local weighted regression method, accurate description and rapid adaptation of various motion modes are realized through adaptive bandwidth adjustment and an online updating mechanism, and the grabbing success rate of a moving target is effectively improved.

The invention discloses an autonomous sewing system and method based on multi-modal sensing and self-adaptive control. The autonomous sewing system comprises a robot module, a sewing machine module, a visual sensing module and an intelligent control platform. According to the cloth pose registration module, high-precision registration of a flexible target is realized through a two-stage registration process, a local area weighting ICP and a non-rigid cloth surface registration method; the intelligent control module adopts a visual servo dynamic path tracking control method, dynamically corrects the posture of the robot, improves the path tracking precision and the sewing thread consistency, and ensures machine-machine collaborative synchronization and uniform sewing thread uniformity by constructing a speed, track and tension three-factor coupling synchronous control method. By adopting an error adjustment and tension stabilization mechanism based on self-adaptive control, the robustness of the system in a complex sewing state is improved; and through the data analysis module and the exception handling module, full-process monitoring, quality evaluation and exception grading response are realized, and the stability and reliability of the system are improved.

The invention relates to the technical field of robots, and provides an image-based simplified robot visual servo control method, in the aspects of image acquisition and illumination compensation, adaptive multi-scale histogram equalization is combined with illumination simulation compensation based on a generative adversarial network (GAN), so that the problems of illumination diversity and complexity are effectively solved, image distortion is avoided, and the visual servo control precision of a robot is improved. The image feature identification degree is greatly improved in a complex illumination environment, a robot can obtain clear visual information, a multi-modal feature fusion optimization part and attention mechanism fusion automatically adjust different modal feature weights according to a scene, a cross-modal feature interaction network breaks the independence of each modal feature, and the multi-modal feature fusion optimization part and the attention mechanism fusion optimization part are integrated. The target recognition accuracy and robustness are enhanced through cooperation of the two methods, the feature fusion effect is improved, and for the shielding problem, the robot can optimize the shielding processing strategy according to the environment state and the decision result through the shielding decision based on reinforcement learning, and the adaptability is enhanced.