66 results about "Visual servoing" patented technology

This application relates to the field of robotics, and particularly to a visual servo control method and apparatus based on heterogeneous joint weighted motion decomposition. The method includes: weighting the pose deviation of the target pose measurement data based on the target pose measurement data of the agile arm; planning the expected joint angle data for the next cycle of the agile arm, and weighting the expected joint angle data to obtain the joint motion data of the agile arm; converting the joint motion data into motor motion data based on the target kinematic transformation relationship, and weighting the motor motion data according to the motor response characteristics to obtain weighted motor motion data, thereby controlling the agile arm to perform corresponding visual servo control actions. This application can weight the pose information of the target in each direction, the planned joint angle data, and the corresponding motor angle data, ensuring that the response characteristics of each motor are basically consistent, thus enabling the visual servo method to be well adapted to the motion characteristics of the tethered agile arm.

The application discloses an end gripper for adaptive gripping of photovoltaic modules, comprising an end gripper frame, a visual relative pose measurement module, a multi-axis linkage pose compensation mechanism, a vacuum suction execution module and a main controller. The visual relative pose measurement module solves the spatial relative pose parameters of the photovoltaic module relative to the end gripper frame in real time. The main controller extracts the rotational deviation amount in the parameters, generates a servo adjustment instruction to drive the multi-axis linkage pose compensation mechanism to perform iterative approximation type pose correction, forming a visual servo closed-loop control. When the rotational deviation amount converges within the preset tolerance range, it is determined that the pose matching state is reached and the vacuum suction execution module is triggered to work. The application realizes real-time adaptive compensation for dynamic angle deviation of the photovoltaic module during conveying, solves the problems that the existing end gripper lacks integrated multi-dimensional pose adjustment capability and deep collaborative closed-loop control, and significantly improves the gripping precision and automation level.

The present application relates to underwater vehicle control technology field, specifically to a kind of machine manta ray visual servo autonomous precision benthic control method and system for narrow space, comprising: setting the initialization parameter of machine manta ray fixed-point benthic;Obtain the relative pose information between machine manta ray and the mark between fixed-point benthic position;Obtain the real-time phase difference corresponding to each pectoral fin of machine manta ray;Obtain the buoyancy of machine manta ray;Machine manta ray is controlled benthically.The present application can not only realize trajectory convergence and position convergence, but also can guarantee attitude stable convergence, embodies the good convergence and operating stability of the present application in actual operation.

This invention discloses a method for autonomous operation of humanoid robots based on ROS. It employs a decision-making and control framework that fuses hierarchical finite state machines with YOLOv8 vision to achieve continuous autonomous execution of multiple tasks in complex unstructured environments. The method includes: system initialization and standardized processing of multimodal perception data; target 3D localization fused with YOLOv8 and coordinate transformation; task scheduling and state transitions using hierarchical finite state machines; segmented navigation and motion planning of joint trajectories; and ROS command issuance and full-process closed-loop control. This invention addresses the rigidity of traditional script-based control by fusing hierarchical state machines with visual servoing. Combined with a full-process anomaly handling mechanism and multi-target scene processing rules, it can dynamically adapt to changes in complex unstructured environments, effectively avoiding problems such as state freezes and erroneous grasping.

The application discloses a humanoid robot pouring method and device based on visual servo, and relates to the technical field of industrial vision intelligence. The method comprises the following steps: acquiring first posture information of a target object in a working environment; determining a pouring target position above a water cup based on a preset height and a water cup position; iteratively adjusting the end posture of the mechanical arm of the humanoid robot to reduce the position deviation between the spout position and the pouring target position by using a first visual servo control algorithm; when the position deviation meets a first preset condition, controlling the mechanical arm to perform a pouring action; during the execution of the pouring action, acquiring a real-time updated spout position, and compensating and adjusting the end posture of the mechanical arm based on the real-time deviation between the real-time updated spout position and the pouring target position by using a second visual servo control algorithm. The application improves the accurate alignment and real-time control of the robot pouring action, and has stronger stability and adaptability.

The application relates to the technical field of robot control, and discloses a terminal position dynamic compensation method based on fusion of visual servoing and force feedback, which comprises the following steps: collecting a top view image, segmenting an irregular residual hole contour, and calculating a dynamic alignment center coordinate; then, a reverse conical spiral descending trajectory is generated in combination with a lateral force vector to guide a fiber tail fiber connector into a guide port; in the insertion stage, the curvature radius characteristic of a force-displacement curve resistance peak point is used to determine the authenticity of bottom touching; for the case that the bottom touching is determined to be false, the side projection chord length is used to deduce an under-insertion depth, and a high-frequency micro-oscillation compensation insertion is driven until a real buckle locking feature is captured; the application realizes accurate alignment and reliable positioning of fiber plug-in and plug-out actions under complex working conditions such as shielding and mechanical wear.

This invention discloses a robust visual servo control method for robots based on an adaptive update law, comprising: associating the pixel coordinates of feature points with their 3D coordinates in the camera coordinate system according to a pinhole camera projection model, and establishing a global linear regression model containing unknown position parameter vectors; designing an adaptive update law for the estimated values ​​of the unknown position parameter vectors based on the global linear regression model, used to estimate the unknown depth information parameters of feature points in real time, and reconstructing the visual Jacobian matrix online based on the estimated depth information parameters; constructing a visual state observer to obtain smoothed image feature estimates; defining a joint space-assisted tracking error based on the smoothed image feature estimates based on the reconstructed visual Jacobian matrix; and designing an integral sliding mode robust feedback term based on the integral sliding mode in the robot joint driving torque expression based on the lumped disturbance in robot dynamics, combined with feedforward compensation, to generate the final joint driving torque to drive the robot to perform servo tracking tasks. This invention can achieve highly robust and high-precision asymptotic tracking of target feature points under complex working conditions.

This invention belongs to the field of mechanical processing and manufacturing, and more specifically, relates to a flexible eccentric drilling end-effector based on three-dimensional vision. This equipment achieves automated positioning for drilling through visual servoing; the drill bit adopts an eccentric structure design, enabling it to smoothly extend into extremely narrow semi-enclosed spaces such as stringers; it employs a pressure foot design based on parallel finger cylinders to solve the problem of difficulty in clamping and holding the drill bit in narrow drilling positions; the installation angle of the three-dimensional camera can be adjusted as needed to adapt to different working environments; the overall equipment integrates functions such as visual servoing, automatic clamping, high-precision feeding, and offset machining, realizing automated drilling operations in narrow spaces; the overall equipment adopts a modular design, and while meeting drilling requirements, it achieves miniaturization and compactness through optimized design and component selection, enabling it to adapt to the size and weight requirements of machining tools in large aircraft cabins, and possessing good economy, safety, and maintainability.

This invention discloses a visual servo grasping and assembly method for tunnel boring machine (TBM) segments adapted to the motion of an observation platform. The method includes the following steps: mounting a visual sensor on the observation platform to collect the three-dimensional coordinates of reference markers on the assembly machine, the assembly machine, the tunnel segments, and feature points at the target assembly position; unifying the local coordinate system of the observation platform with the base coordinate system of the assembly machine at the current moment; constructing an error vector field; based on the spatial distribution pattern of the error vector field, according to geometric decoupling mapping rules, allocating and mapping the overall error to the target motion increments of each independent joint of the assembly machine; iterative servo execution: executing the target motion increment commands in priority order, repeating the above steps until the magnitudes of all first spatial error vectors / second spatial error vectors are less than a set threshold. This invention relates to the field of automated equipment technology for tunnel construction and can solve the problem of poor engineering adaptability caused by traditional control methods relying on mechanism models and absolute pose calculations.

This invention provides a fire-fighting robot control method and system based on AI vision, relating to the technical field of fire-fighting robot control technology. The method includes: acquiring a first instruction and executing preemptive instruction arbitration based on the first instruction, wherein the first instruction carries a target component identifier; performing spatial coarse positioning according to the deployment topology type of the fire-fighting robot; after spatial coarse positioning, performing dimensionality-reduced visual servoing based on the visual template of the target component to align the end effector of the fire-fighting robot with the target component; after the end effector is aligned with the target component, performing a first state verification; after the first state verification is passed, controlling the end effector to perform physical operations on the target component; and after the physical operations are performed, performing a second state verification. This invention solves the problem of low robot control accuracy, thereby improving the robot control accuracy.

This invention belongs to the field of electric robot technology. It proposes a visual servoing control method and system driven by pose perception at the oil extraction port. The method generates a frustum region by mapping a target detection box to filter point cloud noise; it uses a network with embedded self-attention mechanism for semantically enhanced instance segmentation to obtain the target mask; and it combines residual compensation and 3D bounding box prediction to transform the point cloud to the real coordinate system to calculate the accurate pose. Finally, a hybrid visual servoing strategy is adopted: pose error control is used at long distances, while image feature-based control is used at close distances, achieving efficient and accurate automatic docking of the oil extraction robot.

The application discloses a kind of flexible production line-oriented multi-robot vision cooperative positioning and grabbing method, it is related to flexible production line robot vision cooperative technical field, comprising: deployment includes industrial camera, 3D laser radar distributed vision node and wrist camera, cover operation area and guarantee field of view overlap and real-time feedback;Multi-source vision data space-time calibration, control synchronization error, unified coordinate and optimize registration accuracy, introduce temperature compensation correction parameter;Cooperative detection tracking workpiece, share results and dynamically adjust tracking mode;Plan global optimal grabbing pose, generate candidate pose and screen;Cooperative control robot motion trajectory, predict avoidance interference;Through vision servo correction pose deviation, trigger grabbing and confirmation, complete transfer.The application improves positioning and grabbing precision and efficiency, enhances multi-robot cooperativity, reduces interference, adapts production line workpiece replacement demand, guarantees long-term operation stability, provides strong technical support for flexible production line.

This application relates to the field of robotics, and particularly to a visual servo control method and device based on heterogeneous joint segmented motion decomposition. The method includes: processing target pose measurement data stepwise to obtain target pose deviation data; planning the desired joint angle motion endpoint of the agile arm in subsequent target cycles, and planning the joint motion data of the agile arm in subsequent target cycles during the online phase in an offline phase; converting the joint motion data into motor motion data, and segmenting the motor motion data within subsequent target cycles to control the agile arm to perform corresponding visual servo control actions using the segmented motor motion data. This application can subdivide and interpolate the planned motion data according to the control cycle of the underlying motor motion controller, achieving efficient and accurate visual servo control. Furthermore, it allows for the development of multi-threading for visual servo planning calculations, with offline planning performed in the main thread, offering high flexibility and practicality.

The application discloses a kind of self-adapting mechanical arm grabbing method based on visual servoing, and the grabbing method includes S1, rough approach stage, obtains the point cloud and image data of work scene, carries out preliminary identification and pose estimation to target object;S2, initial trajectory planning, utilizes five Bezier curves to carry out initial trajectory planning;S3, fine servo stage, mechanical arm moves along initial trajectory planning, and adopts visual servo closed-loop control;S4, blind grabbing stage, when the end of mechanical arm approaches target object and causes partial or complete loss of visual feature, the system starts multimodal fusion control to complete grabbing.The application divides the grabbing process into four stages of rough approach, trajectory planning, fine servo and blind grabbing, and uses different control strategies for the characteristics of each stage, uses prediction filter to correct the error caused by visual delay in advance, smoothly connects low-speed vision and high-speed control, and improves dynamic grabbing precision.

This application relates to the field of circuit maintenance technology, specifically to a visual positioning system and method for a robot used in live-line substation maintenance. The system includes a multimodal visual perception unit mounted on a maintenance base, a potential-synchronized edge computing and processing unit, and positioning and navigation software running within them. The method employs a hierarchical positioning strategy: first, global coarse positioning and target guidance are performed, controlling the perception unit to scan the environment, identify the target, and plan the path; then, local fine positioning and task preparation are performed, and after stabilizing the robot, the precise pose of the target is calculated using a zoom camera; finally, servo-level micro-positioning and task execution are performed, planning the robotic arm path based on the precise pose, and switching to visual servo mode to complete precise alignment and operation. This invention solves the problem of high-precision autonomous positioning and guidance for maintenance robots in complex environments, improving operational safety and automation.

The present application relates to a kind of based on conjugate double position and visual servoing pump valve body full sequence processing method and five surface machining center.The method includes: by industrial camera, pump valve body blank real-time image is acquired, input pre-training convolutional neural network, output instance segmentation mask and reference point pixel coordinate;Based on PnP algorithm, workpiece pose is solved in real time;Through visual servo control, control signal is generated, and vertical turning module and horizontal machining module are driven along the correction path and parallelly cooperated cutting;Real-time acquisition multi-dimensional force sensing data, combined with processing error compensation model, dynamically compensates servo shaft displacement;By the phase conversion of precision indexing table, until completing full sequence processing.Five surface machining center includes base, precision indexing table and the vertical turning module and horizontal machining module arranged around, is driven to realize parallel cutting by same numerical control system.The present application improves processing precision and efficiency, realizes full sequence automatic processing.

This application relates to a visual servo control method, apparatus, equipment, and non-volatile storage medium for cutterhead changing in a tunnel boring machine. The improved method includes: acquiring image data and obtaining an image to be processed; processing and compressing the image to be processed and outputting a feature vector and a compressed image; processing the feature vector and compressed image and outputting motion control commands; performing feedback checks on the execution of the motion control commands and obtaining a first feedback error, and determining the first feedback error; obtaining the current execution action and obtaining a second feedback error, and determining the second feedback error. Under harsh conditions such as changing lighting, pollution, and partial occlusion, the servo control system of this application can achieve high-precision and highly robust visual servo control, effectively improving the operational capabilities of the tunnel boring machine cutterhead changing robot.

The application discloses a time-varying adaptive depth estimation visual servo method, and belongs to the technical field of hand-eye robot control, which comprises the following steps: constructing a hand-eye robot visual servo system model, designing a fixed depth estimation method with an estimation error as a driving force, designing a time-varying depth estimation method, and controlling robot movement. The method introduces a new auxiliary variable, which can smooth the time-varying influence of parameters to a certain extent, effectively reduce the interference of noise and interference on the estimation result, avoid the estimation deviation caused by abnormal data points, and accurately estimate the dynamic target parameter change in real time, so as to help realize the precise visual servo control of the dynamic target.

The application discloses a kind of mechanical arm safety vision servo control method and system based on reinforcement learning, belong to the intersection technical field of robot control and artificial intelligence. The visual feature information of target object in image plane is acquired using mechanical arm hand-eye camera;Define the state space of mechanical arm body state, visual feature state and obstacle observation, mechanical arm action space and position control mode;Sparse safety reward function is set, including vision servo success reward, visual feature point out camera field of view penalty and CBF inspired safety penalty;Sparse safety reward function is used to train mechanical arm to avoid obstacle servo movement, construct visual feature sequence dataset, and offline train gated recurrent unit GRU visual feature completion model, so that the target is missing for not more than 3 seconds in the process of mechanical arm obstacle avoidance servo movement, obstacle avoidance servo movement can still be realized to reach target object, the application can realize the stable control of mechanical arm vision servo in multi-obstacle and hand-eye camera short-time shielding environment, improve system robustness, and effectively reduce collision risk.

This invention discloses a control system and method for a disc cutter gripping robot in a tunnel boring machine. The system includes a central processing unit, a robotic arm, and an end effector. The end effector integrates a vision sensor, an ultrasonic sensor array, and a tactile sensor array. The method employs a progressive multi-sensor fusion control strategy: first, coarse positioning is achieved through path planning and visual servoing; second, precise positioning and guidance with six degrees of freedom are performed by activating the ultrasonic sensor array at close range; and finally, adaptive compliant adjustment is achieved using the tactile sensor array during gripping contact. This invention effectively overcomes the adverse effects of harsh environments such as dust, vibration, and low light within the tunnel cutterhead through the collaborative work of time-activated heterogeneous sensors, achieving robust and adaptive gripping of disc cutters with precision ranging from decimeter to sub-millimeter levels, thus improving the automation level and safety of cutter changing operations.

The application provides a kind of facility agriculture picking robot vision-haptics fusion control method, the method comprises: by obtaining the RGB-D image and laser radar point cloud of facility environment, extract crop ridge center line, real-time solution chassis linear velocity vector and angular velocity vector;Obtain the three-dimensional point cloud and surface image of fruit, solve the maturity grade of fruit, and construct visual stiffness prior model, estimate the elastic modulus of fruit;According to the elastic modulus on-line mapping generation mechanical arm end impedance controller's target stiffness matrix and damping matrix;Adopt the position-based visual servo strategy, the pose error between end effector and target fruit is converted into the expected angular velocity of each joint of mechanical arm, drive end to approach target;When the contact force detected by end six-dimensional force sensor reaches the preset threshold, the system is switched from pure visual position control to adaptive impedance control mode, and the visual planning path is dynamically corrected using force feedback.

This invention relates to a paper-inspired bionic dual-segment flexible continuous robotic arm. The robotic arm employs a paper-inspired bionic dual-segment composite main structure design, with a one-piece molded flexible main body as its core. The elastic deformation of flexible hinges simulates the crease movement of paper folds, and combined with non-vector visual servo control, it uses line drive to achieve bending and precise positioning in confined spaces. This robotic arm is integrally molded using 3D printing technology, eliminating the mechanical gaps and connection failure risks associated with traditional assembly. Furthermore, it adopts an axial stiffness gradient design, offering advantages in high dexterity, high stability, and compact structure in confined space operations such as minimally invasive surgery and precision procedures.

An anti-interference line-of-sight stabilization control method for dynamic target tracking belongs to the field of visual servoing and photoelectric tracking technology. This invention addresses the problem that existing line-of-sight stabilization systems cannot effectively handle time-varying and unknown target maneuvers. It includes: mounting a camera on the line-of-sight stabilization device to acquire target images and obtain the image miss distance; using a UDE-based outer-loop programmable logic controller to obtain an interference estimation signal; using an MPC-based outer-loop visual servo controller to calculate the outer-loop control signal; a UDE-based inner-loop attitude controller to obtain a control signal for the line-of-sight stabilization device based on the outer-loop control signal and the current attitude of the line-of-sight stabilization device; and the actuator of the line-of-sight stabilization device adjusting its attitude according to the control signal, ensuring that the image miss distance of the target position coordinates in the target image is within a set threshold. This invention is applicable to scenarios such as UAV monitoring, UAV countermeasures, and low-altitude security.