308 results about "Robot vision" patented technology

The invention relates to the technical field of robot visual perception and motion control, and discloses a motion control method and device based on physical reference, equipment and a medium, and the method comprises the steps: obtaining instruction information, a multi-view image and movable assembly pose information; processing the multi-view image according to the instruction information to generate target segmentation information; generating a scale normalization point cloud and a model estimation baseline; determining a physical reference baseline and generating a scale calibration factor; converting the scale normalization point cloud into a physical space point cloud by using a scale calibration factor; extracting a three-dimensional relative position of the target object relative to the movable component in combination with the target segmentation information; an action instruction is generated based on the multi-modal input. According to the method, physical scale alignment of the point cloud is realized through physical reference baseline calibration, so that a visual reconstruction result has real space significance, an accurate action instruction is generated, and the robot space understanding and operation precision is improved.

The invention relates to the technical field of tumor interventional operations, and discloses a visual navigation method based on a tumor interventional operation robot. The method comprises the following steps: acquiring real-time medical image data of a tumor area containing multi-modal imaging information so as to comprehensively present anatomical details; and performing three-dimensional reconstruction on the image data to generate a tumor area three-dimensional anatomical structure model capable of visually displaying a space structure. Key anatomical feature points are extracted based on the model, space coordinates are calculated, a surgical robot intervention path is planned according to the coordinates, and an initial navigation track is generated; and continuously collecting real-time pose data of the robot in an operation, dynamically matching the real-time pose data with the initial navigation trajectory, adjusting motion parameters according to a matching result, and generating a corrected navigation instruction. The method can reflect the intraoperative anatomy condition in real time, dynamically optimize the path, solve the problems that traditional navigation depends on preoperative static images and lacks real-time adjustment, reduce operative complications and improve the treatment effect of patients.

The invention relates to the field of robot vision positioning, and discloses an intelligent hand-eye calibration and self-adaptive correction system and method. According to the method, a mechanical arm is controlled to drive a camera to collect multi-modal calibration data, a convolutional neural network is utilized to identify a calibration plate mark point, and a three-dimensional coordinate under a camera coordinate system is calculated in combination with depth information; meanwhile, on the basis of an encoder and torque data, flexible deformation of the mechanical arm is compensated through a kinematic model and a self-adaptive rigidity model, and three-dimensional coordinates under a base coordinate system are obtained. And obtaining a hand-eye transformation matrix by solving a transformation relation between the two point sets. And repeatedly calibrating before and after operation, inputting the difference of the two transformation matrixes into the fault diagnosis neural network, outputting fault type probability distribution, and generating a maintenance strategy. According to the invention, high-precision hand-eye calibration, automatic compensation of flexible deformation and intelligent diagnosis of system state change can be realized, so that the long-term precision and reliability of a visual positioning system are improved.

The invention relates to the technical field of agricultural picking robot visual perception, and discloses a strawberry fruit detection and picking key point positioning method and model based on deep learning. The method comprises the following steps: acquiring strawberry RGB-D images to construct a data set; on the basis of a YOLOv8-Pose architecture, global feature extraction is enhanced by constructing a C2f-MAMBA module of a fusion state space model, an HWD module is introduced to retain detail features, and a Faster Net Block is used for weight reduction, so that an STRAW-MAMBA model is constructed; outputting coordinates of five key points such as a fruit bounding box and a shear point by the model, and performing geometric compensation on a shielding point; and finally, calculating three-dimensional space coordinates of the key points in combination with depth information. According to the method, a C2f-MAMBA module based on Mama is innovatively introduced, global feature association is effectively modeled, a multi-scale attention mechanism is combined, the defect that long-range dependence is difficult to capture in a traditional CNN is overcome, and the detection and positioning precision of strawberry fruits and picking key points thereof in a complex background is remarkably improved.

Apparatuses, systems, and techniques are disclosed for controlling a robot to execute a task. In at least one embodiment, a current image of the robot in an environment and a text describing the task are obtained. A future image of the robot in the environment is predicted based on the current image and the text. Subsequently, one or more actions are predicted based on the current image, the future image, and the text. The one or more actions can move the robot from a first state corresponding to the current image to a second state corresponding to the future image. The robot executes the sequence of actions to move in the environment.

The application discloses a mechanical arm cable assembly method and system, and relates to the technical field of robot vision perception and control, and comprises the following steps: acquiring point cloud data of a working environment; fitting a cable center line according to the point cloud data and determining a relative pose matrix of a cable terminal and a target slot; determining an initial alignment attitude matrix of a mechanical arm end according to the relative pose matrix, so as to obtain initial joint angles of the mechanical arm and use the initial joint angles as a starting point of an assembly action; after selecting a grabbing point on the cable center line and planning an insertion path, entering an insertion stage from the starting point; according to detection of an insertion force and an insertion direction, performing error calculation and compensation until the insertion force change rate and the end pose error meet a set condition, and then determining that the assembly is completed. The application realizes self-perception, self-judgment and self-adjustment flexible assembly, improves assembly precision and robustness, and is suitable for automatic grabbing, guiding and insertion operation of flexible cables in industrial automation production.

The invention relates to the technical field of underwater robot control, and particularly discloses a visual servo scanning track control system for a deepwater ROV intelligent electromagnetic detection robot, which synchronously acquires visual, acoustic and inertial motion data of an underwater robot and prior geometric data of a to-be-detected structure; visual sparse light spot features and acoustic contour and abnormal region features are extracted, and credibility evaluation is carried out; by establishing a unified state vector and fusing multi-source data and geometric constraints, the accurate pose, the motion state and the perception uncertainty measurement of the robot relative to the underwater structure are estimated in real time; according to the uncertainty measurement, a local expected trajectory used for precise tracking or pose stabilization is generated in a self-adaptive mode; a motion control instruction is calculated through a control law capable of automatically adjusting gain according to uncertainty, and closed-loop control is achieved; according to the invention, continuous, self-adaptive and high-precision execution of detection operation under complex working conditions is realized.

The invention discloses a mechanical arm cable assembly method and system, and relates to the technical field of robot visual perception and control, and the method comprises the steps: obtaining the point cloud data of a working environment; fitting a cable center line according to the point cloud data and determining a relative pose matrix of the cable terminal and the target slot; an initial alignment attitude matrix of the tail end of the mechanical arm is determined according to the relative pose matrix, and an initial joint angle of the mechanical arm is obtained through solving and serves as a starting point of the assembly action; after a grabbing point is selected on the center line of the cable and a plug-in mounting path is planned, starting to enter a plug-in mounting stage from a starting point; and error calculation and compensation are carried out according to detection of the insertion force and the insertion direction, and it is judged that assembly is completed until the insertion force change rate and the tail end pose error meet set conditions. Self-sensing, self-judging and self-adjusting flexible assembling is achieved, the assembling precision and robustness are improved, and the flexible cable automatic grabbing, guiding and inserting device is suitable for automatic grabbing, guiding and inserting operation of flexible cables in industrial automatic production.

The invention discloses a tomato picking point positioning and feasibility analysis system based on RGB-D attitude estimation, and belongs to the technical field of agricultural robot visual perception, and the system comprises an RGB-D data collection module which is used for synchronously collecting an RGB image and a depth image of a tomato cluster; a data preprocessing module; a multi-scale feature extraction and fusion module; the multi-task prediction module comprises a detection and key point prediction head, a picking point depth regression prediction head and a picking feasibility analysis module which are arranged in parallel; the tomato cluster three-dimensional picking system has the advantages that synchronous output of tomato cluster structure attitude information, picking point three-dimensional positions and picking feasibility judgment is achieved, tomato cluster attitude sensing stability and picking point three-dimensional positioning precision are improved, and the tomato cluster three-dimensional picking system is high in practicability and high in practicability. And the robot is endowed with an intelligent picking decision-making capability, and the operation success rate of the tomato picking robot is improved.

The invention relates to the technical field of robot visual guidance, and discloses a visual guidance robot material box grabbing module, which comprises a mechanical coarse positioning and state sensing unit for physically clamping a material box and collecting physical state data; a prediction type pre-positioning and dynamic reference generation unit receives the data, calls a historical deviation database, and generates a prediction target position and a temporary standard reference in parallel; the nested visual search and calibration unit moves to a prediction position and searches to obtain a complete feature image; the accurate deviation calculation and fine adjustment unit compares the image with a reference, calculates a final residual deviation and generates an accurate grabbing position; and the closed-loop learning and data feedback unit executes capturing and feeds back the final residual deviation to the historical deviation database. According to the method, a predicted target position and a dynamic temporary reference are established by using a physical state data vector and a historical deviation database, and parallel compensation of system errors and mechanical errors is realized.

The invention discloses a robot grabbed target recognition and positioning method and system based on a knowledge graph, and the method comprises the steps: constructing a triple knowledge graph containing object appearance features and grabbed point pose parameters through multi-source data integration, and achieving a structured node network; environment perception data of a robot vision module is obtained, vector representation of a target object is determined through processing, if the similarity exceeds a threshold value, a matching node is inquired through the entity linking technology, recognition ambiguity is solved, and an accurate result is obtained; based on a result, retrieving pose parameters in the knowledge graph, performing coordinate conversion to generate a fine-grained grabbing sequence, combining real-time visual feedback, adopting a dynamic adjustment algorithm to optimize a path, and updating the knowledge graph through simulation evaluation to adapt to a new scene. According to the method, the core function of the knowledge graph in grabbing optimization is highlighted, the accuracy, robustness and adaptive capacity of robot operation are improved, and the method is suitable for the fields of industrial automation and the like.

The invention discloses a double-arm cooperative bolt alignment method and system based on hybrid visual servo, and relates to the field of robot visual servo and cooperative control, and the method comprises a visual perception module which is used for obtaining part space pose information based on first image information of a part where a target bolt is located; according to the second image information of the target bolt, obtaining a two-dimensional image coordinate and space depth information of a bolt angular point of the target bolt; the hybrid visual servo control module is used for generating a motion control instruction of the main mechanical arm by utilizing a staged visual alignment control strategy based on the spatial pose information of the part and the two-dimensional image coordinates and spatial depth information of the angular points of the bolts; and the double-arm cooperative control module is used for generating a cooperative motion instruction of the slave mechanical arm based on a cooperative control algorithm in the process that the master mechanical arm carries out visual servo alignment according to the motion control instruction. According to the method, the bolt alignment task under the complex working condition can be completed in a self-adaptive mode, and the intelligent level of overhauling operation is improved.

The invention discloses a tactile image domain migration method and device based on a multi-scale generative adversarial network, and the method comprises the steps: constructing the multi-scale generative adversarial network, and achieving the domain migration between a simulation tactile image and a real tactile image. The generator takes U-Net as a trunk, introduces a multi-scale stacking module, a multi-stage attention gate mechanism and a channel-space attention module, and improves the reconstruction capability of simulation image textures, illumination and contact areas. And the discriminator adopts a multi-scale discrimination structure to realize the discrimination of image authenticity and detail consistency. Through joint training of joint adversarial loss, loop consistency loss, contact area consistency loss, illumination balance loss and frequency domain loss functions, it is ensured that an output image is consistent with a real image in visual and semantic levels. The method can be widely applied to a robot vision-touch fusion perception task, and the migration performance and robustness of the perception model are remarkably improved under the condition of non-paired data.

The invention discloses a video target tracking system based on historical information dynamic propagation and future frame prediction collaborative optimization, which belongs to the field of visual target tracking and comprises a dynamic attention token module, a multi-constraint future prediction module, a real-time bidirectional calibration module, a tracking decision module and a scene adaptive memory module. The dynamic attention token module is used for receiving continuous video frames and generating a multi-scale token sequence dynamically adjusted along with target states, and the target states comprise a shielding area, a movement speed and an appearance scale; the target tracking robustness problem in a complex scene is solved by dynamically fusing the historical trajectory features of the target and future state prediction, and the method can be applied to the fields of intelligent monitoring, automatic driving visual perception, unmanned aerial vehicle inspection, robot visual navigation and the like. A bidirectional optimization mechanism based on historical token propagation and future frame prediction breaks through the limitations of insufficient utilization of time sequence information and insufficient dynamic adaptability in the prior art.

The invention discloses a rain and fog environment multi-scale target detection optimization model for a mobile robot, and belongs to the technical field of mobile robot vision, and the model comprises the following steps: collecting road target images in severe rain and fog weather, constructing a data set, constructing a rain and fog weather target detection model based on improved YOLO11n, and obtaining a target detection result. And training and verifying the constructed model by using the data set, deploying the trained and verified model on a mobile robot platform, performing target detection on the rain and fog image acquired in real time, and outputting a target category, a bounding box and confidence, thereby solving the problems of insufficient target detection accuracy and poor real-time performance of the mobile robot in the rain and fog environment.

The invention discloses an SLAM (Simultaneous Localization and Mapping) method and system based on visual inertial guidance and laser cascade registration, and belongs to the technical field of environmental perception, and the method comprises the following steps: acquiring multi-source data, predicting a pose by adopting an IMU (Inertial Measurement Unit) pre-integration method, taking the pose as an initial value, and outputting a robot visual-inertial pose by combining a camera image and utilizing a visual-inertial odometer technology; constructing a visual inertia factor; taking the vision-inertia pose as an initial value of registration, performing multi-stage registration on the input laser radar point cloud frame, outputting a laser estimation pose, and constructing a laser factor; visual loopback detection and laser loopback detection are respectively carried out, an effective closed loop is determined according to the space-time consistency of the two kinds of loopback detection, loopback factors are constructed, a factor graph is constructed, graph optimization is executed, and globally consistent tracks and map points are obtained. According to the method, the real-time performance is maintained, the registration convergence is improved, and the method is suitable for positioning and mapping in a complex degraded scene.

The invention discloses a bionic robot facial expression synchronization method and device based on visual perception, equipment and a medium, and relates to the technical field of man-machine interaction, and the method comprises the steps: controlling a target bionic robot to execute a target expression control instruction used for achieving a preset expected robot facial expression, and capturing a robot visual facial expression of the robot; based on the robot visual facial expression and a preset expected robot facial expression, carrying out calibration on the target bionic robot so as to construct a hardware compensation parameter vector; executing a facial expression synchronization operation based on the hardware compensation parameter vector and a pre-trained target expression mapping model by asynchronously running a preset sensing thread and a preset driving thread; the preset sensing thread is used for collecting a target face image in real time and outputting a target steering engine control vector through a target expression mapping model; and the preset driving thread is used for outputting a steering engine control instruction at a preset output frequency based on a preset frame compensation algorithm, the hardware compensation parameter vector and the target steering engine control vector.

The invention discloses a robot visual identification decision control method based on deep learning, and relates to the technical field of deep learning, image data of each target object in a target area is collected, a data training sample and a data verification sample are established according to the collected image data, a conversion relation between coordinate systems of a binocular camera is constructed, and a target object is identified according to the conversion relation. A distortion coefficient, an internal parameter matrix and an external parameter matrix of the binocular camera are obtained, a convolutional neural network model is constructed, training is performed according to the data training sample and the data verification sample, a target object is identified through the trained convolutional neural network model, the horizontal distance between the target object and the robot is calculated, and clamping operation is executed. Feature extraction is carried out on image data through deep learning, the defect that a traditional method depends on manual setting of feature parameters is avoided, and the accuracy and adaptability of target object recognition are improved by importing the standard feature value of the target object and combining a comprehensive similarity calculation method.

This invention discloses a robot visual-tactile perception method and system based on transformer and contrastive learning, belonging to the field of robotics technology. The method introduces an enhanced Swing Transformer model as the perception model. This enhanced model uses the Swing Transformer as the backbone network and incorporates a coordinate attention mechanism. Global information is extracted through the sliding window attention mechanism of the backbone network, while local feature extraction is enhanced through the coordinate attention mechanism. This effectively extracts and fuses global and local features from visual and tactile data. In the pre-training stage, the enhanced Swing Transformer model undergoes multimodal contrastive learning training, specifically intra-modal and inter-modal contrastive learning, generating richer and more accurate discriminative features, thereby effectively improving the robot's intelligent perception and manipulation capabilities. This method can effectively extract features in multimodal data fusion and utilize the correlation and synergistic effects of features between different modalities to achieve deeper multimodal data fusion.

The application relates to the technical field of robot vision control, and discloses a robot air attitude real-time correction method based on vision prediction. The method collects a vision data sequence of a robot air attitude, calculates attitude prediction parameters at each time point through a vision processing algorithm, and generates an initial data set; subsequently, adjacent attitude points are analyzed through clustering based on a space-time proximity relationship, consistent sections and conflict sections of motion modes are identified, and a partitioned labeling result is formed. Then, the motion speed and time sequence data of the attitude points in the consistent sections are extracted, the fluctuation characteristics are analyzed in combination with the attitude angle change amount, the motion influence strength under the influence of external environment interference is evaluated, and motion influence superposition evaluation data is generated. Finally, the attitude deviation points in the evaluation data whose response values exceed a reference value and are located in the conflict sections are detected, a deviation point set is constructed, the points needing correction are marked according to the detailed attitude information of the deviation points, real-time attitude correction instructions are generated and output.

The invention discloses a robot vision grabbing method and system for small-size fasteners. The robot vision grabbing method comprises the following steps that an OBB model labeling data set is constructed; a virtual environment is constructed, and a robot body path is verified; the robot body grabs parts with an uncertain number from the part box and places the parts on the flexible pad; then, the robot body moves the tail end to a detection position, data collection and OBB detection are conducted on the part located on the flexible pad through a visual detection head, the posture of the part is judged, and a proper grabbing strategy is selected for grabbing according to the posture of the part; and after being grabbed, the part is placed to a designated position, and cyclic grabbing is conducted till no part exists in the part box. According to the robot vision grabbing method and system for the small-size fasteners, the OBB vision detection model and the path planning algorithm are combined, the robot automatic grabbing algorithm process of the small fasteners is provided, and efficient grabbing of the small-size fasteners in the fixed posture can be achieved only by arranging detection points on the vision probe and the flexible pad.

The utility model relates to the technical field of automatic recharging, in particular to an automatic recharging system adopting a robot visual navigation and infrared receiving mode, which comprises a robot main body and a charging seat, and the robot main body comprises a power supply module, an infrared receiving module, a camera, an electric quantity detection module, a main control module, a driving module and a load. The charging seat comprises a charging circuit and an infrared transmitting module, and the infrared receiving module and the infrared transmitting module are matched with each other. According to the utility model, the robot visual navigation and the infrared receiving are combined, the advantages of the robot visual navigation and the infrared receiving are brought into full play, in an environment with good light, the camera collects surrounding environment images, and after the images are analyzed by the image processing unit, accurate position information is provided for the robot, and the robot is guided to move towards the charging seat; when light is poor or blocked, the infrared receiving module is matched with the infrared transmitting module of the charging base, it is ensured that the robot can stably receive signals, the environmental adaptability of the system is effectively enhanced, and the recharging success rate is increased.

The utility model discloses a POE network camera module for visual binocular splicing of an outdoor courtyard robot, and relates to the technical field of camera modules. The main board is fixedly connected to one side of the mounting substrate; the two lenses are fixedly connected to one side of the main board; and the stabilizing module is located on the mounting base plate, the stabilizing module comprises two mounting frames, the two mounting frames are fixedly connected to one side of the mounting base plate, and fixing ring pieces are fixedly connected to the two mounting frames. According to the outdoor courtyard robot vision binocular splicing POE network camera module disclosed by the utility model, elastic potential energy pre-stored by the coil spring is converted into traction force of the winding rope, and separation detection triggering is combined, so that the clamping piece can be driven to keep the lens stable when the lens is separated, the lens displacement risk in an outdoor bumpy environment is effectively dealt with, and the service life of the lens is prolonged. And damage to a mainboard interface or a connector caused by pulling of a flat cable due to lens separation is avoided.

This invention discloses an automatic workpiece positioning and protection device in a robot vision-guided punching production line, belonging to the technical field of automatic positioning and protection devices for punching workpieces. It includes an insulated transverse mounting plate disposed at both ends of the upper surface of the feeding assembly line support. This device, through the installation of conductive rotating limiting components that cooperate with conductive telescopic rollers, can clamp and fix both sides of the sheet metal, preventing vibration of the sheet metal during punching and thus avoiding displacement of the sheet metal position in subsequent punchings. The installation of conductive slip rings facilitates the detection of the end position of the sheet metal by the punching machine control cabinet. When the remaining sheet metal is insufficient to form a qualified product, the end of the sheet metal does not contact the conductive telescopic rollers, and the punching machine control cabinet can detect the insufficient sheet metal, stopping the punching of the remaining sheet metal. This prevents sheet metal from splashing onto workers and causing irreparable injury.

1. The name of the design product: modular humanoid robot vision embedded support (multi-hole position). 2. The use of the design product: for humanoid robot vision embedded support. 3. The design points of the design product: in shape. 4. The picture or photo that best indicates the design points: perspective view.

The utility model belongs to the technical field of visual experiment platforms, and particularly relates to a composite robot visual experiment platform which comprises a detection platform, a visual mechanism and a rotating disc, a first driving motor; a fixing mechanism; a Z direction slide block, wherein the Z direction slide block is movably arranged above the detection platform; a Z-direction power mechanism; the X-direction sliding block is movably connected to the Z-direction sliding block, and the X-direction sliding block is movably connected to the Z-direction sliding block; an X-direction power mechanism; the rotating air cylinder is fixed to the X-direction sliding block, and the visual mechanism is fixed to an output shaft of the rotating air cylinder; according to the utility model, the plurality of fixing mechanisms can simultaneously fix a plurality of different objects to be tested, the first driving motor is utilized to drive the rotating disc to rotate and transfer the objects to be tested, and the height, the horizontal position and the inclination angle of the visual mechanism can be adjusted to adapt to different experiment requirements and meet real processing scenes; and the experiment result is closer to the real situation.

This invention belongs to the field of robot vision processing technology, specifically a method for preventing robot falls based on a depth camera and a single-point laser. The method includes the following steps: The robot's main controller receives environmental data collected by the depth camera and the laser sensor, respectively, and processes and analyzes the data to determine whether the environment in front of the delivery robot is a fall-prone environment. If either the processed data from the depth camera or the laser sensor indicates a fall-prone environment, the robot is deemed to be in a fall-prone environment, and the current dangerous state of the robot is reported to the backend server, which then stops the robot's operation. If neither the processed data from the depth camera nor the laser sensor indicates a fall-prone environment, the robot continues operation. This invention prevents robots from tipping over and causing damage in complex application scenarios, such as on stairs where height changes rapidly.

The application discloses a kind of robot vision guidance grabbing devices, it is related to industrial automation and robot technical field, including: robot main body, visual perception module, control module, end execution module and calibration module.The robot vision guidance grabbing device of the embodiment of the application, through the collaborative design of each module, effectively improve the precision of the grabbing of the application, anti-interference ability and working condition adaptability, with the advantages of convenient operation, stable operation, can adapt to workpiece grabbing demand in different scenes.

A robot vision‑based automatic grasping device, comprising a base (1). A driving arm (2) is mounted at the top of the base (1); a support column (3) is mounted at the other end of the driving arm (2); a vision detection probe (4) is provided at the top of the driving arm (2); a grasping assembly is provided at the top of the base (1); the grasping assembly comprises an adjustment rod (5); the adjustment rod (5) is fixedly connected to the bottom of the support column (3); a support plate (6) is fixedly connected to the surface of the adjustment rod (5); clamping plates (7) are provided at the bottom of the support plate (6). By means of the provision of the grasping assembly, during material grasping, the driving arm (2) can drive the clamping plates (7) to move to the surface of the material, then a driving motor (14) drives the clamping plates (7) to move, and when the three clamping plates (7) move relative to each other at the same time, the material can be clamped from three sides at the same time, thereby improving the stability of material grasping; moreover, rubber pads (31) are used to provide a cushioning gap between the clamping plates (7) and the material, thereby providing a protective effect on the material. Also provided is a robot vision‑based automatic grasping method.

The invention provides a robot visual positioning and path planning method and system, and the method comprises the steps: extracting static obstacle features and dynamic target motion features from an environment image, and constructing an environment feature map containing time-space correlation information; generating a probability distribution prediction area of the dynamic target; rasterizing and layering a space area between the current pose and the target position of the robot, and endowing grids of different levels with passage cost values; generating an initial path from the current pose to the target position, and performing smooth optimization on the initial path; in the movement process of the robot, monitoring the change of an environment characteristic spectrum, and when it is detected that the actual movement track of the dynamic target deviates from a prediction area and exceeds a deviation threshold value, adjusting a local path section; and calculating a speed and steering control instruction of the robot according to the adjusted local path section, and sending the control instruction to a robot motion control module. According to the invention, the accuracy, safety and real-time performance of robot path planning can be ensured.